Adventures in Linear Types

Lately, I have dedicated a large part of my free time to audio software. I have done this mainly out of interest in the subject due to my history in music. But at the same time, I also thought writing audio software could be a fun passion project or even a small business that I could work on alongside my day job. I don’t see myself replacing my current job with this, but maybe I could dedicate 20% of my work time to it.

The world of audio software is a pretty exciting place. It involves a lot of low-level systems like signals and real-time operations, complex math at times, and something you can feel or at least hear. And what’s great, I don’t have any background in this stuff!

Now I have programmed most of my life and played around with RTOS, but when it comes to writing algorithms for manipulating digital signals, that’s new stuff for me. However, I have experience with the topic from the user point of view since I have been making music for almost as long as I have programmed. This experience involves playing instruments, how effects affect the sound, how mixing and mastering works etc. But what do linear types have to do with any of this?

Signals in the Wild

Like I said earlier, signal processing (not necessarily just audio) is very low-level stuff. So when working with signals in software, you often need to work with C or C++. This is mainly due to the performant and close-to-hardware nature of the languages required to handle and manipulate signals optimally and efficiently.

Digital signal processing is also full of algorithms. The standard workflow for people in this industry seems to be that these applications are prototyped on some high-level language before being produced. Often in languages/tools like MATLAB, Octave, Mathematica, and similar, very heavily math-oriented languages and tools. Julia has appeared to grow in popularity also in this world. These high-level languages are mainly used due to the speed of development.

It is also not uncommon to see FPGA being used in these applications. For several reasons, they are reconfigurable hardware, so you can tailor and deploy computation units and data buses specifically designed for your particular needs. So if you’re working with digital hardware, you can’t go wrong with FPGAs. In this world, VHDL or Verilog comes in handy.

As you can see, overall, the applications tend to involve a lot of different low-level concepts but, at the same time, high-level topics in terms of prototyping. But, as the post’s header might hint, I’m not interested in the prototyping aspects of signal processing since I think those are all well and good. Instead, I’m interested in having a small thought experiment on whether the low-level elements could be improved somehow.

I would consider myself a functional programmer first and foremost, even though I mainly write imperative and/or object-oriented code, at least professionally. Now in my free time and in non-trivial side projects (that are not signal processing related), I like to work with weird languages like Haskell or Common Lisp. Unfortunately, as I mentioned above, almost all the work in this signal-processing world is written in C or C++, emphasizing the latter. However, I completely understand why these languages are used since we talk about real-time programming, so latency needs to be minimized.

“Real-time” can be understood that the program has to produce the correct result but also on a certain amount of time (which varies between systems).

If we use audio processing as an example, typically, you would have some sort of processing function in your code that would work in the audio callback:

process :: BufferRef -> ()

This function would get its callback from a sound card or some input device, e.g. microphone. After it has received its callback, this block of code (whatever might be inside it) would write the corresponding audio data into the given buffer. Which would then be played by the speakers or vice-versa when recording.

This procedure is basically what should happen in real-time lots of times when we are doing audio processing. Audio software is often set up to send these audio callbacks from a high-priority “real-time” thread with a very short latency between the callbacks, ~1-10ms (varies between systems).

To achieve this minimal latency between callbacks, you often can’t rely on garbage collection since you can’t be sure when your program launches it. I dare to say that most of the software benefits from GC significantly, but in the audio, making GC right is very hard. What makes it hard is that if GC launches at the wrong time or the latency between callbacks gets too large, garbage data will leak into the buffer, causing unwanted sounds.

Most other software might only see a slight latency in their computations if they do profiling, so that might not be the end of the world, of course, depending on your context. But in audio, you cannot let that happen since you can literally hear that glitch, which is unforgivable.

When it comes to C or C++, I think everyone knows their foot guns that involve memory management. Thankfully in modern C++, it’s not that bad (as long as you follow core guidelines, ) but there is still a lot of unnecessary baggage when it comes to safe code in these languages.

Could there be any way we could use garbage-collected language while doing “real-time” operations and how that could be achieved?

Linear Types

GHC 9.0 introduced support for Linear Haskell, which can be enabled with -XLinearTypes. One of the significant use cases for linear types is implementing latency-sensitive real-time services and analytics jobs. As I mentioned earlier a major issue in this use case is GC pauses, which can happen at arbitrary points for reasonably long periods. The problem is exacerbated as the size of the working set increases. The goal is to partially or entirely eliminate garbage collection by controlling aliasing and making memory management explicit but safe.

So what, then, are linear types. Henry Baker described linear types and their benefits in his paper Lively Linear Lisp — ‘Look Ma, No Garbage!’ and also on “Use-once” variables and linear objects: storage management, reflection and multi-threading . As you can see, we are not talking about a new topic. We are talking about types whose instances must be held in a linear variable. A variable is linear if it’s accessed exactly once in its scope. Same for a linear object, their reference count is always 1. When we have this safety guarantee on the type-level, we can avoid synchronization and GC and also, we could update linear objects in place since that would be referentially transparent.

Avoiding Garbage Collection

So why can we avoid synchronization and GC with linear types? If we would consider the following function as an example:

linearFunc :: a %1-> b

On their own, linear types only gives a type to functions that consume their argument exactly once when their result is consumed precisely once. So, alone, they don’t make your programs any faster or safer for resources. Still, they allow you to write many optimizations and resource-safe abstractions that weren’t possible before.

First, since linear values can only be used once, these values cannot be shared. This means that, in principle, they shouldn’t be subject to GC. But this depends on the value consumer since that may very well do some sort of de-allocation on the spot. One way to mitigate this could be to store these values to heap outside of GC’s control.

While utilizing heap for these values alone would diminish the GC, it would introduce some overhead to your program, which could increase the total running time of your application. But if we continue using real-time systems as an example, this isn’t necessarily bad.

In real-time systems, optimizations often happen only to the worst-case scenarios. This is because you don’t really care about your latencies as long as they stay within the particular window. But you watch that those latencies should never go above your maximum limit, and this is primarily where optimisation utilizing linear types could come in handy.

Practical Linear Types

Linear types are a blessing in GC languages if you intend to do anything safely in the low-level world. I would like to continue this post with some practical examples of how Haskell utilizes these types and how they can make low-level optimizations and resources safer in your Haskell code, but that deserves its own post.

Missing Technical Writing

This blog is a few years old, and during its time, it has had a couple of different style changes (and a couple of domain changes). Initially, this blog started in 2018 when I was in school, and I thought some blogging platform could improve my writing.

Most of the posts back then were some assignments for various school related works. The blog was a clever publishing platform for those and was accepted in many courses. Unfortunately, most of those have since been deleted and forgotten.

During those days, this was very tech-oriented, focusing more on writing various tutorials about different subjects, mainly around distributed computing. Then I landed my first job as a software engineer, where I could put the tutorials I had written to practice. Unfortunately, at about the same time, I started losing my motivation on regular writing, mainly due to time constraints.

Around late 2020, or early 2021, I started to rekindle my writing habit with different styles and topics. Then I began to write down my thoughts about the world around me. So many topics were about music, art, and some tech stuff here and there but not really “technical writing”. I was mainly sharing my opinions about the various topics that interested me. While I enjoyed this little bit more pondering writing style - some could call ranting - for me, it felt that it was missing something.

I missed the technical writing. But now, since I had established a particular style in my writing, it felt out of place to start writing about various geeky subjects revolving around computer science. That being said, all those writings in the tutorial and guidance way or these recent little bits of the more analyzing way of writing haven’t gone to waste. However, I feel that now I know what I want to write.

I think the combination of both will suit me the best. At the same time, I don’t necessarily see myself writing some basic how-to tutorials anymore. But I feel technical analysis with various concrete implementations would be interesting to research and write. My main interests generally lie in music and arts and how these intertwine with computer science. However, my professional life is linked to designing and implementing distributed systems, distributed computing in big data environments, systems engineering, performance, reliability, compilers and programming languages. I feel that those might also be regular visitors to my ramblings.

My professional life has also revolved around these topics. So I feel I can write a lot in conjunction with my work. Especially since working with large distributed systems, you’re bound to stumble upon some weird things, making this analysis-oriented technical writing an excellent tool for thought. I would want to have this semi-regular to regular writing habits that I once had, but that is something I can’t promise. Post at a time!

But since this is my blog, I feel that occasional rambles and shitposts about various topics are in place. Do people want to read those? Probably not, but it’s fun to vent every once in a while. So going forward, I think you can expect a significant portion of more technical commentary with a creative angle alongside a smaller portion of things that entertain and occasionally horrifies me.

Views on Digital Declutter

I’m very prone to procrastination. While I wouldn’t say that I have focus issues, I have noticed that I can easily spend hours on non-essential sites that don’t bring anything to my life. Social media has been one of them. I have always had a pretty weird relationship with social media. I joined Facebook and Instagram a long time ago because many of my friends and family were already there. While I never did post stuff actively, I always noticed that I just ended up mindlessly surfing these, especially on Instagram.

A couple of years back, I became conscious of this and decided to delete my accounts on these platforms without giving too much thought to it. While leaving these platforms was pretty easy for me, I noticed that I had just replaced these with some other platform, YouTube in my case. After which, I started spending countless hours on that platform instead. Back then, I didn’t consider this habit as bad as mindlessly browsing Instagram or Facebook despite it being the same. I think I just rationalized it to myself as being educational or informative in a better way than other platforms.

A year or two passes without Instagram or Facebook completely fine, but I wanted to start using them again for some reason. Maybe I thought I had already been cured of this disease, so I could have a healthy relationship with them from now on. I also had professional reasons behind this since I thought these platforms offer a great way of marketing your art to others, which is true in some cases. However, I quickly noticed similar behaviour when I was last on these platforms. So after a couple of months of trying to get back in, I just felt repulsed by them and decided to leave them again. When it comes to marketing, that is not for me. I understand the benefits of being an artist in social media. Still, since I mainly enjoy that as a passionate hobby, I don’t see the need for being on social media.

So at the time of writing this, I think it has been about six months or so of living without these. Still, I’m conscious of my unnecessarily large usage of YouTube, News etc. While comparing my use with Instagram, I still wouldn’t consider watching YouTube or regularly checking news as bad as mindlessly scrolling through your feeds. I still noticed similar behaviour on those I struggled with, for example, with Instagram. I became conscious about randomly picking up my phone and scrolling through the news even though I had just read them or letting YouTube’s autoplay roll for long periods without giving it too much thought. So I wanted to tackle these habits.

I have noticed that the most extreme methods work the best when fixing some bad habit, at least in my case. So I didn’t want to ease when trying to have a healthy relationship with these applications but instead went cold turkey immediately.

So how has this worked for me? I think great! In the beginning, I noticed how much free time I have when I don’t spend on useless things. Also, initially, I occasionally picked up my phone by instinct. However, I quickly realized I had no applications to spend mindlessly surfing, so I quickly grew out of this habit. At first, I felt boredom slightly when I couldn’t spend time on these apps, but thankfully I realized that this spare time needed to be used elsewhere. Before this, I was already reading relatively a lot, about three to four books per month, but I have almost doubled that number nowadays. I also wrote about time management between multiple passions a while ago, where I pondered how I manage time between, for example, programming and music. After ditching distractive sites entirely, I have felt that the time management between these activities and my work life hasn’t been an issue. Finding time for various pet projects and serious work outside my life is straightforward since I don’t spend my time on useless stuff anymore.

Do I see myself using these applications in the future? Well, I want to read the news and continue to do so, not just constantly. I usually catch up with recent events in the morning, but I don’t desire to install any news apps on my telephone. When it comes to these streaming platforms, YouTube, Netflix, etc., I could live without them. There are lots of good information on these platforms, so if I need to watch some videos, I can allow myself to do so. However, I don’t want them to control my life in a way that I’m uncomfortable with.

Passions and Time Management

I have always enjoyed reading about other people’s productivity hacks and workflows, in general, regarding whatever they might be doing. However, I often stumble upon reading how people maintain an extravagant lifestyle with dozens of different hobbies, interests, and passions with ease. So it makes me wonder how they manage their time to maintain a healthy level of participation in their interests without burning out.

I don’t have dozens of different passions or interests in my life, but my passions tend to be quite large on their own, so when combining those with 40 hour work week, I need to think about my time management thoroughly. These passions I would consider to be writing, music and programming. Fortunately, I currently work in the tech industry, so I can make a good living by doing one of my passions. Writing on its own isn’t necessarily a huge topic/interest in the sense that it only consumes time, but the practice itself is pretty straightforward. On the other hand, music is time-consuming, and it involves many different activities, in my case. Sure, you could argue that writing includes other practices, too, like planning what to write, but music is on another level. I play multiple instruments, which I record for myself and others in my home studio. I enjoy composing tunes, adding some mixing and mastering to this, and you need to sacrifice a lot of time for this. Programming is also something that I enjoy spending my time on. While I do it for a living, what makes me genuinely like it is the projects I work on in my free time, whether it’s my pet projects of various sizes or some open-source projects.

Finding the time and focusing on the task at hand isn’t necessarily an issue for me, but it’s more about maintaining a healthy balance between all these passions that I hold dear to me. Although that being said, I would consider myself a very gifted procrastinator, so focusing on the task at hand can often be tricky. However, focusing becomes no more an issue once I’ve gotten into the flow. A more significant problem here is often finishing projects rather than starting a new one (which I feel is entirely another issue to improve).

When I work on my passions, I tend to focus for days or weeks on one passion, e.g. programming, neglecting my other passions like writing and music. This on its own isn’t necessarily a bad thing since I don’t feel that I’m wasting my time when I’m doing something that I enjoy and can get rewarded in multiple ways. But I would like to maintain an equal balance between my passions.

Tools to the Rescue

A while ago, I started reading about how other people have managed their time with multiple passions/hobbies, and almost unanimously, everybody used various schedules for this. So I have already used a “life management” system for a long time to handle all my to-do lists and schedules related to my home and work life (insert praising words about emacs’ org-mode here).

I’m not going into details about how I manage my life with org-mode, but if you’re interested in the tool, I would recommend going through articles found at Org for GTD and other Task management systems and from Rainer König’s OrgMode tutorial

I realise that I have been missing for a long time in my current setup because I haven’t scheduled when to work on what project. While I’ve split my free-time projects into sub-tasks and occasionally schedule and deadline when to work/finish those, the work has always been very sporadic. The result has often been that I work for an X period on one project and then move on to another, so I usually just forget what I was supposed to do on the earlier project. For me, this often leads to unnecessary postponing or cancelling/removing tasks completely. I also quickly start saying, “I’ll do it tomorrow”, which everyone knows won’t happen.

Starting Light

So I started to approach the whole concept of time management between multiple different interests: to make dedicated timeslots and days for whatever I might be working on. I maintain numerous ongoing projects that don’t necessarily have deadlines but are just larger projects that I want to work on it from time to time. Then these projects have sub-projects which are usually scheduled with deadlines. These projects and sub-projects might include something related to work, home, open-source work, recording, or simply writing something.

I nowadays approach working these by dedicating timeslots for something on a specific day. My work life and day-to-day home stuff take a good portion of my days, but I try to use it as efficiently as possible the rest of the time. So on Monday, I might work on some programming-related endeavours based on my backlog, Tuesday something else and so on. The way I still approach tasks haven’t changed in any way, as in I still manage my tasks and TODOs and keep track of them, but nowadays, I just dedicate specific days to specific interest/passion.

Conclusion

This way, I don’t feel that I’m neglecting the stuff I want to work on. Issues with this kind of approach are the context switching almost daily. However, this kind of switching isn’t necessarily a bad thing. I don’t want to think about work-related topics after I’ve “clocked in the hours”, but I want to do something to relax or move my thoughts elsewhere. I’ve mainly stumbled with issues that when you might focus on programming one day, you focus on music. While this switch on its own hasn’t been too bad in my case, when I get back to, for example, programming after doing something else for several days, it always takes a while to get back to the flow. But I do believe that this is just missing practice in the world of managing multiple different passions.

I have now split my time between multiple passions for several weeks, which is an excellent way to go. It has also taught me about the stuff that I genuinely want to work with since when you write down what you want to do and when it’s easy to spot the stuff that you don’t want to work on or just don’t have an interest in it. So this also works in my case to find the topics that genuinely interest me. Will I continue to manage my time like this for long? Well hopefully. I feel that this way, I can contribute to all the stuff that makes my life interesting, so obviously, I wouldn’t want to miss that.

Music, AI, and the Future

Since the discussion about artificial intelligence has become mainstream, I have started to ponder AI’s possible impacts on our day-to-day lives. While I work in tech, I come from this “culture and arts” background, at least a little bit. I did some theatre when I was young and have worked with music in one way or another for most of my life. This background has got me thinking about how AI could affect these fields.

We have seen multiple interdisciplinary works mixing artificial intelligence with various art forms, like drawings, paintings, music etc. Already drawings and paintings generated with AI present a superb quality in those, in which you cannot distinguish whether these were created by AI or an actual human. On the other hand, music is not quite at that level yet, in my opinion. At least in the form of an entirely generated song by AI. That being said, I have heard great pieces utilizing both human touch and AI, where AI plays a supportive role in the whole work. Similar things can be seen in all creative endeavours where AI could be utilized.

If AI gets used more and more in these creative projects with great success, to me, it raises a question, can human art be entirely replaced with AI? I believe it would be naive to say that it couldn’t. But, considering the possible future where we cannot distinguish humans from computers, how could we determine this kind of smaller medium like song or book on how it was created or who created it? As a consumer of these kinds of mediums, does it matter if some algorithms made your new favourite novel to provide the same feeling that you might get from reading a regular author’s book?

They Are Taking Our Jobs

To put it shortly, AI can replace anyone’s job who happens to handle bits in one way or another. AI can do it way better than you ever can in these jobs. So when we talk about “creative jobs”, how can you do it better than someone else? Are you possibly better at drawing than someone else? Or can you compose better symphonies than someone else? What makes you better? Is it purely a technical thing, or is there something else? When we talk about painting or drawing’s technicality, sure, you could argue that your “pen strokes”, etc., might be better than someone else’s. But does this make it better art?

Already there has been a trend of AI-generated music populating different streaming platforms. Currently, that music has almost always been something simple in which AI can excel. This could be called elevator music or Muzak. This kind of music is most likely something that many people wouldn’t mind because it’s generated with a computer and lacks the human touch. But how would people feel if there were a chart-topping song entirely generated with AI? Again, I believe many people wouldn’t like that, other than a few tech geeks who might think it could be cool (me included).

Could AI then fully replace the human touch in our art forms? We might wait for that to happen for a very long time. Still, as I said earlier, it would be naive to think that this couldn’t happen, especially in the future, where we have reached a certain level of intelligence where we can’t distinguish each other from humans and machines.

So what could this mean for our “blue-collar” artists? What could be the driving force for them to create new art if the audience doesn’t know if it was created by a computer or a human? To me, that seems very grim.

Creative Programming

If we can’t beat them, join ’em? Right? If we think this will be the future, while it might not be a very uplifting thing to consider, it’ll most likely be very realistic. While AI will have dire repercussions on our life in the future, I also believe it can be used for great good. Whether AI is used in health, fighting climate change etc., there are many good use cases. In my opinion, utilizing AI in the arts is also one. Should you create your next song or novel entirely with AI? Possibly not, although GPT-3 has shown some great results on how good text it can write.

I like to write or play music, so I don’t want to replace the artificial process I enjoy so much. So I could utilize it in my creative endeavours by working with it side by side. It could possibly generate some ideas for my next blog post, novel, poem or whatever. For example, AI could be taught with the text of a long list of your favourite authors or songs by your favourite bands. Based on this knowledge, maybe some of the possible ideas it could generate could be finished by a human giving the final piece that human touch.

Conclusion

So while the future might look dark and grim for us, maybe we could make some use of it, so at least we might have a little bit of enjoyment. Thankfully we are a long way from this singularity that many people tend to talk about, but the trend has shown to be moving towards that kind of future. So rather than fighting against it, at least personally, I want to make the best use of our technical achievements in one way or another. Who knows if the next big novel or piece will be created with AI or other great technological invention. That said, I have already found many great ways to utilize AI in my creative projects, so who knows what might come from those.