Building GPT From Scratch

Stumbled upon an interesting video by Andrej Karpathy talking about building Generatively Pretrained Transformer (GPT). I definitely need to take some time to start looking through the papers linked in the video on wonder how things work out.

Youtube video kCc8FmEb1nY

Internet Is Empty and Devoid of People

Proving you’re a human on a web flooded with generative AI content

Humans who want to engage in informal, unoptimised, personal interactions have to hide in closed spaces like invite-only Slack channels, Discord groups, email newsletters, small-scale blogs, and digital gardens. Or make themselves illegible and algorithmically incoherent in public venues. […]

We’re about to drown in a sea of pedestrian takes. An explosion of noise that will drown out any signal. Goodbye to finding original human insights or authentic connections under that pile of cruft.

Many people will say we already live in this reality. We’ve already become skilled at sifting through unhelpful piles of “optimised content” designed to gather clicks and advertising impressions. […]

As the forest grows darker, noisier, and less human, I expect to invest more time in in-person relationships and communities. And while I love meatspace, this still feels like a loss.

Am I even real anymore?

Music, AI, and the Future

Since the discussion about artificial intelligence has become mainstream, I have started to ponder AI’s possible impacts on our day-to-day lives. While I work in tech, I come from this “culture and arts” background, at least a little bit. I did some theatre when I was young and have worked with music in one way or another for most of my life. This background has got me thinking about how AI could affect these fields.

We have seen multiple interdisciplinary works mixing artificial intelligence with various art forms, like drawings, paintings, music etc. Already drawings and paintings generated with AI present a superb quality in those, in which you cannot distinguish whether these were created by AI or an actual human. On the other hand, music is not quite at that level yet, in my opinion. At least in the form of an entirely generated song by AI. That being said, I have heard great pieces utilizing both human touch and AI, where AI plays a supportive role in the whole work. Similar things can be seen in all creative endeavours where AI could be utilized.

If AI gets used more and more in these creative projects with great success, to me, it raises a question, can human art be entirely replaced with AI? I believe it would be naive to say that it couldn’t. But, considering the possible future where we cannot distinguish humans from computers, how could we determine this kind of smaller medium like song or book on how it was created or who created it? As a consumer of these kinds of mediums, does it matter if some algorithms made your new favourite novel to provide the same feeling that you might get from reading a regular author’s book?

They Are Taking Our Jobs

To put it shortly, AI can replace anyone’s job who happens to handle bits in one way or another. AI can do it way better than you ever can in these jobs. So when we talk about “creative jobs”, how can you do it better than someone else? Are you possibly better at drawing than someone else? Or can you compose better symphonies than someone else? What makes you better? Is it purely a technical thing, or is there something else? When we talk about painting or drawing’s technicality, sure, you could argue that your “pen strokes”, etc., might be better than someone else’s. But does this make it better art?

Already there has been a trend of AI-generated music populating different streaming platforms. Currently, that music has almost always been something simple in which AI can excel. This could be called elevator music or Muzak. This kind of music is most likely something that many people wouldn’t mind because it’s generated with a computer and lacks the human touch. But how would people feel if there were a chart-topping song entirely generated with AI? Again, I believe many people wouldn’t like that, other than a few tech geeks who might think it could be cool (me included).

Could AI then fully replace the human touch in our art forms? We might wait for that to happen for a very long time. Still, as I said earlier, it would be naive to think that this couldn’t happen, especially in the future, where we have reached a certain level of intelligence where we can’t distinguish each other from humans and machines.

So what could this mean for our “blue-collar” artists? What could be the driving force for them to create new art if the audience doesn’t know if it was created by a computer or a human? To me, that seems very grim.

Creative Programming

If we can’t beat them, join ’em? Right? If we think this will be the future, while it might not be a very uplifting thing to consider, it’ll most likely be very realistic. While AI will have dire repercussions on our life in the future, I also believe it can be used for great good. Whether AI is used in health, fighting climate change etc., there are many good use cases. In my opinion, utilizing AI in the arts is also one. Should you create your next song or novel entirely with AI? Possibly not, although GPT-3 has shown some great results on how good text it can write.

I like to write or play music, so I don’t want to replace the artificial process I enjoy so much. So I could utilize it in my creative endeavours by working with it side by side. It could possibly generate some ideas for my next blog post, novel, poem or whatever. For example, AI could be taught with the text of a long list of your favourite authors or songs by your favourite bands. Based on this knowledge, maybe some of the possible ideas it could generate could be finished by a human giving the final piece that human touch.

Conclusion

So while the future might look dark and grim for us, maybe we could make some use of it, so at least we might have a little bit of enjoyment. Thankfully we are a long way from this singularity that many people tend to talk about, but the trend has shown to be moving towards that kind of future. So rather than fighting against it, at least personally, I want to make the best use of our technical achievements in one way or another. Who knows if the next big novel or piece will be created with AI or other great technological invention. That said, I have already found many great ways to utilize AI in my creative projects, so who knows what might come from those.