Extravagancy in Tech

I’ve started to ponder the repercussions of this trend of extravagant architectural choices in the tech industry. But unfortunately, these options seem prevalent in this current era of cloud computing. At least, I seem to stumble upon these regularly when working with a wide variety of distributed systems. Great examples of this kind of trend are various Kubernetes setups in projects where you could easily manage to progress without it or some data infrastructure solution that feels like a sledgehammer for hitting a small nail.

I’m not bashing these technologies since I enjoy working with them, and I work with them daily. They have their purpose, but this purpose is often meant for a larger picture in mind. Now, if we focus on the example of Kubernetes, sure, it can bring many benefits, like easier deployments, reduced complexity on large projects, and often reduced costs. But no one can argue that it can be overkill in many projects. If it’s not needed, it mainly brings unnecessary complexity and reduces productivity in these projects. So it can be a double-edged sword. But I don’t want to focus on these singular technologies in this topic since they feel minor on the grand scale.

Implications on Our Evolution

When we move more to this science-fiction picture of the future, we need to start thinking more about topics such as transhumanism and how we will live with machines that’ll outsmart us. Understandably, issues associated with transhumanism, like the singularity, AI, nanotechnologies, cybernetics, and much more, are challenging to discuss, first of all, on a technological level and on a moral and ethical level. But, on the other hand, it is also hard to say that we will even ever see the rise of these kinds of technologies. It could be that our civilization can see that these inventions are possible, but we cannot implement these. On the other hand, it could also be that technological evolution has also started to get so rapid that we will see a significant turn of events in these topics in the near future. [[https://www.kurzweilai.net/the-law-of-accelerating-returns ][Overall technological evolution grows exponentially, so the time between significant inventions gets shorter and shorter]]. So, we can only speculate on how things might turn out.

Whatever the outcome may be, I believe that some degree of optimism is in place. However, I think the singularity is inevitable, and most of the industry’s actions indicate that the path is not good. These actions are the main reason why these over-the-top architectural choices might hint at something that might be inevitably bad.

When I talk about some projects using these “sledgehammer” solutions in projects where they aren’t necessary, I’m overall talking about a small pesky thing. What worries me about this topic is that we are using these kinds of hyped-up tools, which happen to be the month’s flavour in every project; what could this mean, for example, in the development of AI or other future technologies? Could we seem to have endless resources cause of something that cannot be reverted? Bill Joy wrote a great essay about the future not needing us , which makes it scary to think that we run these extravagant systems just mainly because we can. A similar thing applies to data collection and many other issues in privacy. Most big platforms that utilize some tracking tend to collect a lot of data, which often isn’t used thoroughly, so the data is collected to build minimal information about the user. Possibly the rest are saved for later.

Clever Usage of Limited Resources

Back in the olden days, when I wasn’t even born, computers tended to be understandably very limited in terms of resources. Computing has evolved tremendously since allowing us to use these kinds of larger-than-life solutions in environments where they wouldn’t necessarily be needed. Has the quality of systems or programs evolved directly proportional to the increase in computing power? Definitely not. The fact that these kinds of powers are available to us everywhere has possibly increased the number of innovations since more people can start thinking of possible uses for these machines that are all around us because they are in contact with them regularly. Although you could think that since more people are in contact with these machines daily, it would equal more interest in programming, etc. This doesn’t seem to be the case.

Where I’m getting with this is the fact that the quality tends to be going down when we go towards the future; how could this be tackled? Clearly, this kind of wild west design in these crucial systems can’t continue.

Strategic Approach in the Development

When we talk about this extravagancy phenomenon in tech projects, it tends to affect the program/system developers the most. Often, they are not making these decisions since it tends to be someone from the ivory tower who often plans these decisions. Thankfully, these people have at least some background in these systems relatively frequently but not always. So should the developer’s opinions matter more when considering various options for your project? Sun Microsystems had a great idea when they marketed Java to people. Sun was a hardware company that figured out that they had to please programmers first to sell more hardware, which resulted in Java being one of the most widely used languages today. Now, did Java please programmers? Maybe back when people hated C++, but opinions seem to have shifted recently, although both languages still enjoy immense support.

Overall, I think these large systems have their places in many domains, but these domains where their power could use efficiently are very rare. This ends up in a situation where we either have a lot of unnecessary computing power just lying there or used for something unnecessary. Now systems have this unnecessary complexity that mainly hinders the people’s workflow in developing the whole system.

I also think that doing something because “this might be needed in the future” is a bad practice since this tends to end up in an infinite loop of unnecessary work. Since more straightforward solutions tend to be quite often good enough for most projects with much better developer experience and much better efficiency. These solutions also often allow effortless migration to a bigger and better solution if needed. So don’t optimize if it’s not necessary.

Contemplating Web Analytics

I started to rekindle my, unfortunately, lost writing habit a couple of weeks ago. I set up Google Analytics for this page mainly due to its easy use to see simple analytics. I was only interested in visitor count and possibly where my readers’ were coming from. Google Analytics is a massive tool with massive amounts of data going into it. I tried to restrict this collection as much as possible, which suits my personal blog’s needs.

Then my page rose to the front page of Hacker News, and it started to get a lot of traction. Suddenly, thousands of readers came every day to my pesky little page with just a few posts as I followed the visitor counts rising in my Google Analytics view. That got me thinking about the ethics of this kind of tracking. Which then ended up with me deleting my account and data from it.

Discomfort With Tracking

Before I deleted my data and account from Google Analytics, I looked for alternatives. I stumbled upon many other privacy-oriented and GDPR-compliant analytics platforms, which at first seemed promising. Also, having good options for ever-prevalent Google Analytics is a great thing. But despite these features, they don’t remove the uneasiness mining your users’ data causes. Of course, we are talking about spying here. Thankfully there are now some restrictions regarding personally identifiable information (PII), at least in the GDPR, limiting the shadiness quite a lot. But that brings new issues in handling this kind of information since you need to be sure that your software doesn’t leak this information. Thankfully, opting out entirely from collecting PII in your software is an option.

I understand why people might want to add at least simplistic tracking to their sites since it can provide helpful information about your content, companies can see how users use their site, and the list goes on. Especially when you combine Google Analytics, or similar analytics tool, with ads, companies can reap significant benefits from this kind of tracking. But 9 of 10 sites shouldn’t need this. You could argue that most administrators use this tracking only for dopamine fixes and don’t utilize the tracked data. Even though they might use it somehow, how do they inform the user? I dare to say that information about data usage is almost always written in some shallow boilerplate text or in no way at all.

Informed Consent

GDPR highlights mainly four things about data usage:

It gives EU citizens the final say on how their data is used. If your company handles PIIs, there are tighter restrictions on handling these. Companies can store/use data only if the person consents to it. User has rights to their data.

Consent is the crucial part here since many sites lack on this front. There has been a lot of discussion about what should be considered consent. GDPR Art. 6.1(f) says that “processing is necessary for the legitimate interests pursued by the controller or by a third party”. Now legitimate interest is relatively shallow, and quite a few authorities in Germany, for example, consider that third-party analytics do not fall under “legitimate interest” . You can utilize consent management platforms to ensure the user’s consent before dropping the tracking code on your page. But this then raises the question of what can be considered consent.

Drew DeVault wrote a great post about web analytics and informed consent . Informed consent is a principle from healthcare, but it still can offer significant elements to be utilized, especially in technology and privacy. Drew split up the essential elements of informed consent in tracking to these three points:

Disclosure of the nature and purpose of the research and its implications (risks and benefits) for the participant and the confidentiality of the collected information. An adequate understanding of these facts on the part of the participant, requiring an accessible explanation in lay terms and an assessment of understanding. The participant must exercise voluntary agreement, without coercion or fear of repercussions (e.g. not being allowed to use your website).

Considering these essential elements of informed consent, we agree that most tracking sites don’t follow these guidelines.

Thankfully trivial tracker blocking is supported already in many browsers, which makes this issue slightly more bearable, and also, you’re able to download external tools to do it. But still, this kind of approach is pretty upside down.

All Kinds of Cookies

Unfortunately, ad-tech companies have tried to make blocking these harder and harder by constantly evolving these cookies to evercookies, supercookies, etc. The way these have worked is that trackers have stored these harder-to-detect and delete cookies in different obscure places in the browser, like Flash storage or HSTS flags. Evercookies were a big thing in early 2010 since many sites were using Flash and Silverlight, and those were very exploitable. Today those technologies aren’t used anymore, but that doesn’t mean the evolution of cookies has stopped. On the other hand, Supercookies work on the network level of your service provider.

Thankfully lately, for example, Firefox has been able to start tackling these . In that post, the Firefox team discloses what they had to do to take some action against this, and it is wild. First, they had to re-architect the whole connection handling in the browser, which was first made to increase user experience by reducing overhead to eliminate these pesky cache-based cookies.

Still, browser fingerprinting could be considered the evilest cookie of them all. Browser fingerprinting identifies everything it can from your system. Like some cookies, this has real use cases, e.g., preventing fraud in financial institutions. Still, principally this is just another intrusive way to track people. Thankfully some modern browsers offer at least some ways to avoid this, but not a full-fledged solution (other than disposable systems).

Future of Cookies

Lately, there has been some news about privacy-friendly substitutes to cookies by tech giants. Cookies have been a relatively significant issue privacy-wise for decades, and since the ad industry is so large, finding a replacement for these has been hard. So only time will tell. We cannot get rid of cookies entirely in the near future. They might change into something else, maybe this kind of API utilizing machine learning to analyze user data. Which I don’t know is better or worse. So cannot wait! tin-foil hat tightens

Conclusion

So what is the conclusion here? Probably nothing. Recently started small-time blogger just got scared from big numbers coming into his site collecting all kinds of data which ended up with him stopping this kind of action, at least on his site. Since for most users/sites, this kind of tracking is just a silly monkey-get-banana dopamine fix.

Don’t track unless you need to; if you do, inform it thoroughly.

Leap of Faith in Email Providers

When talking about the tools of the trade, almost regardless of the industry, email seems to be a vital tool. The same applies to me. Obviously, in the tech industry, everything goes by email. But also in music. If I happen to write, record, mix or master something, I always share these via email.

Unfortunately, email is a crucial part of my workflow, so I care about my productivity while using it. So recently, I started to look for options for my two different GSuite accounts. One was used for my personal domain, and another was for my music publishing company. A big reason behind the migration was that I found GSuite too much for my needs. I don’t necessarily have anything against Google’s product, albeit I agree they have a bit too big of a footprint on the internet, so I at least try to limit my contributions to it.

Requirements for Provider

I only have two requirements for my provider: IMAP/SMTP support and the ability to use my domain(s). Given these requirements, there are probably hundreds of providers that would fit these requirements. But after a while of skimming through different providers, I ended up with FastMail and ProtonMail.

FastMail

FastMail seemed like a good fit when I first looked into it: easily manageable domains and reasonable pricing. I quickly tested it with their offered trial account and was pretty pleased with their product. However, concerns arose when I learned that the company is from Australia. Not that I hate Australia by any means, but their hostile and subversive laws regarding encryption are pretty sketchy. The assistance and access act allows, under Australia’s legislation, police to force companies to create a technical function that would give them access to encrypted messages without the user’s knowledge, which made FastMail pretty much a no-go for me.

ProtonMail

After finding Australia’s laws against encryption, it seemed like a natural choice. I had already heard of them before, and their security stand. Unfortunately, ProtonMail doesn’t support IMAP/SMTP access, at least in the standard way, mainly because of encryption, which is why I didn’t want to go that route when I first heard of them. However, they offer a kind of unorthodox solution via their ProtonMail Bridge. By my understanding, this only handles the authentication to your mail and provider localhost-only endpoints to IMAP4/SMTP. Then you can configure your mail client of choice with these new endpoints.

Attractive solution, and at least for me, it seems to work and doesn’t hinder my workflow that much. Albeit, this conveniently enables vendor lock-in, which is not very good in my books. Still, I’m pretty happy with their product and decided to migrate my emails there.

Honorable Mention: Migadu

Migadu is on the smaller end of the spectrum when talking about email providers, but overall they seemed to have great values. I didn’t go that route (yet?) because I read that they have had some outages in their services in the past. This doesn’t mean that your email has been lost since the global mail system is pretty tolerant of that, but not logging into your mail can be pretty annoying. Also, their bandwidth-based pricing and daily mail limits made them unsuitable for me. I work a lot with email and send and receive a lot of them, so they offered pricing ideal for my needs, but it was a little bit too expensive at that point.

Dishonorable Mention: Self-hosting

No.

Conclusion

FastMail at first seemed like a good fit, but due to Australia’s legislation, it just doesn’t work for me. ProtonMail overall seems like a pretty exciting provider, at least on paper. But the vendor lock-in aspect of their bridge is rather odd, although I understand why they have done it. Still, this seemed minor to me, so I’ll continue to use their service, at least for a while.

Reawakening Long Lost Habit (Or Forming a New One)

A few years ago, I had a habit of semi-regularly writing about various exciting topics. Unfortunately, time passed, and I began to write less and less, and recently I’ve gotten out of the habit altogether. This is a shame in many ways since I’ve always felt writing to be immensely therapeutic.

At the time of writing, this world is also in a very odd place. Most countries are quarantined due to COVID-19, and people stay in their homes. Yours truly included! So to pass the time during these times, I’m trying to reawaken this habit.

Habitual writing has been on my mind for a long time, especially since it has been so present in my life. I’ve also somehow lost a few other healthy habits lately, which have made me think about how I can reawaken them in my daily life. Healthy practices that come to mind that I’ve lost would definitely be workouts and meditation. Although you could argue that the lost habit of working out is mainly related to the current difficult times, I’m not too worried. I believe that eventually when the world calms down in terms of this pandemic, I can relearn that habit quite quickly. But losing the regular meditation practice is really a shame, in my opinion. Like working out, meditation has played a big part in my life for years.

Even though my meditation practices have been irregular lately, the earlier “hard work” has helped me in my everyday life. But recently, I’ve started thinking about how I could relearn this habit. I’ve learned that, at least in my own case, the best way to learn habits has definitely been to do something often but not in an excessive amount. So in meditation, this was easy. Start for 5 or 10 minutes (which is nothing, everyone can find time for this) and just do it. Current times support relearning this since people are primarily working remotely. Hence, it is easy to start your day with this practice. With these simple steps, I feel like I’ve been able to reawaken this practice that was once very present in my life.

This got me thinking about utilizing a similar approach in other habits I’ve forgotten. The habits that came to mind were music and writing. Although some could argue that these are more or less the same thing. For some reason, I’ve struggled to pick up my instruments and write some new music during the pandemic. Many others have the same feelings in their own area of interest. I don’t know the cause for this; maybe the constant staring at the same four walls for over a year is the culprit. Who knows? A similar thing has also happened in my writing.

What really got me wanting to reawaken these habits was when I stumbled upon Richard P. Gabriel’s poetry. Gabriel is a legendary Lisp programmer. As a Lisp programmer myself, I’m always interested in what other like-minded people are up to. Gabriel started a project of writing one poem a day on March 18, 2000 to end a lengthy poetry-writing slump. Gabriel agrees that he is not necessarily a great poet, even though many could argue otherwise, but I think that is non-essential. While forming this habit, you don’t necessarily need to be the new Robert Frost. But since writing poetry (or anything) is a technical skill, constant practice is bound to help you in your journey. I stumbled upon a similar approach while reading Pat Pattison’s Writing Better Lyrics , where he talked about “daily object writing” in terms of getting better at writing. Pattison also noted that forming a habit is the big thing in this, which will eventually improve writing.

This approach is more or less similar to how I learned the healthy habit of regular meditation. How could I apply a similar approach to my composing and writing? Knowing myself, I cannot do this kind of creative work sporadically (or wait for the creative slump to end), or I’ll never do it. If I tried to write one piece and post every day, I feel that doing both daily would be slightly excessive (mainly timewise). So I need to find a healthy balance in practice and not be over-encumbered.

In my case, I believe that some time-boxed, very focused practice on something works the best. So what I intend to do is I’ll focus on a period (half an hour, an hour or so) on the given task, whether it is composing, writing, or programming (another healthy habit that I practice, which thankfully hasn’t been lost, but I always feel I could do more of it). I’ll set a healthy goal for this time box, so I don’t expect to write some new groundbreaking sonata, earth-shattering blog post, or the next big open-source project. Instead, I want to do something in these fields regularly to hone my skills in the given area. Since I’m trying to work on multiple habits, I also understand that I might not always have time to do everything. That’s okay. I can most likely squeeze in a smaller session to have at least some practice. Or if I just simply cannot do anything, that’s fine too. I just don’t want to see myself doing something excessively one day and then slacking off the next day since “I did so much yesterday.” (learned from Pattison).

Productivity has been really close to my heart, even though I occasionally lack significantly in that area. But maybe with small steps, everyone can benefit from a slight boost in their productivity.

Or just procrastinate… As long as you’re happy.

FreeBSD Jails For Fun and Profit

Docker has recently stormed into software development. While its concepts are powerful and valuable, similar tools have been used in systems for decades. FreeBSD’s jails in one of those tools which build upon even older chroot(2). To put it shortly, with these tools, you can make a safe environment separated from the rest of the system.

Jails in FreeBSD are by no means a new tool (introduced in 4.x), but for one reason or another, I haven’t used them that often, which is a shame since they are so powerful. So I wanted to explore this concept in a concise and summarized manner.

Templates

ZFS datasets are a great way of creating templates for jails since, after the template creation, you can easily create new jails with zfs clone or zfs send/receive. Typically, people divide jails to complete and service jails, where the former resembles a real FreeBSD system, and the latter is often dedicated to applications/services. I’ll cover complete jails for now.

Creating templates starts with creating a dataset for your jail and template. Here I’ll make a new dataset for the base installation of FreeBSD 12.2.

$ sudo zfs create -o mountpoint=/vm zroot/vm
$ sudo zfs create zroot/vm/tmpl
$ sudo zfs create zroot/vm/tmpl/12.2

After that, fetch the base installation itself:

$ fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.2-RELEASE/base.txz
# Fetch all the necessary stuff for your template, e.g. lib32 if needed
$ sudo tar -xJvpf base.txz -C /vm/tmpl/12.2

After that, you should write a minimum viable /etc/rc.conf for the template:

$ sudo emacs /vm/tmpl/12.2/etc/rc.conf

# Start or stop services
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
syslogd_flags="-ss"
cron_flags="-J 60"

You can also disable some unnecessary jobs for jails:

$ sudo emacs /vm/tmpl/12.2/etc/periodic.conf

# No output for successful script runs.
daily_show_success="NO"
weekly_show_success="NO"
monthly_show_success="NO"
security_show_success="NO"

# Output to log files which are rotated by default.
daily_output="/var/log/daily.log"
daily_status_security_output="/var/log/daily.log"
weekly_output="/var/log/weekly.log"
weekly_status_security_output="/var/log/weekly.log"
monthly_output="/var/log/monthly.log"
monthly_status_security_output="/var/log/monthly.log"

# No need for those without sendmail
daily_clean_hoststat_enable="NO"
daily_status_mail_rejects_enable="NO"
daily_status_mailq_enable="NO"
daily_queuerun_enable="NO"

# Host does those
daily_status_disks_enable="NO"
daily_status_zfs_zpool_list_enable="NO"
daily_status_network_enable="NO"
daily_status_uptime_enable="NO"
daily_ntpd_leapfile_enable="NO"
weekly_locate_enable="NO"
weekly_whatis_enable="NO"
security_status_chksetuid_enable="NO"
security_status_neggrpperm_enable="NO"
security_status_chkuid0_enable="NO"
security_status_ipfwdenied_enable="NO"
security_status_ipfdenied_enable="NO"
security_status_ipfwlimit_enable="NO"
security_status_ipf6denied_enable="NO"
security_status_tcpwrap_enable="NO"

You also might want to enable ports in your jail:

$ sudo mkdir /vm/tmpl/12.2/usr/ports
$ sudo mkdir -p /vm/tmpl/12.2/var/ports/{distfiles,packages}
$ sudo emacs /vm/tmpl/12.2/etc/make.conf

WRKDIRPREFIX = /var/ports
DISTDIR = /var/ports/distfiles
PACKAGES = /var/ports/packages

Apply system updates to the template:

$ sudo freebsd-update -b /vm/tmpl/12.2 fetch install

Lastly, take a snapshot:

Strictly speaking, a template is a snapshot, not a dataset. The snapshot can be cloned or sent/received to generate new datasets for production jails.

$ sudo zfs snapshot zroot/vm/tmpl/12.2@complete

This creates a snapshot of zroot/vm/tmpl/12.2 named complete. You can then check your current snapshots with the following:

$ sudo zfs list -t snapshot

Creating jails from the template

Now you should create a new jail based on that snapshot. You can do it either with zfs clone or zfs send/receive:

Difference Between the Two

“A clone is a writable volume or file system whose initial contents are the same as the dataset from which it was created. As with snapshots, creating a clone is nearly instantaneous and initially consumes no additional disk space. In addition, you can snapshot a clone.” [1]

“The zfs send command creates a stream representation of a snapshot that is written to standard output. By default, a full stream is generated. You can redirect the output to a file or to a different system. The zfs receive command creates a snapshot whose contents are specified in the stream that is provided on standard input. If a full stream is received, a new file system is created as well. You can send ZFS snapshot data and receive ZFS snapshot data and file systems with these commands. See the examples in the next section.” [2]

$ sudo zfs clone zroot/vm/tmpl/12.2@complete zroot/vm/jail1

# OR

$ sudo sh -c "zfs send zroot/vm/tmpl/12.2@complete | zfs receive zroot/vm/jail1"

Jail configurations

# /etc/rc.conf

cloned_interfaces="lo0"

# PF is used for NAT and port forwarding.
pf_enable="YES"
pflog_enable="YES"

jail_enable="YES"
jail_list="jail1"

### /etc/jail.conf

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;

host.hostname = $name;
path = "/vm/$name";
exec.consolelog = "/var/log/jail_${name}_console.log";
exec.prestart = "cp /etc/resolv.conf $path/etc";
exec.poststop = "rm $path/etc/resolv.conf";

jail1 {
        ip4.addr = "lo0|127.1.1.1/32";
        ip6.addr = "lo0|fd00:1:1:1::1/64";
        allow.chflags;
        allow.raw_sockets;
}

# /etc/hosts

...

127.1.1.1 jail1
fd00:1:1:1::1 jail1

Jail management

FreeBSD provides nifty built-in tools for jail management:

Start all jails.

$ sudo service jail start

Start a specific jail(s).

$ sudo service jail start jail1

Log in to jail.

$ sudo jexec jail1

Run a command on a jail.

$ sudo jexec jail1 ifconfig

List running jails.

$ jls
$ jls -v
$ jls -s

So that’s how you can spin up a simple restricted environment on your FreeBSD system. Of course, this topic still has many things to cover, e.g., in-depth networking and configurations.

Notes

• [1] Overview of ZFS Clones: https://docs.oracle.com/cd/E19253-01/819-5461/gbcxz/index.html

• [2] Sending and Receiving ZFS Data: https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.html

References

• FreeBSD Jails