The title is a bit tongue in cheek. The subtitle would be something like How
looking at ideas behind architectural patterns can yield good solutions even
when common implementations would be overly complex.
I struggled with this article, and it’s because it tries to make two points at
the same time1 Which ironically means it has low cohesion and high
coupling.:
- It tries to be a rant against overcomplicating implementations. This will be
covered briefly, and then I hope a more complete treatment can be a separate
article. - It also tries to be an example of how software following the Unix philosophy
doesn’t have to be hard to write. In fact, the design can evolve in that
direction almost accidentally.
Write programs that do one thing and do it well. Write programs to work
together.
This is easy if you’re not afraid of keeping things simple.
The title promises event sourcing and microservices. The article will deliver on
them in spirit, but not perhaps in the technical detail you would expect. We are
going to see 78 lines of code, so it will seem a bit silly to talk about
architectural patterns. If you want, you can imagine this to be a larger, more
complicated system. But the truth is good ideas help even small systems like
this one.
The two ideas to rule them all are cohesion and coupling. Many of the other
good practises we find in software engineering are really instances or
variations of high cohesion and/or low coupling.2 Single responsibility,
dry, wet, interface segregation, structured programming, pure functional
programming, cqrs, microservices, message passing, and the list goes on. This
is not an accident, it comes down to what makes systems complex in general. If
we can reason about components in isolation, the system is less complex, and
less complex systems are much easier to work with.
The code we’ll see makes up the following system.
If we wrap this system in buzzwords, we have
- The three services, LogTrain, LogAnal, and LogGen have a single
responsibility and can be deployed independently as long as they continue to
conform to the protocol the other services expect. They perform their aggregate
functionality by talking to each other. - LogLog is an append-only event log that serves as the authoritative
source for the state of the system. It is updated live by LogTrain every
time the user makes a guess, and any other application can read from it to
find out what the state of the application is and compute derived state (e.g.
LogDist).
One could imagine implementing this with message buses and containerised web API
server applications orchestrated by Kubernetes. I believe that would be
confusing the tools with the ideas. It’s easy to lose sight of the core ideas:
- Maintain high cohesion in components by having separate components for
separate responsibilities. - Maintain low coupling by isolating component runtimes and having them
communicate asynchronously.
When you back out of the shiny tools and return to the core ideas, it’s easy to
see there might a simpler way to implement the system: regular scripts executed
at the command line, that communicate by writing and reading plain files. Unless
you really need some of the promises made by event buses and Kubernetes (which
in many cases you don’t) you can get the same benefits with plain command line
scripts and text files. The advanced tools serve their purpose in specific
contexts, but they should not be mistaken for embodiments of the core ideas.
I would argue that even if you think you need event buses and Kubernetes, try
implementing the mvp with command line scripts and text files. You might be
surprised how quick you can go, how far that will take you, and once you have
something working you can optimise for edge cases.
Onwards to the example. I mentioned previously that it would be nice with a
script to practise mentally taking the logarithms of random numbers. 3 I have
long been looking for a way to create these kinds of small one-off applications
for my phone to use on the go, but without having to learn all of Android
development. It just struck me that Termux doesn’t just run ssh; it can also
run any script I care to write. I can even install nvim on it to edit the
scripts and fix bugs in them. The exact details of these scripts aren’t
important, but they provide a concrete example we can discuss.
You have already seen the system diagram above, but we can look at it again with
the real filenames we’ll see in this section, and the real communication
mechanisms.
The design didn’t start out looking like that. It grew there organically, as I
realised there were opportunities for cheap wins.
A guessing loop is actually all we need
I wrote logtrain.pl on my phone when I had some waiting time to kill. Even
with the awkward keyboard, it probably took less than 20 minutes in
total.4 On the phone, I ended up using a lot of single-character variable
names and didn’t write many comments. I have fixed some of that for the code in
this article. While writing I also noticed some inconsistencies and small latent
bugs that are fixed now. I remain convinced that self-reviewing code gets you at
least 20 % of the value of having a trusted colleague review code.
I won’t dwell on this first version because it’s simple enough to be an
introductory tutorial to a scripting language: generate a random number, ask the
user for the logarithm of it, then print what the actual logarithm was so the
user can check their result, then repeat until eof.5 One thing of note,
perhaps, is that this script generates a random value for (log{x}) and then
displays (10^{log{x
