This post will describe how I design my programs in Go. I needed this
for work, and while I searched for a link, nothing quite fits my
coding practices out there. The word “Layered” can pull up some fairly
close descriptions, but I want to lay out what I do.

Deriving Some Requirements

Go has a rule that I believe is underappreciated in its utility and
whose implications are often not fully grasped, which is: Packages may
not circularly reference each other. It is strictly forbidden. A
compile error.

Packages are also the primary way of hiding information within Go,
through the mechanism of exported and unexported fields and
identifiers in the package. Some people
will pile everything into a single package, and while I’m not quite
ready to call this unconditionally a bad idea, it does involve
sacrificing all ability to use information hiding to maintain
invariants, and that is a heck of a tool to put down. At any sort of
scale, you’d better have some concept of the discipline you’re going
to replace that with.

So for the purposes of this discussion, I’m going to discard the
“one large package approach”.

We also know that Go uses a package named main that contains a
function named main to define the entry point for a given
executable. The resulting package import structure is a directed acyclic
graph, where
the packages are the nodes and the imports are the directed edges, and
there is a distinguished “top node” for each executable.

So how do I deal with this requirement in Go?

Layered Design In Go

This is a portion of a package hierarchy extracted from a real
project, with many of the names substituted and rubbed off, but the
relationships are approximately correct:

All imports of external modules are automatically not part of a loop
in the base package, so we can just look at the import patterns of the
application currently being written. Since loops are forbidden that
implies there must be packages that do not import any other package in
the application. Pull those out and put them on the bottom:

There is now a set of packages in the remaining set that only
reference the packages we just pulled out and put on the bottom. Put
them in their own layer:

You can repeat this process until you’ve layered everything by the
depth of the import stack:

All package imports now point downwards, though it can be hard to
tell.

If you look at the bottom, you see very basic things, like
“the package that provides metrics to everything else”,
“the package that refines the logging for our system”, and
“a set data structure”.

These are composed up into slightly higher level functionality that
uses logging, metrics, etc. and puts together things like header
functionality, or information about users (imagine permissions or
metadata are stored here).

Throughout this post I will refer “higher level packages”; in this
case, it literally refers to the way packages will appear “higher” on
this graph than any package it imports. It is not the definition of
“higher” we often use to mean “higher level of abstraction”; yes,
if a package offers a “higher level of abstraction” it will generally
of necessity also be a “higher level package”, but a
“higher level package” is often just a package that is using the lower
level package, not an abstraction, e.g., a “crawler” may have
net/http show up as a “lower level package” but a crawler is not any
sort of abstraction around net/http so much as it is just an
application that is using net/http.

These things are then composed into higher layer objects, and so
forth, and so forth, until you finally get to the desired application
functionality.

This Is Descriptive, Not Prescriptive

If you are the sort of person who reads things about software
architecture, you are used to people making prescriptive statements
about design. For instance, one such statement I found in another
article about layered design in Go has the prescriptive statement that
layers should be organized into very formal layers, and that no layer
should ever reach down more than one level to import a package. That
is not a requirement imposed by Go, it is a prescriptive statement
by the author. You can see in my graph above I don’t agree with that
particular prescription, as I have imports that reach down multiple
levels.

However, the previous section is not prescriptive. It may sound like
prescriptions you’ve heard before, but it’s actually required. You
can graph all Go modules in the layers I described. It’s a
mathematical consequence of the rules for how packages are allowed to
import each other. It is descriptive of the reality in Go.

Which means that any other prescriptive design for a Go program must
sit on top of this structure. It has no choice. It may not be
necessary, but it is certainly convenient in MVC design for the
Model, View, and Controller to be able to mutually reference each
other, at least a little. In Go, that’s not an option, if you want
them separate from each other. You can do MVC, but you must do “MVC on
top of Go layering”. If you slam all of the MVC stuff into one package
for convenience, or for it to work at all, you’re increasing the odds
that you’ll still end up with circular loops between the packages
implementing other related Models or Views or Controllers.

You can “hexagonal architecture”, but you must do it on top of Go
layering.

You can do any design methodology you want, but it must always rigidly
fit the layered design described in the previous section, because it
is not an option.

They do not all go on top of this design constraint equally well.

So What’s The Best Prescription?

Naturally, this raises the question of what’s the best methodology to
sit on top of this.

My personal contention is that the answer is “none”. This is a
perfectly sensible and adequate design methodology on its own.

It harmonizes well with a lot of the points I make in Functional
Programming Lessons in Imperative
Code. Particularly
the section about purifiable
subcomponents and
the ability to compose multiple component’s purifications together. If
you design a shim for the metrics for testing the metrics in a pure
manner, you can then use that to design the purification for the
bodypart2 above. Then you can use that to design the purification for
the body package, which can then be fed into the classify package, and
through all that, you can get classification tests that don’t have to
depend on any external state, despite the fact it is sitting on top of
a lot of other packages.

Sometimes a purification component will “absorb” the tree under it, so
for instance, the contentmodel module may absorb all metrics and
other packages into itself so the contentmodel can ignore all that and
just use the test shim provided by the contentmodel package. Other
times it may be necessary to have multiple shims related to the
packages being imported. They’re already coupled by the import, so
this is not additional coupling. Generally it’s a bit of both in my
experience.

Any codebase can be decomposed this way with enough work. The work can
be nontrivial, but it gets a lot easier as you practice.

My favorite advantage of this methodology is that for any package you point
at, there is a well-defined and limited number of packages you need to
understand in order to understand the package you are looking at, even
considering the transitive closure of imports. It is effectively
impossible to write code that requires you to understand the entire
rest of the cod