November 20, 2022
9 min read
Why? That’s weird
No, it’s not! Google Maps is probably the most amazing service we get for free1
It’s something I use almost every day and is incredibly useful for getting around.
But what if I didn’t need Google at all?
OpenStreetMap provides crowdsourced mapping
data available for free to the world. But this isn’t to say I can just use OSM.
The organization does provide the data, but its usage policy encourages users
to not rely on their servers for personal use,
and instead take responsibility for hosting themselves.
And based on this project, I can see why. The hardware requirements are not for
the faint of heart.
Hardware
I had access to a gaming PC that was mostly unused.
These days my girlfriend and I prefer to play couch
co-op games on the Switch, rather than on PCs in separate rooms.
So I thought I’d make use of this fairly beefy desktop and turn it
into a self-hosted map server.
This particular desktop2 has:
- Intel i7 gen 10700kf CPU
- 32 GB of RAM
- 1TB NVMe SSD
- GTX 2070 GPU
- 750 Watt PSU
It cost $1,270 when I bought it (open box) from Microcenter
about two years ago.
To meet the demanding requirements3
for running a map server for the entire planet,
I also bought 128 GB of RAM to replace the 32 GB it came with.
This brought the total hardware cost to $1,670.
This is quite a large price for self-hosting, but it should
be noted that the requirements are roughly proportional to the
geographic boundaries of the data. If you wanted to self-host
a map service for a smaller region (a European country or a few
US states), this could be accomplished with a much more modest
computer.
In the end, I compromised and lowered my expectations
to only hosting the map data for North America.
Requirements
What does it take to match Google Maps? In my mind, there are 3 main
requirements:
- A “slippy map” which can be dragged and zoomed interactively
- A way to route from place to place
- A way to search for locations
These can be solved with 3-ish services:
- a tile server4 (technically this involves multiple services)
- valhalla5, a routing service
- nominatim6, a geocoding service
Implementation
To accomplish this, I set up a github repo to group together
these core services and run them using docker-compose.
This repo can be found
here.
It was surprisingly easy to piece together these dockerized
services in a single docker-compose.
The READMEs for these services suggest creating a separate
docker volume before starting the containers.
This is useful because the import can take a very long time
(multiple days for the whole planet).
I initially tried to import the entire planet.osm.pbf file,
which is currently 66 GB. However, the import kept crashing, so I
limited the scope of my project to only use data in North America.
If I need to travel abroad anytime soon I’ll just have to use Google Maps.
The North America extract is only 12GB in its compressed protobuf format,
so I was able to run all 3 services on this data.
Remarkably, the services extract this data into a much more verbose
(but more read efficient) format.
To see how much final disk space was being used, I ran:
docker system df -v
From the output, it was clear why I couldn’t import the entire planet.
VOLUME NAME LINKS SIZE
nominatim-data 1 291.9GB
osm-data 2 337.8GB
osm-tiles 1