Maps and gist

So the other day, while googling for a simple plot-some-data-on-a-map solution, I accidentally stumbled upon a gist with a map inside and had to stop to investigate this further. I found that gist are automatically rendering maps when you insert/upload topojson data. Ok great, let’s try to use this for something fun…

TL;DR

If you are in a hurry and just want to test this without reading all the text. Here is the short howto list, or you can view my crud US president election 2016 poll hack made with the same method.

1. Municipalities in Norway as geojson
2. Employed people in all municipalities in Norway.

3. Do the command line magic:

$ sudo npm install -g topojson
$ topojson -o municipality.topojson -p navn,komm kommuner.geojson
$ iconv -f iso-8859-1 -t utf-8 dataset.csv > dataset-utf8.csv
$ git clone git@github.com:d812cb66ac9f6ff6b8bf01cf7dfe9f39.git
$ cd d812cb66ac9f6ff6b8bf01cf7dfe9f39/
$ node ./topo-json-number-of-employed.js
$ brew install gist
$ export LC_ALL=en_US.UTF-8
$ export LANG=en_US.UTF-8
$ gist <filename>

4. Done. Now click the link in your terminal and enjoy your beautiful map.

TopoJSON

Ok, so this is the place to start if you want to read about all the details. First let’s find some useful tools for dealing with topojson.

Install topojson

To be able to handle topojson files you need a tool. Luckily this tool is called TopoJSON. With this tool you are able to convert files from geojson to topojson command line. To install this tool with npm:

$ sudo npm install -g topojson

Find some useful map data

I’ve chosen to download a geojson file with all the borders for all municipalities in Norway. You can download the data from Kartverket:

Municipalities in Norway as geojson

This is a rather big file, 24MB unzipped. Let’s try to convert it to topojson with the command line tool we just installed.

$ topojson -o municipality.topojson -p navn,komm kommuner.geojson

Yay! Now my file is only 0.77MB and much more useable for web pages and mobile devices.

Enrich map data with some insight

I found a data set containing a count of all employed people in all municipalities in Norway.

These dataset are mainly encoded randomly (most of them are iso-8859-1), so you need to verify the encoding before you proceed.

$ file -I dataset.csv

If you get something other than utf-8 you need to convert to utf-8.

$ iconv -f iso-8859-1 -t utf-8 dataset.csv > dataset-utf8.csv

Merging datasets

To merge the topojson with the other dataset you need to write some lines of code. It’s not too hard but I can give you a flying start with my hack. Don’t shoot me for the lack of testing and other best practices.

5orenso/topo-json-number-of-employed.js

Upload the file to a gist

To be able to get the topojson file into a gist without having to copy-paste it you need a tool. As far as I know, it’s not possible to upload files in the gist webinterface, so I found a nice tool for uploading files to a gist on the command line. This tools is called Gist. To install the tool you do like this.

$ brew install gist

And if you environment encoding is wrong you need to set this before running it. If you have the wrong environment setting you will probably get a nasty error like the one I got: Error: "\xC3" on US-ASCII.

$ export LC_ALL=en_US.UTF-8
$ export LANG=en_US.UTF-8

To upload the new topojson file you just type.

$ gist <filename>

Cmd + click the new link to see your shiny new gist. Mine looks like this:

There are some mapping issues for some of the municipalities, but I haven’t bothered to fix them yet. Maybe later… or if you want to help me, give me a shoutout on Twitter @sorenso

5orenso/municipality-enriched-2014.topojson

Happy gisting :-)