A great way of sharing data quickly
So the other day, while googling for a simple plot-some-data-on-a-map solution, I accidentally stumbled upon a gist with a map inside and had to stop to investigate this further. I found that gist are automatically rendering maps when you insert/upload topojson data. Ok great, let’s try to use this for something fun…
If you are in a hurry and just want to test this without reading all the text. Here is the short howto list, or you can view my crud US president election 2016 poll hack made with the same method.
Do the command line magic:
$ sudo npm install -g topojson $ topojson -o municipality.topojson -p navn,komm kommuner.geojson $ iconv -f iso-8859-1 -t utf-8 dataset.csv > dataset-utf8.csv $ git clone email@example.com:d812cb66ac9f6ff6b8bf01cf7dfe9f39.git $ cd d812cb66ac9f6ff6b8bf01cf7dfe9f39/ $ node ./topo-json-number-of-employed.js $ brew install gist $ export LC_ALL=en_US.UTF-8 $ export LANG=en_US.UTF-8 $ gist <filename>
Ok, so this is the place to start if you want to read about all the details. First let’s find some useful tools for dealing with topojson.
To be able to handle topojson files you need a tool. Luckily this tool is called TopoJSON. With this tool you are able to convert files from geojson to topojson command line. To install this tool with npm:
$ sudo npm install -g topojson
I’ve chosen to download a geojson file with all the borders for all municipalities in Norway. You can download the data from Kartverket:
This is a rather big file, 24MB unzipped. Let’s try to convert it to topojson with the command line tool we just installed.
$ topojson -o municipality.topojson -p navn,komm kommuner.geojson
Yay! Now my file is only 0.77MB and much more useable for web pages and mobile devices.
I found a data set containing a count of all employed people in all municipalities in Norway.
These dataset are mainly encoded randomly (most of them are iso-8859-1), so you need to verify the encoding before you proceed.
$ file -I dataset.csv
If you get something other than
utf-8 you need to convert to
$ iconv -f iso-8859-1 -t utf-8 dataset.csv > dataset-utf8.csv
To merge the topojson with the other dataset you need to write some lines of code. It’s not too hard but I can give you a flying start with my hack. Don’t shoot me for the lack of testing and other best practices.
To be able to get the topojson file into a gist without having to copy-paste it you need a tool. As far as I know, it’s not possible to upload files in the gist webinterface, so I found a nice tool for uploading files to a gist on the command line. This tools is called
Gist. To install the tool you do like this.
$ brew install gist
And if you environment encoding is wrong you need to set this before running it. If you have the wrong environment setting you will probably get a nasty error like the one I got:
Error: "\xC3" on US-ASCII.
$ export LC_ALL=en_US.UTF-8 $ export LANG=en_US.UTF-8
To upload the new topojson file you just type.
$ gist <filename>
Cmd + click the new link to see your shiny new gist. Mine looks like this:
There are some mapping issues for some of the municipalities, but I haven’t bothered to fix them yet. Maybe later… or if you want to help me, give me a shoutout on Twitter @sorenso
Happy gisting :-)