Google Maps and Dutch Elections – Working demo

Finally I’ve got something to show. The results of the parliamentary elections of 2006 of the Christian Democrats can be found on this page.

Things I’ve changed since my last post are, the loading of the kml-file uses jQuery. Looks more polished, maintainable and doesn’t bug in Firefox. I love the $.each method. Performance is ok, after you click on the link below the map it loads on my computer in a second or so. Not benchmarked. It works for me in Firefox,Chrome and Safari under OS X. I haven’t tested other operating systems thoroughly though it seemed to work using IE7, WinXP.

There seem to be a few glitches on the polygons. I’m not sure how to fix it, or where the problem originates. Have to look into that.

The article on FlowingData I posted earlier also has a link in the comments to a library handling this kind of stuff. Should have found out earlier, would’ve saved me a lot of work. Might check it out though, it uses the encoded polygons for Google Maps to speed it up, something I’d like to implement. Right now only the US states are in there I belief. Also, nice website. These (encoded) polygon really should be in a repository somewhere. I’ll happily donate my data, after the trouble I went through to convert the shapefiles to something sensible with the right projection. Nevertheless, thank you CBS for map data.

 

Progress report: Google Maps and Dutch elections

I managed to make quite a bit of progress on this little experiment, and learn a bunch of stuff on the way.

Last time I ended with a kml-file that was viewable in Google Maps. All nice and well but not really useful, yet. Next step was to take the kml-file, extract the coordinates and use the Google Maps API to show them on a Google Maps. This all went reasonably well, there is built-in support for polygons and the API is easy to work with. The javascript right now is still rather slow and needs optimizing. Also, I might change bits of the AJAX-request to use jQuery instead of the API for readability and maintainability.

The speed of the loading might further be improved by using encoded polylines. Google has a format which renders the polygons faster than the kml-file and there is an algorithm for the conversion.  I’ll definitely look into this for speed-ups.

After this was done I had to convert the election results (a .csv) file to json to work with it in javascript. This was not too difficult using python. Since the elections for parliament in 2006 (the set I use for testing) some municipalities have merged or joined others. My kml-file therefore had different municipalities than the json file.  Looking for differences between the data sets I was able to find the incomplete municipalities. Wikipedia revealed the details of the mergers etc.  and I was able to merge them in the csv using Python.  The percentages still need to be calculated again but that’s no biggy (and can also be done in javascipt).

Last step was to assign colors to the polygons based on the values of the json file. I have this working although there is no clear algorithm yet. Through a stroke of luck my feedreader turned up this item today giving me all the information I needed for the coloring, especially through this application. When I have time I’ll use it to further patch things up.

Steps remaining is using the encoded polylines, use jQuery (which should also fix a bug in Firefox of rendering the polygons a bit strange) and give more election results as input.  I also stumbled upon Raphael, it looks excellent and might use it for some extra graphs.

Google Maps and Dutch election results (wip)

Right now I’m working on visualising the Dutch election results (parliament, local etc.) in Google Maps. It’s far from done but this might will serve as the documentation of my efforts for myself and hopefully for others as well. This also the reason being this post is in English.

My goal is relatively simple, visualise turnout, votes for party X as percentage of total votes, winning party etc. per municipality. For that purpose I have collected the results of the elections at http://www.verkiezingsuitslagen.nl/, I understand the Kiesraad is planning to visualise the results themselves but I decided not to wait for that, this little project will probably not be as extensive as theirs. Besides, I have some ideas of my own I’d like to implement.

Next step was a map of Dutch municipalities, in itself not difficult but adapting it to serve my needs was. I had first decided to stick to a map in svg-format (from http://commons.wikimedia.org/wiki/File:Nederland_gemeenten_2007.svg), based on the information by the CBS (The Dutch Bureau of Statistics). In fact this worked rather well, using the jquery-svg library I was able to select specific municipalities and color them. I might even go back to using that. Nonetheless, the thought of using Google Maps for this experiment prevailed.

I downloaded the borders of municipalities in the Netherlands from the CBS and then the trouble started. I have no experience using Geodata whatsoever and figured it would be easy peasy from here on. Trouble however was that the CBS map uses a different projection that Google Maps. The projection is basically the way in which the earth is projected onto a 2d-surface. On maps Greenland is about the same size as South-America, but in fact it’s 8 times smaller, the projection is what’s causing that because there will always be distortions when you try to put a sphere on a flat piece of paper. Pretty much each country has its own projection for domestic uses and I needed to translate the Dutch one (Rijksdriehoekcoordinaten) to the one used by Google Maps.

I tried several programs across three different platforms (my native Os X, virtualized Windows 7 and Ubuntu running on a laptop) to achieve this. I tried ogr2ogr commandline tool to convert a shape-file (.shp) delivered by CBS. It failed and only then I found out about the projections. I moved on.

One excellent piece of open-source software MapWindow GIS, I first tried to use it to convert the projection of the shape-files but it failed on me. Still, I’m not sure what I did wrong. Nevertheless, it looked great and MapWindow came in handy a little later. I installed a bunch of stuff to no avail until I stumbled on Quantum G (Qgis). There was a version for OS X, Windows and Linux. What could go wrong? The OS X version worked pretty neat, till the conversion to the Google Maps projection where it crashed. The Windows version wouldn’t install and the Linux version missed the projection-conversion menu. I went back to the OS X version, fiddled with some settings and this time it worked. What did I do? (this is the documentation for myself since I won’t remember this in a year).

First, open the shapefile and set the projection right (Tools->Data Management->Define current projection). In this case to the Dutch system (Amersfoort/RD New: EPSG 28992). Then go to Tools->Data Management->Export to new projection and choose the Google Maps projection (Popular Visualisation CRS/Mercator:EPSG 3785) Next step, select ‘browse’ to give the filename for the output. In the save dialog, set the encoding to UTF-8. I think this was the cause of it failing on me the first time. Click OK and tada, a shapefile with the right projection.

Now Google Maps still needed to get the data in a format it would understand. This format is the Keyhole Markup Language or KML and many conversion tools exist for it. Qgis also has one (the ogr converter) but it didn’t work as good for me as the Shape2earth plugin in MapWindow GIS. The reason for this is the number of settings in the plugin. In the Shape2earth plugin I was able to include the extra data (# of inhabitants, ages etc.) in the kml-file and set opacity levels. Not sure yet what use it will be but it’s easier to delete than add in later.

This about concludes what has been done so far. What’s left is to delve into the Google Maps API to show the election results and map the election results to the municipalities. It should all be doable. I’m a bit afraid of the speed the application will have in the end, especially since I plan on handling the colouring on the client-side. Possible plans are to animate the election results so you can see the difference between elections visually, and visualise the extra data embedded in the kml files. In any case, here is what I have so far. Google Maps doesn’t show all municipalities with polygons because it only shows the current results, you’ll have to click next. Of course you are free to pose questions on how I did this.

Dutch Municipalities weergeven op een grotere kaart