Interactive: Haavisto’s great challenge

The first round of the presidential election in Finland was held on Sunday. It was a super-exciting race with Sauli Niinistö coming out on top, but with Pekka Haavisto of the Green Party as the great surprise. Haavisto finished second, just before the grand ol’ man of the Center Party, Paavo Väyrynen.

Haavisto is having great momentum and is quickly rising in the polls. But can he really take on Sauli Niinistö who has been the favorite for years already?

With this interactive visualization I will show that Pekka Haavisto face a great challenge in the second round. In the first round he got 570.000 votes. Niinistö got twice as much. 1,4 million voters will have to find a new candidate in the second round. Haavisto will have to get about 70 percent of those votes, which won’t be easy considering he is the liberal alternative of the two finalists and a lot of the undecided voters are conservatives.

Anyway here is the visualization. It lets you drag and drop the votes of the candidates that didn’t make it to the second round. Hopefully it gives you an idea of the effort that Haavisto will have to go through to stand a chance. But who knows? He has surprised us once already.

Open interactive visualization in new window.

Campaign funding times two

These two interactive visualizations has been in the drawer all summer. I made them in June already and did a small effort to get them published, but then I when that didn’t happen they were sort of forgotten about.

The starting point was the campaign funding data that was published after the parliamentary elections here in Finland. All MPs have to publicly declare all donations above 1500 euros. The data can be found here, or in a slightly refined form here (thanks Helsingin Sanomat!). Helsingin Sanomat has already provided their own visualization, check it out here.

The network

I started by approaching the data as a network using Protovis. This was the result:

Click to open in new window. Note that it takes a while to load.

I think the output is not too shabby, although the loading time here is really not acceptable. I couldn’t find a way to fasten up the rendering. The JavaScript code could also have been better, but I learned a lot in the process of putting it all together and would have been able to write a much smoother code today I think.

The explorer

The network approach above might be pretty, but not as informative as it could be. Again I used Protovis to build an interface that quickly lets you browse through all the reports.

Click to open in new window. The explorer itself is in Finnish.

I think this visualization has a lot of strengths. It is “click-less” which means you can quickly browse the candidates. Life is too short to be clicking. The loading time is also much, much shorter than in the network visualization.

Any thoughts?

How open data improved election coverage in Finland

This is a guest post I’ve written at the Open Knowledge Foundation Blog.

Parliamentary elections in Finland are usually rather dull. Rarely does the rest of the world bother to pay any attention. But this year was different. The elections in April were the most exciting ones in decades with the incredible rise (from 5 to 19 percent) of the populist party True Finns as the main attraction. But the intricate political puzzle that followed the success of the True Finns was not the only source of excitement in the elections. Especially not if you happen to be an open data enthusiast.

Since the mid-90s so called voter advice applications have played an increasingly important role in the Finnish elections. Voter advice applications are questionnaires about political issues put together by media outlets and NGOs for candidates to answer. Voters can then see which candidate match their own views best.

The by-product of these applications is a very interesting set of data. Here you got all the opinions of (almost) all the candidates gathered in an easily accesible format. I don’t know about you, but this surely gets me going.

Up until now this data has been a completely vested resource. The news rooms have kept it to themselves and not managed to take the analysis past the level of “lets see what the candidates think about nuclear power”. But this year things changed. The leading newspaper Helsingin Sanomat decided to publish their data openly a week before the elections. And within a couple of days The Crowd (bloggers, programmers etc.) managed to do more with the data than journalist had done in fifteen years:

  • Kansan Muisti (“the memory of the people”), a site resembling It’s Your Parliament, used the data to investigate if the MPs had voted in accordance with the promises made in the voter advice applications before the elections.

A few weeks after the election the public broadcaster YLE followed the example of Helsingin Sanomat and published their data as well.

Journalism is changing. The immediate reaction of a traditional journalist is often resistance when someone asks a newsroom to publicly share data. “Someone might steal a story that we haven’t yet done!!!” argues the Journalist 1.0.

But it is time to realize that journalistic output does not only have to be 700 word stories, neatly structured with headings, preamble and text. Journalism can also be publishing a set of data that will be refined and possibly developed into new stories by readers. As the case of Finland shows, there is still a great amount of unused journalistic potential in The Crowd.

The opinions of our new parliament – in 30 seconds

Yesterdays elections turned out to be even more exciting than everyone had expected. True Finns shocked everyone with their third position. Trying to get a government together now is not easy.

But what opinions do the new Finnish parliament represent? I’ve put together an interactive tool that lets you explore the opinions of the new MP’s – in about 30 seconds. It is based on the answers given in Yle’s vaalikone. I had to put it on a different server because this WordPress blog doesn’t support own Javascript adventures. Open it here.

Application opens in new window. Requires an updated web browser.

This was my first real visualization in the Javascript based Protovis. I strongly recommend it so far. I hardly have any prior Javascript experience, but with assistance from the tutorials at Knight Digital Media Center I have been able to figure it out quite quickly. I especially appreciate the possibilities to create interactivity. Made me realized what a waste of time mouse-clicking is (compared to mouseover interaction).

  • Get the data. (Note that the answers might be slightly outdated, I noted in the reporting from Yle that a few new candidates seem to have participated since I did the scrape. However, the big picture should be accurate.)

Campaign funding: How much do know?

The number one political scandal in Finland during the past few years has been about campaign funding. Especially the Center party has been exposed to a lot of criticism due to its incomplete accounting of the campaign funding. The legislation has been tightened and today every MP has to disclose every campaign contribution over 1.500 euros. But has this really made the political system more transparent?

The candidates can voluntarily make a declaration of the campaign funding before the elections. Less than a week before the parliamentary elections a fourth of the candidates have decided to do so. The funding and spending reports are presented at Puolue ja vaalirahoitusvalvonta. I have gathered the data from this site (partly by scraping, partly by copy-pasting) to get an idea of how the campaigns are financed. This is the result:

As you can see the campaigns are mostly paid from the pocket of the candidate. The average candidate spends 7.275 euros on the campaign (of which 3.379 euro is own money).

How is the money spent?

Mostly on advertisement in newspapers (2.838 euro in average) and outdoors (910 euros). And also on printing own campaign papers (1.297 euros).

This is all interesting, but does not really answer the initial question: how transparent is the campaign funding? Do we know who is paying for the campaigns? Answer: merely a part of it.

Only 30 percent of the support given from companies, associations and private persons is publicly disclosed, mostly because many of the donations do not exceed the 1.500 euro limit (in some cases smaller donations are also disclosed). But 70 percent of the support come from unknown actors. Not the transparency one would expect after the scandals that we have been through.

A few comments:

  • Get the data (Google Docs).
  • There seems to be something fishy with at least part of the data on Puolue ja vaalirahoitusvalvonta. When you list donations according to size Sauli Sakari Ahvenjärvi gets both a 39.000 euro donation from the district association of his party (KD Satakunnan piiri) and a 15.000 euro donation from the party. However, the latter is not account for in his own report.
  • Note that this is all based on volonatary accounting. The majority of the candidates has not said anything about how their campaigns are financed.


Why vaalikone data wants to be free

Helsingin Sanomat confirmed today that they will publish the data from their voters advice application (or in Finnish “vaalikone”, as I will call it from here on) openly next week under a Creative Commons 3.0 license. For a while I thought they would hold the data until after the elections. That is why I chose to scrape one of their questions myself the other day (which actually resulted in a story on today).

This is great news. Why? Because, as I will argue in this post, vaalikone data wants to be free.

1. Why it just wouldn’t hurt

Lets start by trying to turn this argument around. Why should this data not be distributed publicly? I think the main reason why this does not happen today is pure ignorance and old-fashioned thinking. Most media outlets just haven’t thought of it. However, if you do think about it I suppose one of the main concerns would be that you give something away to your competitors for free. The data can be used to write stories (“what do candidates think about Nato?”) and you don’t want to break your information monopoly by sharing the data. After all you have probably spent both time and money to gather the answers from all the candidates.

This is a very traditional way of thinking about journalism. Let me present you with a different perspective.

Suppose you do give the data away for free. Would your opponents use it to fill their papers? As a reporter with plenty of newsroom experience I would say probably not. No paper would like to build story after story on a material gathered by an opponent (I do think that most newsrooms would have the decency to acknowledge their sources). Anyone that has worked in a newspaper knows that you don’t mind spending an afternoon trying to get hold of the same politician that was interviewed in the competing paper just because you don’t want to quote your rival. It’s a matter of pride.

So who would use the data? Well, for example bloggers like me. My previous post was a mashup of data from a question in the vaalikone of Helsingin Sanomat about who Finland should be friends with on Facebook. Helsingin Sanomat used the post to write their own story, which probably did not take more than half an hour. Or at least much less than it would have taken them to do all the data work themself. One could say that they managed to crowdsource the refining of the data. Cost for them? Nada.

For the media outlet the real value of an application such as a “vaalikone” is in the application itself, which hopefully attracts thousands of voters looking for the right candidate. More visitors = more potential advertisement. Sharing the data doesn’t change this.

Ergo, it just wouldn’t hurt to give away the data.

2. Why there really is no option

Even if you don’t agree with my argument so far, the option of keeping the data to yourself might not even be an option in practice. If you want voters to be able to see what candidates think about various questions you have to publish the answers. And if you publish the answers there is always a risk that someone will go through all the questions and record the responses.

With more than 2000 candidates and 20-30 questions this would of course be a lot of work. However, with a simple screen scraping script the process of going through every answer to every question could be done in a matter of minutes. We are not talking about War Games style hacking here, just a small script that runs through all the (public!) pages. The same thing you could do manually yourself if you would have an extraordinary amount of spare time.

This is what I did when I recently scraped the vaalikone of Yle. Is this not stealing? Nope, not if you ask me. One could also say it is good old-fashioned investigative reporting. After all, is going through a large number of (public!) files and publishing the results not what we usually call investigative journalism? Is it somehow different if you let an automated script do all the work? I would say it’s more clever.

Ergo, even if you don’t want to publish your data, there really might not be an option. If you don’t share, someone else will.

3. Why it is the new (and right) way of doing things

Once upon a time journalism was a profession reserved for people working in more or less fancy offices. Reporters did not hesitate to take a certain pride in their position. Today this traditional role of the journalist is being challenged by bloggers and other online spectators – or citizen journalists as some might call them. It is not as easy as it used to be to define who is a journalist. In Sweden the web forum Flashback was nominated for a the journalist award of the year after a collective investigation of a severe case of school bullying. Were they journalists?

One can argue about wheter a thread on a discussion board is journalism or not, but any newsroom with serious ambitions of pursuing modern investigative reporting should consider engaging the public in one way or another. Workshops such as HS Open shows that the innovative potential is likely to be much bigger outside, than inside the newsroom. The more eyes that get to run through the data, the greater the chance of finding interesting and meaningful patterns. The more programmers that get to play around with the numbers, the cooler the mash-ups. What could they accomplish? I don’t know. And that is sort of the point with innovation and investigation.

Ergo, we need to start thinking in new ways about doing journalism and publishing open vaalikone data would be a good start. Information wants to be free, also the one behind a vaalikone.

If you live in Helsinki and you want to continue this discussion in real life, join the debate “Vaalikoneet auki!” on Wednesday 30th March.


Finland on Facebook – according to candidates

With whom should Finland be friends on Facebook? Helsingin Sanomat asked this question of all the candidates in the parliamentary elections. I screen scraped that the 1747 answers to see what they thought. If the candidates would get to choose, Finland’s Facebook profile would look something like this:

1057 friends in common
885 friends in common
595 friends in common
591 friends in common
373 friends in common
219 friends in common
145 friends in common
119 friends in common
71 friends in common
61 friends in common

So Russia is apparently our best friend. Or at least that is what we want to make them believe. Cuba ends up surprisingly high, but that is much beacuse of the Communist Party that is still keeping it real.

You’ll find the data on Google Docs if you want to examine it yourself.