Updated with some corrections/clarifications from Paul Tagliamonte, who hacked on the newly-released python-sunlight library and blogs here.
Updated with new code snippet to use terminal input to search for campaign contributions to legislators and write to a csv.
A little less than a year ago I tried to play with the Open States legislator API, and with a bunch of help came away with a simple php-based search of our legislators.
If you haven’t played with the Open States API before, you should sign up for an API key and start, but there is a wealth of information — 47 states worth — just waiting to be unlocked by your imagination.
If you at least a passing interest in python, here are some steps that a beginner can use to grab legislator data and write it to a csv. I’ll follow up later with a method of sending that data to Google’s Fusion Tables where you could integrate into a map.
So where to start?
On Monday, Sunlight Labs — part of the Sunlight Foundation with created Open States — announced an updated python library to interact with it’s Congress, Open States and Capitol Words APIs.
You will need an API key to use it, but it’s just a matter of registering and waiting for the email.
Once you have the API key, Sunlight Labs has a couple easy steps to get setup. This is straight from their blog post:
If you’re on a UNIX-type (MacOS, GNU/Linux, *BSD, AIX or Solaris (or any of the other POSIX-ey systems)) machine, you should be able to run a command that looks like the following:
echo "your-api-key-here" > ~/.sunlight.key
It’s worth mentioning that your-api-key-here should actually be your API key that was emailed to you up above.
Via Paul, here’s what is happening at this step:
…this actually creates a “dot” (really: hidden on UNIX systems, so it doesn’t cloud up your folder) file in your home directory
The tilde (~) expands to where your home folder is — so on my system, it’d be something like
So this actually writes a file to
so that each user on the system can have their own key.
The lib (during runtime) will look to see if this one file is present, and use that if it is. You can see if it’s there (as well as tons of other dot-files) by running something like:
Basically — and correct me if I am wrong here — this you want to make a “dot” file .sunlight.key and place it in your root directory and tell python where to find it.
Then using the python package installer pip, bring the library to your system.
Then using the python package installer pip you can install the Sunlight library.
However, pip doesn’t come with a default Python installation, so if you receive an error when you type in the command above, you can install setup tools first, which will give you access to the easy_install package manager.
Then it’s a matter of using easy_install to grab pip, and pip to grab sunlight.
easy_install pip
pip install sunlight
Now whenever a walkthrough suggests placing a file that begins with a dot in a directory that has ~ stand in for it, I get a little nervous; I’m not gonna lie. This one was fairly painless.
The documentation and GitHub repo has some sample code — a lot of sample code — and I repurposed a portion for this little snippet. You can save as legi-return.py, and it will return all the legislators for Wisconsin.
#imports the library
import sunlight
#query for wisco legislators
legis = sunlight.openstates.legislators(
state='wi'
)
#identify each legislator in the output
for legi in legis:
#return each legislator
print "%s %s (%s) District: %s Party: %s ID: %s" % (
legi['first_name'], legi['last_name'], legi['chamber'], legi['district'], legi['party'], legi['leg_id'])
Just cd into the directory and type python legi-return.py into the terminal and you should see some output.
If you get an error, it might be of the indentation variety, so I’ve also made a gist of the code.
Once I got this up and running, and being a python beginner curious of my own abilities, I started to wonder if I could create a search of legislators from the terminal. So I made this code snippet to search for Wisconsin state legislators by last name. You can call this legi-lookup.py if you want to.
#import library
import sunlight
#ask for input
legi_name = raw_input("Enter the Legislator's Last Name...")
#pull wisco legis with last name entered by user
legis = sunlight.openstates.legislators(
state='wi',
last_name=legi_name
)
#tell the user what you found
print "We found these legislators"
for legi in legis:
print "%s %s (%s) District: %s Party: %s ID: %s" % (
legi['first_name'], legi['last_name'], legi['chamber'], legi['district'], legi['party'], legi['leg_id'])
As you can see, this borrows quite heavily from the first snippet. The difference is I’m asking for input from the terminal — use Fitzgerald — and then narrowing the results based on the input.
Also, this area here…
legis = sunlight.openstates.legislators(
state='wi',
last_name=legi_name
)
…can take all kinds of parameters to narrow your query. There’s a list here you can use.
So I’m feeling all good about thing, and now I want to take my initial query — for all of the state legislators — and I want to write them to a csv. I had this from a prior walkthrough…
import csv
#opens the csv writer, specifies the file name
writer = csv.writer(open('stocks.csv', 'wb', buffering=0))
#data that should be written, separated by comma
writer.writerows([
('GOOG', 'Google, Inc.', 505.24, 0.47, 0.09),
('YHOO', 'Yahoo! Inc.', 27.38, 0.33, 1.22),
('CNET', 'CNET Networks, Inc.', 8.62, -0.13, -1.49)
])
…so I figured the principals were the same.
It actually wasn’t that difficult to figure out since I was already identifying the specific fields that I wanted to be output. You can save this snippet as legi-to-csv.py.
#import libraries
import sunlight
import csv
#pull wisco legis
legis = sunlight.openstates.legislators(
state='wi'
)
#open csv writer
writer = csv.writer(open('legi.csv', 'wb', buffering=0), delimiter=';', quoting=csv.QUOTE_ALL)
#open loop
for legi in legis:
#write csv rows
writer.writerows([
(legi['first_name'], legi['last_name'], legi['chamber'], legi['party'] )
])
Then assuming you are in the same directory, type python levi-to-csv.py into the terminal and you should see a file called levi.csv added to your directory. Hopefully it has a series of comma delimited fields of legislator goodness.
UPDATE
So here is one more for you…
Firing this snippet in the terminal will ask you for a legislator’s last name, state and campaign cycle, query influence explorer and then write the results to a csv.
#!/usr/bin/env python
#import library
import sunlight
from sunlight import influenceexplorer
import csv
#ask for input
legi_name = raw_input("Input Legislator's last name...")
legi_state = raw_input("Enter state...")
legi_cycle = raw_input("Enter campaign cycle...")
#pull legis with last name entered by user
legis = sunlight.openstates.legislators(
state=legi_state,
last_name=legi_name
)
#queries against influence explorer
contrib = influenceexplorer.contributions(
contributor_state=legi_state,
recipient_ft=legi_name,
cycle=legi_cycle
)
#open csv writer
writer = csv.writer(open('money.csv', 'wb', buffering=0), delimiter=';', quoting=csv.QUOTE_ALL)
print "Writing CSV file for you"
for legi in legis:
(
legi['first_name'],
legi['last_name'],
legi['district'],
legi['party'])
for contributor in contrib:
(
contributor['contributor_name'],
contributor['contributor_city'],
contributor['amount'],
contributor['seat'],
)
#write csv rows
writer.writerows([
(
contributor['contributor_name'],
contributor['contributor_city'],
contributor['amount'],
contributor['seat'],
legi['first_name'],
legi['last_name'],
legi['district'],
legi['party']
)
])
print "Finished"
One thing I need to learn is how to write header rows, so I’ll work on that and update.
From here, your imagination is really the limit. With the Open States you can query legislation, committee assignments, committee hearings – all kinds of things. And don’t forget, this library interacts with the Congressional and Capitol Words API, which means there is so much data at your fingertips. So a web producer could take this and build a search from it, or a journalist could use it to search for information right from their computer.
Let this settle in and next we’ll talk about feeding data from Open States to Google Fusion Tables to make a legi mashup map.
