Joining the mass of election reviewers

Like many Americans and non-Americans alike, I've been glued to the terrifying coverage of the US 2016 Presidential Election.  With only a day to go, I decided to join the large array of blogs and news outlets that have performed a review of the vast quantities of polling data available.

The source

First of all I had to get hold of some polling data.  Of course there are plenty of sources available on the internet, so I had a look at the following, but decided to go with the poll results collated by electionprojection.com, mainly because of the ease of obtaining.  Other such sources I found useful:

• realclearpolitics.com
• politico.com
• nytimes.com

In order to get hold of the state by state data I wrote a small python script to parse the data from html tables on the electionprojection website:

# Get poll data
data = []
# 
for loop in range(0,len(electoral_college)):
    state = electoral_college['State'].iloc[loop]
    state = state.replace(" ","-")
    if state.lower() == 'district-of-columbia':
        continue
    url = 'http://www.electionprojection.com/latest-polls/'+state.lower()+\
    '-presidential-polls-trump-vs-clinton-vs-johnson-vs-stein.php'
    the_content = urlopen(url)
    soup = BeautifulSoup(the_content,"html.parser")
    table = soup.find('table',attrs={'class':'mb pollblock'})
    table.prettify()
    rows = table.find_all('tr')
    for row in rows:
        cols = row.find_all('td')
        cols = [ele.text.strip() for ele in cols]
        cols.insert(0,state)
        data.append([ele for ele in cols if ele])

data = pd.DataFrame(data)
data.columns=['State','Firm','Dates','Sample','Clinton','Trump','Johnson','Stein','Spread']
poll_data = data[data.Firm.notnull()] # remove null lines
poll_df = poll_data[poll_data.Firm != 'EP Poll Average']

Given the US system uses the electoral collage for determining the next president I needed to get the a list of votes for each state.  Wikipedia could help:

The simulation

To determine tomorrow's winner I took the poll results for each state, applied a random variable for each candidate, and ran it a tonne of times (Monte Carlo).  This approach estimates the winning candidate of each state which can then be attributed to a number of electoral votes.  Simply summing up the votes gives us the winner.  

This chart shows the percentage chance of victory based on the number of simulations run.  It's clear to see this quickly converges to 74% for Clinton, and 26% for Trump.  My random variable wasn't quite random enough to give Johnson or Stein a chance.

The conclusion

Well Nate Silver of fivethirtyeight.com estimates a 72% chance for a Clinton victory... So look at that.. Maybe I should be a look for a career change.