notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kevin de León district election results analysis\n",
    "\n",
    "By [Ben Welsh](https://palewi.re/who-is-ben-welsh/)\n",
    "\n",
    "A November 2018 contest for California's U.S. Senate seat matched incumbent Sen. Dianne Feinstein against former state Sen. Kevin de León. This analysis investigates how de León fared in his own district. It was conducted for the February 11, 2019, Los Angeles Times story [\"Former state Sen. Kevin de León will run to replace Jose Huizar on L.A. City Council.\"](https://www.latimes.com/local/lanow/la-me-ln-de-leon-city-council-20190211-story.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Python tools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import geopandas as gpd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.simplefilter(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze election results\n",
    "\n",
    "Election results published in PDF form by the [Los Angeles County Registrar/Recorder](https://lavote.net/home/voting-elections/current-elections/election-results/past-election-results#11062018) are available as comma-separated values thanks to the [OpenElections](http://openelections.net/) project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "results = pd.read_csv('https://raw.githubusercontent.com/openelections/openelections-data-ca/master/2018/20181106__ca__general__los_angeles__precinct.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The precinct numbers need to be split off from city names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "parse_precinct_number = lambda val: val.split(\" \")[0].strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "results['precinct_number'] = results.precinct.apply(parse_precinct_number)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filter down to results from the U.S. Senate contest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "us_senate = results[results.office == 'US Senate']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filter down to results from de León's constituency, the 24th State Senate district."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "senate24 = df[\n",
    "    (df.office == 'State Senate') &\n",
    "    (df.district == 24)\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filter the U.S. Senate results down to de León district."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "filtered = us_senate[us_senate.precinct_number.isin(senate24.precinct_number)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clean up the columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed = filtered[[\n",
    "    'precinct_number',\n",
    "    'candidate',\n",
    "    'votes'\n",
    "]].rename(columns={\"precinct_number\": \"precinct\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate the total votes for each candidate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>candidate</th>\n",
       "      <th>votes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Dianne Feinstein</td>\n",
       "      <td>111860</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Kevin DeLeon</td>\n",
       "      <td>105236</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          candidate   votes\n",
       "0  Dianne Feinstein  111860\n",
       "1      Kevin DeLeon  105236"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trimmed.groupby(\"candidate\").votes.sum().reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate the difference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6624"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "111860 - 105236"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert the precint results to a pivot table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "pivot = trimmed.pivot(index=\"precinct\", columns=\"candidate\", values=\"votes\").reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clean up the column headers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_pivot = pivot[[\n",
    "    'precinct',\n",
    "    'Dianne Feinstein',\n",
    "    'Kevin DeLeon'\n",
    "]].rename(columns={\n",
    "    'Dianne Feinstein': \"feinstein\",\n",
    "    'Kevin DeLeon': \"deleon\"\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add a column with the winner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_winner(row):\n",
    "    if row.feinstein == row.deleon:\n",
    "        return 'tie'\n",
    "    elif row.feinstein > row.deleon:\n",
    "        return 'feinstein'\n",
    "    else:\n",
    "        return 'deleon'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_pivot['winner'] = trimmed_pivot.apply(get_winner, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add up how many precincts each candidate won."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "feinstein    164\n",
       "deleon       151\n",
       "tie            4\n",
       "Name: winner, dtype: int64"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trimmed_pivot.winner.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate the margin of victory in each precinct."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_margin = lambda row: abs(row.feinstein - row.deleon)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_pivot['margin'] = trimmed_pivot.apply(get_margin, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write out the pivot to a CSV for inspection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_pivot.to_csv(\"./output/sd-24-results.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Merge results to map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read in a shapefile of precincts from the Los Angeles County Registrar/Recorder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "shp = gpd.read_file(\"./input/RRCC_PRECINCTS_ELECTION.shp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clean up the columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_shp = shp[[\n",
    "    'Precinct',\n",
    "    'geometry'\n",
    "]].rename(columns={\"Precinct\": \"precinct\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Correct a couple bad precinct numbers discovered by visual inspection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "def correct_shp(precinct):\n",
    "    if precinct == '9001668B':\n",
    "        return '9001668A'\n",
    "    elif precinct == '9005012B':\n",
    "        return '9005012A'\n",
    "    else:\n",
    "        return precinct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_shp['precinct'] = trimmed_shp.precinct.apply(correct_shp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Merge the shapefile to the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "merged_shp = trimmed_shp.merge(trimmed_pivot, on=\"precinct\", how=\"inner\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write out the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "merged_shp.to_crs({\"init\": \"EPSG:4236\"}).to_file(\"./output/sd-24-polygons.shp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert polygons to points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "merged_shp['geometry'] = merged_shp.geometry.centroid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write that out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "merged_shp.to_crs({\"init\": \"EPSG:4236\"}).to_file(\"./output/sd-24-points.shp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Map results in QGIS\n",
    "\n",
    "The [map.qgs](./map.qgs) file included with this repository plots the results. Circles are sized by margin of victory. Yellow precincts were won by Feinstein. Green precincts were won by de León."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./output/screenshot.png\">"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}