Merge branch 'develop'

CONTRIB.md

@@ -29,8 +29,11 @@ with the password `terriblepassword`.
 For the webapp, the credentials for the testing/development environment are expected to be in a file `$PGPASS`, so set that up: 
 
 `echo "localhost:5432:openoversight-dev:openoversight:terriblepassword" >> ~/.pgpass`
+
 `echo "export PGPASS=~/.pgpass" >> ~/.bashrc`
 
+`source ~/.bashrc`
+
 In the `/vagrant/OpenOversight` directory, there is a script to create the database:
 
 `python create_db.py`
@@ -59,4 +62,4 @@ If you e.g. add a new column or table, you'll need to migrate the database. You
 `python migrate_db.py`
 
 to do this.
-`python upgrade_db.py` and `python downgrade_db.py` can also be used as necessary. Note that I followed [this tutorial](http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-iv-database) to set this up.
+`python upgrade_db.py` and `python downgrade_db.py` can also be used as necessary. Note that I followed [this tutorial](http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-iv-database) to set this up.

DEPLOY.md

@@ -75,7 +75,8 @@ SECRET_KEY = 'changemeplzorelsehax'
 
 # Systemd
 
-You can write a simple systemd unit file to launch OpenOversight on boot. We defined ours in `/etc/systemd/system/openoversight.service`. You should create the proper usernames and groups that are defined in the unit file since this allows you to drop privileges on boot. This unit file was adopted from this [DigitalOcean guide](https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-centos-7).
+You can write a simple systemd unit file to launch OpenOversight on boot. We defined ours in `/etc/systemd/system/openoversight.service`. You should create the proper usernames and groups that are defined in the unit file since this allows you to drop privileges on boot. This unit file was adopted from this [DigitalOcean guide](https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-centos-7). We define a static environment variable called `PGPASS` which is loaded by systemd before it drops priviledges for ngingx. More details can be found in `CONTRIB.md`.
+
 ```
 [Unit]
 Description=Gunicorn instance to serve OpenOversight
@@ -84,9 +85,10 @@ After=network.target
 [Service]
 User=nginx
 Group=nginx
-WorkingDirectory= /home/nginx/oovirtenv/OpenOversight/OpenOversight
+WorkingDirectory=/home/nginx/oovirtenv/OpenOversight/OpenOversight
 Environment="PATH=/home/nginx/oovirtenv/bin"
-ExecStart=/usr/local/bin/gunicorn -w 4 -b 127.0.0.1:4000 app:app &
+Environment="PGPASS=~/.pgpass"
+ExecStart=/usr/local/bin/gunicorn -w 4 -b 127.0.0.1:4000 app:app 
 
 [Install]
 WantedBy=multi-user.target

OpenOversight/app/static/theme.css

@@ -17,3 +17,6 @@ body {
   width: auto;
 }
 
+.glyphicon {
+    font-size: 30px;
+}

OpenOversight/app/templates/about.html

@@ -3,13 +3,14 @@
 
 <div class="container theme-showcase" role="main">
 <h1>About OpenOversight</h1>
-<p>OpenOversight is an accountability platform for helping the public identify police officers for the purpose of filing complaints. We rely on crowdsourced and public data to build a database of police officers in a city, allowing the public to filter through the dataset to find the name and badge number of the offending officer.</p>
-
+<br>
+<p>In support of demands for greater police accountability in Chicago and nationwide, Illinois nonprofit The Lucy Parsons Labs launched OpenOversight an interactive web tool that makes it easier for Chicago residents to file complaints against police officers. OpenOversight is an accountability platform for helping the public identify police officers for the purpose of filing complaints. We rely on crowdsourced and public data to build a database of police officers in a city, allowing the public to filter through the dataset to find the name and badge number of the offending officer.</p>
+<p>Using OpenOversight, members of the public can search for the names and badge numbers of Chicago police they have negative interactions with based on where that interaction took place and the officer's estimated age, race and gender. Using this information, the OpenOversight web application returns a digital gallery of potential matches and, when possible, includes pictures of officers in uniform to assist in identification. "The deck is stacked against people harmed by Chicago police," says Jennifer Helsby, CTO of the Lucy Parsons Labs and lead developer on the OpenOversight project. "Despite a long history of proven abuses, including torture, Chicago police are almost never held accountable for misconduct or crimes they commit. To file a misconduct complaint, the burden is on the public to provide as much detailed data about the officer as possible. OpenOversight aims to empower Chicagoans with tools that make it easier to identify officers and hold them accountable."</p>
 <br>
 <p>Facts and Figures about Chicago Police Department:</p>
 <ul>
   <li>To file a police complaint in Chicago, a member of the public needs to know as much <a href="http://ipraportal.iprachicago.org/pls/htmldb/f?p=1503:12:5406784243929236">detailed data</a> about the officer as possible. Based on complaints data from the <a href="http://invisible.institute/">Invisible Institute</a>, from March 2011 - March 2015, 28% of complaints (4,000 total complaints) were immediately dropped due to no officer identification.  <a href="http://cpdp.co/data/D8or5A/only-20525-28595-have-id-for-the-accused-officer">Source: Citizen Police Data Project</a></li>
-  <li>• All complaints against officers must be supported by a sworn affidavit. False complaints can result in perjury charges, a Class 3 felony.  <a href="http://home.chicagopolice.org/inside-the-cpd/internal-affairs-division/the-complaint/">Source: Chicago Police</a></li>
+  <li>• All complaints against officers must be supported by a sworn affidavit. False complaints can result in perjury charges, a Class 3 felony.  <a href="https://web.archive.org/web/20160829203856/http://home.chicagopolice.org/inside-the-cpd/internal-affairs-division/the-complaint/">Source: Chicago Police</a></li>
    <li>Less than 2% of the 28,567 complaints filed against the Chicago police department from March 2011 to September 2015 resulted in discipline. Most officers who do face discipline are suspended for a week or less.  <a href="http://cpdb.co/data/D1zdwL/citizens-police-data-project">Source: Citizens Police Data Project</a></li>
    <li>Chicago spent over $500 million from 2004 to 2014 on settlements, legal fees and other costs related to complaints against police officers.  <a href="http://www.bettergov.org/news/beyond-burge">Source: Better Government Association</a></li>
 </ul>
@@ -28,6 +29,8 @@ <h1>About OpenOversight</h1>
 
 <p>To review, an image is one that we have collected or have been given that may contain police officers - it could contain many police officers. A “tag” is what we call when a volunteer looks at that image, can read the badge number(s) or name(s) in that image and and thus can associate an officer in the database (name and badge number) with a face in the image. There may be many tags for a given image if there are many officers in the image. </p>
 
+<p>OpenOversight is the first project of its kind in the United States and is a direct response to the failure of Chicago's leaders to take proactive steps that allow the public to identify and report problem police officers. OpenOversight is released as free and open source software so others can launch similar police accountability projects in their own cities. The software is available for download on <a href="https://github.com/lucyparsons/OpenOversight">GitHub</a>.
+
 </div>
 
 {% endblock %}

OpenOversight/app/templates/base.html

@@ -75,11 +75,24 @@
 
     <hr>
     <footer class="footer">
-      <div class="container">
-         <img src="{{url_for('static', filename='images/github.ico')}}"/><p class="text-muted">OpenOversight is hosted on <a href="https://github.com/lucyparsons/OpenOversight"> GitHub</a>.</p>
-         <img src="{{url_for('static', filename='images/lplfavicon.png')}}"/><p class="text-muted"> A <a href="https://twitter.com/lucyparsonslabs">Lucy Parsons Labs</a> project </p>
-         <img src="{{url_for('static', filename='images/securedrop.png')}}"/><p class="text-muted">Send us documents anonymously on our <a href="https://lucyparsonslabs.com/securedrop">SecureDrop server</a>.</p>
+    <div class="container theme-showcase" role="main">
+      <div class="row">
+        <div class="col-xs-6 col-md-4">
+          <a class="text-muted" href="https://github.com/lucyparsons/OpenOversight"><img src="{{url_for('static', filename='images/github.ico')}}"/>OpenOversight is hosted on GitHub.</a>
+        </div>
+        <div class="col-xs-6 col-md-4">
+          <a class="text-muted" href="https://lucyparsonslabs.com"><img src="{{url_for('static', filename='images/lplfavicon.png')}}"/>A Lucy Parsons Labs project.</a>
+        </div>
+        <div class="pull-right col-xs-6 col-md-4">
+          <a class="text-muted" href="https://twitter.com/lucyparsonslabs"><img src="{{url_for('static', filename='images/twitterfavicon.ico')}}"/>Follow @lucyparsonslabs on Twitter.</a>
+        </div>
+      </div>
+      <div class="row">
+        <div class="col-xs-6 col-md-3"></div>
+        <div class="col-xs-6 col-md-6"><a class="text-muted" href="https://lucyparsonslabs.com/securedrop"><img src="{{url_for('static', filename='images/securedrop.png')}}"/>Send us documents or messages anonymously using our SecureDrop server.</a></div>
+        <div class="col-xs-6 col-md-3"></div>
       </div>
+    </div>
     </footer>
 
     <!-- Bootstrap core JavaScript

OpenOversight/app/templates/gallery.html

@@ -70,6 +70,10 @@ <h4>Sorry, no officers found for your query! <a href="/find">Try again</a></h4>
 
         </div>
 
+     <div class="jumbotron">
+      <span class="glyphicon glyphicon glyphicon-alert" aria-hidden="true"></span><br>
+      <b>See any errors?</b> OpenOversight is currently in public beta. Please submit any issues or suggestions you have for us on <a href="https://github.com/lucyparsons/OpenOversight">GitHub</a>, <a href="https://twitter.com/lucyparsonslabs">Twitter</a>, or by email to <a href="info@lucyparsonslabs.com">info@lucyparsonslabs.com</a>. We are very interested in any data corrections or inaccuracies you would like to report and correct.</div>
+
 <div class="page-header">
 <h1>Your Query</h1>
 </div>

OpenOversight/app/templates/index.html

@@ -9,6 +9,11 @@ <h1>OpenOversight</h1>
         <p>OpenOversight is a Lucy Parsons Labs project to improve police accountability using public and crowdsourced data. We maintain a database of police officers and provide a digital gallery that allows the public to identify the name and badge number of a police officer they would like to file a complaint about. <a href="/find"><button type="button" class="btn btn-info"">Try it</button></a></p>
       </div>
 
+      <div class="jumbotron">
+      <span class="glyphicon glyphicon glyphicon-alert" aria-hidden="true"></span><br>
+      <b>Note:</b> OpenOversight is currently in public beta. Please submit any issues or suggestions you have for us on <a href="https://github.com/lucyparsons/OpenOversight">GitHub</a>, <a href="https://twitter.com/lucyparsonslabs">Twitter</a>, or by email to <a href="info@lucyparsonslabs.com">info@lucyparsonslabs.com</a>. We are currently requesting pictures of Chicago Police Department officers in uniform with their name or badge number displayed for our database. We are also very interested in any data corrections or inaccuracies you would like to report and correct.</div>
+
+
       <div class="page-header">
         <h1>Find an Officer</h1>
       </div>

OpenOversight/app/templates/input_find_officer.html

@@ -17,7 +17,7 @@ <h4>Department: {{ form.dept }}</h4>
     {% for error in form.dept.errors %}
         <p><span style="color: red;">[{{ error }}]</span></p>
     {% endfor %}
-    <p>Don't see your department? Want to bring OpenOversight to your city? Email us at <a href="mailto:openoversight@redshiftzero.com">OpenOversight</a>.</p>
+    <p>Don't see your department? Want to bring OpenOversight to your city? Email us at <a href="mailto:info@lucyparsonslabs.com">OpenOversight</a>.</p>
 
     <h4>Rank: {{ form.rank }}</h4>
     {% for error in form.rank.errors %}

OpenOversight/app/utils.py

@@ -47,11 +47,23 @@ def grab_officer_faces(officer_ids):
         if len(faces) > 0:
             officer_images.update({officer_id: faces[0].Image.filepath})
         else:   # use placeholder image
-            officer_images.update({officer_id: 'https://placehold.it/200x200'})
+            officer_images.update({officer_id: 'static/images/placeholder.png'})
 
     return officer_images
 
 
+def sort_officers_by_photos(officers, officer_images):
+    identified_officers = []
+    unidentified_officers = []
+    for officer in officers:
+        if officer_images[officer.Officer.id] == 'static/images/placeholder.png':
+            unidentified_officers.append(officer)
+        else:
+            identified_officers.append(officer)
+
+    officers = identified_officers + unidentified_officers
+    return officers
+
 def allowed_file(filename):
     return '.' in filename and \
         filename.rsplit('.', 1)[1] in config.ALLOWED_EXTENSIONS

OpenOversight/app/views.py

@@ -4,9 +4,10 @@
 from werkzeug import secure_filename
 from app import app
 
-from utils import allowed_file, grab_officers, grab_officer_faces
+from utils import (allowed_file, grab_officers, grab_officer_faces,
+                  sort_officers_by_photos)
 from forms import FindOfficerForm
-
+import config
 
 @app.route('/')
 @app.route('/index')
@@ -26,13 +27,15 @@ def get_officer():
 @app.route('/gallery', methods=['GET', 'POST'])
 def get_gallery():
     form_values = request.form
-
     officers = grab_officers(form_values)
     officer_ids = [officer.Officer.id for officer in officers]
     officer_images = grab_officer_faces(officer_ids)
+    sorted_officers = sort_officers_by_photos(officers, officer_images)
 
-    return render_template('gallery.html', officers=officers, form=form_values,
-                           officer_images=officer_images)
+    return render_template('gallery.html',
+                           officers=sorted_officers,
+                           form=form_values,
+		           officer_images=officer_images)
 
 
 @app.route('/complaint', methods=['GET', 'POST'])

OpenOversight/create_db.py

@@ -1,3 +1,5 @@
+#!/usr/bin/python
+
 from migrate.versioning import api
 from config import SQLALCHEMY_DATABASE_URI
 from config import SQLALCHEMY_MIGRATE_REPO
@@ -13,4 +15,4 @@
 else:  # the database already exists
     api.version_control(SQLALCHEMY_DATABASE_URI,
     	                SQLALCHEMY_MIGRATE_REPO,
-    	                api.version(SQLALCHEMY_MIGRATE_REPO))
+    	                api.version(SQLALCHEMY_MIGRATE_REPO))

OpenOversight/test_data.py

@@ -1,4 +1,8 @@
+#!/usr/bin/python
+
+import argparse
 from datetime import datetime
+import sys
 
 from app import db, models
 
@@ -57,20 +61,48 @@ def populate():
 
 def cleanup():
     """ Cleanup database"""
+
+    faces = models.Face.query.all()
+    for face in faces:
+        db.session.delete(face)
+
     officers = models.Officer.query.all()
     for po in officers:
         db.session.delete(po)
 
     assignments = models.Assignment.query.all()
     for assn in assignments:
         db.session.delete(assn)
-
+   
     faces = models.Face.query.all()
     for face in faces:
         db.session.delete(face)
 
-    images = models.Image.query.all()
-    for image in images:
-        db.session.delete(image)
-
+    # TODO: Reset primary keys on all these tables
     db.session.commit()
+
+
+if __name__=="__main__":
+   parser = argparse.ArgumentParser()
+   parser.add_argument("-p", "--populate", action='store_true',
+                       help="populate the database with test data")
+   parser.add_argument("-c", "--cleanup", action='store_true',
+                       help="delete all test data from the database")
+   args = parser.parse_args()
+
+   if args.populate:
+       print("[*] Populating database with test data...")
+       try:
+           populate()
+           print("[*] Completed successfully!")
+       except:
+           print("[!] Encountered an unknown issue, exiting.")
+           sys.exit(1)
+   if args.cleanup:
+       print("[*] Cleaning up database...")
+       try:
+           cleanup()
+           print("[*] Completed successfully!")
+       except:
+           print("[!] Encountered an unknown issue, exiting.")
+           sys.exit(1)

README.md

@@ -6,7 +6,7 @@ OpenOversight is a Lucy Parsons Labs project to improve police accountability th
 
 As a proof of concept, OpenOversight currently uses the Chicago Police Department but this infrastructure will be used to extent the project to other cities where it is needed. Interested in helping bring OpenOversight to your city? Email us at [info@lucyparsonslabs.com](mailto:info@lucyparsonslabs.com).  
 
-This project is written and maintained by [@lucyparsonslabs](https://twitter.com/lucyparsonslabs.com) with collaboration, partnerships, and contributions welcome. If you would like to contribute code or documentation, please see our contributing guide. If you prefer to contribute in other ways, please submit images to our platform or use our crowd-sourced human labeling tool to help build training datasets.
+This project is written and maintained by [@lucyparsonslabs](https://twitter.com/lucyparsonslabs.com) with collaboration, partnerships, and contributions welcome. If you would like to contribute code or documentation, please see our contributing guide. If you prefer to contribute in other ways, please submit images to our platform or talk to us about how to help sort and tag images. This project is in public beta, and we are currently soliciting photographs to add to the database.
 
 ## Note to Law Enforcement
 
@@ -18,8 +18,7 @@ Please use [our issue tracker](https://github.com/lucyparsons/OpenOversight//iss
 
 ## Technologies 
 
- * We use Python 2.7 as we have deps not yet Python 3 compatible 
- * Face detection: openface, opencv
+ * We use Python 2.7
  * Web frontend: Flask, WTForms for input validation
  * Database backend: psycopg2, PostgreSQL
  * ETL toolchain: pandas, sqlalchemy