This is a set of R scripts used for analyzing Google Health data for the "Searching for news: The Flint water crisis" report, published April 27, 2017. These scripts combine multiple samples, minimize noise and identify points at which search volume changes. Please see our posts describing how we did this research and how to use these scripts for more information.
This repository contains a generalized version of code used for collecting and analyzing data from the Google Health API for Pew Research Center's project, "Searching for News: The Flint Water Crisis", published on April 27, 2017.
The project explored what aggregated search behavior can tell us about how news spreads and how public attention shifts in today's fractured information environment, using the water crisis in Flint, Michigan, as a case study.
The study delves into the kinds of searches that were most prevalent as a proxy for public interest, concerns and intentions about the crisis, and tracks the way search activity ebbed and flowed alongside real world events and their associated news coverage.
Researchers collected the data via Google's Health API, to which the Center requested and gained special access for this project. For more information, read our Medium post on how we used Google Trends data to conduct our research. Note that this requires access to the Health API; to apply, click here.
R v3.4+
- dplyr
- purrr
- mgcv
- lubridate
- ggplot2
- changepoint
- stringr
- lme4
- tidyverse
- reshape2
NOTE: Use of this tool requires an API key from Google, with special access for the Health API. To request access, please contact the Google News Lab via this form.
To combine samples, load the impute_samples.r script and call process_region() with the directory the data is stored in and the name of the region.
starting_dir = <main data folder>
combine_region(starting_dir, ‘513’)
To create changepoints, load the create_changepoints.r script and call run_plots() with the data folder and the name of the region (for charts):
run_plots(“<folder>”, “Flint”)
For more information, see the post on how to use these scripts here.
This report was made possible by The Pew Charitable Trusts. Pew Research Center is a subsidiary of The Pew Charitable Trusts, its primary funder. This report is a collaborative effort based on the input and analysis of a number of individuals and experts at Pew Research Center. Google's data experts provided valuable input during the course of the project, from assistance in understanding the structure of the data to consultation on methodological decisions. While the analysis was guided by our consultations with the advisers, Pew Research Center is solely responsible for the interpretation and reporting of the data.
In addition to the license, Users must abide by the following conditions:
- User may not use the Center's logo
- User may not use the Center's name in any advertising, marketing or promotional materials.
- User may not use the licensed materials in any manner that implies, suggests, or could otherwise be perceived as attributing a particular policy or lobbying objective or opinion to the Center, or as a Center endorsement of a cause, candidate, issue, party, product, business, organization, religion or viewpoint.
Pew Research Center, April, 2017, "Searching for News: The Flint Water Crisis"
Pew Research Center, September 2018, "Searching For News" Available at: github.com/pewresearch/searching_for_news
-
September 13, 2018 "Sharing the code we used to study the public's interest in the Flint water crisis"
-
April 27, 2017 "Searching for News: The Flint Water Crisis"
-
April 27, 2017 "Using Google Trends data for research? Here are 6 questions to ask"
-
April 27, 2017 "Q&A: Using Google search data to study public interest in the Flint water crisis"
This code is provided as-is for use in your own projects. You are free to submit issues and pull requests with any questions or suggestions you may have. We will do our best to respond within a 30-day time period.
Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It does not take policy positions. The Center conducts public opinion polling, demographic research, content analysis and other data-driven social science research. It studies U.S. politics and policy; journalism and media; internet, science and technology; religion and public life; Hispanic trends; global attitudes and trends; and U.S. social and demographic trends. All of the Center's reports are available at www.pewresearch.org. Pew Research Center is a subsidiary of The Pew Charitable Trusts, its primary funder.
For all inquiries, please email info@pewresearch.org. Please be sure to specify your deadline, and we will get back to you as soon as possible. This email account is monitored regularly by Pew Research Center Communications staff.