1_Evolution_of_Community_Norms.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 1: Evolution of Community Norms\n",
    "\n",
    "## Contents\n",
    "1. [Setup](#Section-1%3A-Setup)\n",
    "    1. [Import](#1.1-Import-Packages)\n",
    "    1. [Data](#1.2-Data)\n",
    "1. [Activity over Time](#Section-2%3A-Activity-over-Time)\n",
    "    1. [Measuring Overall Conflict](#2.1-Measuring-Overall-Conflict)\n",
    "    1. [Drivers of Change](#2.2-Drivers-of-Change)\n",
    "    1. [Joining after Default Day](#2.3-Joining-after-Default-Day)\n",
    "    1. [Being New to the Community](#2.4-Being-New-to-the-Community)\n",
    "    1. [Coming from Hostile Communities](#2.5-Coming-from-Hostile-Communities)    \n",
    "\n",
    "## Section 0: Background\n",
    "- `r/TwoXChromosomes` (TwoX) is a feminist subreddit community that started in 2009. People interested in the topic found it by searching or following links that other people posted.\n",
    "- On May 7, 2014, it was made a \"default subreddit,\" which means people saw it on the front page of reddit. \n",
    "- A flood of new people went to the subreddit. They didn't know the community, and many of them didn't like feminism.\n",
    "    - Users of anti-feminist subreddits, in particular, were angry that a feminist subreddit made the list of defaults. \n",
    "- This was a big contraversy, both within reddit and in broader newsmedia.\n",
    "    - For example: \"[Reddit women protest at new front-page position](https://www.theguardian.com/technology/2014/may/13/reddit-women-protest-front-page-subforum-subreddit-position)\"\n",
    "- We're going to investigate what happened using data from this subreddit.\n",
    "\n",
    "### 0.1 Topics\n",
    "- Changes in online communities over time\n",
    "- Changes in communities due to sudden events\n",
    "- Whether community change is driven by longtime users changing their behavior, or new users bringing new behaviors\n",
    "\n",
    "## Section 1: Setup\n",
    "### 1.1 Import Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "tqdm.pandas()\n",
    "import numpy as np\n",
    "import urllib.request\n",
    "import os.path\n",
    "from datetime import datetime, timedelta\n",
    "import matplotlib.pyplot as plt\n",
    "pd.plotting.register_matplotlib_converters()\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Data\n",
    "- Here we load the data and do a little cleaning up. \n",
    "- Don't worry about how the cleanup is done, just run the code and scroll down."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download from https://drive.google.com/drive/folders/1vTmSKLrq-Io1E935VzAqVSGnqftSBNTY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install gdown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gdown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if os.path.isfile(\"data/merged/TwoXChromosomes.tsv.gz\"):\n",
    "    print(\"You already downloaded the data. Great!\")\n",
    "else:\n",
    "    print('Downloading file...')\n",
    "    gdown.download(id=\"106wrL0XqvZp7gDd9CwIiLxf-L7WFQP3a\",\n",
    "                               output=\"data/merged/TwoXChromosomes.tsv.gz\")\n",
    "    print('Done!')\n",
    "    \n",
    "print(\"Loading data...\")\n",
    "twox_comments = pd.read_csv('data/merged/TwoXChromosomes.tsv.gz', sep='\\t')\n",
    "\n",
    "print('Sampling data...')\n",
    "#the full data requires more than 4GB RAM to run this notebook\n",
    "twox_comments = twox_comments.sample(frac=.6)\n",
    "\n",
    "print(\"Converting dates...\")\n",
    "twox_comments['date'] = pd.to_datetime(twox_comments.date)\n",
    "\n",
    "print('Compressing data...')\n",
    "twox_comments.drop_duplicates(subset=['id'], inplace=True)\n",
    "twox_comments['id'] = twox_comments.index.values\n",
    "f = twox_comments.select_dtypes(include=['int']).apply(pd.to_numeric,downcast='unsigned')\n",
    "twox_comments[f.columns] = f\n",
    "f = twox_comments.select_dtypes(include=['float']).round(3).apply(pd.to_numeric,downcast='float')\n",
    "twox_comments[f.columns] = f\n",
    "if 'parent_id' in twox_comments.columns:\n",
    "    twox_comments.drop(columns=['parent_id'], inplace=True)\n",
    "twox_comments['author'] = twox_comments.author.astype('category')\n",
    "\n",
    "print(\"Done!\")\n",
    "twox_comments.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "twox_comments.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Helper function for plotting\n",
    "Don't worry about what's in it, just run the code and scroll down."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_plot(grouped, columns='id', title=None, top=None, bottom=None, \n",
    "             events=None, agg='mean', years=[2012,2016]):\n",
    "    \n",
    "    plt.close('all')\n",
    "    \n",
    "    fig, axs = plt.subplots(figsize=(14,10))\n",
    "    if bottom is not None:\n",
    "        axs.set_ylim(bottom=bottom)\n",
    "        \n",
    "    if top is not None:\n",
    "        axs.set_ylim(top=top)\n",
    "\n",
    "    if years is not None:\n",
    "        start = datetime(year=years[0], month=1, day=1)\n",
    "        end = datetime(year=years[1], month=1, day=1)\n",
    "        axs.set_xlim(left=start, right=end)\n",
    "\n",
    "    if isinstance(columns, str):\n",
    "        columns = [columns]\n",
    "        \n",
    "    if agg == 'mean':\n",
    "        for c in columns:\n",
    "            means = grouped[c].mean()\n",
    "            sems = grouped[c].sem()\n",
    "            axs.plot(means.index, means, label=c)\n",
    "            axs.fill_between(sems.index, means-(1.96*sems), \n",
    "                             means+(1.96*sems), alpha=0.5)\n",
    "        if title is None:\n",
    "            title = 'Average scores with 95% confidence interval'\n",
    "        axs.legend()\n",
    "        axs.set_ylabel('Score')\n",
    "            \n",
    "    elif agg == 'count':\n",
    "        for c in columns:\n",
    "            counts = grouped.count()[c]\n",
    "            axs.plot(counts, label=c)\n",
    "        if title is None:\n",
    "            title = 'Number of comments per month'\n",
    "        axs.set_ylabel('Count')\n",
    "            \n",
    "    elif agg == 'unique':\n",
    "        for c in columns:\n",
    "            counts = grouped[c].nunique()\n",
    "            axs.plot(counts, label=c)\n",
    "        if title is None:\n",
    "            title = 'Number of unique ___ per month'\n",
    "        axs.legend()\n",
    "        axs.set_ylabel('Count')\n",
    "            \n",
    "        \n",
    "    if events is not None:\n",
    "        if years is not None:\n",
    "            events = events[events.date >= start]\n",
    "            events = events[events.date <= end]\n",
    "        for e in events.iterrows():\n",
    "            if e[1].event == 1:\n",
    "                axs.axvline(e[1].date, color='k', linestyle='dashed', \n",
    "                            alpha=.5)\n",
    "                spot = axs.get_ylim()[1] - axs.get_ylim()[0]\n",
    "                spot *= .1 \n",
    "                spot += axs.get_ylim()[0]\n",
    "                axs.text(e[1].date, spot, e[1].description)            \n",
    "            \n",
    "    axs.set_title(title)      \n",
    "                \n",
    "    plt.show()\n",
    "    return\n",
    "\n",
    "events = [{'date': datetime(year=2014, month=5, day=7),\n",
    "          'event': 1,\n",
    "          'description': 'Default Day'},\n",
    "          {'date': datetime(year=2017, month=2, day=1),\n",
    "          'event': 1,\n",
    "          'description': 'Ban Day'}]\n",
    "events = pd.DataFrame(events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Section 2: Activity over Time\n",
    "- When TwoX was added to Reddit's front page, people say there was a surge of new users. We can test this by looking at the data.\n",
    "- This code puts the comments into groups: one group for every month. Then it gets the count of comments and the count of people commenting in each month. Finally, it plots them for us to see how the level of activity changes over time. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Group the comments by month\n",
    "monthly = twox_comments.resample('M', on='date')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_plot(monthly, agg='count', events=events)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_plot(monthly, columns='author', agg='unique', events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Notice anything?\n",
    "- There's a huge jump up in the number of comments and users in May 2014, when it was added to the front page. \n",
    "\n",
    "### 2.1 Measuring Overall Conflict\n",
    "- If there is conflict among people in the subreddit, we might expect more posts to get deleted. \n",
    "- The first graph shows us what percent of posts are deleted.\n",
    "- The other graphs show different measures of the comments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_plot(monthly, columns='deleted', \n",
    "          title=\"Percent of comments deleted\",\n",
    "          events=events, bottom=0, top=.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_plot(monthly, columns=['OBSCENE', 'TOXICITY', \n",
    "                            'ATTACK_ON_COMMENTER', \n",
    "                            'INFLAMMATORY'], \n",
    "          events=events, top=.4, bottom=.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "make_plot(monthly, columns=['sentiment'], \n",
    "          events=events, top=.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Drivers of Change\n",
    "- We saw above that the community changed dramatically after it became a default community. But that leaves unanswered questions:\n",
    "    - Did everyone start behaving differently after default day?\n",
    "    - Is the change because there are different people commenting than before?\n",
    "    - Is the change because newcomers haven't adapted to the community's norms yet?  \n",
    "    - Does it matter what other communities a user is in if we want to know their behavior here?\n",
    "- Next we're going to compare users who joined before default day with those who joined after it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "\n",
    "#### Short Answer 1:\n",
    "- Before running the code below, answer these questions with a sentence or two each:\n",
    "    - Summarize the changes before and after the Default Day. \n",
    "        - First, summarize the overall change of the percentage of deleted posts. (1 sentence) \n",
    "        - Next, summarize the changes in (toxticity, .... ). (1 sentence per measure, 4 measures total)\n",
    "        - Then, summarize the overall change of the sentiment score. (1 sentence)\n",
    "    - What do you think might be driving the changes in average scores that we saw above?\n",
    "    - What ways might users who joined after it was a default community be different in our measures of comment civility than the users who joined before it was a default community?\n",
    "    \n",
    "    \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🤔 **Write your response here:**\n",
    "...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Helper functions\n",
    "- The code below labels users based on when they joined TwoX.\n",
    "- Don't worry about how the code in this cell works, just run it and scroll down."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def p(top, bottom, events, years, axs):    \n",
    "    if bottom is not None:\n",
    "        axs.set_ylim(bottom=bottom)\n",
    "    if top is not None:\n",
    "        axs.set_ylim(top=top)\n",
    "    \n",
    "    if years is not None:\n",
    "        start = datetime(year=years[0], month=1, day=1)\n",
    "        end = datetime(year=years[1], month=1, day=1)\n",
    "            \n",
    "    if events is not None:\n",
    "        if years is not None:\n",
    "            events = events[events.date >= start]\n",
    "            events = events[events.date <= end]\n",
    "        else:\n",
    "            start = events.iloc[0].date - timedelta(days=30)\n",
    "            end = events.iloc[0].date + timedelta(days=30)\n",
    "        for e in events.iterrows():\n",
    "            if e[1].event == 1:\n",
    "                axs.axvline(e[1].date, color='k', linestyle='dashed', \n",
    "                            alpha=.5)\n",
    "                spot = axs.get_ylim()[1] - axs.get_ylim()[0]\n",
    "                spot *= .1 \n",
    "                spot += axs.get_ylim()[0]\n",
    "                axs.text(e[1].date, spot, e[1].description) \n",
    "    axs.set_xlim(left=start, right=end)\n",
    "    axs.legend()\n",
    "    plt.show()\n",
    "    return \n",
    "\n",
    "def plot_users(counts, column='id', title=None, top=None, \n",
    "               bottom=None, events=None,\n",
    "               years=[2012,2016], cohorts='once'):\n",
    "    \n",
    "    plt.close('all')\n",
    "    fig, axs = plt.subplots(figsize=(14,10))\n",
    "    \n",
    "    if isinstance(cohorts, str):\n",
    "        cohorts = [cohorts]\n",
    "        \n",
    "    for c in cohorts:\n",
    "        axs.plot(counts[column][c], \n",
    "                 label=c)\n",
    "        \n",
    "    if title is None:\n",
    "        title = 'Number of comments per month'\n",
    "    axs.set_title(title)\n",
    "    axs.set_ylabel(column)\n",
    "    \n",
    "    p(top, bottom, events, years, axs)\n",
    "    return\n",
    "\n",
    "def plot_users_mean(means, sems, title=None, top=None, \n",
    "               bottom=None, events=None,column='id',\n",
    "               years=[2012,2016], cohorts='once'):\n",
    "    \n",
    "    plt.close('all')\n",
    "    fig, axs = plt.subplots(figsize=(14,10))\n",
    "    \n",
    "    if isinstance(cohorts, str):\n",
    "        cohorts = [cohorts]\n",
    "        \n",
    "    for c in cohorts:\n",
    "        axs.plot(means[column][c].index, \n",
    "                 means[column][c],\n",
    "                 label=c)\n",
    "        axs.fill_between(sems[column][c].index, \n",
    "                         means[column][c]-(sems[column][c]), \n",
    "                         means[column][c]+(sems[column][c]), \n",
    "                         alpha=0.5)\n",
    "    if title is None:\n",
    "        title = 'Average '+column+' with 95% confidence interval'\n",
    "    axs.set_title(title)\n",
    "    axs.set_ylabel(column)\n",
    "    \n",
    "    p(top, bottom, events, years, axs)\n",
    "    return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "users = {}\n",
    "groups = [{'name': 'before', 'direction': 'lt',\n",
    "           'date': datetime(year=2014, month=5, day=7)},\n",
    "          {'name': 'after', 'direction': 'gt',\n",
    "           'date': datetime(year=2014, month=5, day=7)}]\n",
    "\n",
    "print('Finding one-time posters...')\n",
    "#select people who only posted one comment ever\n",
    "tmp = twox_comments.groupby(by='author').count()\n",
    "users['once'] = set(tmp[tmp.id == 1].index.values)\n",
    "del tmp\n",
    "\n",
    "#select all other people, \n",
    "firsts = twox_comments[~twox_comments.author.isin(users['once']\n",
    "                                                 )][['author',\n",
    "                                                     'date']]\n",
    "\n",
    "print('Finding initial posts...')\n",
    "#figure out their first post date\n",
    "firsts.sort_values(by='date', inplace=True)\n",
    "firsts.drop_duplicates(subset='author', keep='first', inplace=True)\n",
    "\n",
    "print('Dividing users into groups...')\n",
    "#select the users in each group\n",
    "for g in groups:\n",
    "    if g['direction'] == 'lt':\n",
    "        users[g['name']] = set(firsts[firsts.date < g['date']].author)\n",
    "    elif g['direction'] == 'gt':\n",
    "        users[g['name']] = set(firsts[firsts.date > g['date']].author)\n",
    "\n",
    "def get_cohort(user):\n",
    "    c = np.nan\n",
    "    # these need to be in chronological order\n",
    "    cohorts = ['once', 'before', 'after']\n",
    "    for co in cohorts:\n",
    "        if user == '[deleted]':\n",
    "            c = np.nan\n",
    "        elif user in users[co]:\n",
    "            c = co\n",
    "    \n",
    "    return c\n",
    "\n",
    "#label the comments by their author's cohort\n",
    "twox_comments['cohort'] = twox_comments.author.apply(get_cohort).astype('category')\n",
    "\n",
    "del users\n",
    "\n",
    "print('Calculating statistics....')\n",
    "#group comments by both date and cohort\n",
    "cohorts = twox_comments.groupby(by='cohort').resample('M', on='date') \n",
    "cohort_mean = cohorts.mean(numeric_only=True)\n",
    "cohort_error = cohorts.sem(numeric_only=True) * 1.96"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Joining after Default Day\n",
    "\n",
    "#### What fraction of posts were made by people who joined before vs after default day?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users(cohorts.count(), column='id', \n",
    "           cohorts=['before', 'after'], \n",
    "           bottom=0, top=90000,\n",
    "           events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "\n",
    "#### Short Answer 2:\n",
    "\n",
    "- Observe the graph above, and document the pattern of comments per month for people who joined before and after the Default Day. (2-3 sentences)\n",
    "    \n",
    "- Were the people who came to TwoX after default day different than those who came to the community before it? (Would making the community default change the type of people it will attract? Discuss from a social perspective, and use your intuition)\n",
    "- Next, we are going to look at who is to be accounted for the increased conflict. Based on your intuition, who (people who joined before the Default Day, or people who joined after the Default Day) do you think are...\n",
    "    - generating more toxic posts?\n",
    "    - attacking commentors more often?\n",
    "    - generating more inflammatory posts?\n",
    "    - posting more negatively? \n",
    "- Explain your prediction briefly. \n",
    "    \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🤔 **Write your response here:**\n",
    "...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### What do the posts of people who joined before and after default day look like?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(cohort_mean, cohort_error, column='TOXICITY', \n",
    "           cohorts=['before', 'after'], \n",
    "           events=events, bottom=.2, top=.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "    \n",
    "#### Coding Question 1\n",
    "\n",
    "Fill in the \"BITE_ME\" below to generate a graph for \"Average ATTACK_ON_COMMENTER with 95% confidence interval\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(cohort_mean, cohort_error, column='BITE_ME', \n",
    "           cohorts=['before', 'after'], \n",
    "           events=events, bottom=.2, top=.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "\n",
    "#### Coding Question 2\n",
    "    \n",
    "Fill in the \"BITE_ME\" below to generate a graph for \"Average INFLAMMATORY with 95% confidence interval\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(cohort_mean, cohort_error, column='BITE_ME', \n",
    "           cohorts=['before', 'after'], \n",
    "           events=events, bottom=.2, top=.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "    \n",
    "#### Coding Question 3\n",
    "\n",
    "Fill in the \"BITE_ME\" below to generate a graph for \"Average sentiment with 95% confidence interval\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(cohort_mean, cohort_error, column='BITE_ME', \n",
    "           cohorts=['before', 'after'], \n",
    "           events=events, bottom=.0, top=.15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "\n",
    "#### Short Answer 3:\n",
    "- Write a sentence or two for each question:\n",
    "    \n",
    "    2. How do these graphs compare with what you predicted before you looked?\n",
    "    3. Did either group seem to contribute more to the overall changes in the community we saw in the beginning?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🤔 **Write your response here:**\n",
    "...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 2.4 Being New to the Community\n",
    "Is it about new users?\n",
    "- We saw that people who joined after default day behaved differently than those who joined before. In the next section, we'll look at whether *new* users are always different than users who have been around longer. \n",
    "    - ** Here, we define comments from new users as comments posted within the first 7 days a user is in the community. All other comments are from \"old\" users, who have been in the community more than one week. **\n",
    "- Run the helper code below and scroll down."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "firsts.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Finding when users joined...')\n",
    "firsts = firsts[firsts.author != '[deleted]']\n",
    "tmp = twox_comments[['author', 'date']].merge(firsts, \n",
    "                                              how='left', \n",
    "                                              on='author',\n",
    "                                             )\n",
    "\n",
    "print('Getting time since first post...')\n",
    "tmp = (tmp.date_x - tmp.date_y) < timedelta(days=7)\n",
    "twox_comments['new_user'] = tmp.replace({True: 'new', False:'old'}).astype('category')\n",
    "\n",
    "del tmp\n",
    "del firsts\n",
    "\n",
    "print('Calculating statistics...')\n",
    "#group comments by both date and cohort\n",
    "newbies = twox_comments.groupby(by='new_user').resample('M', on='date')\n",
    "newbies_mean = newbies.mean(numeric_only=True)\n",
    "newbies_error = newbies.sem(numeric_only=True) * 1.96"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(newbies_mean, newbies_error, column='TOXICITY', \n",
    "           cohorts=['old', 'new'], \n",
    "           events=events, bottom=.2, top=.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "    \n",
    "#### Coding Question 4\n",
    "\n",
    "Fill in the \"BITE_ME\" below to generate a graph for \"ATTACK_ON_COMMENTER with 95% confidence interval\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(newbies_mean, newbies_error, column='BITE_ME', \n",
    "           cohorts=['old', 'new'], \n",
    "           events=events, bottom=.2, top=.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "    \n",
    "#### Coding Question 5\n",
    "\n",
    "Fill in the \"BITE_ME\" below to generate a graph for \"INFLAMMATORY with 95% confidence interval\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(newbies_mean, newbies_error, column='BITE_ME', \n",
    "           cohorts=['old', 'new'], \n",
    "           events=events, bottom=.2, top=.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 Coming from Hostile Communities\n",
    "Is it about what other communities users belong to?\n",
    "- We saw above that posts from people who just joined aren't really different, on average, from posts by more established users. In the next section, we'll look at whether it matters what other communities a users belong to.\n",
    "    - Much of the debate was about *who* was joining and *why*, not when.\n",
    "- In the next part, we divide users into two groups:\n",
    "    - Users who post in *both* TwoX and anti-feminist (e.g. MRA) communities. \n",
    "    - Users who post in TwoX, but not in anti-feminist communities. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "    \n",
    "#### Short Answer 4:\n",
    "- Which of these two group do you expect to be more toxic / inflammatory? Write a few sentences explaining why."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🤔 **Write your response here:**\n",
    "...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- This next cell loads data about the users and splits them into groups.\n",
    "- Run the helper code below and scroll down."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Loading list of MRA suthors...')\n",
    "mra_authors = set(pd.read_csv('data/mra_authors.tsv.gz', sep='\\t').author.values)\n",
    "\n",
    "def get_mra(user):\n",
    "    c = 'only TwoX'\n",
    "    if user == '[deleted]':\n",
    "        c = np.nan\n",
    "    elif user in mra_authors:\n",
    "        c = 'TwoX and MRA'\n",
    "    return c\n",
    "\n",
    "print('Matching with TwoX authors...')\n",
    "twox_comments['MRA'] = twox_comments.author.apply(get_mra).astype('category')\n",
    "del mra_authors\n",
    "\n",
    "print('Calculating statistics...')\n",
    "mra = twox_comments.groupby(by='MRA').resample('M', on='date')\n",
    "mra_mean = mra.mean(numeric_only=True)\n",
    "mra_error = mra.sem(numeric_only=True) * 1.96"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### How many posts come from each group?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users(mra.count(), \n",
    "           cohorts=['only TwoX', 'TwoX and MRA'], \n",
    "           events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### How many users come from each group?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users(mra.apply(lambda x: len(x[\"author\"].unique())).reset_index().melt(id_vars=\"MRA\").set_index([\"MRA\",\"date\"]),\n",
    "           column=\"value\",\n",
    "           title=\"Number of users per month\",\n",
    "           cohorts=['only TwoX', 'TwoX and MRA'], \n",
    "           events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### How does the behavior of each group compare?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(mra_mean, mra_error, column='TOXICITY', \n",
    "           cohorts=['only TwoX', 'TwoX and MRA'], \n",
    "           events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "    \n",
    "#### Coding Question 6\n",
    "\n",
    "Fill in the \"BITE_ME\" below to generate a graph for \"ATTACK_ON_COMMENTER with 95% confidence interval\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(mra_mean, mra_error, column='BITE_ME', \n",
    "           cohorts=['only TwoX', 'TwoX and MRA'], \n",
    "           events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "    \n",
    "#### Coding Question 7\n",
    "\n",
    "Fill in the \"BITE_ME below to generate a graph for \"sentiment with 95% confidence interval\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(mra_mean, mra_error, column='BITE_ME', \n",
    "           cohorts=['only TwoX', 'TwoX and MRA'], \n",
    "           events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert-info\">\n",
    "    \n",
    "#### Coding Question 8\n",
    "\n",
    "Fill in the \"BITE_ME\" below to generate a graph for \"INFLAMMATORY with 95% confidence interval\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_users_mean(mra_mean, mra_error, column='BITE_ME', \n",
    "           cohorts=['only TwoX', 'TwoX and MRA'], \n",
    "           events=events)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<div class=\"alert-info\">\n",
    "\n",
    "#### Short Answer 5:\n",
    "- Write a few sentences answering each of these questions:\n",
    "    1. What do these comparisons tell us?\n",
    "    2. Overall, what happened on default day?\n",
    "    3. How are users who joined TwoX after default day different from those who joined before?\n",
    "    4. How did users who joined before default day change when it was made a default subreddit? \n",
    "        - In what ways did they stay the same?\n",
    "    5. How are users who post only in the feminist community TwoX different from those who also post in anti-feminist (MRA) communities? \n",
    "        - Why might that be? \n",
    "    6. How do you think the users who post only in the feminist community TwoX may have responded to the influx of those who post in anti-feminist (MRA) reddits and their behavior?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🤔 **Write your response here:**\n",
    "...."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}