PyPI/PyPI - Get_number_of_downloads_any_package.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "72a216be-10dc-4c1d-a3fa-a9ce992d1c27",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "<img width=\"10%\" alt=\"Naas\" src=\"https://landen.imgix.net/jtci2pxwjczr/assets/5ice39g4.png?w=160\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25f38b3f-5cb5-4700-ba73-fb24646dd3ca",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "# PyPI - - Get number of downloads any package\n",
    "<a href=\"https://app.naas.ai/user-redirect/naas/downloader?url=https://raw.githubusercontent.com/jupyter-naas/awesome-notebooks/master/PyPI/PyPI%20-%20Get_number_of_downloads_any_package.ipynb\" target=\"_parent\"><img src=\"https://naasai-public.s3.eu-west-3.amazonaws.com/open_in_naas.svg\"/></a><br><br><a href=\"https://github.com/jupyter-naas/awesome-notebooks/issues/new?assignees=&labels=&template=template-request.md&title=Tool+-+Action+of+the+notebook+\">💡 Template request</a> | <a href=\"https://github.com/jupyter-naas/awesome-notebooks/issues/new?assignees=&labels=bug&template=bug_report.md&title=PyPI+-+-+Get+number+of+downloads+any+package:+Error+short+description\">🚨 Bug report</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4da75a0d-6fba-4115-8667-429956d5ffa0",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Tags:** #pypi #downloads #package #operations #analytics #plotly #html #csv #image #png"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab9c2b57-cd13-4086-8a8c-676006cdaebf",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Author:** [Sanjeet Attili](https://linkedin.com/in/sanjeet-attili-760bab190/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41c21843-7073-4538-9a8f-539ee90d86bb",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "This notebook enables you to get a plot of number of downloads of any package since past 180 days"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "922c974a-a4f5-41b1-af0e-83db3798dee4",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b1b919a-fe68-41c6-8401-1c0395149f33",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    import pypistats\n",
    "except:\n",
    "    !pip install -U pypistats --user\n",
    "    !pip install --upgrade pypistats\n",
    "    import pypistats\n",
    "# from pprint import pprint\n",
    "from datetime import datetime\n",
    "import plotly.graph_objects as go\n",
    "import naas"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c3161ff-a300-4b0a-a7ab-d63cd762fead",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c16f635-fbb6-4921-9777-4a4013dcfa37",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Inputs\n",
    "package = \"pandas\"\n",
    "\n",
    "# Outputs\n",
    "name_output = f\"{package}_downloads\"\n",
    "csv_output = f\"{name_output}.csv\"\n",
    "html_output = f\"{name_output}.html\"\n",
    "image_output = f\"{name_output}.png\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36f9d891-dd69-49e0-9288-ba6cf1566198",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f380f3f-7731-4b63-bad8-a042d0ca9709",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Difference between with_mirrors and without_mirrors**\n",
    "\n",
    "The with_mirrors and without_mirrors are not mutually exclusive sets of download counts like the other segmentations provided.\n",
    "In fact, the without_mirrors downloads are a subset of the downloads in with_mirrors.\n",
    "\n",
    "i.e If you sum the with (a+b) and without (a) mirrors, you count the without mirrors twice (a+b+a)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83227828-b6ca-4ebe-8ace-4344975b695c",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "df = pypistats.overall(package, total=False, format=\"pandas\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4da3d345-bb29-4b44-a1b1-8ebc840255d7",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Gives us the cumulative number of downloads over a period of 180 days\n",
    "def get_cumulative_dataframe(df):\n",
    "    \n",
    "    data = df.groupby('category').get_group('with_mirrors').sort_values(\n",
    "        'date').reset_index(drop='index').groupby(\n",
    "        'date').agg({'downloads':'sum'}).reset_index()\n",
    "    \n",
    "    cum_sum = 0\n",
    "    for idx, num in enumerate(data['downloads']):\n",
    "        cum_sum+=num\n",
    "        data.loc[idx, 'cumulative_downloads'] = cum_sum\n",
    "\n",
    "    data['cumulative_downloads'] = data.cumulative_downloads.astype('int')\n",
    "    data.drop(columns = 'downloads', inplace=True)\n",
    "    \n",
    "    return data\n",
    "\n",
    "df_downloads = get_cumulative_dataframe(df)\n",
    "df_downloads.tail(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7077755-36d0-4e95-a581-86c5764cc383",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0b568d5-3904-4dbd-b1e8-476ad4ae9a61",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Plotting a line chart for number of downloads"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70713839-101a-4d57-af31-3340bc13b857",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "def create_linechart(df, package, date, value):\n",
    "    # Get last value\n",
    "    last_value = \"{:,.0f}\".format(df.loc[df.index[-1], value]).replace(\",\", \" \")\n",
    "    \n",
    "    # Init\n",
    "    fig = go.Figure()\n",
    "    \n",
    "    # Create fig\n",
    "    fig.add_trace(\n",
    "        go.Scatter(\n",
    "            x=df[date].to_list(),\n",
    "            y=df[value].to_list(),\n",
    "            mode=\"lines+text\",\n",
    "            line=dict(color=\"black\"),\n",
    "        )\n",
    "    )\n",
    "    fig.update_traces(marker_color='black')\n",
    "    fig.update_layout(\n",
    "        title=f\"⭐<b> Number of downloads for {package} </b><br><span style='font-size: 13px;'> Total Downloads as of today: {last_value}</span>\",\n",
    "        title_font=dict(family=\"Arial\", size=18, color=\"black\"),\n",
    "        plot_bgcolor=\"#ffffff\",\n",
    "        width=1200,\n",
    "        height=800,\n",
    "        paper_bgcolor=\"white\",\n",
    "        xaxis_title=\"Date\",\n",
    "        xaxis_title_font=dict(family=\"Arial\", size=13, color=\"black\"),\n",
    "        yaxis_title='No. of downloads',\n",
    "        yaxis_title_font=dict(family=\"Arial\", size=13, color=\"black\"),\n",
    "        margin_pad=10,\n",
    "    )\n",
    "    fig.show()\n",
    "    return fig\n",
    "\n",
    "fig = create_linechart(df_downloads, package, \"date\", \"cumulative_downloads\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0396b1b-00d3-45c0-af1d-af07daa6f07c",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Save and share your csv file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52b4015d-f6ab-4bb8-a0a5-f7ac5d824bbc",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Save your dataframe in CSV\n",
    "df_downloads.to_csv(csv_output, index=False)\n",
    "\n",
    "# Share output with naas\n",
    "naas.asset.add(csv_output)\n",
    "\n",
    "#-> Uncomment the line below to remove your asset\n",
    "# naas.asset.delete(csv_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2992dcb0-23e3-419d-b6e9-ee626ce86bf5",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Save and share your graph in HTML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a57d1784-1c1b-47d8-ad3a-bdc0ddfc9eb3",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Save your graph in HTML\n",
    "fig.write_html(html_output)\n",
    "\n",
    "# Share output with naas\n",
    "naas.asset.add(html_output, params={\"inline\": True})\n",
    "\n",
    "#-> Uncomment the line below to remove your asset\n",
    "# naas.asset.delete(html_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef4ee648-1da0-4b15-9bd3-86a2e0c4ab0d",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Save and share your graph in image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9042998c-348e-451c-b9c7-74bb157abe3a",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Save your graph in PNG\n",
    "fig.write_image(image_output)\n",
    "\n",
    "# Share output with naas\n",
    "naas.asset.add(image_output, params={\"inline\": True})\n",
    "\n",
    "#-> Uncomment the line below to remove your asset\n",
    "# naas.asset.delete(image_output)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "papermill": {
   "default_parameters": {},
   "environment_variables": {},
   "parameters": {},
   "version": "2.3.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}