sxs_catalog_download_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# sxs_catalog_download_example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates how to use the `sxs` python library to download data from the SXS Catalog of waveforms hosted on Zenodo (https://github.com/moble/sxs). The catalog is available at https://black-holes.org/waveforms and is described in https://arxiv.org/abs/1904.04831.\n",
    "\n",
    "You can install `sxs` via `pip install sxs`. You'll need version `2019.4.15.16.32.10` or later.\n",
    "\n",
    "Note: You should use a recent version of python 3, such as 3.6.5, to run this notebook. I installed python 3.6.5 using anaconda (https://www.anaconda.com). \n",
    "\n",
    "Note: I found I had to also install tqdm, which you can install similarly. This should be installed when you install the `sxs` library with `pip`, but in case you see errors about being unable to import `tqdm`, `pip install tqdm` solves this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to download data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section demonstrates how to download simulation data from Zenodo. You can download data from one simulation or from multiple simulations at once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For downloading data\n",
    "import sxs\n",
    "from sxs import zenodo as zen\n",
    "\n",
    "# For interacting with the data\n",
    "import h5py\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This line attempts to download select files from a specific simulation. \n",
    "\n",
    "`dry_run = True` means that the function does everything except actually download the data; set this to false to download the data into the same directory as this notebook.\n",
    "\n",
    "You can download dat for multiple simulations by changing `sxs_ids` to include more simulations, or to include an expression that matches multiple simulations. For instance, `sxs_ids = ['SXS:BBH:']` would download all binary-black-hole waveforms from the catalog.\n",
    "\n",
    "Set `highest_lev_only = True` to download only the highest resoution of each file that is available at multiple resolutions, instead of downloading files from all resolutions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zen.download.matching(\"common-metadata.txt\", \"metadata.txt\", \\\n",
    "                      \"rhOverM_Asymptotic_GeometricUnits_CoM.h5\", \\\n",
    "                      \"Horizons.h5\", \\\n",
    "                      sxs_ids = ['SXS:BBH:0444'], \\\n",
    "                      dry_run = True, \\\n",
    "                      highest_lev_only = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the help text for a function that downloads data from the catalog, for more details on how the function works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "?zen.download.matching"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Examples of using data from the catalog"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download the highest resolution Horizons and waveform files for a specific simulation. This will download the data in the same notebook as this notebook, overwriting other files of the same name if they exist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zen.download.matching(\"common-metadata.txt\", \"metadata.txt\", \"metadata.json\", \\\n",
    "                      \"rhOverM_Asymptotic_GeometricUnits_CoM.h5\", \\\n",
    "                      \"Horizons.h5\", \\\n",
    "                      sxs_ids = ['SXS:BBH:0444'], \\\n",
    "                      dry_run = False, \\\n",
    "                      highest_lev_only = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "horizons = h5py.File(\"SXS_BBH_0444/Horizons.h5\", 'r')\n",
    "rhOverM = h5py.File(\"SXS_BBH_0444/rhOverM_Asymptotic_GeometricUnits_CoM.h5\", 'r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keys of the waveform file specify the extrapolation order or a text file giving version history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list(rhOverM.keys()))\n",
    "print(np.array(rhOverM['VersionHist.ver']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each extrapolation order has some metadata given as attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list(rhOverM['Extrapolated_N2.dir'].attrs.keys()))\n",
    "for key in rhOverM['Extrapolated_N2.dir'].attrs.keys():\n",
    "    print(key + \" = \" + str(rhOverM['Extrapolated_N2.dir'].attrs[key]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an example that plots waveform data from the downloaded H5 files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "timeReIm = np.array(rhOverM['Extrapolated_N2.dir']['Y_l2_m2.dat'])\n",
    "plt.clf()\n",
    "plt.plot(timeReIm[:,0], timeReIm[:,1])\n",
    "plt.xlabel(\"Time (code units)\")\n",
    "plt.ylabel(\"Strain\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a similar example plotting the irreducible mass vs. time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(horizons.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(horizons['AhA.dir'].keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.clf()\n",
    "plt.plot(horizons['AhA.dir']['ArealMass.dat'][:,0], horizons['AhA.dir']['ArealMass.dat'][:,1], label='AhA')\n",
    "plt.plot(horizons['AhB.dir']['ArealMass.dat'][:,0], horizons['AhB.dir']['ArealMass.dat'][:,1], label='AhB')\n",
    "plt.plot(horizons['AhC.dir']['ArealMass.dat'][:,0], horizons['AhC.dir']['ArealMass.dat'][:,1], label='AhC')\n",
    "plt.xlabel(\"Time (code units)\")\n",
    "plt.ylabel(\"Irreducible mass (code units)\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example of using metadata in the catalog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"SXS_BBH_0444/metadata.json\") as file:\n",
    "    metadata_json = json.load(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is code to parse the metadata json file `metadata.json`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(metadata_json.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata_json['remnant_mass']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here is an example getting the same metadata from `metadata.txt`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"SXS_BBH_0444/metadata.json\") as file:\n",
    "    metadata_txt_raw = file.readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Simple code to kind of parse the text file. Room for improvement.\n",
    "metadata_txt = {}\n",
    "for line in metadata_txt_raw:\n",
    "    if ':' in line:\n",
    "        key = line.split('\":')[0].strip().split('\"')[1]\n",
    "        value = line.split('\":')[-1].split(',')[0].strip()\n",
    "        metadata_txt[key] = value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "float(metadata_txt['remnant_mass'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}