Sentinel_2_RGB.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "338d2487",
   "metadata": {},
   "source": [
    "## MOSAIKS feature extraction\n",
    "\n",
    "This tutorial demonstrates the **MOSAIKS** method for extracting _feature vectors_ from satellite imagery patches for use in downstream modeling tasks. It will show:\n",
    "- How to extract 1km$^2$ patches of Sentinel 2 multispectral imagery for a list of latitude, longitude points\n",
    "- How to extract summary features from each of these imagery patches\n",
    "- How to use the summary features in a linear model of the population density at each point\n",
    "\n",
    "### Background\n",
    "\n",
    "Consider the case where you have a dataset of latitude and longitude points assosciated with some dependent variable (for example: population density, weather, housing prices, biodiversity) and, potentially, other independent variables. You would like to model the dependent variable as a function of the independent variables, but instead of including latitude and longitude directly in this model, you would like to include some high dimensional representation of what the Earth looks like at that point (that hopefully explains some of the variance in the dependent variable!). From the computer vision literature, there are various [representation learning techniques](https://en.wikipedia.org/wiki/Feature_learning) that can be used to do this, i.e. extract _features vectors_ from imagery. This notebook gives an implementation of the technique described in [Rolf et al. 2021](https://www.nature.com/articles/s41467-021-24638-z), \"A generalizable and accessible approach to machine learning with global satellite imagery\" called Multi-task Observation using Satellite Imagery & Kitchen Sinks (**MOSAIKS**). For more information about **MOSAIKS** see the [project's webpage](http://www.globalpolicy.science/mosaiks).\n",
    "\n",
    "\n",
    "**Notes**:\n",
    "- This example uses [Sentinel-2 Level-2A data](https://planetarycomputer.microsoft.com/dataset/sentinel-2-l2a). The techniques used here apply equally well to other remote-sensing datasets.\n",
    "- If you're running this on the [Planetary Computer Hub](http://planetarycomputer.microsoft.com/compute), make sure to choose the **GPU - PyTorch** profile when presented with the form to choose your environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e33320a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -q git+https://github.com/geopandas/dask-geopandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8e98f0ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "import time\n",
    "import os\n",
    "import gc\n",
    "import calendar\n",
    "from IPython.display import FileLink\n",
    "\n",
    "RASTERIO_BEST_PRACTICES = dict(  # See https://github.com/pangeo-data/cog-best-practices\n",
    "    CURL_CA_BUNDLE=\"/etc/ssl/certs/ca-certificates.crt\",\n",
    "    GDAL_DISABLE_READDIR_ON_OPEN=\"EMPTY_DIR\",\n",
    "    AWS_NO_SIGN_REQUEST=\"YES\",\n",
    "    GDAL_MAX_RAW_BLOCK_CACHE_SIZE=\"200000000\",\n",
    "    GDAL_SWATH_SIZE=\"200000000\",\n",
    "    VSI_CURL_CACHE_SIZE=\"200000000\",\n",
    ")\n",
    "os.environ.update(RASTERIO_BEST_PRACTICES)\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "\n",
    "import contextily as ctx\n",
    "import rasterio\n",
    "import rasterio.warp\n",
    "import rasterio.mask\n",
    "import shapely.geometry\n",
    "import geopandas\n",
    "import dask_geopandas\n",
    "from dask.distributed import Client\n",
    "\n",
    "warnings.filterwarnings(action=\"ignore\", category=UserWarning, module=\"torch\")\n",
    "\n",
    "import pystac_client\n",
    "import planetary_computer as pc"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8014cb18",
   "metadata": {},
   "source": [
    "## Set Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a7c8cf08",
   "metadata": {},
   "outputs": [],
   "source": [
    "num_features = 1000\n",
    "satellite = \"sentinel-2-l2a\"\n",
    "country_code = 'ZMB'\n",
    "use_file = True\n",
    "# use_file = False"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "928608cd",
   "metadata": {},
   "source": [
    "## Create grid and sample points to featurize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "aef5d183",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(19598, 4)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "if use_file:\n",
    "    gdf = pd.read_feather('data/keep/ZMB_crop_weights_20k-points.feather')\n",
    "    gdf = (\n",
    "        geopandas\n",
    "        .GeoDataFrame(\n",
    "            gdf, \n",
    "            geometry = geopandas.points_from_xy(x = gdf.lon, y = gdf.lat), \n",
    "            crs='EPSG:4326')\n",
    "    )\n",
    "else:\n",
    "    cell_size = 0.01  # Roughly 1 km\n",
    "    ### get country shape\n",
    "    country_file_name = f\"{data_dir}/boundaries/africa_adm0.geojson\"\n",
    "    africa = geopandas.read_file(country_file_name)\n",
    "    country = africa[africa.adm0_a3 == country_code]\n",
    "    #### This would be simpler, but throws an error down the line if used \n",
    "    # world = geopandas.read_file(geopandas.datasets.get_path('naturalearth_lowres'))\n",
    "    # country = world.query(f'iso_a3 == \"{country_code}\"')\n",
    "    ### Create grid of points\n",
    "    cell_size = .01  # Very roughly 1 km\n",
    "    xmin, ymin, xmax, ymax = country.total_bounds\n",
    "    xs = list(np.arange(xmin, xmax + cell_size, cell_size))\n",
    "    ys = list(np.arange(ymin, ymax + cell_size, cell_size))\n",
    "    def make_cell(x, y, cell_size):\n",
    "        ring = [\n",
    "            (x, y),\n",
    "            (x + cell_size, y),\n",
    "            (x + cell_size, y + cell_size),\n",
    "            (x, y + cell_size)\n",
    "        ]\n",
    "        cell = shapely.geometry.Polygon(ring).centroid\n",
    "        return cell\n",
    "    center_points = []\n",
    "    for x in xs:\n",
    "        for y in ys:\n",
    "            cell = make_cell(x, y, cell_size)\n",
    "            center_points.append(cell)\n",
    "    ### Put grid into a GeDataFrame for cropping to country shape\n",
    "    gdf = geopandas.GeoDataFrame({'geometry': center_points}, crs = 'EPSG:4326')\n",
    "    gdf['lon'], gdf['lat'] = gdf.geometry.x, gdf.geometry.y\n",
    "    ### Subset to country \n",
    "    ### This buffer ensures that no points are take at the border \n",
    "    ### which would lead to duplication with neighboring countries\n",
    "    gdf = gdf[gdf.within(country.unary_union.buffer(-0.005))]\n",
    "    gdf = gdf[['lon', 'lat', 'geometry']].reset_index(drop = True)\n",
    "    gdf = gdf.sample(frac = 0.1, random_state=42, ignore_index=False)\n",
    "    points = gdf[[\"lon\", \"lat\"]].to_numpy()\n",
    "pt_len = gdf.shape[0]\n",
    "gdf.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f40e13b0",
   "metadata": {},
   "source": [
    "First we define the pytorch model that we will use to extract the features and a helper method. The **MOSAIKS** methodology describes several ways to do this and we use the simplest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e85ddac5",
   "metadata": {},
   "outputs": [],
   "source": [
    "class RCF(nn.Module):\n",
    "    \"\"\"A model for extracting Random Convolution Features (RCF) from input imagery.\"\"\"\n",
    "    def __init__(self, num_features=16, kernel_size=3, num_input_channels=3):\n",
    "        super(RCF, self).__init__()\n",
    "        # We create `num_features / 2` filters so require `num_features` to be divisible by 2\n",
    "        assert num_features % 2 == 0, \"Please enter an even number of features.\"\n",
    "        # Applies a 2D convolution over an input image composed of several input planes.\n",
    "        self.conv1 = nn.Conv2d(\n",
    "            num_input_channels,\n",
    "            num_features // 2,\n",
    "            kernel_size=kernel_size,\n",
    "            stride=1,\n",
    "            padding=0,\n",
    "            dilation=1,\n",
    "            bias=True,\n",
    "        )\n",
    "        # Fills the input Tensor 'conv1.weight' with values drawn from the normal distribution\n",
    "        nn.init.normal_(self.conv1.weight, mean=0.0, std=1.0) \n",
    "        # Fills the input Tensor 'conv1.bias' with the value 'val = -1'.\n",
    "        nn.init.constant_(self.conv1.bias, -1.0)\n",
    "    def forward(self, x):\n",
    "        # The rectified linear activation function or ReLU for short is a piecewise linear function \n",
    "        # that will output the input directly if it is positive, otherwise, it will output zero.\n",
    "        x1a = F.relu(self.conv1(x), inplace=True)\n",
    "        # The below step is where we take the inverse which is appended later\n",
    "        x1b = F.relu(-self.conv1(x), inplace=True)\n",
    "        # Applies a 2D adaptive average pooling over an input signal composed of several input planes.\n",
    "        x1a = F.adaptive_avg_pool2d(x1a, (1, 1)).squeeze()\n",
    "        x1b = F.adaptive_avg_pool2d(x1b, (1, 1)).squeeze()\n",
    "        if len(x1a.shape) == 1:  # case where we passed a single input\n",
    "            return torch.cat((x1a, x1b), dim=0)\n",
    "        elif len(x1a.shape) == 2:  # case where we passed a batch of > 1 inputs\n",
    "            return torch.cat((x1a, x1b), dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f0600b1",
   "metadata": {},
   "source": [
    "Next, we initialize the model and pytorch components"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "eece8ad5",
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device(\"cuda\")\n",
    "model = RCF(num_features).eval().to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cd81227",
   "metadata": {},
   "source": [
    "### Extract features from the imagery around each point\n",
    "\n",
    "We need to find a suitable Sentinel 2 scene for each point. As usual, we'll use `pystac-client` to search for items matching some conditions, but we don't just want do make a `.search()` call for each of the 67,968 remaining points. Each HTTP request is relatively slow. Instead, we will *batch* or points and search *in parallel*.\n",
    "\n",
    "We need to be a bit careful with how we batch up our points though. Since a single Sentinel 2 scene will cover many points, we want to make sure that points which are spatially close together end up in the same batch. In short, we need to spatially partition the dataset. This is implemented in `dask-geopandas`.\n",
    "\n",
    "So the overall workflow will be\n",
    "\n",
    "1. Find an appropriate STAC item for each point (in parallel, using the spatially partitioned dataset)\n",
    "2. Feed the points and STAC items to a custom Dataset that can read imagery given a point and the URL of a overlapping S2 scene\n",
    "3. Use a custom Dataloader, which uses our Dataset, to feed our model imagery and save the corresponding features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d8fdc9d9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NPARTITIONS = 250\n",
    "\n",
    "ddf = dask_geopandas.from_geopandas(gdf, npartitions=1)\n",
    "hd = ddf.hilbert_distance().compute()\n",
    "gdf[\"hd\"] = hd\n",
    "gdf = gdf.sort_values(\"hd\")\n",
    "\n",
    "dgdf = dask_geopandas.from_geopandas(gdf, npartitions=NPARTITIONS, sort=False)\n",
    "\n",
    "del ddf\n",
    "del hd\n",
    "del gdf\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9028e818",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using:\n",
      "   sentinel-2-l2a imagery \n",
      "   Looking for monthly images for 19598 points \n",
      "   Calulating 1000 features \n",
      "   between 2016 and 2021 \n",
      "\n",
      "Matching images to points for: 1-2016\n",
      "Found acceptable images for 13181 points in 18.33 seconds\n",
      "Featurizing: 01-2016\n",
      "0/13181 -- 0.00% -- 0.45 seconds\n",
      "1000/13181 -- 7.59% -- 4.63 seconds\n",
      "2000/13181 -- 15.17% -- 5.76 seconds\n",
      "3000/13181 -- 22.76% -- 6.21 seconds\n",
      "4000/13181 -- 30.35% -- 6.43 seconds\n",
      "5000/13181 -- 37.93% -- 6.18 seconds\n",
      "6000/13181 -- 45.52% -- 7.47 seconds\n",
      "7000/13181 -- 53.11% -- 4.98 seconds\n",
      "8000/13181 -- 60.69% -- 7.80 seconds\n",
      "9000/13181 -- 68.28% -- 8.49 seconds\n",
      "10000/13181 -- 75.87% -- 5.87 seconds\n",
      "11000/13181 -- 83.45% -- 6.49 seconds\n",
      "12000/13181 -- 91.04% -- 6.89 seconds\n",
      "13000/13181 -- 98.63% -- 5.01 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 2-2016\n",
      "Found acceptable images for 2631 points in 17.45 seconds\n",
      "Featurizing: 02-2016\n",
      "0/2631 -- 0.00% -- 0.73 seconds\n",
      "1000/2631 -- 38.01% -- 4.41 seconds\n",
      "2000/2631 -- 76.02% -- 3.65 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 3-2016\n",
      "Found acceptable images for 5700 points in 17.58 seconds\n",
      "Featurizing: 03-2016\n",
      "0/5700 -- 0.00% -- 0.56 seconds\n",
      "1000/5700 -- 17.54% -- 3.81 seconds\n",
      "2000/5700 -- 35.09% -- 3.54 seconds\n",
      "3000/5700 -- 52.63% -- 3.80 seconds\n",
      "4000/5700 -- 70.18% -- 5.04 seconds\n",
      "5000/5700 -- 87.72% -- 2.94 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 4-2016\n",
      "Found acceptable images for 16413 points in 18.91 seconds\n",
      "Featurizing: 04-2016\n",
      "0/16413 -- 0.00% -- 0.73 seconds\n",
      "1000/16413 -- 6.09% -- 3.60 seconds\n",
      "2000/16413 -- 12.19% -- 4.51 seconds\n",
      "3000/16413 -- 18.28% -- 3.91 seconds\n",
      "4000/16413 -- 24.37% -- 3.53 seconds\n",
      "5000/16413 -- 30.46% -- 2.86 seconds\n",
      "6000/16413 -- 36.56% -- 4.49 seconds\n",
      "7000/16413 -- 42.65% -- 4.07 seconds\n",
      "8000/16413 -- 48.74% -- 3.55 seconds\n",
      "9000/16413 -- 54.83% -- 4.09 seconds\n",
      "10000/16413 -- 60.93% -- 3.72 seconds\n",
      "11000/16413 -- 67.02% -- 2.97 seconds\n",
      "12000/16413 -- 73.11% -- 3.42 seconds\n",
      "13000/16413 -- 79.21% -- 3.91 seconds\n",
      "14000/16413 -- 85.30% -- 3.49 seconds\n",
      "15000/16413 -- 91.39% -- 3.99 seconds\n",
      "16000/16413 -- 97.48% -- 3.01 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 5-2016\n",
      "Found acceptable images for 19420 points in 19.50 seconds\n",
      "Featurizing: 05-2016\n",
      "0/19420 -- 0.00% -- 0.62 seconds\n",
      "1000/19420 -- 5.15% -- 3.65 seconds\n",
      "2000/19420 -- 10.30% -- 4.18 seconds\n",
      "3000/19420 -- 15.45% -- 4.14 seconds\n",
      "4000/19420 -- 20.60% -- 3.77 seconds\n",
      "5000/19420 -- 25.75% -- 3.93 seconds\n",
      "6000/19420 -- 30.90% -- 4.24 seconds\n",
      "7000/19420 -- 36.05% -- 4.30 seconds\n",
      "8000/19420 -- 41.19% -- 3.65 seconds\n",
      "9000/19420 -- 46.34% -- 3.46 seconds\n",
      "10000/19420 -- 51.49% -- 3.78 seconds\n",
      "11000/19420 -- 56.64% -- 3.59 seconds\n",
      "12000/19420 -- 61.79% -- 4.10 seconds\n",
      "13000/19420 -- 66.94% -- 3.60 seconds\n",
      "14000/19420 -- 72.09% -- 3.79 seconds\n",
      "15000/19420 -- 77.24% -- 4.32 seconds\n",
      "16000/19420 -- 82.39% -- 4.06 seconds\n",
      "17000/19420 -- 87.54% -- 3.01 seconds\n",
      "18000/19420 -- 92.69% -- 2.93 seconds\n",
      "19000/19420 -- 97.84% -- 3.59 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 6-2016\n",
      "Found acceptable images for 19544 points in 20.25 seconds\n",
      "Featurizing: 06-2016\n",
      "0/19544 -- 0.00% -- 0.65 seconds\n",
      "1000/19544 -- 5.12% -- 4.02 seconds\n",
      "2000/19544 -- 10.23% -- 4.03 seconds\n",
      "3000/19544 -- 15.35% -- 4.33 seconds\n",
      "4000/19544 -- 20.47% -- 3.36 seconds\n",
      "5000/19544 -- 25.58% -- 2.70 seconds\n",
      "6000/19544 -- 30.70% -- 4.34 seconds\n",
      "7000/19544 -- 35.82% -- 3.99 seconds\n",
      "8000/19544 -- 40.93% -- 3.61 seconds\n",
      "9000/19544 -- 46.05% -- 3.60 seconds\n",
      "10000/19544 -- 51.17% -- 4.19 seconds\n",
      "11000/19544 -- 56.28% -- 3.38 seconds\n",
      "12000/19544 -- 61.40% -- 4.49 seconds\n",
      "13000/19544 -- 66.52% -- 4.21 seconds\n",
      "14000/19544 -- 71.63% -- 4.16 seconds\n",
      "15000/19544 -- 76.75% -- 4.57 seconds\n",
      "16000/19544 -- 81.87% -- 3.58 seconds\n",
      "17000/19544 -- 86.98% -- 3.19 seconds\n",
      "18000/19544 -- 92.10% -- 3.18 seconds\n",
      "19000/19544 -- 97.22% -- 2.51 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 7-2016\n",
      "Found acceptable images for 19598 points in 19.80 seconds\n",
      "Featurizing: 07-2016\n",
      "0/19598 -- 0.00% -- 0.65 seconds\n",
      "1000/19598 -- 5.10% -- 3.65 seconds\n",
      "2000/19598 -- 10.21% -- 4.30 seconds\n",
      "3000/19598 -- 15.31% -- 4.37 seconds\n",
      "4000/19598 -- 20.41% -- 3.93 seconds\n",
      "5000/19598 -- 25.51% -- 3.87 seconds\n",
      "6000/19598 -- 30.62% -- 4.67 seconds\n",
      "7000/19598 -- 35.72% -- 4.40 seconds\n",
      "8000/19598 -- 40.82% -- 4.09 seconds\n",
      "9000/19598 -- 45.92% -- 4.40 seconds\n",
      "10000/19598 -- 51.03% -- 5.06 seconds\n",
      "11000/19598 -- 56.13% -- 4.01 seconds\n",
      "12000/19598 -- 61.23% -- 4.32 seconds\n",
      "13000/19598 -- 66.33% -- 4.90 seconds\n",
      "14000/19598 -- 71.44% -- 3.95 seconds\n",
      "15000/19598 -- 76.54% -- 5.66 seconds\n",
      "16000/19598 -- 81.64% -- 4.32 seconds\n",
      "17000/19598 -- 86.74% -- 3.95 seconds\n",
      "18000/19598 -- 91.85% -- 2.94 seconds\n",
      "19000/19598 -- 96.95% -- 2.64 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 8-2016\n",
      "Found acceptable images for 19466 points in 19.49 seconds\n",
      "Featurizing: 08-2016\n",
      "0/19466 -- 0.00% -- 0.67 seconds\n",
      "1000/19466 -- 5.14% -- 4.66 seconds\n",
      "2000/19466 -- 10.27% -- 4.67 seconds\n",
      "3000/19466 -- 15.41% -- 4.68 seconds\n",
      "4000/19466 -- 20.55% -- 4.78 seconds\n",
      "5000/19466 -- 25.69% -- 4.18 seconds\n",
      "6000/19466 -- 30.82% -- 4.51 seconds\n",
      "7000/19466 -- 35.96% -- 4.65 seconds\n",
      "8000/19466 -- 41.10% -- 4.32 seconds\n",
      "9000/19466 -- 46.23% -- 5.38 seconds\n",
      "10000/19466 -- 51.37% -- 5.14 seconds\n",
      "11000/19466 -- 56.51% -- 6.32 seconds\n",
      "12000/19466 -- 61.65% -- 4.51 seconds\n",
      "13000/19466 -- 66.78% -- 4.87 seconds\n",
      "14000/19466 -- 71.92% -- 3.75 seconds\n",
      "15000/19466 -- 77.06% -- 5.30 seconds\n",
      "16000/19466 -- 82.19% -- 4.03 seconds\n",
      "17000/19466 -- 87.33% -- 4.04 seconds\n",
      "18000/19466 -- 92.47% -- 4.27 seconds\n",
      "19000/19466 -- 97.61% -- 4.43 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 9-2016\n",
      "Found acceptable images for 19598 points in 19.51 seconds\n",
      "Featurizing: 09-2016\n",
      "0/19598 -- 0.00% -- 0.67 seconds\n",
      "1000/19598 -- 5.10% -- 4.45 seconds\n",
      "2000/19598 -- 10.21% -- 4.34 seconds\n",
      "3000/19598 -- 15.31% -- 4.21 seconds\n",
      "4000/19598 -- 20.41% -- 4.26 seconds\n",
      "5000/19598 -- 25.51% -- 3.79 seconds\n",
      "6000/19598 -- 30.62% -- 4.86 seconds\n",
      "7000/19598 -- 35.72% -- 4.76 seconds\n",
      "8000/19598 -- 40.82% -- 4.23 seconds\n",
      "9000/19598 -- 45.92% -- 5.18 seconds\n",
      "10000/19598 -- 51.03% -- 5.78 seconds\n",
      "11000/19598 -- 56.13% -- 4.28 seconds\n",
      "12000/19598 -- 61.23% -- 4.04 seconds\n",
      "13000/19598 -- 66.33% -- 5.89 seconds\n",
      "14000/19598 -- 71.44% -- 3.87 seconds\n",
      "15000/19598 -- 76.54% -- 4.45 seconds\n",
      "16000/19598 -- 81.64% -- 3.85 seconds\n",
      "17000/19598 -- 86.74% -- 4.73 seconds\n",
      "18000/19598 -- 91.85% -- 3.96 seconds\n",
      "19000/19598 -- 96.95% -- 4.27 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 10-2016\n",
      "Found acceptable images for 18908 points in 19.63 seconds\n",
      "Featurizing: 10-2016\n",
      "0/18908 -- 0.00% -- 0.73 seconds\n",
      "1000/18908 -- 5.29% -- 4.66 seconds\n",
      "2000/18908 -- 10.58% -- 4.56 seconds\n",
      "3000/18908 -- 15.87% -- 4.42 seconds\n",
      "4000/18908 -- 21.16% -- 4.23 seconds\n",
      "5000/18908 -- 26.44% -- 4.18 seconds\n",
      "6000/18908 -- 31.73% -- 4.89 seconds\n",
      "7000/18908 -- 37.02% -- 4.63 seconds\n",
      "8000/18908 -- 42.31% -- 4.22 seconds\n",
      "9000/18908 -- 47.60% -- 4.42 seconds\n",
      "10000/18908 -- 52.89% -- 6.79 seconds\n",
      "11000/18908 -- 58.18% -- 4.70 seconds\n",
      "12000/18908 -- 63.47% -- 5.28 seconds\n",
      "13000/18908 -- 68.75% -- 4.17 seconds\n",
      "14000/18908 -- 74.04% -- 4.64 seconds\n",
      "15000/18908 -- 79.33% -- 4.32 seconds\n",
      "16000/18908 -- 84.62% -- 4.04 seconds\n",
      "17000/18908 -- 89.91% -- 4.38 seconds\n",
      "18000/18908 -- 95.20% -- 4.62 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 11-2016\n",
      "Found acceptable images for 17476 points in 19.15 seconds\n",
      "Featurizing: 11-2016\n",
      "0/17476 -- 0.00% -- 0.64 seconds\n",
      "1000/17476 -- 5.72% -- 4.92 seconds\n",
      "2000/17476 -- 11.44% -- 4.71 seconds\n",
      "3000/17476 -- 17.17% -- 4.05 seconds\n",
      "4000/17476 -- 22.89% -- 4.40 seconds\n",
      "5000/17476 -- 28.61% -- 4.73 seconds\n",
      "6000/17476 -- 34.33% -- 4.23 seconds\n",
      "7000/17476 -- 40.05% -- 4.34 seconds\n",
      "8000/17476 -- 45.78% -- 4.28 seconds\n",
      "9000/17476 -- 51.50% -- 6.08 seconds\n",
      "10000/17476 -- 57.22% -- 4.13 seconds\n",
      "11000/17476 -- 62.94% -- 5.55 seconds\n",
      "12000/17476 -- 68.67% -- 3.94 seconds\n",
      "13000/17476 -- 74.39% -- 4.33 seconds\n",
      "14000/17476 -- 80.11% -- 4.21 seconds\n",
      "15000/17476 -- 85.83% -- 3.92 seconds\n",
      "16000/17476 -- 91.55% -- 4.42 seconds\n",
      "17000/17476 -- 97.28% -- 4.21 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 12-2016\n",
      "Found acceptable images for 6408 points in 17.95 seconds\n",
      "Featurizing: 12-2016\n",
      "0/6408 -- 0.00% -- 0.70 seconds\n",
      "1000/6408 -- 15.61% -- 5.07 seconds\n",
      "2000/6408 -- 31.21% -- 4.53 seconds\n",
      "3000/6408 -- 46.82% -- 4.16 seconds\n",
      "4000/6408 -- 62.42% -- 4.77 seconds\n",
      "5000/6408 -- 78.03% -- 3.82 seconds\n",
      "6000/6408 -- 93.63% -- 4.77 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Saving file as: data/sentinel-2-l2a_bands-3-4-5_ZMB_20k-points_1000-features_2016.feather\n",
      "Save finished!\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 1-2017\n",
      "Found acceptable images for 578 points in 17.34 seconds\n",
      "Featurizing: 01-2017\n",
      "0/578 -- 0.00% -- 0.73 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 2-2017\n",
      "Found acceptable images for 991 points in 17.31 seconds\n",
      "Featurizing: 02-2017\n",
      "0/991 -- 0.00% -- 0.63 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 3-2017\n",
      "Found acceptable images for 5196 points in 17.57 seconds\n",
      "Featurizing: 03-2017\n",
      "0/5196 -- 0.00% -- 0.62 seconds\n",
      "1000/5196 -- 19.25% -- 4.47 seconds\n",
      "2000/5196 -- 38.49% -- 4.22 seconds\n",
      "3000/5196 -- 57.74% -- 4.25 seconds\n",
      "4000/5196 -- 76.98% -- 3.85 seconds\n",
      "5000/5196 -- 96.23% -- 4.28 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 4-2017\n",
      "Found acceptable images for 6552 points in 18.11 seconds\n",
      "Featurizing: 04-2017\n",
      "0/6552 -- 0.00% -- 0.88 seconds\n",
      "1000/6552 -- 15.26% -- 4.38 seconds\n",
      "2000/6552 -- 30.53% -- 4.99 seconds\n",
      "3000/6552 -- 45.79% -- 4.09 seconds\n",
      "4000/6552 -- 61.05% -- 4.77 seconds\n",
      "5000/6552 -- 76.31% -- 5.00 seconds\n",
      "6000/6552 -- 91.58% -- 4.29 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 5-2017\n",
      "Found acceptable images for 18687 points in 19.41 seconds\n",
      "Featurizing: 05-2017\n",
      "0/18687 -- 0.00% -- 0.62 seconds\n",
      "1000/18687 -- 5.35% -- 4.18 seconds\n",
      "2000/18687 -- 10.70% -- 4.68 seconds\n",
      "3000/18687 -- 16.05% -- 4.04 seconds\n",
      "4000/18687 -- 21.41% -- 4.98 seconds\n",
      "5000/18687 -- 26.76% -- 3.87 seconds\n",
      "6000/18687 -- 32.11% -- 5.26 seconds\n",
      "7000/18687 -- 37.46% -- 4.30 seconds\n",
      "8000/18687 -- 42.81% -- 4.35 seconds\n",
      "9000/18687 -- 48.16% -- 4.33 seconds\n",
      "10000/18687 -- 53.51% -- 4.60 seconds\n",
      "11000/18687 -- 58.86% -- 4.21 seconds\n",
      "12000/18687 -- 64.22% -- 4.00 seconds\n",
      "13000/18687 -- 69.57% -- 4.94 seconds\n",
      "14000/18687 -- 74.92% -- 6.04 seconds\n",
      "15000/18687 -- 80.27% -- 4.37 seconds\n",
      "16000/18687 -- 85.62% -- 4.15 seconds\n",
      "17000/18687 -- 90.97% -- 4.16 seconds\n",
      "18000/18687 -- 96.32% -- 4.30 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 6-2017\n",
      "Found acceptable images for 19425 points in 19.27 seconds\n",
      "Featurizing: 06-2017\n",
      "0/19425 -- 0.00% -- 0.64 seconds\n",
      "1000/19425 -- 5.15% -- 4.50 seconds\n",
      "2000/19425 -- 10.30% -- 4.52 seconds\n",
      "3000/19425 -- 15.44% -- 4.50 seconds\n",
      "4000/19425 -- 20.59% -- 4.89 seconds\n",
      "5000/19425 -- 25.74% -- 3.91 seconds\n",
      "6000/19425 -- 30.89% -- 5.26 seconds\n",
      "7000/19425 -- 36.04% -- 4.67 seconds\n",
      "8000/19425 -- 41.18% -- 4.16 seconds\n",
      "9000/19425 -- 46.33% -- 4.45 seconds\n",
      "10000/19425 -- 51.48% -- 5.31 seconds\n",
      "11000/19425 -- 56.63% -- 4.79 seconds\n",
      "12000/19425 -- 61.78% -- 4.30 seconds\n",
      "13000/19425 -- 66.92% -- 5.14 seconds\n",
      "14000/19425 -- 72.07% -- 3.71 seconds\n",
      "15000/19425 -- 77.22% -- 4.35 seconds\n",
      "16000/19425 -- 82.37% -- 3.87 seconds\n",
      "17000/19425 -- 87.52% -- 3.93 seconds\n",
      "18000/19425 -- 92.66% -- 3.89 seconds\n",
      "19000/19425 -- 97.81% -- 4.27 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 7-2017\n",
      "Found acceptable images for 19598 points in 20.30 seconds\n",
      "Featurizing: 07-2017\n",
      "0/19598 -- 0.00% -- 0.64 seconds\n",
      "1000/19598 -- 5.10% -- 4.59 seconds\n",
      "2000/19598 -- 10.21% -- 4.95 seconds\n",
      "3000/19598 -- 15.31% -- 5.37 seconds\n",
      "4000/19598 -- 20.41% -- 5.66 seconds\n",
      "5000/19598 -- 25.51% -- 4.64 seconds\n",
      "6000/19598 -- 30.62% -- 4.91 seconds\n",
      "7000/19598 -- 35.72% -- 5.16 seconds\n",
      "8000/19598 -- 40.82% -- 4.33 seconds\n",
      "9000/19598 -- 45.92% -- 4.92 seconds\n",
      "10000/19598 -- 51.03% -- 5.88 seconds\n",
      "11000/19598 -- 56.13% -- 5.41 seconds\n",
      "12000/19598 -- 61.23% -- 4.67 seconds\n",
      "13000/19598 -- 66.33% -- 4.83 seconds\n",
      "14000/19598 -- 71.44% -- 5.04 seconds\n",
      "15000/19598 -- 76.54% -- 5.44 seconds\n",
      "16000/19598 -- 81.64% -- 5.12 seconds\n",
      "17000/19598 -- 86.74% -- 4.60 seconds\n",
      "18000/19598 -- 91.85% -- 4.28 seconds\n",
      "19000/19598 -- 96.95% -- 4.29 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 8-2017\n",
      "Found acceptable images for 19598 points in 19.83 seconds\n",
      "Featurizing: 08-2017\n",
      "0/19598 -- 0.00% -- 0.75 seconds\n",
      "1000/19598 -- 5.10% -- 4.01 seconds\n",
      "2000/19598 -- 10.21% -- 4.57 seconds\n",
      "3000/19598 -- 15.31% -- 4.76 seconds\n",
      "4000/19598 -- 20.41% -- 4.04 seconds\n",
      "5000/19598 -- 25.51% -- 3.93 seconds\n",
      "6000/19598 -- 30.62% -- 4.69 seconds\n",
      "7000/19598 -- 35.72% -- 4.40 seconds\n",
      "8000/19598 -- 40.82% -- 4.16 seconds\n",
      "9000/19598 -- 45.92% -- 6.02 seconds\n",
      "10000/19598 -- 51.03% -- 5.32 seconds\n",
      "11000/19598 -- 56.13% -- 4.68 seconds\n",
      "12000/19598 -- 61.23% -- 4.39 seconds\n",
      "13000/19598 -- 66.33% -- 5.04 seconds\n",
      "14000/19598 -- 71.44% -- 3.82 seconds\n",
      "15000/19598 -- 76.54% -- 4.75 seconds\n",
      "16000/19598 -- 81.64% -- 4.30 seconds\n",
      "17000/19598 -- 86.74% -- 4.04 seconds\n",
      "18000/19598 -- 91.85% -- 3.98 seconds\n",
      "19000/19598 -- 96.95% -- 3.70 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 9-2017\n",
      "Found acceptable images for 19598 points in 19.74 seconds\n",
      "Featurizing: 09-2017\n",
      "0/19598 -- 0.00% -- 0.72 seconds\n",
      "1000/19598 -- 5.10% -- 4.16 seconds\n",
      "2000/19598 -- 10.21% -- 4.66 seconds\n",
      "3000/19598 -- 15.31% -- 4.04 seconds\n",
      "4000/19598 -- 20.41% -- 4.30 seconds\n",
      "5000/19598 -- 25.51% -- 3.77 seconds\n",
      "6000/19598 -- 30.62% -- 4.39 seconds\n",
      "7000/19598 -- 35.72% -- 4.24 seconds\n",
      "8000/19598 -- 40.82% -- 4.10 seconds\n",
      "9000/19598 -- 45.92% -- 4.79 seconds\n",
      "10000/19598 -- 51.03% -- 4.91 seconds\n",
      "11000/19598 -- 56.13% -- 4.80 seconds\n",
      "12000/19598 -- 61.23% -- 4.36 seconds\n",
      "13000/19598 -- 66.33% -- 5.41 seconds\n",
      "14000/19598 -- 71.44% -- 4.20 seconds\n",
      "15000/19598 -- 76.54% -- 4.25 seconds\n",
      "16000/19598 -- 81.64% -- 3.99 seconds\n",
      "17000/19598 -- 86.74% -- 4.19 seconds\n",
      "18000/19598 -- 91.85% -- 4.51 seconds\n",
      "19000/19598 -- 96.95% -- 4.10 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 10-2017\n",
      "Found acceptable images for 19519 points in 19.59 seconds\n",
      "Featurizing: 10-2017\n",
      "0/19519 -- 0.00% -- 0.61 seconds\n",
      "1000/19519 -- 5.12% -- 4.40 seconds\n",
      "2000/19519 -- 10.25% -- 4.32 seconds\n",
      "3000/19519 -- 15.37% -- 4.23 seconds\n",
      "4000/19519 -- 20.49% -- 4.57 seconds\n",
      "5000/19519 -- 25.62% -- 4.32 seconds\n",
      "6000/19519 -- 30.74% -- 4.58 seconds\n",
      "7000/19519 -- 35.86% -- 4.95 seconds\n",
      "8000/19519 -- 40.99% -- 4.55 seconds\n",
      "9000/19519 -- 46.11% -- 4.72 seconds\n",
      "10000/19519 -- 51.23% -- 5.82 seconds\n",
      "11000/19519 -- 56.36% -- 5.57 seconds\n",
      "12000/19519 -- 61.48% -- 4.55 seconds\n",
      "13000/19519 -- 66.60% -- 5.54 seconds\n",
      "14000/19519 -- 71.72% -- 4.29 seconds\n",
      "15000/19519 -- 76.85% -- 4.74 seconds\n",
      "16000/19519 -- 81.97% -- 4.10 seconds\n",
      "17000/19519 -- 87.09% -- 4.44 seconds\n",
      "18000/19519 -- 92.22% -- 4.83 seconds\n",
      "19000/19519 -- 97.34% -- 4.82 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 11-2017\n",
      "Found acceptable images for 14690 points in 18.95 seconds\n",
      "Featurizing: 11-2017\n",
      "0/14690 -- 0.00% -- 0.66 seconds\n",
      "1000/14690 -- 6.81% -- 4.89 seconds\n",
      "2000/14690 -- 13.61% -- 4.12 seconds\n",
      "3000/14690 -- 20.42% -- 5.13 seconds\n",
      "4000/14690 -- 27.23% -- 4.44 seconds\n",
      "5000/14690 -- 34.04% -- 4.37 seconds\n",
      "6000/14690 -- 40.84% -- 4.83 seconds\n",
      "7000/14690 -- 47.65% -- 4.09 seconds\n",
      "8000/14690 -- 54.46% -- 4.62 seconds\n",
      "9000/14690 -- 61.27% -- 4.09 seconds\n",
      "10000/14690 -- 68.07% -- 4.15 seconds\n",
      "11000/14690 -- 74.88% -- 4.08 seconds\n",
      "12000/14690 -- 81.69% -- 3.96 seconds\n",
      "13000/14690 -- 88.50% -- 4.20 seconds\n",
      "14000/14690 -- 95.30% -- 4.18 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 12-2017\n",
      "Found acceptable images for 4192 points in 17.42 seconds\n",
      "Featurizing: 12-2017\n",
      "0/4192 -- 0.00% -- 0.59 seconds\n",
      "1000/4192 -- 23.85% -- 5.02 seconds\n",
      "2000/4192 -- 47.71% -- 5.25 seconds\n",
      "3000/4192 -- 71.56% -- 3.99 seconds\n",
      "4000/4192 -- 95.42% -- 3.67 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Saving file as: data/sentinel-2-l2a_bands-3-4-5_ZMB_20k-points_1000-features_2017.feather\n",
      "Save finished!\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 1-2018\n",
      "Found acceptable images for 13492 points in 18.63 seconds\n",
      "Featurizing: 01-2018\n",
      "0/13492 -- 0.00% -- 0.66 seconds\n",
      "1000/13492 -- 7.41% -- 3.94 seconds\n",
      "2000/13492 -- 14.82% -- 4.38 seconds\n",
      "3000/13492 -- 22.24% -- 4.17 seconds\n",
      "5000/13492 -- 37.06% -- 3.92 seconds\n",
      "6000/13492 -- 44.47% -- 4.37 seconds\n",
      "7000/13492 -- 51.88% -- 4.45 seconds\n",
      "8000/13492 -- 59.29% -- 4.45 seconds\n",
      "9000/13492 -- 66.71% -- 3.87 seconds\n",
      "10000/13492 -- 74.12% -- 3.92 seconds\n",
      "11000/13492 -- 81.53% -- 4.02 seconds\n",
      "12000/13492 -- 88.94% -- 4.06 seconds\n",
      "13000/13492 -- 96.35% -- 4.27 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 2-2018\n",
      "Found acceptable images for 8531 points in 17.76 seconds\n",
      "Featurizing: 02-2018\n",
      "0/8531 -- 0.00% -- 0.60 seconds\n",
      "1000/8531 -- 11.72% -- 4.82 seconds\n",
      "2000/8531 -- 23.44% -- 4.67 seconds\n",
      "3000/8531 -- 35.17% -- 5.11 seconds\n",
      "4000/8531 -- 46.89% -- 3.79 seconds\n",
      "5000/8531 -- 58.61% -- 5.13 seconds\n",
      "6000/8531 -- 70.33% -- 4.25 seconds\n",
      "7000/8531 -- 82.05% -- 4.06 seconds\n",
      "8000/8531 -- 93.78% -- 4.09 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 3-2018\n",
      "Found acceptable images for 8961 points in 18.32 seconds\n",
      "Featurizing: 03-2018\n",
      "0/8961 -- 0.00% -- 0.69 seconds\n",
      "1000/8961 -- 11.16% -- 6.21 seconds\n",
      "2000/8961 -- 22.32% -- 4.10 seconds\n",
      "3000/8961 -- 33.48% -- 4.74 seconds\n",
      "4000/8961 -- 44.64% -- 4.26 seconds\n",
      "5000/8961 -- 55.80% -- 5.47 seconds\n",
      "6000/8961 -- 66.96% -- 3.81 seconds\n",
      "7000/8961 -- 78.12% -- 4.68 seconds\n",
      "8000/8961 -- 89.28% -- 4.16 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 4-2018\n",
      "Found acceptable images for 19312 points in 19.42 seconds\n",
      "Featurizing: 04-2018\n",
      "0/19312 -- 0.00% -- 0.66 seconds\n",
      "1000/19312 -- 5.18% -- 4.76 seconds\n",
      "2000/19312 -- 10.36% -- 5.89 seconds\n",
      "3000/19312 -- 15.53% -- 5.49 seconds\n",
      "4000/19312 -- 20.71% -- 4.82 seconds\n",
      "5000/19312 -- 25.89% -- 5.09 seconds\n",
      "6000/19312 -- 31.07% -- 7.09 seconds\n",
      "7000/19312 -- 36.25% -- 6.71 seconds\n",
      "8000/19312 -- 41.43% -- 5.10 seconds\n",
      "9000/19312 -- 46.60% -- 5.91 seconds\n",
      "10000/19312 -- 51.78% -- 5.36 seconds\n",
      "11000/19312 -- 56.96% -- 5.76 seconds\n",
      "12000/19312 -- 62.14% -- 6.23 seconds\n",
      "13000/19312 -- 67.32% -- 4.42 seconds\n",
      "14000/19312 -- 72.49% -- 4.38 seconds\n",
      "15000/19312 -- 77.67% -- 5.82 seconds\n",
      "16000/19312 -- 82.85% -- 4.87 seconds\n",
      "17000/19312 -- 88.03% -- 4.33 seconds\n",
      "18000/19312 -- 93.21% -- 4.58 seconds\n",
      "19000/19312 -- 98.38% -- 4.28 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 5-2018\n",
      "Found acceptable images for 19598 points in 19.70 seconds\n",
      "Featurizing: 05-2018\n",
      "0/19598 -- 0.00% -- 0.64 seconds\n",
      "1000/19598 -- 5.10% -- 4.87 seconds\n",
      "2000/19598 -- 10.21% -- 4.06 seconds\n",
      "3000/19598 -- 15.31% -- 3.93 seconds\n",
      "4000/19598 -- 20.41% -- 3.74 seconds\n",
      "5000/19598 -- 25.51% -- 3.97 seconds\n",
      "6000/19598 -- 30.62% -- 4.88 seconds\n",
      "7000/19598 -- 35.72% -- 4.89 seconds\n",
      "8000/19598 -- 40.82% -- 5.17 seconds\n",
      "9000/19598 -- 45.92% -- 4.67 seconds\n",
      "10000/19598 -- 51.03% -- 5.41 seconds\n",
      "11000/19598 -- 56.13% -- 5.89 seconds\n",
      "12000/19598 -- 61.23% -- 4.22 seconds\n",
      "13000/19598 -- 66.33% -- 6.26 seconds\n",
      "14000/19598 -- 71.44% -- 4.04 seconds\n",
      "15000/19598 -- 76.54% -- 4.09 seconds\n",
      "16000/19598 -- 81.64% -- 4.09 seconds\n",
      "17000/19598 -- 86.74% -- 4.24 seconds\n",
      "18000/19598 -- 91.85% -- 4.68 seconds\n",
      "19000/19598 -- 96.95% -- 4.19 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 6-2018\n",
      "Found acceptable images for 19598 points in 19.58 seconds\n",
      "Featurizing: 06-2018\n",
      "0/19598 -- 0.00% -- 0.82 seconds\n",
      "1000/19598 -- 5.10% -- 4.50 seconds\n",
      "2000/19598 -- 10.21% -- 4.30 seconds\n",
      "3000/19598 -- 15.31% -- 3.99 seconds\n",
      "4000/19598 -- 20.41% -- 4.09 seconds\n",
      "5000/19598 -- 25.51% -- 3.92 seconds\n",
      "6000/19598 -- 30.62% -- 4.72 seconds\n",
      "7000/19598 -- 35.72% -- 4.41 seconds\n",
      "8000/19598 -- 40.82% -- 4.21 seconds\n",
      "9000/19598 -- 45.92% -- 5.32 seconds\n",
      "10000/19598 -- 51.03% -- 5.30 seconds\n",
      "11000/19598 -- 56.13% -- 5.77 seconds\n",
      "12000/19598 -- 61.23% -- 4.39 seconds\n",
      "13000/19598 -- 66.33% -- 4.64 seconds\n",
      "14000/19598 -- 71.44% -- 3.74 seconds\n",
      "15000/19598 -- 76.54% -- 4.25 seconds\n",
      "16000/19598 -- 81.64% -- 4.15 seconds\n",
      "17000/19598 -- 86.74% -- 4.30 seconds\n",
      "18000/19598 -- 91.85% -- 4.08 seconds\n",
      "19000/19598 -- 96.95% -- 4.21 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 7-2018\n",
      "Found acceptable images for 19598 points in 19.53 seconds\n",
      "Featurizing: 07-2018\n",
      "0/19598 -- 0.00% -- 0.65 seconds\n",
      "1000/19598 -- 5.10% -- 4.34 seconds\n",
      "2000/19598 -- 10.21% -- 4.17 seconds\n",
      "3000/19598 -- 15.31% -- 3.92 seconds\n",
      "4000/19598 -- 20.41% -- 3.97 seconds\n",
      "5000/19598 -- 25.51% -- 3.99 seconds\n",
      "6000/19598 -- 30.62% -- 4.77 seconds\n",
      "7000/19598 -- 35.72% -- 4.56 seconds\n",
      "8000/19598 -- 40.82% -- 4.07 seconds\n",
      "9000/19598 -- 45.92% -- 4.74 seconds\n",
      "10000/19598 -- 51.03% -- 4.64 seconds\n",
      "11000/19598 -- 56.13% -- 4.79 seconds\n",
      "12000/19598 -- 61.23% -- 3.86 seconds\n",
      "13000/19598 -- 66.33% -- 4.64 seconds\n",
      "14000/19598 -- 71.44% -- 3.76 seconds\n",
      "15000/19598 -- 76.54% -- 4.24 seconds\n",
      "16000/19598 -- 81.64% -- 4.60 seconds\n",
      "17000/19598 -- 86.74% -- 4.11 seconds\n",
      "18000/19598 -- 91.85% -- 3.87 seconds\n",
      "19000/19598 -- 96.95% -- 4.20 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 8-2018\n",
      "Found acceptable images for 19598 points in 20.06 seconds\n",
      "Featurizing: 08-2018\n",
      "0/19598 -- 0.00% -- 0.69 seconds\n",
      "1000/19598 -- 5.10% -- 4.43 seconds\n",
      "2000/19598 -- 10.21% -- 4.51 seconds\n",
      "3000/19598 -- 15.31% -- 3.83 seconds\n",
      "4000/19598 -- 20.41% -- 4.06 seconds\n",
      "5000/19598 -- 25.51% -- 4.00 seconds\n",
      "6000/19598 -- 30.62% -- 4.49 seconds\n",
      "7000/19598 -- 35.72% -- 4.74 seconds\n",
      "8000/19598 -- 40.82% -- 4.50 seconds\n",
      "9000/19598 -- 45.92% -- 4.75 seconds\n",
      "10000/19598 -- 51.03% -- 4.60 seconds\n",
      "11000/19598 -- 56.13% -- 4.88 seconds\n",
      "12000/19598 -- 61.23% -- 4.47 seconds\n",
      "13000/19598 -- 66.33% -- 5.32 seconds\n",
      "14000/19598 -- 71.44% -- 3.99 seconds\n",
      "15000/19598 -- 76.54% -- 4.61 seconds\n",
      "16000/19598 -- 81.64% -- 4.64 seconds\n",
      "17000/19598 -- 86.74% -- 5.51 seconds\n",
      "18000/19598 -- 91.85% -- 3.85 seconds\n",
      "19000/19598 -- 96.95% -- 4.14 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 9-2018\n",
      "Found acceptable images for 19598 points in 19.63 seconds\n",
      "Featurizing: 09-2018\n",
      "0/19598 -- 0.00% -- 0.73 seconds\n",
      "1000/19598 -- 5.10% -- 5.05 seconds\n",
      "2000/19598 -- 10.21% -- 4.46 seconds\n",
      "3000/19598 -- 15.31% -- 3.86 seconds\n",
      "4000/19598 -- 20.41% -- 4.32 seconds\n",
      "5000/19598 -- 25.51% -- 3.72 seconds\n",
      "6000/19598 -- 30.62% -- 5.17 seconds\n",
      "7000/19598 -- 35.72% -- 4.23 seconds\n",
      "8000/19598 -- 40.82% -- 4.32 seconds\n",
      "9000/19598 -- 45.92% -- 4.49 seconds\n",
      "10000/19598 -- 51.03% -- 5.12 seconds\n",
      "11000/19598 -- 56.13% -- 5.37 seconds\n",
      "12000/19598 -- 61.23% -- 4.22 seconds\n",
      "13000/19598 -- 66.33% -- 4.90 seconds\n",
      "14000/19598 -- 71.44% -- 3.98 seconds\n",
      "15000/19598 -- 76.54% -- 4.83 seconds\n",
      "16000/19598 -- 81.64% -- 4.13 seconds\n",
      "17000/19598 -- 86.74% -- 4.09 seconds\n",
      "18000/19598 -- 91.85% -- 3.93 seconds\n",
      "19000/19598 -- 96.95% -- 4.21 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 10-2018\n",
      "Found acceptable images for 19519 points in 19.63 seconds\n",
      "Featurizing: 10-2018\n",
      "0/19519 -- 0.00% -- 1.27 seconds\n",
      "1000/19519 -- 5.12% -- 5.43 seconds\n",
      "2000/19519 -- 10.25% -- 5.93 seconds\n",
      "4000/19519 -- 20.49% -- 5.00 seconds\n",
      "5000/19519 -- 25.62% -- 4.01 seconds\n",
      "6000/19519 -- 30.74% -- 6.33 seconds\n",
      "7000/19519 -- 35.86% -- 5.94 seconds\n",
      "8000/19519 -- 40.99% -- 5.26 seconds\n",
      "9000/19519 -- 46.11% -- 4.89 seconds\n",
      "10000/19519 -- 51.23% -- 5.91 seconds\n",
      "11000/19519 -- 56.36% -- 5.26 seconds\n",
      "12000/19519 -- 61.48% -- 4.77 seconds\n",
      "13000/19519 -- 66.60% -- 5.81 seconds\n",
      "14000/19519 -- 71.72% -- 3.89 seconds\n",
      "15000/19519 -- 76.85% -- 4.45 seconds\n",
      "16000/19519 -- 81.97% -- 5.28 seconds\n",
      "17000/19519 -- 87.09% -- 5.02 seconds\n",
      "18000/19519 -- 92.22% -- 5.04 seconds\n",
      "19000/19519 -- 97.34% -- 4.48 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 11-2018\n",
      "Found acceptable images for 18585 points in 19.34 seconds\n",
      "Featurizing: 11-2018\n",
      "0/18585 -- 0.00% -- 0.70 seconds\n",
      "1000/18585 -- 5.38% -- 4.74 seconds\n",
      "2000/18585 -- 10.76% -- 4.76 seconds\n",
      "3000/18585 -- 16.14% -- 4.24 seconds\n",
      "4000/18585 -- 21.52% -- 4.58 seconds\n",
      "5000/18585 -- 26.90% -- 4.05 seconds\n",
      "6000/18585 -- 32.28% -- 4.84 seconds\n",
      "7000/18585 -- 37.66% -- 4.47 seconds\n",
      "8000/18585 -- 43.05% -- 4.68 seconds\n",
      "9000/18585 -- 48.43% -- 4.62 seconds\n",
      "10000/18585 -- 53.81% -- 5.83 seconds\n",
      "11000/18585 -- 59.19% -- 3.99 seconds\n",
      "12000/18585 -- 64.57% -- 5.59 seconds\n",
      "13000/18585 -- 69.95% -- 3.97 seconds\n",
      "14000/18585 -- 75.33% -- 5.09 seconds\n",
      "15000/18585 -- 80.71% -- 6.78 seconds\n",
      "16000/18585 -- 86.09% -- 4.37 seconds\n",
      "17000/18585 -- 91.47% -- 4.22 seconds\n",
      "18000/18585 -- 96.85% -- 4.01 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 12-2018\n",
      "Found acceptable images for 5281 points in 17.52 seconds\n",
      "Featurizing: 12-2018\n",
      "0/5281 -- 0.00% -- 0.58 seconds\n",
      "1000/5281 -- 18.94% -- 4.60 seconds\n",
      "2000/5281 -- 37.87% -- 3.78 seconds\n",
      "3000/5281 -- 56.81% -- 4.56 seconds\n",
      "4000/5281 -- 75.74% -- 4.47 seconds\n",
      "5000/5281 -- 94.68% -- 4.32 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Saving file as: data/sentinel-2-l2a_bands-3-4-5_ZMB_20k-points_1000-features_2018.feather\n",
      "Save finished!\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 1-2019\n",
      "Found acceptable images for 5188 points in 17.64 seconds\n",
      "Featurizing: 01-2019\n",
      "0/5188 -- 0.00% -- 0.76 seconds\n",
      "1000/5188 -- 19.28% -- 4.92 seconds\n",
      "2000/5188 -- 38.55% -- 4.02 seconds\n",
      "3000/5188 -- 57.83% -- 4.94 seconds\n",
      "4000/5188 -- 77.10% -- 5.18 seconds\n",
      "5000/5188 -- 96.38% -- 3.86 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 2-2019\n",
      "Found acceptable images for 13030 points in 18.76 seconds\n",
      "Featurizing: 02-2019\n",
      "0/13030 -- 0.00% -- 0.72 seconds\n",
      "1000/13030 -- 7.67% -- 4.48 seconds\n",
      "2000/13030 -- 15.35% -- 4.22 seconds\n",
      "3000/13030 -- 23.02% -- 4.19 seconds\n",
      "4000/13030 -- 30.70% -- 4.69 seconds\n",
      "5000/13030 -- 38.37% -- 4.36 seconds\n",
      "6000/13030 -- 46.05% -- 4.48 seconds\n",
      "7000/13030 -- 53.72% -- 4.42 seconds\n",
      "8000/13030 -- 61.40% -- 5.04 seconds\n",
      "9000/13030 -- 69.07% -- 4.81 seconds\n",
      "10000/13030 -- 76.75% -- 4.81 seconds\n",
      "11000/13030 -- 84.42% -- 4.96 seconds\n",
      "12000/13030 -- 92.10% -- 4.03 seconds\n",
      "13000/13030 -- 99.77% -- 4.11 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 3-2019\n",
      "Found acceptable images for 16426 points in 19.13 seconds\n",
      "Featurizing: 03-2019\n",
      "0/16426 -- 0.00% -- 0.68 seconds\n",
      "1000/16426 -- 6.09% -- 4.22 seconds\n",
      "2000/16426 -- 12.18% -- 4.41 seconds\n",
      "3000/16426 -- 18.26% -- 4.35 seconds\n",
      "4000/16426 -- 24.35% -- 4.44 seconds\n",
      "5000/16426 -- 30.44% -- 4.26 seconds\n",
      "6000/16426 -- 36.53% -- 4.70 seconds\n",
      "7000/16426 -- 42.62% -- 4.49 seconds\n",
      "8000/16426 -- 48.70% -- 3.92 seconds\n",
      "9000/16426 -- 54.79% -- 4.11 seconds\n",
      "10000/16426 -- 60.88% -- 5.34 seconds\n",
      "11000/16426 -- 66.97% -- 4.56 seconds\n",
      "12000/16426 -- 73.05% -- 4.23 seconds\n",
      "13000/16426 -- 79.14% -- 4.30 seconds\n",
      "14000/16426 -- 85.23% -- 4.12 seconds\n",
      "15000/16426 -- 91.32% -- 4.23 seconds\n",
      "16000/16426 -- 97.41% -- 4.47 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 4-2019\n",
      "Found acceptable images for 18768 points in 19.19 seconds\n",
      "Featurizing: 04-2019\n",
      "0/18768 -- 0.00% -- 0.67 seconds\n",
      "1000/18768 -- 5.33% -- 4.36 seconds\n",
      "2000/18768 -- 10.66% -- 4.57 seconds\n",
      "3000/18768 -- 15.98% -- 4.09 seconds\n",
      "4000/18768 -- 21.31% -- 4.03 seconds\n",
      "5000/18768 -- 26.64% -- 3.94 seconds\n",
      "6000/18768 -- 31.97% -- 4.28 seconds\n",
      "7000/18768 -- 37.30% -- 4.20 seconds\n",
      "8000/18768 -- 42.63% -- 4.68 seconds\n",
      "9000/18768 -- 47.95% -- 4.31 seconds\n",
      "10000/18768 -- 53.28% -- 5.26 seconds\n",
      "11000/18768 -- 58.61% -- 4.91 seconds\n",
      "12000/18768 -- 63.94% -- 4.96 seconds\n",
      "13000/18768 -- 69.27% -- 4.80 seconds\n",
      "14000/18768 -- 74.60% -- 4.33 seconds\n",
      "15000/18768 -- 79.92% -- 4.66 seconds\n",
      "16000/18768 -- 85.25% -- 3.79 seconds\n",
      "17000/18768 -- 90.58% -- 4.15 seconds\n",
      "18000/18768 -- 95.91% -- 4.23 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 5-2019\n",
      "Found acceptable images for 19478 points in 19.85 seconds\n",
      "Featurizing: 05-2019\n",
      "0/19478 -- 0.00% -- 0.68 seconds\n",
      "1000/19478 -- 5.13% -- 4.66 seconds\n",
      "2000/19478 -- 10.27% -- 4.47 seconds\n",
      "3000/19478 -- 15.40% -- 4.92 seconds\n",
      "4000/19478 -- 20.54% -- 4.18 seconds\n",
      "5000/19478 -- 25.67% -- 3.87 seconds\n",
      "6000/19478 -- 30.80% -- 5.54 seconds\n",
      "7000/19478 -- 35.94% -- 6.34 seconds\n",
      "8000/19478 -- 41.07% -- 4.79 seconds\n",
      "9000/19478 -- 46.21% -- 5.43 seconds\n",
      "10000/19478 -- 51.34% -- 4.99 seconds\n",
      "11000/19478 -- 56.47% -- 5.42 seconds\n",
      "12000/19478 -- 61.61% -- 4.60 seconds\n",
      "13000/19478 -- 66.74% -- 5.09 seconds\n",
      "14000/19478 -- 71.88% -- 4.03 seconds\n",
      "15000/19478 -- 77.01% -- 4.96 seconds\n",
      "16000/19478 -- 82.14% -- 4.32 seconds\n",
      "17000/19478 -- 87.28% -- 4.26 seconds\n",
      "18000/19478 -- 92.41% -- 4.38 seconds\n",
      "19000/19478 -- 97.55% -- 4.71 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 6-2019\n",
      "Found acceptable images for 19598 points in 19.77 seconds\n",
      "Featurizing: 06-2019\n",
      "0/19598 -- 0.00% -- 0.65 seconds\n",
      "1000/19598 -- 5.10% -- 3.97 seconds\n",
      "2000/19598 -- 10.21% -- 4.56 seconds\n",
      "3000/19598 -- 15.31% -- 4.06 seconds\n",
      "4000/19598 -- 20.41% -- 4.53 seconds\n",
      "5000/19598 -- 25.51% -- 4.00 seconds\n",
      "6000/19598 -- 30.62% -- 4.36 seconds\n",
      "7000/19598 -- 35.72% -- 4.37 seconds\n",
      "8000/19598 -- 40.82% -- 3.90 seconds\n",
      "9000/19598 -- 45.92% -- 4.60 seconds\n",
      "10000/19598 -- 51.03% -- 4.78 seconds\n",
      "11000/19598 -- 56.13% -- 4.85 seconds\n",
      "12000/19598 -- 61.23% -- 4.00 seconds\n",
      "13000/19598 -- 66.33% -- 5.37 seconds\n",
      "14000/19598 -- 71.44% -- 3.94 seconds\n",
      "15000/19598 -- 76.54% -- 4.53 seconds\n",
      "16000/19598 -- 81.64% -- 4.41 seconds\n",
      "17000/19598 -- 86.74% -- 3.81 seconds\n",
      "18000/19598 -- 91.85% -- 3.90 seconds\n",
      "19000/19598 -- 96.95% -- 4.08 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 7-2019\n",
      "Found acceptable images for 19598 points in 20.12 seconds\n",
      "Featurizing: 07-2019\n",
      "0/19598 -- 0.00% -- 0.58 seconds\n",
      "1000/19598 -- 5.10% -- 4.25 seconds\n",
      "2000/19598 -- 10.21% -- 3.91 seconds\n",
      "3000/19598 -- 15.31% -- 4.03 seconds\n",
      "4000/19598 -- 20.41% -- 4.13 seconds\n",
      "5000/19598 -- 25.51% -- 3.97 seconds\n",
      "6000/19598 -- 30.62% -- 4.53 seconds\n",
      "7000/19598 -- 35.72% -- 4.49 seconds\n",
      "8000/19598 -- 40.82% -- 4.18 seconds\n",
      "9000/19598 -- 45.92% -- 4.61 seconds\n",
      "10000/19598 -- 51.03% -- 5.09 seconds\n",
      "11000/19598 -- 56.13% -- 5.35 seconds\n",
      "12000/19598 -- 61.23% -- 4.15 seconds\n",
      "13000/19598 -- 66.33% -- 5.54 seconds\n",
      "14000/19598 -- 71.44% -- 3.59 seconds\n",
      "15000/19598 -- 76.54% -- 4.26 seconds\n",
      "16000/19598 -- 81.64% -- 4.36 seconds\n",
      "17000/19598 -- 86.74% -- 3.77 seconds\n",
      "18000/19598 -- 91.85% -- 4.16 seconds\n",
      "19000/19598 -- 96.95% -- 3.86 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 8-2019\n",
      "Found acceptable images for 19598 points in 19.48 seconds\n",
      "Featurizing: 08-2019\n",
      "0/19598 -- 0.00% -- 0.64 seconds\n",
      "1000/19598 -- 5.10% -- 4.23 seconds\n",
      "2000/19598 -- 10.21% -- 4.58 seconds\n",
      "3000/19598 -- 15.31% -- 4.42 seconds\n",
      "4000/19598 -- 20.41% -- 5.65 seconds\n",
      "5000/19598 -- 25.51% -- 3.94 seconds\n",
      "6000/19598 -- 30.62% -- 4.88 seconds\n",
      "7000/19598 -- 35.72% -- 5.49 seconds\n",
      "8000/19598 -- 40.82% -- 4.96 seconds\n",
      "9000/19598 -- 45.92% -- 5.68 seconds\n",
      "10000/19598 -- 51.03% -- 7.02 seconds\n",
      "11000/19598 -- 56.13% -- 5.72 seconds\n",
      "12000/19598 -- 61.23% -- 5.46 seconds\n",
      "13000/19598 -- 66.33% -- 5.85 seconds\n",
      "14000/19598 -- 71.44% -- 4.07 seconds\n",
      "15000/19598 -- 76.54% -- 4.53 seconds\n",
      "16000/19598 -- 81.64% -- 4.59 seconds\n",
      "17000/19598 -- 86.74% -- 4.43 seconds\n",
      "18000/19598 -- 91.85% -- 4.06 seconds\n",
      "19000/19598 -- 96.95% -- 4.22 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 9-2019\n",
      "Found acceptable images for 19598 points in 20.04 seconds\n",
      "Featurizing: 09-2019\n",
      "0/19598 -- 0.00% -- 0.73 seconds\n",
      "1000/19598 -- 5.10% -- 4.22 seconds\n",
      "2000/19598 -- 10.21% -- 4.26 seconds\n",
      "4000/19598 -- 20.41% -- 4.64 seconds\n",
      "5000/19598 -- 25.51% -- 3.96 seconds\n",
      "6000/19598 -- 30.62% -- 4.41 seconds\n",
      "7000/19598 -- 35.72% -- 4.45 seconds\n",
      "8000/19598 -- 40.82% -- 4.06 seconds\n",
      "9000/19598 -- 45.92% -- 5.32 seconds\n",
      "10000/19598 -- 51.03% -- 4.87 seconds\n",
      "11000/19598 -- 56.13% -- 4.69 seconds\n",
      "12000/19598 -- 61.23% -- 4.20 seconds\n",
      "13000/19598 -- 66.33% -- 4.63 seconds\n",
      "14000/19598 -- 71.44% -- 4.03 seconds\n",
      "15000/19598 -- 76.54% -- 4.35 seconds\n",
      "16000/19598 -- 81.64% -- 4.22 seconds\n",
      "17000/19598 -- 86.74% -- 4.70 seconds\n",
      "18000/19598 -- 91.85% -- 4.16 seconds\n",
      "19000/19598 -- 96.95% -- 4.00 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 10-2019\n",
      "Found acceptable images for 19574 points in 19.59 seconds\n",
      "Featurizing: 10-2019\n",
      "0/19574 -- 0.00% -- 0.76 seconds\n",
      "1000/19574 -- 5.11% -- 3.95 seconds\n",
      "2000/19574 -- 10.22% -- 4.31 seconds\n",
      "3000/19574 -- 15.33% -- 4.63 seconds\n",
      "4000/19574 -- 20.44% -- 4.12 seconds\n",
      "5000/19574 -- 25.54% -- 3.96 seconds\n",
      "6000/19574 -- 30.65% -- 5.40 seconds\n",
      "7000/19574 -- 35.76% -- 5.48 seconds\n",
      "8000/19574 -- 40.87% -- 5.47 seconds\n",
      "9000/19574 -- 45.98% -- 4.70 seconds\n",
      "10000/19574 -- 51.09% -- 6.30 seconds\n",
      "11000/19574 -- 56.20% -- 4.83 seconds\n",
      "12000/19574 -- 61.31% -- 4.61 seconds\n",
      "13000/19574 -- 66.41% -- 5.87 seconds\n",
      "14000/19574 -- 71.52% -- 4.44 seconds\n",
      "15000/19574 -- 76.63% -- 5.34 seconds\n",
      "16000/19574 -- 81.74% -- 5.44 seconds\n",
      "17000/19574 -- 86.85% -- 5.01 seconds\n",
      "18000/19574 -- 91.96% -- 4.10 seconds\n",
      "19000/19574 -- 97.07% -- 4.03 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 11-2019\n",
      "Found acceptable images for 18495 points in 19.56 seconds\n",
      "Featurizing: 11-2019\n",
      "0/18495 -- 0.00% -- 0.64 seconds\n",
      "1000/18495 -- 5.41% -- 4.33 seconds\n",
      "2000/18495 -- 10.81% -- 4.26 seconds\n",
      "3000/18495 -- 16.22% -- 6.15 seconds\n",
      "4000/18495 -- 21.63% -- 4.42 seconds\n",
      "5000/18495 -- 27.03% -- 3.94 seconds\n",
      "6000/18495 -- 32.44% -- 4.33 seconds\n",
      "7000/18495 -- 37.85% -- 4.91 seconds\n",
      "8000/18495 -- 43.25% -- 4.25 seconds\n",
      "9000/18495 -- 48.66% -- 4.42 seconds\n",
      "10000/18495 -- 54.07% -- 4.82 seconds\n",
      "11000/18495 -- 59.48% -- 4.28 seconds\n",
      "12000/18495 -- 64.88% -- 4.55 seconds\n",
      "13000/18495 -- 70.29% -- 4.40 seconds\n",
      "14000/18495 -- 75.70% -- 4.10 seconds\n",
      "15000/18495 -- 81.10% -- 4.05 seconds\n",
      "16000/18495 -- 86.51% -- 4.30 seconds\n",
      "17000/18495 -- 91.92% -- 4.40 seconds\n",
      "18000/18495 -- 97.32% -- 4.61 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 12-2019\n",
      "Found acceptable images for 11393 points in 18.61 seconds\n",
      "Featurizing: 12-2019\n",
      "0/11393 -- 0.00% -- 0.72 seconds\n",
      "1000/11393 -- 8.78% -- 4.54 seconds\n",
      "2000/11393 -- 17.55% -- 5.40 seconds\n",
      "3000/11393 -- 26.33% -- 5.74 seconds\n",
      "4000/11393 -- 35.11% -- 4.39 seconds\n",
      "5000/11393 -- 43.89% -- 6.84 seconds\n",
      "6000/11393 -- 52.66% -- 5.35 seconds\n",
      "7000/11393 -- 61.44% -- 10.35 seconds\n",
      "8000/11393 -- 70.22% -- 5.80 seconds\n",
      "9000/11393 -- 79.00% -- 4.86 seconds\n",
      "10000/11393 -- 87.77% -- 6.53 seconds\n",
      "11000/11393 -- 96.55% -- 5.87 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Saving file as: data/sentinel-2-l2a_bands-3-4-5_ZMB_20k-points_1000-features_2019.feather\n",
      "Save finished!\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 1-2020\n",
      "Found acceptable images for 5677 points in 17.35 seconds\n",
      "Featurizing: 01-2020\n",
      "0/5677 -- 0.00% -- 0.71 seconds\n",
      "1000/5677 -- 17.61% -- 4.34 seconds\n",
      "2000/5677 -- 35.23% -- 4.66 seconds\n",
      "3000/5677 -- 52.84% -- 3.94 seconds\n",
      "4000/5677 -- 70.46% -- 4.41 seconds\n",
      "5000/5677 -- 88.07% -- 4.24 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 2-2020\n",
      "Found acceptable images for 5473 points in 17.72 seconds\n",
      "Featurizing: 02-2020\n",
      "0/5473 -- 0.00% -- 0.74 seconds\n",
      "1000/5473 -- 18.27% -- 3.97 seconds\n",
      "2000/5473 -- 36.54% -- 3.91 seconds\n",
      "3000/5473 -- 54.81% -- 4.56 seconds\n",
      "4000/5473 -- 73.09% -- 4.16 seconds\n",
      "5000/5473 -- 91.36% -- 5.05 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 3-2020\n",
      "Found acceptable images for 9405 points in 18.41 seconds\n",
      "Featurizing: 03-2020\n",
      "0/9405 -- 0.00% -- 0.64 seconds\n",
      "1000/9405 -- 10.63% -- 5.32 seconds\n",
      "2000/9405 -- 21.27% -- 4.54 seconds\n",
      "3000/9405 -- 31.90% -- 4.10 seconds\n",
      "4000/9405 -- 42.53% -- 4.79 seconds\n",
      "5000/9405 -- 53.16% -- 4.44 seconds\n",
      "6000/9405 -- 63.80% -- 4.01 seconds\n",
      "7000/9405 -- 74.43% -- 3.85 seconds\n",
      "8000/9405 -- 85.06% -- 4.44 seconds\n",
      "9000/9405 -- 95.69% -- 4.50 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 4-2020\n",
      "Found acceptable images for 19010 points in 19.38 seconds\n",
      "Featurizing: 04-2020\n",
      "0/19010 -- 0.00% -- 0.71 seconds\n",
      "1000/19010 -- 5.26% -- 4.00 seconds\n",
      "2000/19010 -- 10.52% -- 4.95 seconds\n",
      "3000/19010 -- 15.78% -- 4.74 seconds\n",
      "4000/19010 -- 21.04% -- 4.29 seconds\n",
      "5000/19010 -- 26.30% -- 4.15 seconds\n",
      "6000/19010 -- 31.56% -- 4.45 seconds\n",
      "7000/19010 -- 36.82% -- 4.52 seconds\n",
      "8000/19010 -- 42.08% -- 3.92 seconds\n",
      "9000/19010 -- 47.34% -- 4.50 seconds\n",
      "10000/19010 -- 52.60% -- 5.61 seconds\n",
      "11000/19010 -- 57.86% -- 4.14 seconds\n",
      "12000/19010 -- 63.12% -- 5.57 seconds\n",
      "13000/19010 -- 68.39% -- 4.31 seconds\n",
      "14000/19010 -- 73.65% -- 4.80 seconds\n",
      "15000/19010 -- 78.91% -- 4.18 seconds\n",
      "16000/19010 -- 84.17% -- 4.25 seconds\n",
      "17000/19010 -- 89.43% -- 4.32 seconds\n",
      "18000/19010 -- 94.69% -- 4.43 seconds\n",
      "19000/19010 -- 99.95% -- 5.22 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 5-2020\n",
      "Found acceptable images for 19598 points in 19.21 seconds\n",
      "Featurizing: 05-2020\n",
      "0/19598 -- 0.00% -- 0.93 seconds\n",
      "1000/19598 -- 5.10% -- 7.61 seconds\n",
      "2000/19598 -- 10.21% -- 4.74 seconds\n",
      "3000/19598 -- 15.31% -- 5.93 seconds\n",
      "4000/19598 -- 20.41% -- 6.59 seconds\n",
      "5000/19598 -- 25.51% -- 5.42 seconds\n",
      "6000/19598 -- 30.62% -- 6.46 seconds\n",
      "7000/19598 -- 35.72% -- 5.71 seconds\n",
      "8000/19598 -- 40.82% -- 4.82 seconds\n",
      "9000/19598 -- 45.92% -- 6.60 seconds\n",
      "10000/19598 -- 51.03% -- 6.67 seconds\n",
      "11000/19598 -- 56.13% -- 7.76 seconds\n",
      "12000/19598 -- 61.23% -- 5.60 seconds\n",
      "13000/19598 -- 66.33% -- 5.39 seconds\n",
      "14000/19598 -- 71.44% -- 4.17 seconds\n",
      "15000/19598 -- 76.54% -- 5.82 seconds\n",
      "16000/19598 -- 81.64% -- 5.41 seconds\n",
      "17000/19598 -- 86.74% -- 4.87 seconds\n",
      "18000/19598 -- 91.85% -- 4.30 seconds\n",
      "19000/19598 -- 96.95% -- 4.32 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 6-2020\n",
      "Found acceptable images for 19466 points in 19.83 seconds\n",
      "Featurizing: 06-2020\n",
      "0/19466 -- 0.00% -- 0.63 seconds\n",
      "1000/19466 -- 5.14% -- 4.14 seconds\n",
      "2000/19466 -- 10.27% -- 4.62 seconds\n",
      "3000/19466 -- 15.41% -- 4.32 seconds\n",
      "4000/19466 -- 20.55% -- 4.41 seconds\n",
      "5000/19466 -- 25.69% -- 4.04 seconds\n",
      "6000/19466 -- 30.82% -- 4.69 seconds\n",
      "7000/19466 -- 35.96% -- 4.53 seconds\n",
      "8000/19466 -- 41.10% -- 4.47 seconds\n",
      "9000/19466 -- 46.23% -- 4.45 seconds\n",
      "10000/19466 -- 51.37% -- 4.58 seconds\n",
      "11000/19466 -- 56.51% -- 4.82 seconds\n",
      "12000/19466 -- 61.65% -- 4.04 seconds\n",
      "13000/19466 -- 66.78% -- 4.89 seconds\n",
      "14000/19466 -- 71.92% -- 3.83 seconds\n",
      "15000/19466 -- 77.06% -- 4.55 seconds\n",
      "16000/19466 -- 82.19% -- 4.49 seconds\n",
      "17000/19466 -- 87.33% -- 4.18 seconds\n",
      "18000/19466 -- 92.47% -- 4.42 seconds\n",
      "19000/19466 -- 97.61% -- 4.29 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 7-2020\n",
      "Found acceptable images for 19544 points in 19.78 seconds\n",
      "Featurizing: 07-2020\n",
      "0/19544 -- 0.00% -- 0.74 seconds\n",
      "1000/19544 -- 5.12% -- 4.43 seconds\n",
      "2000/19544 -- 10.23% -- 4.30 seconds\n",
      "3000/19544 -- 15.35% -- 4.26 seconds\n",
      "4000/19544 -- 20.47% -- 4.37 seconds\n",
      "5000/19544 -- 25.58% -- 4.71 seconds\n",
      "6000/19544 -- 30.70% -- 4.67 seconds\n",
      "7000/19544 -- 35.82% -- 4.57 seconds\n",
      "8000/19544 -- 40.93% -- 4.73 seconds\n",
      "9000/19544 -- 46.05% -- 5.00 seconds\n",
      "10000/19544 -- 51.17% -- 5.27 seconds\n",
      "11000/19544 -- 56.28% -- 5.26 seconds\n",
      "12000/19544 -- 61.40% -- 4.53 seconds\n",
      "13000/19544 -- 66.52% -- 5.33 seconds\n",
      "14000/19544 -- 71.63% -- 3.80 seconds\n",
      "15000/19544 -- 76.75% -- 4.03 seconds\n",
      "16000/19544 -- 81.87% -- 4.15 seconds\n",
      "17000/19544 -- 86.98% -- 4.01 seconds\n",
      "18000/19544 -- 92.10% -- 4.32 seconds\n",
      "19000/19544 -- 97.22% -- 5.29 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 8-2020\n",
      "Found acceptable images for 19598 points in 19.23 seconds\n",
      "Featurizing: 08-2020\n",
      "0/19598 -- 0.00% -- 0.68 seconds\n",
      "1000/19598 -- 5.10% -- 4.00 seconds\n",
      "2000/19598 -- 10.21% -- 4.65 seconds\n",
      "3000/19598 -- 15.31% -- 4.66 seconds\n",
      "4000/19598 -- 20.41% -- 4.38 seconds\n",
      "5000/19598 -- 25.51% -- 4.12 seconds\n",
      "6000/19598 -- 30.62% -- 4.85 seconds\n",
      "7000/19598 -- 35.72% -- 4.68 seconds\n",
      "8000/19598 -- 40.82% -- 4.17 seconds\n",
      "9000/19598 -- 45.92% -- 5.15 seconds\n",
      "10000/19598 -- 51.03% -- 5.66 seconds\n",
      "11000/19598 -- 56.13% -- 5.82 seconds\n",
      "12000/19598 -- 61.23% -- 5.54 seconds\n",
      "13000/19598 -- 66.33% -- 6.03 seconds\n",
      "14000/19598 -- 71.44% -- 4.97 seconds\n",
      "15000/19598 -- 76.54% -- 4.97 seconds\n",
      "16000/19598 -- 81.64% -- 4.98 seconds\n",
      "17000/19598 -- 86.74% -- 5.29 seconds\n",
      "18000/19598 -- 91.85% -- 4.65 seconds\n",
      "19000/19598 -- 96.95% -- 5.93 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 9-2020\n",
      "Found acceptable images for 19598 points in 19.17 seconds\n",
      "Featurizing: 09-2020\n",
      "0/19598 -- 0.00% -- 1.24 seconds\n",
      "1000/19598 -- 5.10% -- 4.44 seconds\n",
      "2000/19598 -- 10.21% -- 6.37 seconds\n",
      "3000/19598 -- 15.31% -- 5.43 seconds\n",
      "4000/19598 -- 20.41% -- 5.27 seconds\n",
      "5000/19598 -- 25.51% -- 5.58 seconds\n",
      "6000/19598 -- 30.62% -- 7.08 seconds\n",
      "7000/19598 -- 35.72% -- 7.51 seconds\n",
      "8000/19598 -- 40.82% -- 6.75 seconds\n",
      "9000/19598 -- 45.92% -- 5.62 seconds\n",
      "10000/19598 -- 51.03% -- 8.35 seconds\n",
      "11000/19598 -- 56.13% -- 8.49 seconds\n",
      "12000/19598 -- 61.23% -- 7.81 seconds\n",
      "13000/19598 -- 66.33% -- 8.55 seconds\n",
      "14000/19598 -- 71.44% -- 5.79 seconds\n",
      "15000/19598 -- 76.54% -- 6.52 seconds\n",
      "16000/19598 -- 81.64% -- 5.23 seconds\n",
      "17000/19598 -- 86.74% -- 5.50 seconds\n",
      "18000/19598 -- 91.85% -- 6.41 seconds\n",
      "19000/19598 -- 96.95% -- 6.29 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 10-2020\n",
      "Found acceptable images for 19598 points in 19.62 seconds\n",
      "Featurizing: 10-2020\n",
      "0/19598 -- 0.00% -- 0.64 seconds\n",
      "1000/19598 -- 5.10% -- 3.95 seconds\n",
      "2000/19598 -- 10.21% -- 4.15 seconds\n",
      "3000/19598 -- 15.31% -- 4.31 seconds\n",
      "4000/19598 -- 20.41% -- 4.45 seconds\n",
      "5000/19598 -- 25.51% -- 3.82 seconds\n",
      "6000/19598 -- 30.62% -- 5.09 seconds\n",
      "7000/19598 -- 35.72% -- 4.91 seconds\n",
      "8000/19598 -- 40.82% -- 4.06 seconds\n",
      "9000/19598 -- 45.92% -- 5.40 seconds\n",
      "10000/19598 -- 51.03% -- 4.71 seconds\n",
      "11000/19598 -- 56.13% -- 4.90 seconds\n",
      "12000/19598 -- 61.23% -- 4.02 seconds\n",
      "13000/19598 -- 66.33% -- 5.32 seconds\n",
      "14000/19598 -- 71.44% -- 3.87 seconds\n",
      "15000/19598 -- 76.54% -- 4.32 seconds\n",
      "16000/19598 -- 81.64% -- 4.65 seconds\n",
      "17000/19598 -- 86.74% -- 4.50 seconds\n",
      "18000/19598 -- 91.85% -- 4.60 seconds\n",
      "19000/19598 -- 96.95% -- 4.08 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 11-2020\n",
      "Found acceptable images for 18651 points in 19.14 seconds\n",
      "Featurizing: 11-2020\n",
      "0/18651 -- 0.00% -- 0.64 seconds\n",
      "1000/18651 -- 5.36% -- 3.91 seconds\n",
      "2000/18651 -- 10.72% -- 4.18 seconds\n",
      "3000/18651 -- 16.08% -- 4.39 seconds\n",
      "4000/18651 -- 21.45% -- 4.47 seconds\n",
      "5000/18651 -- 26.81% -- 4.03 seconds\n",
      "6000/18651 -- 32.17% -- 4.60 seconds\n",
      "7000/18651 -- 37.53% -- 4.16 seconds\n",
      "8000/18651 -- 42.89% -- 5.04 seconds\n",
      "9000/18651 -- 48.25% -- 4.08 seconds\n",
      "10000/18651 -- 53.62% -- 5.36 seconds\n",
      "11000/18651 -- 58.98% -- 4.25 seconds\n",
      "12000/18651 -- 64.34% -- 5.14 seconds\n",
      "13000/18651 -- 69.70% -- 3.98 seconds\n",
      "14000/18651 -- 75.06% -- 4.32 seconds\n",
      "15000/18651 -- 80.42% -- 4.90 seconds\n",
      "16000/18651 -- 85.79% -- 4.23 seconds\n",
      "17000/18651 -- 91.15% -- 3.91 seconds\n",
      "18000/18651 -- 96.51% -- 4.23 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 12-2020\n",
      "Found acceptable images for 1572 points in 17.04 seconds\n",
      "Featurizing: 12-2020\n",
      "0/1572 -- 0.00% -- 0.68 seconds\n",
      "1000/1572 -- 63.61% -- 5.09 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Saving file as: data/sentinel-2-l2a_bands-3-4-5_ZMB_20k-points_1000-features_2020.feather\n",
      "Save finished!\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 1-2021\n",
      "Found acceptable images for 673 points in 17.23 seconds\n",
      "Featurizing: 01-2021\n",
      "0/673 -- 0.00% -- 0.58 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 2-2021\n",
      "Found acceptable images for 5058 points in 17.48 seconds\n",
      "Featurizing: 02-2021\n",
      "0/5058 -- 0.00% -- 0.58 seconds\n",
      "1000/5058 -- 19.77% -- 5.24 seconds\n",
      "2000/5058 -- 39.54% -- 4.77 seconds\n",
      "3000/5058 -- 59.31% -- 4.82 seconds\n",
      "4000/5058 -- 79.08% -- 4.47 seconds\n",
      "5000/5058 -- 98.85% -- 5.45 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 3-2021\n",
      "Found acceptable images for 8477 points in 18.39 seconds\n",
      "Featurizing: 03-2021\n",
      "0/8477 -- 0.00% -- 0.66 seconds\n",
      "1000/8477 -- 11.80% -- 4.05 seconds\n",
      "2000/8477 -- 23.59% -- 4.12 seconds\n",
      "3000/8477 -- 35.39% -- 4.41 seconds\n",
      "4000/8477 -- 47.19% -- 4.42 seconds\n",
      "5000/8477 -- 58.98% -- 5.81 seconds\n",
      "6000/8477 -- 70.78% -- 7.11 seconds\n",
      "7000/8477 -- 82.58% -- 6.02 seconds\n",
      "8000/8477 -- 94.37% -- 3.95 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 4-2021\n",
      "Found acceptable images for 15216 points in 19.38 seconds\n",
      "Featurizing: 04-2021\n",
      "0/15216 -- 0.00% -- 0.71 seconds\n",
      "1000/15216 -- 6.57% -- 4.24 seconds\n",
      "2000/15216 -- 13.14% -- 4.00 seconds\n",
      "3000/15216 -- 19.72% -- 3.93 seconds\n",
      "4000/15216 -- 26.29% -- 4.35 seconds\n",
      "5000/15216 -- 32.86% -- 5.12 seconds\n",
      "6000/15216 -- 39.43% -- 5.63 seconds\n",
      "7000/15216 -- 46.00% -- 4.87 seconds\n",
      "8000/15216 -- 52.58% -- 4.15 seconds\n",
      "9000/15216 -- 59.15% -- 4.98 seconds\n",
      "10000/15216 -- 65.72% -- 6.97 seconds\n",
      "11000/15216 -- 72.29% -- 4.93 seconds\n",
      "12000/15216 -- 78.86% -- 4.02 seconds\n",
      "13000/15216 -- 85.44% -- 4.35 seconds\n",
      "14000/15216 -- 92.01% -- 4.35 seconds\n",
      "15000/15216 -- 98.58% -- 4.47 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 5-2021\n",
      "Found acceptable images for 19544 points in 19.71 seconds\n",
      "Featurizing: 05-2021\n",
      "0/19544 -- 0.00% -- 0.76 seconds\n",
      "1000/19544 -- 5.12% -- 5.36 seconds\n",
      "2000/19544 -- 10.23% -- 5.33 seconds\n",
      "3000/19544 -- 15.35% -- 4.94 seconds\n",
      "4000/19544 -- 20.47% -- 4.94 seconds\n",
      "5000/19544 -- 25.58% -- 4.58 seconds\n",
      "6000/19544 -- 30.70% -- 7.24 seconds\n",
      "7000/19544 -- 35.82% -- 5.54 seconds\n",
      "8000/19544 -- 40.93% -- 4.54 seconds\n",
      "9000/19544 -- 46.05% -- 5.22 seconds\n",
      "10000/19544 -- 51.17% -- 5.76 seconds\n",
      "11000/19544 -- 56.28% -- 5.10 seconds\n",
      "12000/19544 -- 61.40% -- 4.66 seconds\n",
      "13000/19544 -- 66.52% -- 5.09 seconds\n",
      "14000/19544 -- 71.63% -- 4.10 seconds\n",
      "15000/19544 -- 76.75% -- 4.86 seconds\n",
      "16000/19544 -- 81.87% -- 5.28 seconds\n",
      "17000/19544 -- 86.98% -- 4.30 seconds\n",
      "18000/19544 -- 92.10% -- 4.51 seconds\n",
      "19000/19544 -- 97.22% -- 4.22 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 6-2021\n",
      "Found acceptable images for 19598 points in 19.55 seconds\n",
      "Featurizing: 06-2021\n",
      "0/19598 -- 0.00% -- 0.66 seconds\n",
      "1000/19598 -- 5.10% -- 4.04 seconds\n",
      "2000/19598 -- 10.21% -- 4.56 seconds\n",
      "3000/19598 -- 15.31% -- 4.17 seconds\n",
      "4000/19598 -- 20.41% -- 4.35 seconds\n",
      "5000/19598 -- 25.51% -- 4.01 seconds\n",
      "6000/19598 -- 30.62% -- 4.99 seconds\n",
      "7000/19598 -- 35.72% -- 4.88 seconds\n",
      "8000/19598 -- 40.82% -- 4.00 seconds\n",
      "9000/19598 -- 45.92% -- 4.42 seconds\n",
      "10000/19598 -- 51.03% -- 4.26 seconds\n",
      "11000/19598 -- 56.13% -- 4.29 seconds\n",
      "12000/19598 -- 61.23% -- 4.11 seconds\n",
      "13000/19598 -- 66.33% -- 4.41 seconds\n",
      "14000/19598 -- 71.44% -- 4.06 seconds\n",
      "15000/19598 -- 76.54% -- 4.28 seconds\n",
      "16000/19598 -- 81.64% -- 4.45 seconds\n",
      "17000/19598 -- 86.74% -- 4.06 seconds\n",
      "18000/19598 -- 91.85% -- 3.99 seconds\n",
      "19000/19598 -- 96.95% -- 4.69 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 7-2021\n",
      "Found acceptable images for 19544 points in 19.38 seconds\n",
      "Featurizing: 07-2021\n",
      "0/19544 -- 0.00% -- 0.64 seconds\n",
      "1000/19544 -- 5.12% -- 3.95 seconds\n",
      "2000/19544 -- 10.23% -- 4.57 seconds\n",
      "3000/19544 -- 15.35% -- 4.48 seconds\n",
      "4000/19544 -- 20.47% -- 4.42 seconds\n",
      "5000/19544 -- 25.58% -- 3.93 seconds\n",
      "6000/19544 -- 30.70% -- 4.63 seconds\n",
      "7000/19544 -- 35.82% -- 4.38 seconds\n",
      "8000/19544 -- 40.93% -- 4.16 seconds\n",
      "9000/19544 -- 46.05% -- 4.63 seconds\n",
      "10000/19544 -- 51.17% -- 4.20 seconds\n",
      "11000/19544 -- 56.28% -- 4.64 seconds\n",
      "12000/19544 -- 61.40% -- 4.17 seconds\n",
      "13000/19544 -- 66.52% -- 4.26 seconds\n",
      "14000/19544 -- 71.63% -- 3.86 seconds\n",
      "15000/19544 -- 76.75% -- 4.57 seconds\n",
      "16000/19544 -- 81.87% -- 4.29 seconds\n",
      "17000/19544 -- 86.98% -- 4.37 seconds\n",
      "18000/19544 -- 92.10% -- 4.15 seconds\n",
      "19000/19544 -- 97.22% -- 4.37 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 8-2021\n",
      "Found acceptable images for 19598 points in 19.57 seconds\n",
      "Featurizing: 08-2021\n",
      "0/19598 -- 0.00% -- 0.73 seconds\n",
      "1000/19598 -- 5.10% -- 4.08 seconds\n",
      "2000/19598 -- 10.21% -- 4.34 seconds\n",
      "3000/19598 -- 15.31% -- 3.99 seconds\n",
      "4000/19598 -- 20.41% -- 4.20 seconds\n",
      "5000/19598 -- 25.51% -- 3.74 seconds\n",
      "6000/19598 -- 30.62% -- 6.13 seconds\n",
      "7000/19598 -- 35.72% -- 4.95 seconds\n",
      "8000/19598 -- 40.82% -- 4.43 seconds\n",
      "9000/19598 -- 45.92% -- 5.14 seconds\n",
      "10000/19598 -- 51.03% -- 4.55 seconds\n",
      "11000/19598 -- 56.13% -- 4.14 seconds\n",
      "12000/19598 -- 61.23% -- 4.15 seconds\n",
      "13000/19598 -- 66.33% -- 4.28 seconds\n",
      "14000/19598 -- 71.44% -- 3.74 seconds\n",
      "15000/19598 -- 76.54% -- 4.34 seconds\n",
      "16000/19598 -- 81.64% -- 4.54 seconds\n",
      "17000/19598 -- 86.74% -- 4.31 seconds\n",
      "18000/19598 -- 91.85% -- 3.99 seconds\n",
      "19000/19598 -- 96.95% -- 4.27 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 9-2021\n",
      "Found acceptable images for 19598 points in 19.54 seconds\n",
      "Featurizing: 09-2021\n",
      "0/19598 -- 0.00% -- 0.77 seconds\n",
      "1000/19598 -- 5.10% -- 3.92 seconds\n",
      "2000/19598 -- 10.21% -- 4.51 seconds\n",
      "3000/19598 -- 15.31% -- 5.34 seconds\n",
      "4000/19598 -- 20.41% -- 4.97 seconds\n",
      "5000/19598 -- 25.51% -- 3.88 seconds\n",
      "6000/19598 -- 30.62% -- 5.81 seconds\n",
      "7000/19598 -- 35.72% -- 4.76 seconds\n",
      "8000/19598 -- 40.82% -- 4.93 seconds\n",
      "9000/19598 -- 45.92% -- 4.37 seconds\n",
      "10000/19598 -- 51.03% -- 5.82 seconds\n",
      "11000/19598 -- 56.13% -- 5.55 seconds\n",
      "12000/19598 -- 61.23% -- 4.03 seconds\n",
      "13000/19598 -- 66.33% -- 5.09 seconds\n",
      "14000/19598 -- 71.44% -- 4.00 seconds\n",
      "15000/19598 -- 76.54% -- 4.37 seconds\n",
      "16000/19598 -- 81.64% -- 4.54 seconds\n",
      "17000/19598 -- 86.74% -- 4.70 seconds\n",
      "18000/19598 -- 91.85% -- 3.96 seconds\n",
      "19000/19598 -- 96.95% -- 4.31 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 10-2021\n",
      "Found acceptable images for 17647 points in 19.16 seconds\n",
      "Featurizing: 10-2021\n",
      "0/17647 -- 0.00% -- 0.67 seconds\n",
      "1000/17647 -- 5.67% -- 4.32 seconds\n",
      "2000/17647 -- 11.33% -- 4.19 seconds\n",
      "3000/17647 -- 17.00% -- 4.06 seconds\n",
      "4000/17647 -- 22.67% -- 4.23 seconds\n",
      "5000/17647 -- 28.33% -- 3.86 seconds\n",
      "6000/17647 -- 34.00% -- 4.73 seconds\n",
      "7000/17647 -- 39.67% -- 4.40 seconds\n",
      "8000/17647 -- 45.33% -- 4.05 seconds\n",
      "9000/17647 -- 51.00% -- 4.99 seconds\n",
      "10000/17647 -- 56.67% -- 4.47 seconds\n",
      "11000/17647 -- 62.33% -- 4.26 seconds\n",
      "12000/17647 -- 68.00% -- 4.02 seconds\n",
      "13000/17647 -- 73.67% -- 4.75 seconds\n",
      "14000/17647 -- 79.33% -- 4.18 seconds\n",
      "15000/17647 -- 85.00% -- 3.64 seconds\n",
      "16000/17647 -- 90.67% -- 4.27 seconds\n",
      "17000/17647 -- 96.33% -- 4.01 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 11-2021\n",
      "Found acceptable images for 17990 points in 19.27 seconds\n",
      "Featurizing: 11-2021\n",
      "0/17990 -- 0.00% -- 0.68 seconds\n",
      "1000/17990 -- 5.56% -- 4.06 seconds\n",
      "2000/17990 -- 11.12% -- 5.84 seconds\n",
      "3000/17990 -- 16.68% -- 4.60 seconds\n",
      "4000/17990 -- 22.23% -- 4.39 seconds\n",
      "5000/17990 -- 27.79% -- 4.28 seconds\n",
      "6000/17990 -- 33.35% -- 4.68 seconds\n",
      "7000/17990 -- 38.91% -- 4.47 seconds\n",
      "8000/17990 -- 44.47% -- 4.56 seconds\n",
      "9000/17990 -- 50.03% -- 5.32 seconds\n",
      "10000/17990 -- 55.59% -- 4.49 seconds\n",
      "11000/17990 -- 61.15% -- 4.63 seconds\n",
      "12000/17990 -- 66.70% -- 4.32 seconds\n",
      "13000/17990 -- 72.26% -- 4.95 seconds\n",
      "14000/17990 -- 77.82% -- 4.48 seconds\n",
      "15000/17990 -- 83.38% -- 4.51 seconds\n",
      "16000/17990 -- 88.94% -- 4.12 seconds\n",
      "17000/17990 -- 94.50% -- 4.21 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Matching images to points for: 12-2021\n",
      "Found acceptable images for 17817 points in 18.73 seconds\n",
      "Featurizing: 12-2021\n",
      "0/17817 -- 0.00% -- 1.00 seconds\n",
      "1000/17817 -- 5.61% -- 3.94 seconds\n",
      "2000/17817 -- 11.23% -- 4.42 seconds\n",
      "3000/17817 -- 16.84% -- 4.88 seconds\n",
      "4000/17817 -- 22.45% -- 4.34 seconds\n",
      "5000/17817 -- 28.06% -- 4.18 seconds\n",
      "6000/17817 -- 33.68% -- 5.43 seconds\n",
      "7000/17817 -- 39.29% -- 4.16 seconds\n",
      "8000/17817 -- 44.90% -- 4.48 seconds\n",
      "9000/17817 -- 50.51% -- 5.26 seconds\n",
      "10000/17817 -- 56.13% -- 4.54 seconds\n",
      "11000/17817 -- 61.74% -- 5.52 seconds\n",
      "12000/17817 -- 67.35% -- 4.73 seconds\n",
      "13000/17817 -- 72.96% -- 4.61 seconds\n",
      "14000/17817 -- 78.58% -- 4.66 seconds\n",
      "15000/17817 -- 84.19% -- 4.07 seconds\n",
      "16000/17817 -- 89.80% -- 4.13 seconds\n",
      "17000/17817 -- 95.41% -- 4.89 seconds\n",
      "Freeing RAM\n",
      "\n",
      "Saving file as: data/sentinel-2-l2a_bands-3-4-5_ZMB_20k-points_1000-features_2021.feather\n",
      "Save finished!\n",
      "Freeing RAM\n",
      "\n",
      "CPU times: user 39min 37s, sys: 5min 14s, total: 44min 52s\n",
      "Wall time: 1h 47min 29s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "year_start = 2015\n",
    "year_end = 2021\n",
    "print(\n",
    "    \"Using:\\n\", \n",
    "    f\"  {satellite} imagery \\n\",\n",
    "    f\"  Looking for monthly images for {pt_len} points \\n\",\n",
    "    f\"  Calulating {num_features} features \\n\",\n",
    "    f\"  between {year_start} and {year_end} \\n\")\n",
    "for yr in range(year_start, year_end+1):\n",
    "    features = pd.DataFrame()\n",
    "    ft = []\n",
    "    if (yr == 2015) :\n",
    "        month_range = range(7, 13)\n",
    "    else:\n",
    "        month_range = range(1, 13)\n",
    "        \n",
    "    for mn in month_range:\n",
    "        if mn < 10:\n",
    "            month = \"0\"+str(mn)\n",
    "        else:\n",
    "            month = mn\n",
    "        def query(points):\n",
    "            \"\"\"\n",
    "            Find a STAC item for points in the `points` DataFrame\n",
    "            \n",
    "            Parameters\n",
    "            ----------\n",
    "            points : geopandas.GeoDataFrame\n",
    "                A GeoDataFrame\n",
    "                \n",
    "            Returns\n",
    "            -------\n",
    "            geopandas.GeoDataFrame\n",
    "                A new geopandas.GeoDataFrame with a `stac_item` column containing the STAC\n",
    "                item that covers each point.\n",
    "            \"\"\"\n",
    "            intersects = shapely.geometry.mapping(points.unary_union.convex_hull)\n",
    "            \n",
    "            catalog = pystac_client.Client.open(\n",
    "                \"https://planetarycomputer.microsoft.com/api/stac/v1\"\n",
    "            )\n",
    "            # Define search date range for query\n",
    "            ending_day = calendar.monthrange(yr, int(mn))[1]\n",
    "            search_start = f\"{yr}-{month}-1\" \n",
    "            search_end = f\"{yr}-{month}-{ending_day}\" \n",
    "            \n",
    "            # The time frame in which we search for non-cloudy imagery\n",
    "            search = catalog.search(\n",
    "                collections=[satellite],\n",
    "                intersects=intersects,\n",
    "                datetime=[search_start, search_end],\n",
    "                query={\"eo:cloud_cover\": {\"lt\": 10}},\n",
    "                limit=500,\n",
    "            )\n",
    "            ic = search.get_all_items_as_dict()\n",
    "            features = ic[\"features\"]\n",
    "            features_d = {item[\"id\"]: item for item in features}\n",
    "            data = {\n",
    "                \"eo:cloud_cover\": [],\n",
    "                \"geometry\": [],\n",
    "            }\n",
    "            index = []\n",
    "            for item in features:\n",
    "                data[\"eo:cloud_cover\"].append(item[\"properties\"][\"eo:cloud_cover\"])\n",
    "                data[\"geometry\"].append(shapely.geometry.shape(item[\"geometry\"]))\n",
    "                index.append(item[\"id\"])\n",
    "            items = geopandas.GeoDataFrame(data, index=index, geometry=\"geometry\").sort_values(\n",
    "                \"eo:cloud_cover\"\n",
    "            )\n",
    "            point_list = points.geometry.tolist()\n",
    "            point_items = []\n",
    "            for point in point_list:\n",
    "                covered_by = items[items.covers(point)]\n",
    "                if len(covered_by):\n",
    "                    point_items.append(features_d[covered_by.index[0]])\n",
    "                else:\n",
    "                    # There weren't any scenes matching our conditions for this point (too cloudy)\n",
    "                    point_items.append(None)\n",
    "            return points.assign(stac_item=point_items)\n",
    "        \n",
    "        tic = time.time()\n",
    "        print(\"Matching images to points for: \", mn, \"-\", yr, sep = \"\")\n",
    "        \n",
    "        with Client(n_workers=16) as client:\n",
    "            meta = dgdf._meta.assign(stac_item=[])\n",
    "            df2 = dgdf.map_partitions(query, meta=meta).compute()\n",
    "        df3 = df2.dropna(subset=[\"stac_item\"])\n",
    "        matching_urls = [\n",
    "            pc.sign(item[\"assets\"][\"visual\"][\"href\"]) for item in df3.stac_item.tolist()\n",
    "        ]\n",
    "        points = df3[[\"lon\", \"lat\"]].to_numpy()\n",
    "        \n",
    "        print(\"Found acceptable images for \", \n",
    "              points.shape[0], \n",
    "              \" points in \", \n",
    "              f\"{time.time()-tic:0.2f} seconds\", \n",
    "              sep = \"\")\n",
    "        \n",
    "        class CustomDataset(Dataset):\n",
    "            def __init__(self, points, fns, buffer=0.005):\n",
    "                self.points = points\n",
    "                self.fns = fns\n",
    "                self.buffer = buffer\n",
    "            def __len__(self):\n",
    "                return self.points.shape[0]\n",
    "            def __getitem__(self, idx):\n",
    "                lon, lat = self.points[idx]\n",
    "                fn = self.fns[idx]\n",
    "                if fn is None:\n",
    "                    return None\n",
    "                else:\n",
    "                    point_geom = shapely.geometry.mapping(shapely.geometry.Point(lon, lat))\n",
    "                    with rasterio.Env():\n",
    "                        with rasterio.open(fn, \"r\") as f:\n",
    "                            point_shape = shapely.geometry.shape(point_geom)\n",
    "                            mask_shape = point_shape.buffer(self.buffer).envelope\n",
    "                            mask_geom = shapely.geometry.mapping(mask_shape)\n",
    "                            mask_geom = rasterio.warp.transform_geom(\n",
    "                                \"epsg:4326\", f.crs.to_string(), mask_geom\n",
    "                            )\n",
    "                            try:\n",
    "                                out_image, out_transform = rasterio.mask.mask(\n",
    "                                    f, [mask_geom], crop=True\n",
    "                                )\n",
    "                            except ValueError as e:\n",
    "                                if \"Input shapes do not overlap raster.\" in str(e):\n",
    "                                    return None\n",
    "                    out_image = out_image / 255.0\n",
    "                    out_image = torch.from_numpy(out_image).float()\n",
    "                    return out_image\n",
    "        dataset = CustomDataset(points, matching_urls)\n",
    "        dataloader = DataLoader(\n",
    "            dataset,\n",
    "            batch_size=8,\n",
    "            shuffle=False,\n",
    "            num_workers=os.cpu_count()*2,\n",
    "            collate_fn=lambda x: x,\n",
    "            pin_memory=False,\n",
    "        )\n",
    "        x_all = np.zeros((points.shape[0], num_features), dtype=float)\n",
    "        tic = time.time()\n",
    "        i = 0\n",
    "        print(\"Featurizing: \", month, \"-\", yr, sep = \"\")\n",
    "        for images in dataloader:\n",
    "            for image in images:\n",
    "                if i % 1000 == 0:\n",
    "                    print(\n",
    "                        f\"{i}/{points.shape[0]} -- {i / points.shape[0] * 100:0.2f}%\"\n",
    "                        + f\" -- {time.time()-tic:0.2f} seconds\"\n",
    "                    )\n",
    "                    tic = time.time()\n",
    "                if image is not None:\n",
    "                    # A full image should be ~101x101 pixels (i.e. ~1km^2 at a 10m/px spatial\n",
    "                    # resolution), however we can receive smaller images if an input point\n",
    "                    # happens to be at the edge of a S2 scene (a literal edge case). To deal\n",
    "                    # with these (edge) cases we crudely drop all images where the spatial\n",
    "                    # dimensions aren't both greater than 20 pixels.\n",
    "                    if image.shape[1] >= 20 and image.shape[2] >= 20:\n",
    "                        image = image.to(device)\n",
    "                        with torch.no_grad():\n",
    "                            feats = model(image.unsqueeze(0)).cpu().numpy()\n",
    "                        x_all[i] = feats\n",
    "                    else:\n",
    "                        # this happens if the point is close to the edge of a scene\n",
    "                        # (one or both of the spatial dimensions of the image are very small)\n",
    "                        pass\n",
    "                else:\n",
    "                    pass  # this happens if we do not find a S2 scene for some point\n",
    "                i += 1\n",
    "        features_monthly = pd.DataFrame(x_all)\n",
    "        features_monthly[[\"lon\", \"lat\"]] = points.tolist()\n",
    "        features_monthly['year'] = yr\n",
    "        features_monthly['month'] = mn\n",
    "        \n",
    "        ft.append(features_monthly)\n",
    "        \n",
    "        # Free memory before loop iterates\n",
    "        print(\"Freeing RAM\")\n",
    "        del meta\n",
    "        del query\n",
    "        del df2\n",
    "        del df3\n",
    "        del points\n",
    "        del dataset\n",
    "        del dataloader\n",
    "        del x_all\n",
    "        del features_monthly\n",
    "        del CustomDataset\n",
    "        del matching_urls\n",
    "        gc.collect()\n",
    "        print('')\n",
    "        \n",
    "    features = pd.concat(ft).reset_index(drop = True)\n",
    "    \n",
    "    features.columns = features.columns.astype(str)\n",
    "    # Save the features to a feather file\n",
    "    file_name = (f'data/{satellite}_bands-3-4-5_{country_code}_{pt_len/1000:.0f}'+\n",
    "                 f'k-points_{num_features}-features_{yr}.feather')\n",
    "    \n",
    "    print(\"Saving file as:\", file_name)\n",
    "    features.to_feather(file_name)\n",
    "    \n",
    "    # display(FileLink(file_name))\n",
    "    \n",
    "    print(\"Save finished!\")\n",
    "    # Free memory before loop iterates\n",
    "    print(\"Freeing RAM\")\n",
    "    del features\n",
    "    del ft\n",
    "    gc.collect()\n",
    "    print('')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}