Using Generic Mapping Tools (GMT) binaries from conda once more now that 6.0.0rc1 is released at https://github.com/GenericMappingTools/gmt/releases/tag/6.0.0rc1! Basically we're reverting the messy manual GMT compile code in our Dockerfile introduced in ee9df38, so less maintainence cost going forward. See actual changes at GenericMappingTools/gmt@20c95af...6.0.0rc1. The Python conda package had to be updated from 3.6.6 to 3.6.7 because the GMT 6.0.0rc1 conda package requires openssl >=1.1.1a, < 1.1.2a. Also had to update PyGMT (personal fork with grdtrack, see changes at weiji14/pygmt@weiji14:0.0.1a0-36-gb1e7b75...0.0.1a0-40-g29220bd) to include the 6.0.0rc1 version fix by GenericMappingTools/pygmt#311.

Creating new groundtruth NetCDF grids using GMT surface, replacing the ones last created in b90bd74 in #112. Besides having updated to the GMT 6.0.0rc1 tagged release, the main change here is with using nicely rounded bounds (to 250 units in EPSG:3031) instead of arbitrary decimal points. This will really help resolve some of the problems with points not being included in our RMSE_test calculations near the grid's edges (see #152), and integer coordinates are just nicer to debug won't you say?

Specifically, the data_prep.get_region was refactored to use `gmt info -I xxx.csv` instead of pure pandas, returning an `xmin/xmax/ymin/ymax` string that has an extended region optimized for `gmt surface`. There is a "surface [WARNING]: Your grid dimensions are mutually prime.  Convergence is very unlikely" which I'm just gonna ignore for now. Note that data_prep.ascii_to_xyz was one-line patched to drop NaNs as there were some points in the WISE_ISODYN_RadarByFlight.XYZ file with missing elevation (z) values (since #112...) that was messing up gmt.info in the refactored data_prep.get_region. Unit tests have been modified accordingly, and the grids in the integration tests are now downloaded/created in folder /tmp to avoid messing with the actual files in highres. Matplotlib plots of the grids in data_prep.ipynb have been updated, and the new grids will be released in v0.9.2.

Chipping out new training tiles from the '250 rounded' new grids. We are down from 2347 to 2275 tiles now... The GeoJSON bounding box polygons have been updated, and new quilt data hash to use now is 0734959aa4f4903a17ed2acdfd53b3c0c826aadfc718e5fdd3c1b04963e1206e.

Recalculating the RMSE_test value of our v0.9.0 trained model (at https://www.comet.ml/weiji14/deepbedmap/0b9b232394da42e394998b112f628696) on the new 250 rounded grids produced in 0681bf3. Basically comparing against the results reported in 75266fc. We now calculate elevation errors on 2007tx.nc using the full 42995 groundtruth points instead of just 37829 before, and the RMSE_test is now 44.34 which is up from 43.58 before. Good to see that results are comparable, but it does highlight how the border predictions need more optimization work. Also added a neat new functionality to the _download_deepbedmap_model_weights_from_comet function, downloading any trained .npz model weights via an experiment_key!

Closes #155 Re-grid and re-tile groundtruth data with coordinates rounded to 250.

Bumps [jupyterlab](https://github.com/jupyterlab/jupyterlab) from 1.0.0a5 to 1.0.0a10. Been meaning to update as the newline rendering was driving me crazy, since fixed in jupyterlab/jupyterlab#6513.
- [Release notes](https://github.com/jupyterlab/jupyterlab/releases/tag/v1.0.0a10)
- [Changelog](https://github.com/jupyterlab/jupyterlab/blob/master/docs/source/getting_started/changelog.rst)
- [Commits](jupyterlab/jupyterlab@v1.0.0a5...v1.0.0a10)

Enable more concurrent runs on 4 GPUs instead of just 2! In order to do so srgan_train.get_deepbedmap_test_result was refactored in such a way that we do not risk saving and reloading the model weights to/from the same file (that might actually clash when 2 processes are running, more so if 4 are doing so)! The best RMSE_test result achieved from the last hyperparameter tuning frenzy is 29.90 at https://www.comet.ml/weiji14/deepbedmap/fd658ce06e81492ea5a6f4b5e1afa028 but that might be severely overfitted, 2nd best achieved is 38.50 at https://www.comet.ml/weiji14/deepbedmap/abc3af8e9abc4080a6b5b44b33c537c2 which might be the one we'll actually use.

This commit extends the dual-GPU functionality introduced in a2866b6. Since our GPUs are on different servers (2x Tesla V100s on tara, 2x Tesla P100s on kahutea), this relies on having GMT==6.0.0rc1 installed from conda (1edb16e) instead of compiling from source as the latter would mean GMT can only work on one server. The model weights are saved to unique temporary folders while training and are not actually loaded by get_deepbedmap_test_result (i.e. we just use the model directly since it is trained already...). Also made up a different TPE seed for each device/GPU based on len(hostname) + $CUDA_VISIBLE_DEVICE, very hacky I know. There were a lot of hyperparameter settings I've tried over the weekend on the new quilt hash 0734959aa4f4903a17ed2acdfd53b3c0c826aadfc718e5fdd3c1b04963e1206e training tiles. The final tuning frenzy involved ~25 experiments each on 4 GPUs with this configuration: residual_scaling between 0.15 and 0.30, learning_rate between 6.5e-5 and 8.5e-5, num_epochs between 60 and 90. These floating point hyperparameters are actually a problem for Optuna, see https://0.30000000000000004.com/.

So that we don't have to worry about using the wrong Generator model hyperparameter settings, we now directly download the ESRGAN model's weights and hyperparameter information from Comet.ML in deepbedmap.ipynb! This commit builds upon 77b4fe1 where we refactored features/environment.py's _download_deepbedmap_model_weights_from_comet to get an arbitrary Comet.ML experiment model's weights. The function's name is now shortened to _download_model_weights_from_comet, and made to return num_residual_blocks and residual_scaling hyperparameters so we know how to build the model.

Also updated snapshot on 2007tx.nc test area prediction (see previous ones at 77b4fe1 and 75266fc) using the trained model at https://www.comet.ml/weiji14/deepbedmap/abc3af8e9abc4080a6b5b44b33c537c2 giving an RMSE_test of 38.50.

Not sure why the deepbedmap.feature integration is failing (just gets stuck for >10min) since it works on the server and even on my old laptop in a docker container! Maybe because we're downloading the .npz model weights file twice? Removed the download part in the deepbedmap integration test fixture. Also re-upload test tiles covering the new 2007tx.nc rounded to 250 area.

Quick update of Pine Island Glacier prediction from e8ae274. By evaluating on the new rounded grids, the bicubic baseline RMSE value has dropped from 72.66 to 67.12 now, making our new ESRGAN model's RMSE of 63.46 look pretty insignificant. The 2007tx/2010tr/istarxx grid combination has a new slice now, and since they all have nicely rounded coordinates, we can pygmt.grdtrack the merged xarray.DataArray grid with all the points in one go! This opens up the possibility to evaluate on other groundtruth tracks crossing the Pine Island Glacier area, but having peeked at those results, I feel like there's really really really a lot more work to do...

New DeepBedMap DEM! Compare this version with the EGU version at 7a5d223 or the better v0.8.0 version at 58d8ebd. There's been a couple of hacks needed to get the full continent to come out nicely, notably by clipping the MEASURES_Ice_Velocity/W2_tile layer to a minimum value of 0.0. The newer data_prep.selective_tile script from 4a074d9 was too memory and CPU intensive for handling REMA/W1_tile (believe me, I've tried dask.distributed, all sorts of parallelization, etc on a 80 core, 200GB RAM server) so we're bringing back the old selective_tile just for that crazy layer.

Also note that the refactored data_prep.selective_tile function's gapfill_raster_filepath is now renamed to 'gapfiller' as it can take in either a string filepath to a raster file, or a floating point number to be used to fill in the blank spaces! In deepbedmap.ipynb, we use this 'gapfiller' to arbitrarily fill in BEDMAP2/X_tile with -5000.0 and Arthern Accumulation/W3_tile with 0.0, noting though that this is just for cosmetic purposes as data gaps are in the ocean area outside of DeepBedMap's intended domain (within the grounding line). Another tweak we've made on the cosmetic front is in changing the colormap from BrBG_r to Blues_r that fits with the one on the README (produced using QGIS with an additional hillshading layer), oh and yes, we've updated the README.md DeepBedMap DEM snapshot too!

Closes #156 Streamline DeepBedMap model tuning, evaluation and Antarctic-wide DEM generation.

Release v0.9.2. So it's gonna be forever, or it's gonna go down in flames!