feat: replace logic in OsmPbfLoader from osmium to QuackOSM

CHANGELOG.md

@@ -14,6 +14,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- Refactored `PbfFileHandler` to use `DuckDB` with `spatial` extension instead of `osmium` and `GDAL` [#405](https://github.com/kraina-ai/srai/pull/405)
+- Changed the default pbf download source from `protomaps` download service to `geofabrik`.
+
 ### Deprecated
 
 ### Removed

examples/loaders/osm_pbf_loader.ipynb

@@ -303,6 +303,198 @@
     "\n",
     "ax.set_axis_off()"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Using OSMPbfLoader to download data for a specific area and transforming it to GeoParquet file"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Download all grouped features based on Geofabrik layers in Reykjavík, Iceland"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "loader = OSMPbfLoader()\n",
+    "reykjavik_gdf = geocode_to_region_gdf(\"Reykjavík, IS\")\n",
+    "reykjavik_features_gpq = loader.load_to_geoparquet(reykjavik_gdf, GEOFABRIK_LAYERS)\n",
+    "reykjavik_features_gpq"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Read those features using DuckDB"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import duckdb\n",
+    "\n",
+    "connection = duckdb.connect()\n",
+    "\n",
+    "connection.load_extension(\"parquet\")\n",
+    "connection.load_extension(\"spatial\")\n",
+    "\n",
+    "features_relation = connection.read_parquet([str(path) for path in reykjavik_features_gpq]).project(\n",
+    "    \"* REPLACE (ST_GeomFromWKB(geometry) AS geometry)\"\n",
+    ")\n",
+    "features_relation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Count all buildings"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "features_relation.filter(\"buildings IS NOT NULL\").count(\"feature_id\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Download main roads for Estonia using `PbfFileHandler`\n",
+    "\n",
+    "`PbfFileHandler` is a special class dedicated for reading `*.osm.pbf` files and is used by `OSMPbfLoader` to load features.\n",
+    "It allows for parsing `pbf` files without any tags or geometry filtering, that is automatically applied by `OSMPbfLoader`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "highways_filter = {\n",
+    "    \"highway\": [\n",
+    "        \"motorway\",\n",
+    "        \"trunk\",\n",
+    "        \"primary\",\n",
+    "        \"secondary\",\n",
+    "        \"tertiary\",\n",
+    "    ]\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from srai.loaders import download_file\n",
+    "from srai.loaders.osm_loaders.pbf_file_handler import PbfFileHandler\n",
+    "\n",
+    "estonia_pbf_url = \"http://download.geofabrik.de/europe/estonia-latest.osm.pbf\"\n",
+    "estonia_pbf_file = \"estonia-latest.osm.pbf\"\n",
+    "download_file(estonia_pbf_url, estonia_pbf_file, force_download=False)\n",
+    "\n",
+    "handler = PbfFileHandler(\n",
+    "    geometry_filter=None, tags_filter=highways_filter\n",
+    ")  # parsing pbf file without tags filtering requires a lot of memory available in the system\n",
+    "estonia_features_gpq = handler.convert_pbf_to_gpq(estonia_pbf_file)\n",
+    "estonia_features_gpq"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "features_relation = connection.read_parquet(str(estonia_features_gpq)).project(\n",
+    "    \"* REPLACE (ST_GeomFromWKB(geometry) AS geometry)\"\n",
+    ")\n",
+    "features_relation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Count loaded roads"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "features_relation.count(\"feature_id\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Calculate roads length\n",
+    "We will transform the geometries to the Estonian CRS - [EPSG:3301](https://epsg.io/3301)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "length_in_meters = (\n",
+    "    features_relation.project(\n",
+    "        \"ST_Length(ST_Transform(geometry, 'EPSG:4326', 'EPSG:3301')) AS road_length\"\n",
+    "    )\n",
+    "    .sum(\"road_length\")\n",
+    "    .fetchone()[0]\n",
+    ")\n",
+    "length_in_km = length_in_meters / 1000\n",
+    "length_in_km"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Plot the roads using GeoPandas\n",
+    "\n",
+    "With fast loading of geoparuqet files using `geoarrow.pyarrow` library."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import geoarrow.pyarrow as ga\n",
+    "from geoarrow.pyarrow import io\n",
+    "\n",
+    "from srai.constants import GEOMETRY_COLUMN\n",
+    "\n",
+    "parquet_table = io.read_geoparquet_table(estonia_features_gpq)\n",
+    "ga.to_geopandas(parquet_table.column(GEOMETRY_COLUMN)).plot()"
+   ]
   }
  ],
  "metadata": {