Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: OSM Tile Loader #217

Merged
merged 42 commits into from
Apr 27, 2023
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
aff8645
osm tile loader
mprzymus Mar 28, 2023
277a828
slippy map regionizer
mprzymus Mar 29, 2023
ee579c5
data collectors to only process data
mprzymus Mar 29, 2023
5d645f0
remove commented code
mprzymus Mar 29, 2023
c02cd3f
osm tile loader using slippy map regionizer
mprzymus Mar 29, 2023
3130584
result to dataframe
mprzymus Mar 29, 2023
267dc12
fix(pre-commit.ci): auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 30, 2023
6edfb1a
fix code check issues
mprzymus Mar 30, 2023
ef8f844
Fix ruff
mprzymus Mar 30, 2023
fdc5fb3
osm tile loader example
mprzymus Mar 30, 2023
a4eba0e
slippy map example
mprzymus Mar 30, 2023
81ad014
union instead of |
mprzymus Mar 31, 2023
b2c8907
fix: fixed unhashable error
mprzymus Apr 2, 2023
4744a1e
refactor: review style issues for tile loader
mprzymus Apr 4, 2023
e5cdf57
refactor: updated examples
mprzymus Apr 4, 2023
7c7b419
refactor: review issues for regionizer
mprzymus Apr 4, 2023
d46ce86
refactor: using path and file_extension renaming
mprzymus Apr 4, 2023
1fdcdb4
refactor: integrate with other loaders interface
mprzymus Apr 4, 2023
6aee6d3
fix: typing Tuple instead of tuple
mprzymus Apr 4, 2023
4ec779c
fix: missed 3.9 typing
mprzymus Apr 4, 2023
00b351a
fix: typing
mprzymus Apr 5, 2023
88802b9
refactor: return in refurb manner
mprzymus Apr 5, 2023
24fda5f
build: pyproject.toml updated
mprzymus Apr 5, 2023
5e61810
feat: string id, x, y, z as columns
mprzymus Apr 5, 2023
b5a9b2a
feat: new indexing in loader
mprzymus Apr 5, 2023
705d8a3
docs: updated examples
mprzymus Apr 5, 2023
b5ef3d8
refactor: imperative in docs
Apr 12, 2023
a48537f
refactor: style issues from review
Apr 14, 2023
c514f68
docs: add tiles to changelog
mprzymus Apr 17, 2023
b8ac122
refactor: dict function
mprzymus Apr 17, 2023
16952b3
build: regenerate lock to make tests pass
mprzymus Apr 19, 2023
d88f037
refactor: docs and example
mprzymus Apr 24, 2023
5ac7755
Merge branch 'main' into main
RaczeQ Apr 26, 2023
1529899
fix(pre-commit.ci): auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 26, 2023
5faa658
fix: add OSMTileLoader to init.py
mprzymus Apr 26, 2023
e9f969f
refactor: requests and pillow as optional
mprzymus Apr 26, 2023
593b590
refactor: osm group
mprzymus Apr 26, 2023
e18add0
Merge branch 'main' into main
Calychas Apr 26, 2023
2516bd9
fix: add import OSMTileLoader to __init__.py of Loaders
Calychas Apr 26, 2023
a035321
test: add OSMTileLoader and others to optional's _test_osm()
Calychas Apr 26, 2023
d109810
refactor: move requests to main dependencies
mprzymus Apr 27, 2023
630da0e
refactor: remove requests from import optional
mprzymus Apr 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- SlippyMapRegionizer
- OSMTileLoader
- GTFS Loader from gtfs2vec paper
- GTFS2Vec Model from gtfs2vec paper
- GTFS2Vec Embedder using gtfs2vec model
Expand Down
114 changes: 114 additions & 0 deletions examples/loaders/osm_tile_loader.ipynb
piotrgramacki marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
{
Calychas marked this conversation as resolved.
Show resolved Hide resolved
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# OSM Tile Loader"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from srai.loaders.osm_loaders import OSMTileLoader\n",
"from srai.utils import geocode_to_region_gdf\n",
"\n",
"ZOOM = 9"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Get tiles from tile server"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to get tiles use load method. Method `load` returns `GeoDataFrame` object with images or path to them, as chosen data_collector. Returned frame contains also geographical coordinates of tile and x, y, z coordinates used by tile servers."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"loader = OSMTileLoader(\"https://tile.openstreetmap.de\", zoom=ZOOM, verbose=True)\n",
"gdf = geocode_to_region_gdf(\"Wroclaw, Poland\")\n",
"tiles = loader.load(gdf)\n",
"tiles"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"To get records conveniently use SlippyMapId"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"tiles.loc[f\"279_170_{ZOOM}\"][\"tile\"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Get one tile"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"loader = OSMTileLoader(\"https://tile.openstreetmap.de\", zoom=2)\n",
"tile = loader.get_tile_by_x_y(2, 1)\n",
"tile"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.10.9 ('nlp_venv')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.10"
},
"vscode": {
"interpreter": {
"hash": "685727852e42551632e0fc1e43e361d886b7ddaddaf1474b2acdf6903161a758"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
129 changes: 129 additions & 0 deletions examples/regionizers/slippy_map_regionizer.ipynb
piotrgramacki marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
{
Calychas marked this conversation as resolved.
Show resolved Hide resolved
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Slippy Map Regionizer Example"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import geopandas as gpd\n",
"from shapely import geometry\n",
"\n",
"from srai.regionizers import SlippyMapRegionizer\n",
"from srai.plotting.folium_wrapper import plot_regions\n",
"from srai.constants import WGS84_CRS"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define starting polygons"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"gdf = gpd.GeoDataFrame(\n",
" geometry=[\n",
" geometry.Polygon(\n",
" shell=[\n",
" (-1, 0),\n",
" (0, 0.5),\n",
" (1, 0),\n",
" (1, 1),\n",
" (0, 1),\n",
" ],\n",
" holes=[\n",
" [\n",
" (0.8, 0.9),\n",
" (0.9, 0.55),\n",
" (0.8, 0.3),\n",
" (0.5, 0.4),\n",
" ]\n",
" ],\n",
" ),\n",
" geometry.Polygon(shell=[(-0.25, 0), (0.25, 0), (0, 0.2)]),\n",
" ],\n",
" crs=WGS84_CRS,\n",
")\n",
"gdf.explore()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define tile resolution"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"zoom = 12"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Regionize using SlippyMapRegionizer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"regionizer = SlippyMapRegionizer(zoom)\n",
"gdf_slippy = regionizer.transform(gdf)\n",
"\n",
"folium_map = gdf.explore()\n",
"plot_regions(gdf_slippy, colormap=[\"red\"], map=folium_map)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.10.10 ('.venv': venv)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.16"
},
"vscode": {
"interpreter": {
"hash": "bef431f694ba250a61f6c29f974e05c16577f910a5abf521c495f491290a7f34"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading