Skip to content

Conversation

@zstadler
Copy link
Contributor

@zstadler zstadler commented Jan 1, 2026

Add a new geometry: relation_members option to YAML custom maps that allows processing individual members of OSM relations as separate features. Instead of emitting one feature per relation, this emits one feature per qualifying member, using each member's geometry and tags.

Key features:

  • Select relations using existing include_when/exclude_when filters
  • Emit one feature per qualifying member (way/node)
  • Filter members by type, role, and tags (with inline script support)
  • Add member-level attributes from member tags
  • Support inline script expressions for member filtering and attributes

New configuration fields:

  • member_types: Filter members by OSM element type (node, way, relation)
  • member_roles: Filter members by their role in the relation
  • member_include_when: Filter members by their tags (structured or inline script)
  • member_exclude_when: Exclude members by their tags (structured or inline script)
  • member_attributes: Add attributes from member tags (with inline script support)

Implementation follows the multipolygon processing pattern:

  • Phase 1: Identify relations matching relation_members features
  • Phase 2: Store member geometries/tags, then process relations to emit member features

Includes comprehensive validation:

  • Member fields can only be used with relation_members geometry
  • member_types values must be valid OSM element types
  • Clear error messages for invalid configurations

Resolves #1426

…members

Add a new `geometry: relation_members` option to YAML custom maps that
allows processing individual members of OSM relations as separate features.
Instead of emitting one feature per relation, this emits one feature per
qualifying member, using each member's geometry and tags.

Key features:
- Select relations using existing include_when/exclude_when filters
- Emit one feature per qualifying member (way/node)
- Filter members by type, role, and tags (with inline script support)
- Add member-level attributes from member tags
- Support inline script expressions for member filtering and attributes

New configuration fields:
- member_types: Filter members by OSM element type (node, way, relation)
- member_roles: Filter members by their role in the relation
- member_include_when: Filter members by their tags (structured or inline script)
- member_exclude_when: Exclude members by their tags (structured or inline script)
- member_attributes: Add attributes from member tags (with inline script support)

Implementation follows the multipolygon processing pattern:
- Phase 1: Identify relations matching relation_members features
- Phase 2: Store member geometries/tags, then process relations to emit member features

Includes comprehensive validation:
- Member fields can only be used with relation_members geometry
- member_types values must be valid OSM element types
- Clear error messages for invalid configurations

Resolves onthegomap#1426
@github-actions
Copy link

github-actions bot commented Jan 1, 2026

This Branch 7174c4b Base 6eb3ab5
0:01:11 DEB [archive] - Tile stats:
0:01:11 DEB [archive] - Biggest tiles (gzipped)
1. 14/4942/6092 (160k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.40015 (poi:88k)
2. 9/154/190 (148k) https://onthegomap.github.io/planetiler-demo/#9.5/41.77078/-71.36719 (landcover:85k)
3. 10/308/381 (137k) https://onthegomap.github.io/planetiler-demo/#10.5/41.63994/-71.54297 (landcover:72k)
4. 10/308/380 (137k) https://onthegomap.github.io/planetiler-demo/#10.5/41.90214/-71.54297 (landcover:66k)
5. 14/4941/6092 (117k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.42212 (poi:69k)
6. 14/4941/6093 (117k) https://onthegomap.github.io/planetiler-demo/#14.5/41.81227/-71.42212 (building:62k)
7. 14/4940/6092 (101k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.44409 (building:92k)
8. 14/4946/6112 (99k) https://onthegomap.github.io/planetiler-demo/#14.5/41.50035/-71.31226 (building:60k)
9. 11/616/762 (99k) https://onthegomap.github.io/planetiler-demo/#11.5/41.7057/-71.63086 (landcover:71k)
10. 11/616/764 (97k) https://onthegomap.github.io/planetiler-demo/#11.5/41.44269/-71.63086 (landcover:74k)
0:01:11 DEB [archive] - Max tile sizes
                      z0    z1    z2    z3    z4    z5    z6    z7    z8    z9   z10   z11   z12   z13   z14   all
           boundary  151   336   409   544   872   350   464   571   808  1.6k    2k  6.8k  6.2k  5.6k  4.4k  6.8k
              water 7.7k  3.7k  8.6k  5.5k  2.6k  5.1k   15k   18k   16k   26k   15k   13k   17k   15k   12k   26k
              place    0     0   487   487   487   707   796    1k  1.6k  3.1k    6k  3.6k  1.8k   845   978    6k
            landuse    0     0     0     0   549   695  1.6k  6.7k   17k   44k   58k   49k   38k   19k   12k   58k
     transportation    0     0     0     0   418   968  1.4k  4.5k  6.4k   21k   15k   17k   64k   43k   34k   64k
           waterway    0     0     0     0   112   119     0     0     0  3.3k  2.4k    2k  2.1k  4.9k  2.4k  4.9k
               park    0     0     0     0     0     0  1.1k    4k  9.7k   18k   13k  8.2k  3.7k  3.4k  4.4k   18k
transportation_name    0     0     0     0     0     0   287   364  1.1k  1.9k  5.6k  4.8k  3.9k  3.5k   18k   18k
          landcover    0     0     0     0     0     0     0  9.7k   29k   85k   72k   81k   53k   30k   25k   85k
      mountain_peak    0     0     0     0     0     0     0  1.1k  1.8k  3.4k  4.4k  2.8k  1.4k  1.4k   869  4.4k
         water_name    0     0     0     0     0     0     0     0     0   486   461   433   452  1.2k  1.5k  1.5k
    aerodrome_label    0     0     0     0     0     0     0     0     0     0   666   289   273   221   221   666
            aeroway    0     0     0     0     0     0     0     0     0     0  1.6k    2k    3k  3.3k  2.8k  3.3k
                poi    0     0     0     0     0     0     0     0     0     0     0     0   589   586   88k   88k
           building    0     0     0     0     0     0     0     0     0     0     0     0     0   59k   92k   92k
        housenumber    0     0     0     0     0     0     0     0     0     0     0     0     0     0   35k   35k
          full tile 7.9k    4k  9.5k  6.5k  3.8k  6.3k   21k   41k   84k  201k  183k  135k  114k  124k  252k  252k
            gzipped 6.2k  3.5k  7.1k  5.2k  3.2k    5k   14k   29k   60k  148k  137k   99k   84k   89k  160k  160k
0:01:11 DEB [archive] -    Max tile: 252k (gzipped: 160k)
0:01:11 DEB [archive] -    Avg tile: 5.5k (gzipped: 4.1k) using weighted average based on OSM traffic
0:01:11 DEB [archive] -     # tiles: 4,115,032
0:01:11 DEB [archive] -  # features: 5,712,608
0:01:11 INF [archive] - Finished in 20s cpu:1m16s avg:3.8
0:01:11 INF [archive] -   read    1x(3% 0.6s wait:18s done:1s)
0:01:11 INF [archive] -   encode  4x(63% 13s wait:2s done:1s)
0:01:11 INF [archive] -   write   1x(22% 4s wait:13s done:1s)
0:01:11 INF [archive] - Finished in 1m12s cpu:4m gc:1s avg:3.4
0:01:11 INF [archive] - FINISHED!
0:01:11 INF [archive] - 
0:01:11 INF [archive] - ----------------------------------------
0:01:11 INF [archive] - data errors:
0:01:11 INF [archive] - 	render_snap_fix_input	16,924
0:01:11 INF [archive] - 	osm_multipolygon_missing_way	397
0:01:11 INF [archive] - 	osm_boundary_missing_way	55
0:01:11 INF [archive] - 	merge_snap_fix_input	16
0:01:11 INF [archive] - 	feature_polygon_osm_invalid_multipolygon_empty_after_fix	8
0:01:11 INF [archive] - 	feature_centroid_if_convex_osm_invalid_multipolygon_empty_after_fix	2
0:01:11 INF [archive] - 	render_snap_fix_input2	1
0:01:11 INF [archive] - 	omt_park_area_osm_invalid_multipolygon_empty_after_fix	1
0:01:11 INF [archive] - 	omt_fix_water_before_ne_intersect	1
0:01:11 INF [archive] - 	feature_point_on_surface_osm_invalid_multipolygon_empty_after_fix	1
0:01:11 INF [archive] - ----------------------------------------
0:01:11 INF [archive] - 	overall          1m12s cpu:4m gc:1s avg:3.4
0:01:11 INF [archive] - 	lake_centerlines 3s cpu:6s avg:2.3
0:01:11 INF [archive] - 	  read     1x(17% 0.5s done:2s)
0:01:11 INF [archive] - 	  process  4x(0% 0s done:2s)
0:01:11 INF [archive] - 	  write    1x(0% 0s done:2s)
0:01:11 INF [archive] - 	water_polygons   15s cpu:42s avg:2.8
0:01:11 INF [archive] - 	  read     1x(42% 6s done:7s)
0:01:11 INF [archive] - 	  process  4x(27% 4s wait:4s done:6s)
0:01:11 INF [archive] - 	  write    1x(3% 0.5s wait:10s done:5s)
0:01:11 INF [archive] - 	natural_earth    7s cpu:14s avg:2.1
0:01:11 INF [archive] - 	  read     1x(95% 6s)
0:01:11 INF [archive] - 	  process  4x(13% 0.8s wait:6s)
0:01:11 INF [archive] - 	  write    1x(0% 0s wait:7s)
0:01:11 INF [archive] - 	osm_pass1        2s cpu:7s avg:3.2
0:01:11 INF [archive] - 	  read     1x(2% 0s wait:2s)
0:01:11 INF [archive] - 	  parse    4x(31% 0.7s wait:1s)
0:01:11 INF [archive] - 	  process  1x(73% 2s)
0:01:11 INF [archive] - 	osm_pass2        22s cpu:1m28s avg:4
0:01:11 INF [archive] - 	  read     1x(0% 0s wait:14s done:8s)
0:01:11 INF [archive] - 	  process  4x(73% 16s)
0:01:11 INF [archive] - 	  write    1x(2% 0.5s wait:22s)
0:01:11 INF [archive] - 	ne_lakes         0s cpu:0s avg:0
0:01:11 INF [archive] - 	boundaries       0s cpu:0s avg:2.6
0:01:11 INF [archive] - 	agg_stop         0s cpu:0s avg:0
0:01:11 INF [archive] - 	sort             1s cpu:4s avg:2.6
0:01:11 INF [archive] - 	  worker  1x(52% 0.8s)
0:01:11 INF [archive] - 	archive          20s cpu:1m16s avg:3.8
0:01:11 INF [archive] - 	  read    1x(3% 0.6s wait:18s done:1s)
0:01:11 INF [archive] - 	  encode  4x(63% 13s wait:2s done:1s)
0:01:11 INF [archive] - 	  write   1x(22% 4s wait:13s done:1s)
0:01:11 INF [archive] - ----------------------------------------
0:01:11 INF [archive] - 	archive	108MB
0:01:11 INF [archive] - 	features	293MB
-rw-r--r-- 1 runner runner 109M Jan  4 07:43 run.jar
0:01:09 DEB [archive] - Tile stats:
0:01:09 DEB [archive] - Biggest tiles (gzipped)
1. 14/4942/6092 (160k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.40015 (poi:88k)
2. 9/154/190 (148k) https://onthegomap.github.io/planetiler-demo/#9.5/41.77078/-71.36719 (landcover:85k)
3. 10/308/381 (137k) https://onthegomap.github.io/planetiler-demo/#10.5/41.63994/-71.54297 (landcover:72k)
4. 10/308/380 (137k) https://onthegomap.github.io/planetiler-demo/#10.5/41.90214/-71.54297 (landcover:66k)
5. 14/4941/6092 (117k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.42212 (poi:69k)
6. 14/4941/6093 (117k) https://onthegomap.github.io/planetiler-demo/#14.5/41.81227/-71.42212 (building:62k)
7. 14/4940/6092 (101k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.44409 (building:92k)
8. 14/4946/6112 (99k) https://onthegomap.github.io/planetiler-demo/#14.5/41.50035/-71.31226 (building:60k)
9. 11/616/762 (99k) https://onthegomap.github.io/planetiler-demo/#11.5/41.7057/-71.63086 (landcover:71k)
10. 11/616/764 (97k) https://onthegomap.github.io/planetiler-demo/#11.5/41.44269/-71.63086 (landcover:74k)
0:01:09 DEB [archive] - Max tile sizes
                      z0    z1    z2    z3    z4    z5    z6    z7    z8    z9   z10   z11   z12   z13   z14   all
           boundary  151   336   409   544   872   350   464   571   808  1.6k    2k  6.8k  6.2k  5.6k  4.4k  6.8k
              water 7.7k  3.7k  8.6k  5.5k  2.6k  5.1k   15k   18k   16k   26k   15k   13k   17k   15k   12k   26k
              place    0     0   487   487   487   707   796    1k  1.6k  3.1k    6k  3.6k  1.8k   845   978    6k
            landuse    0     0     0     0   549   695  1.6k  6.7k   17k   44k   58k   49k   38k   19k   12k   58k
     transportation    0     0     0     0   418   968  1.4k  4.5k  6.4k   21k   15k   17k   64k   43k   34k   64k
           waterway    0     0     0     0   112   119     0     0     0  3.3k  2.4k    2k  2.1k  4.9k  2.4k  4.9k
               park    0     0     0     0     0     0  1.1k    4k  9.7k   18k   13k  8.2k  3.7k  3.4k  4.4k   18k
transportation_name    0     0     0     0     0     0   287   364  1.1k  1.9k  5.6k  4.8k  3.9k  3.5k   18k   18k
          landcover    0     0     0     0     0     0     0  9.7k   29k   85k   72k   81k   53k   30k   25k   85k
      mountain_peak    0     0     0     0     0     0     0  1.1k  1.8k  3.4k  4.4k  2.8k  1.4k  1.4k   869  4.4k
         water_name    0     0     0     0     0     0     0     0     0   486   461   433   452  1.2k  1.5k  1.5k
    aerodrome_label    0     0     0     0     0     0     0     0     0     0   666   289   273   221   221   666
            aeroway    0     0     0     0     0     0     0     0     0     0  1.6k    2k    3k  3.3k  2.8k  3.3k
                poi    0     0     0     0     0     0     0     0     0     0     0     0   589   586   88k   88k
           building    0     0     0     0     0     0     0     0     0     0     0     0     0   59k   92k   92k
        housenumber    0     0     0     0     0     0     0     0     0     0     0     0     0     0   35k   35k
          full tile 7.9k    4k  9.5k  6.5k  3.8k  6.3k   21k   41k   84k  201k  183k  135k  114k  124k  252k  252k
            gzipped 6.2k  3.5k  7.1k  5.2k  3.2k    5k   14k   29k   60k  148k  137k   99k   84k   89k  160k  160k
0:01:09 DEB [archive] -    Max tile: 252k (gzipped: 160k)
0:01:09 DEB [archive] -    Avg tile: 5.5k (gzipped: 4.1k) using weighted average based on OSM traffic
0:01:09 DEB [archive] -     # tiles: 4,115,032
0:01:09 DEB [archive] -  # features: 5,712,608
0:01:09 INF [archive] - Finished in 20s cpu:1m15s avg:3.7
0:01:09 INF [archive] -   read    1x(3% 0.6s wait:19s done:1s)
0:01:09 INF [archive] -   encode  4x(57% 12s wait:2s)
0:01:09 INF [archive] -   write   1x(21% 4s wait:14s)
0:01:09 INF [archive] - Finished in 1m9s cpu:3m50s gc:1s avg:3.3
0:01:09 INF [archive] - FINISHED!
0:01:09 INF [archive] - 
0:01:09 INF [archive] - ----------------------------------------
0:01:09 INF [archive] - data errors:
0:01:09 INF [archive] - 	render_snap_fix_input	16,924
0:01:09 INF [archive] - 	osm_multipolygon_missing_way	397
0:01:09 INF [archive] - 	osm_boundary_missing_way	55
0:01:09 INF [archive] - 	merge_snap_fix_input	16
0:01:09 INF [archive] - 	feature_polygon_osm_invalid_multipolygon_empty_after_fix	8
0:01:09 INF [archive] - 	feature_centroid_if_convex_osm_invalid_multipolygon_empty_after_fix	2
0:01:09 INF [archive] - 	render_snap_fix_input2	1
0:01:09 INF [archive] - 	omt_park_area_osm_invalid_multipolygon_empty_after_fix	1
0:01:09 INF [archive] - 	omt_fix_water_before_ne_intersect	1
0:01:09 INF [archive] - 	feature_point_on_surface_osm_invalid_multipolygon_empty_after_fix	1
0:01:09 INF [archive] - ----------------------------------------
0:01:09 INF [archive] - 	overall          1m9s cpu:3m50s gc:1s avg:3.3
0:01:09 INF [archive] - 	lake_centerlines 2s cpu:6s avg:2.5
0:01:09 INF [archive] - 	  read     1x(20% 0.5s done:2s)
0:01:09 INF [archive] - 	  process  4x(0% 0s done:2s)
0:01:09 INF [archive] - 	  write    1x(0% 0s done:2s)
0:01:09 INF [archive] - 	water_polygons   15s cpu:42s avg:2.7
0:01:09 INF [archive] - 	  read     1x(41% 6s done:7s)
0:01:09 INF [archive] - 	  process  4x(26% 4s wait:4s done:6s)
0:01:09 INF [archive] - 	  write    1x(3% 0.5s wait:10s done:5s)
0:01:09 INF [archive] - 	natural_earth    7s cpu:14s avg:2.1
0:01:09 INF [archive] - 	  read     1x(95% 6s)
0:01:09 INF [archive] - 	  process  4x(13% 0.8s wait:6s)
0:01:09 INF [archive] - 	  write    1x(0% 0s wait:6s)
0:01:09 INF [archive] - 	osm_pass1        2s cpu:7s avg:3.2
0:01:09 INF [archive] - 	  read     1x(2% 0s wait:2s)
0:01:09 INF [archive] - 	  parse    4x(33% 0.7s)
0:01:09 INF [archive] - 	  process  1x(71% 2s)
0:01:09 INF [archive] - 	osm_pass2        20s cpu:1m21s avg:4
0:01:09 INF [archive] - 	  read     1x(0% 0s wait:12s done:8s)
0:01:09 INF [archive] - 	  process  4x(75% 15s)
0:01:09 INF [archive] - 	  write    1x(2% 0.5s wait:20s)
0:01:09 INF [archive] - 	ne_lakes         0s cpu:0s avg:0
0:01:09 INF [archive] - 	boundaries       0s cpu:0s avg:2.6
0:01:09 INF [archive] - 	agg_stop         0s cpu:0s avg:0
0:01:09 INF [archive] - 	sort             1s cpu:4s avg:2.5
0:01:09 INF [archive] - 	  worker  1x(49% 0.7s)
0:01:09 INF [archive] - 	archive          20s cpu:1m15s avg:3.7
0:01:09 INF [archive] - 	  read    1x(3% 0.6s wait:19s done:1s)
0:01:09 INF [archive] - 	  encode  4x(57% 12s wait:2s)
0:01:09 INF [archive] - 	  write   1x(21% 4s wait:14s)
0:01:09 INF [archive] - ----------------------------------------
0:01:09 INF [archive] - 	archive	108MB
0:01:09 INF [archive] - 	features	293MB
-rw-r--r-- 1 runner runner 109M Jan  4 07:44 run.jar

Full logs: https://github.com/onthegomap/planetiler/actions/runs/20689654879

This commit fixes the build failure where test compilation couldn't find
VectorTileProto classes, and improves the relation_members feature
implementation with better test coverage and code quality improvements

- Added new tests for code and functional coverage
- Fix test compilation failure for VectorTileProto classes by adding
  build-helper-maven-plugin to include generated protobuf sources in
  test compilation classpath
- Allow relations to be processed as both multipolygon and relation_members
  by changing conditional logic in OsmReader from else-if to independent
  if statements
- Refactor relation_members processing loop to reduce continue statements
  (SonarQube java:S135 compliance)
- Fix MemberContext.tags() to return mutable copy instead of parent tags
- Add comprehensive integration tests for relation_members (772 lines, 11 tests)
  covering railway routes, boundary relations, member filtering, duplicate
  handling, polygon conversion, missing geometry, inline scripts, and
  dual processing scenarios
- Refactor validation test lambdas to satisfy SonarQube java:S5778 rule
- Remove temporary documentation files (COMMIT_MESSAGE.txt, PR_DESCRIPTION.md)
- Clean up comments and improve code documentation
Copy link
Contributor

@CommanderStorm CommanderStorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(not a maintainer)

🤔 Reealy stupid question:
Is the following possible with this PR, or is this another issue?

- name: 'country boundary'
input:
source: osm
geometry: line
# TODO from relation
tags:
boundary: administrative
admin_level: '2'
maritime: yes
output:
layer: boundaries
geometry: line
min_zoom: 0
min_size: 0
tags:
maritime: true
admin_level: 2
disputed: false
- name: 'country coastline'
input:
source: osm
geometry: line
# TODO from relation
tags:
boundary: administrative
admin_level: '2'
natural: coastline
output:
layer: boundaries
geometry: line
min_zoom: 0
min_size: 0
tags:
maritime: true
admin_level: 2
disputed: false

Currently, the US-Canada and russia-ukraine border are relations, so cannot be incuded via the custommap previously (?).

In general, could you add this to the custommap' Readme (aka the docs for this feature)

@zstadler zstadler force-pushed the relation-members branch 2 times, most recently from ec9d7ba to f989290 Compare January 4, 2026 07:36
- Add comprehensive documentation for relation_members geometry type in README
- Document member_types, member_roles, member_include_when, member_exclude_when, and member_attributes fields
- Add Relation Members section with examples and usage patterns
- Document Member Context variables for inline script expressions
- Update JSON schema to include relation_members in geometry enum
- Add all member-related fields to feature definition in schema
- Include descriptions noting member fields are only valid with relation_members geometry
@zstadler
Copy link
Contributor Author

zstadler commented Jan 4, 2026

Is the following possible with this PR, or is this another issue?

This PR would allow the boundary segments to be taken from the members of boundary relations, while taking attributes such as maritime and disputed from the relation member's tags.

If I understand it correctly, this PR could address this TODO in the boundaries layer.

# TODO: member of a relation with boundary=disputed and (admin_level unset or between 2 and 4)
# see https://github.com/shortbread-tiles/shortbread-docs/issues/43

This PR does not address the other boundaries TODO, as each applicable member creates a separate feature for each applicable relation it is a member of

# TODO get min admin level from relations

It would be a valuable to be able issue a single geometry for each member and be able to aggregate information for tags of the parent relations.

Note that shortbread.spec.yml is a test for shortbread.yml were each name represents an example where the given input should produce the given output. This PR does not affect the schema testing mechanism. The quoted comment should probably be placed in the schema itself.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Jan 4, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add relation_members geometry type for processing OSM relation members

2 participants