feat: implement WKT2 to PROJJSON converter #152
Conversation
…sonstring with pure Julia implementation - Add support for GeographicCRS, ProjectedCRS, CompoundCRS and VerticalCRS - Implement detailed parsing of CRS components including datum, ellipsoid, and axis - Add error handling and warning messages for conversion failures
Co-authored-by: Neven Sajko <4944410+nsajko@users.noreply.github.com>
There was a problem hiding this comment.
That is very promising, David! I appreciate the detailed comments.
Could you please check the failing test? We fallback to Cartesian2D when the code doesn't have a correspondent type. That may be causing issues somewhere.
Besides the failing test, we need two more tests to feel safe about the changes:
- Test against the old GDAL implementation by moving the old implementation to the test suite in
test/utils.jland comparing the resulting strings. - Test that the produced strings match the PROJJSON schema.
Both these tests could be run for a list of CRS codes.
for code in codes
projjson1 = gdalprojjson(EPSG{code})
projjson2 = juliaprojjson(EPSG{code})
@test projjson1 == projjson2
endThe list of all codes can be found here.
|
@juliohm Thanks for the review, about the CI test failure, i got the same error on the main branch before implementing the feature, here is a screenshot from github codespaces: test command (got this from the CI): julia --project=. -e 'using Pkg; Pkg.test(;coverage=true, julia_args=["--check-bounds=yes", "--compiled-modules=yes", "--depwarn=yes"], force_latest_compatible_version=false, allow_reresolve=true)' | cat Testing noattrs.jl...
GeoTables without attributes: Error During Test at /workspaces/GeoIO.jl/test/noattrs.jl:1
Got exception outside of a @test
ArgumentError: Malformed CRS string.
Please make sure that the string follows any of the following Well-Known-Text formats: OGC WKT1, ESRI WKT1, WKT2.
Stacktrace:
[1] parseerror()
@ CoordRefSystems ~/.julia/packages/CoordRefSystems/gCde8/src/strings.jl:121
[2] checkmatch(m::Nothing)
@ CoordRefSystems ~/.julia/packages/CoordRefSystems/gCde8/src/strings.jl:118
[3] |>(x::Nothing, f::typeof(CoordRefSystems.checkmatch))
@ Base ./operators.jl:926
[4] string2code(crsstr::String)
@ CoordRefSystems ~/.julia/packages/CoordRefSystems/gCde8/src/strings.jl:108
[5] get(crsstr::String)
@ CoordRefSystems ~/.julia/packages/CoordRefSystems/gCde8/src/get.jl:10
[6] crstype
@ /workspaces/GeoIO.jl/src/conversion.jl:64 [inlined]
[7] togeometry(::GeoInterface.PointTrait, geom::ArchGDAL.IGeometry{ArchGDAL.wkbPoint}, crs::GeoFormatTypes.WellKnownText{GeoFormatTypes.CRS})
@ GeoIO /workspaces/GeoIO.jl/src/conversion.jl:111
[8] geom2meshes
@ /workspaces/GeoIO.jl/src/conversion.jl:132 [inlined]
[9] geom2meshes
@ /workspaces/GeoIO.jl/src/conversion.jl:131 [inlined]
[10] _broadcast_getindex_evalf
@ ./broadcast.jl:678 [inlined]
[11] _broadcast_getindex
@ ./broadcast.jl:651 [inlined]
[12] getindex
@ ./broadcast.jl:610 [inlined]
[13] copy
@ ./broadcast.jl:911 [inlined]
[14] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, typeof(GeoIO.geom2meshes), Tuple{Vector{ArchGDAL.IGeometry{ArchGDAL.wkbPoint}}, Base.RefValue{GeoFormatTypes.WellKnownText{GeoFormatTypes.CRS}}}})
@ Base.Broadcast ./broadcast.jl:872
[15] asgeotable(table::ArchGDAL.IFeatureLayer)
@ GeoIO /workspaces/GeoIO.jl/src/utils.jl:20
[16] load(fname::String; repair::Bool, layer::Int64, lenunit::Nothing, numbertype::Type, kwargs::@Kwargs{})
@ GeoIO /workspaces/GeoIO.jl/src/load.jl:163
[17] load(fname::String)
@ GeoIO /workspaces/GeoIO.jl/src/load.jl:81
[18] macro expansion
@ /workspaces/GeoIO.jl/test/noattrs.jl:22 [inlined]
[19] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
[20] top-level scope
@ /workspaces/GeoIO.jl/test/noattrs.jl:2
[21] include(fname::String)
@ Main ./sysimg.jl:38
[22] macro expansion
@ /workspaces/GeoIO.jl/test/runtests.jl:65 [inlined]
[23] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
[24] top-level scope
@ /workspaces/GeoIO.jl/test/runtests.jl:63
[25] include(fname::String)
@ Main ./sysimg.jl:38
[26] top-level scope
@ none:6
[27] eval
@ ./boot.jl:430 [inlined]
[28] exec_options(opts::Base.JLOptions)
@ Base ./client.jl:296
[29] _start()
@ Base ./client.jl:531
Test Summary: | Pass Error Total Time
GeoIO.jl | 545 1 546 4m22.0s
Images | 8 8 21.7s
STL | 22 22 12.7s
OBJ | 9 9 1.3s
OFF | 11 11 2.6s
MSH | 25 25 4.4s
PLY | 9 9 5.0s
CSV | 33 33 14.3s
GSLIB | 7 7 3.0s
VTK | 74 74 36.6s
NetCDF | 36 36 24.0s
GRIB | 4 4 6.2s
GeoTiff | 62 62 40.1s
Shapefile | 58 58 19.3s
GeoJSON | 29 29 18.8s
GeoPackage | 37 37 5.2s
GeoParquet | 43 43 19.0s
KML | 6 6 0.7s
formats | 4 4 0.8s
convert | 28 28 10.9s
GIS | 36 36 8.9s
GeoTables without attributes | 4 1 5 4.2s
ERROR: LoadError: Some tests did not pass: 545 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /workspaces/GeoIO.jl/test/runtests.jl:62
ERROR: Package GeoIO errored during testing
Stacktrace:
[1] pkgerror(msg::String)
@ Pkg.Types ~/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Types.jl:68
[2] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
@ Pkg.Operations ~/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:2118
[3] test
@ ~/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:2003 [inlined]
[4] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Vector{String}, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::IOContext{IO}})
@ Pkg.API ~/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:481
[5] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{coverage::Bool, julia_args::Vector{String}, force_latest_compatible_version::Bool, allow_reresolve::Bool})
@ Pkg.API ~/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:159
[6] test(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::@Kwargs{coverage::Bool, julia_args::Vector{String}, force_latest_compatible_version::Bool, allow_reresolve::Bool})
@ Pkg.API ~/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:174
[7] top-level scope
@ none:1
@onyedikachi-david ➜ /workspaces/GeoIO.jl (master) $ |
|
You are correct @onyedikachi-david. Sorry for the confusion. For some reason our recent changes broke this single test. I am investigating it. In the meantime, please feel free to go ahead with the other tests. |
|
@onyedikachi-david I identified the issue and fixed it on the master branch. Please rebase to get all tests passing. The issue comes from a GDAL inconsistency, in the way it treats undefined/unknown CRS. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #152 +/- ##
==========================================
- Coverage 93.30% 86.90% -6.40%
==========================================
Files 18 18
Lines 1269 1581 +312
==========================================
+ Hits 1184 1374 +190
- Misses 85 207 +122 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@onyedikachi-david did you have chance to take a look into this? Perhaps start with the test that compares the output with the previous implementation? |
Hello, @juliohm, Yes, I have been on it, some of my test are failing; still on a fix will push soon, see test logs below: Test logs
EPSG:2193: Test Failed at /Users/onyedikachi/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:223
Expression: normalized1 == normalized2
Evaluated: "{\"name\":\"NZGD2000 / New Zealand Transverse Mercator 2000\",\"method\":{\"name\":\"Transverse Mercator\",\"id\":{\"authority\":\"EPSG\",\"code\":9807}},\"base_crs\":{\"datum\":{\"name\":\"New Zealand Geodetic Datum 2000\"},\"name\":\"NZGD2000\",\"id\":{\"authority\":\"EPSG\",\"code\":4167}},\"id\":{\"authority\":\"EPSG\",\"code\":2193},\"type\":\"ProjectedCRS\"}" == "{\"name\":\"NZGD2000 / New Zealand Transverse Mercator 2000\",\"method\":{\"name\":\"Transverse Mercator\",\"id\":{\"authority\":\"EPSG\",\"code\":9807}},\"base_crs\":{\"datum\":{\"name\":\"New Zealand Geodetic Datum 2000\"},\"name\":\"NZGD2000\",\"id\":{\"authority\":\"EPSG\",\"code\":9001}},\"id\":{\"authority\":\"EPSG\",\"code\":2193},\"type\":\"ProjectedCRS\"}"
Stacktrace:
[1] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:679 [inlined]
[2] macro expansion
@ ~/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:223 [inlined]
[3] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
[4] macro expansion
@ ~/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:203 [inlined]
[5] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
[6] macro expansion
@ ~/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:201 [inlined]
[7] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
[8] top-level scope
@ ~/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:200
EPSG:3003: Test Failed at /Users/onyedikachi/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:223
Expression: normalized1 == normalized2
Evaluated: "{\"name\":\"Monte Mario / Italy zone 1\",\"method\":{\"name\":\"Transverse Mercator\",\"id\":{\"authority\":\"EPSG\",\"code\":9807}},\"base_crs\":{\"datum\":{\"name\":\"Monte Mario\"},\"name\":\"Monte Mario\",\"id\":{\"authority\":\"EPSG\",\"code\":4265}},\"id\":{\"authority\":\"EPSG\",\"code\":3003},\"type\":\"ProjectedCRS\"}" == "{\"name\":\"Monte Mario / Italy zone 1\",\"method\":{\"name\":\"Transverse Mercator\",\"id\":{\"authority\":\"EPSG\",\"code\":9807}},\"base_crs\":{\"datum\":{\"name\":\"Monte Mario\"},\"name\":\"Monte Mario\",\"id\":{\"authority\":\"EPSG\",\"code\":9001}},\"id\":{\"authority\":\"EPSG\",\"code\":3003},\"type\":\"ProjectedCRS\"}"
Stacktrace:
[1] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:679 [inlined]
[2] macro expansion
@ ~/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:223 [inlined]
[3] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
[4] macro expansion
@ ~/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:203 [inlined]
[5] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
[6] macro expansion
@ ~/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:201 [inlined]
[7] macro expansion
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
[8] top-level scope
@ ~/Documents/codes/algora-bounties/GeoIO.jl/test/utils.jl:200
Test Summary: | Pass Fail Total Time
GeoIO.jl | 627 2 629 6m12.5s
Images | 8 8 54.0s
STL | 22 22 24.0s
OBJ | 9 9 1.2s
OFF | 11 11 3.5s
MSH | 25 25 4.2s
PLY | 9 9 9.7s
CSV | 33 33 14.5s
GSLIB | 7 7 2.9s
VTK | 74 74 40.6s
NetCDF | 36 36 31.4s
GRIB | 4 4 6.3s
GeoTiff | 62 62 40.7s
Shapefile | 58 58 23.3s
GeoJSON | 29 29 21.4s
GeoPackage | 37 37 7.2s
GeoParquet | 43 43 39.3s
KML | 6 6 0.7s
formats | 4 4 0.7s
convert | 28 28 12.1s
GIS | 36 36 9.5s
GeoTables without attributes | 16 16 5.0s
PROJJSON Conversion | 70 2 72 3.1s
Comparison with GDAL | 61 2 63 3.1s
EPSG:4326 | 7 7 0.6s
EPSG:3857 | 7 7 0.1s
EPSG:4269 | 7 7 0.0s
EPSG:2193 | 6 1 7 2.3s
EPSG:32631 | 7 7 0.0s
EPSG:3003 | 6 1 7 0.0s
EPSG:5514 | 7 7 0.0s
EPSG:6933 | 7 7 0.0s
EPSG:3035 | 7 7 0.0s
Schema Validation | 9 9 0.0s
ERROR: LoadError: Some tests did not pass: 627 passed, 2 failed, 0 errored, 0 broken.
in expression starting at /Users/onyedikachi/Documents/codes/algora-bounties/GeoIO.jl/test/runtests.jl:63
ERROR: Package GeoIO errored during testing
Stacktrace:
[1] pkgerror(msg::String)
@ Pkg.Types ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Pkg/src/Types.jl:68
[2] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
@ Pkg.Operations ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:2118
[3] test
@ ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:2003 [inlined]
[4] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Vector{String}, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::IOContext{IO}})
@ Pkg.API ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Pkg/src/API.jl:481
[5] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{coverage::Bool, julia_args::Vector{String}, force_latest_compatible_version::Bool, allow_reresolve::Bool})
@ Pkg.API ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Pkg/src/API.jl:159
[6] test(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::@Kwargs{coverage::Bool, julia_args::Vector{String}, force_latest_compatible_version::Bool, allow_reresolve::Bool})
@ Pkg.API ~/.julia/juliaup/julia-1.11.4+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/Pkg/src/API.jl:174
[7] top-level scope
@ none:1 |
|
Thank you @onyedikachi-david. Please feel free to reach out here or over our Zulip channel for interactive chat: https://juliaearth.github.io/GeoStatsDocs/stable/about/community GDAL is inconsistent in many ways, so we shouldn't try to over-fit our results to match theirs. In case some strings diverge, we need to double check if the issue is on our side or not. |
|
@onyedikachi-david how is progress here? |
Signed-off-by: David Anyatonwu <davidanyatonwu@gmail.com>
|
Hello, @juliohm, made alot of improvements, please take a look. |
|
@onyedikachi-david I moved the definitions to the correct files in the test folder and ran the tests locally. None of the tests are passing locally. Please avoid using |
test/testutils.jl
Outdated
|
|
||
| # Helper function to validate PROJJSON against schema | ||
| function isvalidprojjson(jsonstr) | ||
| try |
There was a problem hiding this comment.
Please remove this try-catch from the function.
test/testutils.jl
Outdated
| end | ||
|
|
||
| # Function to normalize JSON for comparison | ||
| function normalize_json_for_comparison(jsonstr) |
There was a problem hiding this comment.
Do we really need this function? Why can't we compare the strings directly?
|
@onyedikachi-david I refactored the tests to keep them simple. Please remove the try-catch blocks from these test functions as they are not helping us find bugs. |
|
Hi @juliohm, I am really finding it difficult having a 1 to 1 gdal <=> projjson exact match. |
|
Can you share the differences you are encountering?
Em sáb., 5 de abr. de 2025, 10:50, David Anyatonwu ***@***.***>
escreveu:
… Hi @juliohm <https://github.com/juliohm>, I am really finding it
difficult having a 1 to 1 gdal <=> projjson exact match.
—
Reply to this email directly, view it on GitHub
<#152 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAZQW3LC37MUN2A3ZFYTXCL2X7NTDAVCNFSM6AAAAABZY6KR3OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBQG4ZTGNRSGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
[image: onyedikachi-david]*onyedikachi-david* left a comment
(JuliaEarth/GeoIO.jl#152)
<#152 (comment)>
Hi @juliohm <https://github.com/juliohm>, I am really finding it
difficult having a 1 to 1 gdal <=> projjson exact match.
—
Reply to this email directly, view it on GitHub
<#152 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAZQW3LC37MUN2A3ZFYTXCL2X7NTDAVCNFSM6AAAAABZY6KR3OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOBQG4ZTGNRSGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
I don't think CoorRefSystem gives the complete and gdal compartible === WKT2 Structure Analysis ===
Full WKT2 string:
PROJCRS["S-JTSK / Krovak East North",BASEGEOGCRS["S-JTSK",DATUM["System of the Unified Trigonometrical Cadastral Network",ELLIPSOID["Bessel 1841",6377397.155,299.1528128,LENGTHUNIT["metre",1,ID["EPSG",9001]],ID["EPSG",7004]],ID["EPSG",6156]],ID["EPSG",4156]],CONVERSION["Krovak East North (Greenwich)",METHOD["Krovak (North Orientated)",ID["EPSG",1041]],PARAMETER["Latitude of projection centre",49.5000000000003,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9102]],ID["EPSG",8811]],PARAMETER["Longitude of origin",24.8333333333336,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9102]],ID["EPSG",8833]],PARAMETER["Co-latitude of cone axis",30.2881397527781,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9102]],ID["EPSG",1036]],PARAMETER["Latitude of pseudo standard parallel",78.5000000000003,ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9102]],ID["EPSG",8818]],PARAMETER["Scale factor on pseudo standard parallel",0.9999,SCALEUNIT["unity",1,ID["EPSG",9201]],ID["EPSG",8819]],PARAMETER["False easting",0,LENGTHUNIT["metre",1,ID["EPSG",9001]],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["metre",1,ID["EPSG",9001]],ID["EPSG",8807]],ID["EPSG",5510]],CS[Cartesian,2,ID["EPSG",4499]],AXIS["Easting (X)",east],AXIS["Northing (Y)",north],LENGTHUNIT["metre",1,ID["EPSG",9001]],ID["EPSG",5514]]
=== Component Analysis ===
Base CRS section:
"S-JTSK",DATUM["System of the Unified Trigonometrical Cadastral Network",ELLIPSOID["Bessel 1841",6377397.155,299.1528128,LENGTHUNIT["metre",1,ID["EPSG",9001
Conversion section:
"Krovak East North (Greenwich)",METHOD["Krovak (North Orientated)",ID["EPSG",1041]
Method:
"Krovak (North Orientated)",ID["EPSG",1041
Parameters:
Coordinate System section:
Cartesian,2,ID["EPSG",4499]
Metadata sections:
Main CRS ID:
Authority: EPSG, Code: 9001
=== End Analysis ===I doesn't have Area and BBox Metadata The function: # Add debug logging function
function debug_wkt2_structure(wkt2str)
println("\n=== WKT2 Structure Analysis ===")
println("Full WKT2 string:")
println(wkt2str)
println("\n=== Component Analysis ===")
# Base CRS section
base_section = match(r"BASEGEOGCRS\[(.*?)(?:,\s*CONVERSION|\])", wkt2str)
if base_section !== nothing
println("\nBase CRS section:")
println(base_section.captures[1])
# Extract datum/ellipsoid info
datum_match = match(r"DATUM\[(.*?)\](?:,|$)", base_section.captures[1])
if datum_match !== nothing
println("\nDatum section:")
println(datum_match.captures[1])
end
# Extract base CRS ID
base_id = match(r"ID\[\"([^\"]+)\",\s*(\d+)\]", base_section.captures[1])
if base_id !== nothing
println("\nBase CRS ID:")
println("Authority: $(base_id.captures[1]), Code: $(base_id.captures[2])")
end
end
# Conversion section
conversion_section = match(r"CONVERSION\[(.*?)\](?:,|$)", wkt2str)
if conversion_section !== nothing
println("\nConversion section:")
println(conversion_section.captures[1])
# Extract method
method_match = match(r"METHOD\[(.*?)\]", conversion_section.captures[1])
if method_match !== nothing
println("\nMethod:")
println(method_match.captures[1])
end
# Extract parameters
println("\nParameters:")
for param_match in eachmatch(r"PARAMETER\[(.*?)\]", conversion_section.captures[1])
println(param_match.captures[1])
end
end
# Coordinate System section
cs_section = match(r"CS\[(.*?)\](?:,|$)", wkt2str)
if cs_section !== nothing
println("\nCoordinate System section:")
println(cs_section.captures[1])
end
# Metadata sections
println("\nMetadata sections:")
scope_match = match(r"USAGE\[.*?SCOPE\[\"([^\"]+)\"", wkt2str)
if scope_match !== nothing
println("Scope: $(scope_match.captures[1])")
end
area_match = match(r"AREA\[\"([^\"]+)\"", wkt2str)
if area_match !== nothing
println("Area: $(area_match.captures[1])")
end
bbox_match = match(r"BBOX\[([-+]?\d*\.?\d+),\s*([-+]?\d*\.?\d+),\s*([-+]?\d*\.?\d+),\s*([-+]?\d*\.?\d+)\]", wkt2str)
if bbox_match !== nothing
println("BBox: S=$(bbox_match.captures[1]), W=$(bbox_match.captures[2]), N=$(bbox_match.captures[3]), E=$(bbox_match.captures[4])")
end
# Main CRS ID
crs_id = match(r"(?:^|\])(?:[^]]*?)ID\[\"([^\"]+)\",\s*(\d+)\](?:\]|$)", wkt2str)
if crs_id !== nothing
println("\nMain CRS ID:")
println("Authority: $(crs_id.captures[1]), Code: $(crs_id.captures[2])")
end
println("\n=== End Analysis ===\n")
endHow I am using it: function wkt2toprojjson(wkt2str; multiline=false)
# First dump the WKT2 structure
debug_wkt2_structure(wkt2str)
# Then proceed with parsing based on CRS type
if startswith(wkt2str, "GEOGCRS")
parsegeogcrs(wkt2str, multiline=multiline)
elseif startswith(wkt2str, "PROJCRS")
parseprojcrs(wkt2str, multiline=multiline)
elseif startswith(wkt2str, "COMPOUNDCRS")
parsecompoundcrs(wkt2str, multiline=multiline)
elseif startswith(wkt2str, "VERTCRS")
parsevertcrs(wkt2str, multiline=multiline)
else
# For unsupported types, try a generic approach
parsegeogcrs(wkt2str, multiline=multiline)
end
end |
As far as I understand it, that shouldn't be a problem. The Area and BBox are not part of the CRS definition, but instead are precomputed in some file formats (e.g., shapefile) to speed up loading, etc. What else is different between our strings and the ones produced by GDAL? If that is the only difference, I believe we just need to adjust our test function to compare the relevant parts of the strings. However, if you are seeing different values in the strings (besides Area and BBox), we need to investigate it further. |
|
btw the current json output in this PR is exactly equivalent to that of epsg.io and spatialreference.org |
That is very good to know. Thanks for sharing @Omar-Elrefaei @onyedikachi-david I believe we are super close from a valid solution. We just need to make sure our tests cover the supported EPSG codes reasonably well. If the string comparison is failing due to Area/BBox, we can define a custom comparison function that compares the other parts of the string that are essential. |
| unsafe_string(wktptr[]) | ||
| end | ||
|
|
||
| # Helper function to validate PROJJSON against schema |
There was a problem hiding this comment.
This doesn't validate against the PROJJSON JSON schema, it just checks a few handpicked properties. Probably you want to use JSONSchema.jl to do this properly.
There was a problem hiding this comment.
Very useful package @nsajko , thanks for sharing!
@onyedikachi-david please try to use JSONSchema.jl in the tests.
I'm sorry, I messed up. Probably a mishap when copy-pasting around. As of yet, the vast majority of projjson generated by this PR does not pass a JSONSchema validation check. As an example of the difference between output of this pr and gdal see: EPSG-4326-screenshot, original formatted wkt for epsg 4326: |
|
Thanks all, for the insightful comments, I've fixed some of the parsing issues, just needs some final touches. |
|
@onyedikachi-david could you please push the changes for additional review? Are you still using regex to parse the wkt2 strings? |
|
I am almost done now, most of the diffs are mainly positional and case (capital and lower)😌, there isn't any missing keys in ours own implementation this time around ✅. I am also now making use of a |
- Introduced a new WKT2 parser to handle coordinate reference systems. - Updated the `wkt2toprojjson` function to utilize the new parser for improved accuracy and structure. - Enhanced validation checks for PROJJSON output in tests. - Added helper functions to streamline JSON normalization and comparison in tests. Signed-off-by: David Anyatonwu <davidanyatonwu@gmail.com>
Signed-off-by: David Anyatonwu <davidanyatonwu@gmail.com>
- Added support for scientific notation in number parsing. - Improved parsing logic to handle large integers and decimals more robustly. - Updated coordinate system subtype handling to ensure correct casing. - Rounded float values in JSON output to minimize precision discrepancies. Signed-off-by: David Anyatonwu <davidanyatonwu@gmail.com>
|
The only diffs now are in handling Floats, all codes are passing but one because of float rounding diffs |
Signed-off-by: David Anyatonwu <davidanyatonwu@gmail.com>
|
Hi, @juliohm, any comment on the latest changes? |
Fixes: #150
/claim #150