Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PROJ_NETWORK: First transform fails, succeeds afterwards #705

Open
snowman2 opened this issue Sep 11, 2020 · 9 comments · Fixed by OSGeo/PROJ#4302
Open

PROJ_NETWORK: First transform fails, succeeds afterwards #705

snowman2 opened this issue Sep 11, 2020 · 9 comments · Fixed by OSGeo/PROJ#4302
Labels
proj Bug or issue related to PROJ

Comments

@snowman2
Copy link
Member

https://gis.stackexchange.com/questions/373550/first-call-to-transform-fails-with-inf-all-subsequent-calls-are-ok-what-cou

Worth looking into later.

@snowman2 snowman2 added the proj Bug or issue related to PROJ label Sep 11, 2020
@snowman2
Copy link
Member Author

Unable to re-produce in the latest version. Removed the local cache and still not able to re-produce.

@cipri-tom
Copy link

@snowman2 this is happening for me on MacOS. I have reached here from geopandas .to_crs().

Seeing your suggestion on the linked GIS question, I am able to consistently make get it to fail for the first call to transform, and fix it by pyproj.network.set_network_enabled(False).

I have installed from conda-forge on MacOS 10.15.7 (please see linked issue for environment file).

I believe the problem may be caused by some incorrect CA on my machine -- this is an enterprise laptop and they have overriden some of the certificates. I am not sure how to use set_ca_bundle_path() to check if using the correct ones would work.

If this is the case, may I suggest a harder error / warning, rather than just returning inf upon failure ?

Thanks,
Ciprian

@snowman2
Copy link
Member Author

may I suggest a harder error / warning, rather than just returning inf upon failure ?

There is a errcheck=True flag you can turn on.

@snowman2
Copy link
Member Author

Also, you can pre-download the grids: https://pyproj4.github.io/pyproj/stable/transformation_grids.html

@cipri-tom
Copy link

Thanks for the fast reply !

There is a errcheck=True flag you can turn on.

Which method has the errcheck=True flag ? I don't see it on Transformer.from_crs().

Sorry, I am a second-hand user, haven't interacted with pyproj directly, only through geopandas. They've suggested to try this code to check if the problem comes from here:

from pyproj import CRS, Transformer
from shapely.ops import transform

origin = CRS.from_user_input(df.crs)
target = CRS.from_user_input('+proj=lcc +lat_1=33 +lat_2=45 +lat_0=39 +lon_0=-96 +x_0=0 +y_0=0 +ellps=GRS80 +datum=NAD83 +units=m +no_defs')

transformer = Transformer.from_crs(origin, target, always_xy=True)

geom = df.geometry.iloc[2]  # shapely.geometry causing a problem

transform(transformer.transform, geom).wkt

Also, you can pre-download the grids

Thanks ! It seems to work with disabling the network, as well. Since I've never downloaded the grids, am I losing in accuracy if I disable network ?

@snowman2
Copy link
Member Author

Which method has the errcheck=True flag ? I don't see it on Transformer.from_crs().

The transform method ref

Since I've never downloaded the grids, am I losing in accuracy if I disable network ?

That is a possibility. It depends on the transformation.

AdeelH added a commit to AdeelH/raster-vision that referenced this issue Oct 10, 2024
For some transformations, pyproj attempts to download transformation
grids from the internet for improved accuracy when
Transformer.transform() is called. If it fails to connect to the
internet, it silently returns (inf, inf) and silently modifies its
behavior to not access the internet on subsequent calls, causing
them to succeed (though possibly with a loss of accuracy). See
pyproj4/pyproj#705 for details.

This workaround forces an error to be raised by setting
errcheck=True and ignoring the first error.
AdeelH added a commit to azavea/raster-vision that referenced this issue Oct 10, 2024
For some transformations, pyproj attempts to download transformation
grids from the internet for improved accuracy when
Transformer.transform() is called. If it fails to connect to the
internet, it silently returns (inf, inf) and silently modifies its
behavior to not access the internet on subsequent calls, causing
them to succeed (though possibly with a loss of accuracy). See
pyproj4/pyproj#705 for details.

This workaround forces an error to be raised by setting
errcheck=True and ignoring the first error.
@snowman2 snowman2 reopened this Oct 31, 2024
@snowman2 snowman2 pinned this issue Oct 31, 2024
@snowman2
Copy link
Member Author

@rouault, I am adding you to this issue so you are aware. I haven't had time to dig into it yet, but thought that the frequency of reports was worth bringing it to your attention.

rouault added a commit to rouault/PROJ that referenced this issue Oct 31, 2024
Currently when when need to use a remote grid that can't be opened, we
return HUGE_VAL coordinates values and a proj_errno = PROJ_ERR_OTHER_NETWORK_ERROR,
(I guess) as expected... But if we do following proj_trans() calls on the same
transformation object, the grid transformation is ignored and we
fallback to other methods (Helmert, ballpark, etc.).
Fix that by consistently returning the same error values as the initial
failed call.

Fixes pyproj4/pyproj#705
rouault added a commit to rouault/PROJ that referenced this issue Oct 31, 2024
Currently when when need to use a remote grid that can't be opened, we
return HUGE_VAL coordinates values and a proj_errno = PROJ_ERR_OTHER_NETWORK_ERROR,
(I guess) as expected... But if we do following proj_trans() calls on the same
transformation object, the grid transformation is ignored and we
fallback to other methods (Helmert, ballpark, etc.).
Fix that by consistently returning the same error values as the initial
failed call.

Fixes pyproj4/pyproj#705
rouault added a commit to rouault/PROJ that referenced this issue Oct 31, 2024
Currently when we need to use a remote grid that can't be opened, we
return HUGE_VAL coordinates values and a proj_errno = PROJ_ERR_OTHER_NETWORK_ERROR,
(I guess) as expected... But if we do following proj_trans() calls on the same
transformation object, the grid transformation is ignored and we
fallback to other methods (Helmert, ballpark, etc.).
Fix that by consistently returning the same error values as the initial
failed call.

Fixes pyproj4/pyproj#705
rouault added a commit to rouault/PROJ that referenced this issue Oct 31, 2024
Currently when we need to use a remote grid that can't be opened, we
return HUGE_VAL coordinates values and a proj_errno = PROJ_ERR_OTHER_NETWORK_ERROR,
(I guess) as expected... But if we do following proj_trans() calls on the same
transformation object, the grid transformation is ignored and we
fallback to other methods (Helmert, ballpark, etc.).
Fix that by consistently returning the same error values as the initial
failed call.

Fixes pyproj4/pyproj#705
@snowman2
Copy link
Member Author

snowman2 commented Nov 1, 2024

OSGeo/PROJ#4302

rouault added a commit to OSGeo/PROJ that referenced this issue Nov 1, 2024
Currently when we need to use a remote grid that can't be opened, we
return HUGE_VAL coordinates values and a proj_errno = PROJ_ERR_OTHER_NETWORK_ERROR,
(I guess) as expected... But if we do following proj_trans() calls on the same
transformation object, the grid transformation is ignored and we
fallback to other methods (Helmert, ballpark, etc.).
Fix that by consistently returning the same error values as the initial
failed call.

Fixes pyproj4/pyproj#705
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
proj Bug or issue related to PROJ
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants