Skip to content

Failed import causes unrecoverable error #354

@ingenieroariel

Description

@ingenieroariel

The actual error is:

ubuntu@ip-10-84-250-65:~$ geonode importlayers data
[created] Layer for 'data/hti_biota_landcover_spot_cnigs_041998_polygon.shp' (1/64)
[created] Layer for 'data/hti_structure_ochasheltersites_point_022010.shp' (2/64)
[created] Layer for 'data/hti_structure_idpcamps_ocha_022010_point.shp' (3/64)
[created] Layer for 'data/hti_location_priority_areas_pap_tabarre_nathat_102011.shp' (4/64)
[created] Layer for 'data/hti_hazardclimate_monthprecipitationstation_point_052010.shp' (5/64)
[created] Layer for 'data/hti_structure_buildings_osm_20111021_polygones.shp' (6/64)
[created] Layer for 'data/hti_boundaries_communes_adm2_cnigs_polygon.shp' (7/64)
[created] Layer for 'data/hti_meteorology_rainstations_nathat_cnm_point.shp' (8/64)
[created] Layer for 'data/hti_boundaries_communes_adm2_cnigs_line.shp' (9/64)
Could not save the layer hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010, there was an upload error: java.io.IOException: Schema 'http://geonode.org/:hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010' does not exist.
[failed] Layer for 'data/hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010.shp' (10/64)
Could not save the layer hti_boundary_country_polygon_062006, there was an upload error: java.io.IOException: Schema 'http://geonode.org/:hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010' does not exist.
[failed] Layer for 'data/hti_boundary_country_polygon_062006.shp' (11/64)
Error deleting PostGIS table hti_hazardgeology_landslidesdetail_point_022010:table "hti_hazardgeology_landslidesdetail_point_022010" does not exist
CONTEXT:  SQL statement "DROP TABLE public.hti_hazardgeology_landslidesdetail_point_022010"
PL/pgSQL function "dropgeometrytable" line 23 at EXECUTE statement

Could not save the layer hti_hazardgeology_landslidesdetail_point_022010, there was an upload error: java.io.IOException: Error occured on rollback
[failed] Layer for 'data/hti_hazardgeology_landslidesdetail_point_022010.shp' (12/64)

If importlayers is cancelled and resumed, then the one of the failed layers can be properly imported:

ubuntu@ip-10-84-250-65:~$ geonode importlayers data/hti_boundary_country_polygon_062006.shp
[created] Layer for 'data/hti_boundary_country_polygon_062006.shp' (1/1)


Finished processing 1 layers in 25.0 seconds.

1 Created layers
0 Updated layers
0 Skipped layers
0 Failed layers
25.000000 seconds per layer

A second attempt to import causes the error to be repeated:

ubuntu@ip-10-84-250-65:~$ geonode importlayers data
[skipped] Layer for 'data/hti_biota_landcover_spot_cnigs_041998_polygon.shp' (1/64)
[skipped] Layer for 'data/hti_structure_ochasheltersites_point_022010.shp' (2/64)
[skipped] Layer for 'data/hti_structure_idpcamps_ocha_022010_point.shp' (3/64)
[skipped] Layer for 'data/hti_location_priority_areas_pap_tabarre_nathat_102011.shp' (4/64)
[skipped] Layer for 'data/hti_hazardclimate_monthprecipitationstation_point_052010.shp' (5/64)
[skipped] Layer for 'data/hti_structure_buildings_osm_20111021_polygones.shp' (6/64)
[skipped] Layer for 'data/hti_boundaries_communes_adm2_cnigs_polygon.shp' (7/64)
[skipped] Layer for 'data/hti_meteorology_rainstations_nathat_cnm_point.shp' (8/64)
[skipped] Layer for 'data/hti_boundaries_communes_adm2_cnigs_line.shp' (9/64)
Could not save the layer hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010, there was an upload error: java.io.IOException: Error occurred creating table
[failed] Layer for 'data/hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010.shp' (10/64)
[skipped] Layer for 'data/hti_boundary_country_polygon_062006.shp' (11/64)
Error deleting PostGIS table hti_hazardgeology_landslidesdetail_point_022010:table "hti_hazardgeology_landslidesdetail_point_022010" does not exist
CONTEXT:  SQL statement "DROP TABLE public.hti_hazardgeology_landslidesdetail_point_022010"
PL/pgSQL function "dropgeometrytable" line 23 at EXECUTE statement

Could not save the layer hti_hazardgeology_landslidesdetail_point_022010, there was an upload error: java.io.IOException: Schema 'http://geonode.org/:hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010' does not exist.
[failed] Layer for 'data/hti_hazardgeology_landslidesdetail_point_022010.shp' (12/64)
[skipped] Layer for 'data/hti_inlandwaters_rivers_cnigs_line_062006.shp' (13/64)
Could not save the layer hti_boundaries_sections_communales_adm3_cnigs_polygon, there was an upload error: java.io.IOException: Error occurred creating table
[failed] Layer for 'data/hti_boundaries_sections_communales_adm3_cnigs_polygon.shp' (14/64)

An attempt to import only the offending one causes an error:

ubuntu@ip-10-84-250-65:~$ geonode importlayers data/hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010.shp
Could not save the layer hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010, there was an upload error: java.io.IOException: Error occured on rollback
[failed] Layer for 'data/hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010.shp' (1/1)


Finished processing 1 layers in 0.0 seconds.

0 Created layers
0 Updated layers
0 Skipped layers
1 Failed layers
0.000000 seconds per layer

A second attempt causes a different error:

ubuntu@ip-10-84-250-65:~$ geonode importlayers data/hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010.shp
Error deleting PostGIS table hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010:table "hti_hazardhydrology_exceptionalflood_westdepartment_nathat_poly" does not exist
CONTEXT:  SQL statement "DROP TABLE public.hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010"
PL/pgSQL function "dropgeometrytable" line 23 at EXECUTE statement

Could not save the layer hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010, there was an upload error: java.io.IOException: Error occured on rollback
[failed] Layer for 'data/hti_hazardhydrology_exceptionalflood_westdepartment_nathat_polygon_052010.shp' (1/1)


Finished processing 1 layers in 1.0 seconds.

0 Created layers
0 Updated layers
0 Skipped layers
1 Failed layers
1.000000 seconds per layer

Restarting postgres does not fix the issue:

ubuntu@ip-10-84-250-65:~$ sudo service postgresql restart
 * Restarting PostgreSQL 9.1 database server
   ...done.

This is how the GeoServer log looks like:

2012-08-04 03:04:26,634 INFO [catalog.rest] - PUT file, mimetype: application/zip
2012-08-04 03:04:26,763 WARN [geotools.jdbc] - There is code leaving feature readers/iterators open, this is leaking statements and connections!
2012-08-04 03:04:26,768 WARN [geotools.jdbc] - There is code leaving feature readers/iterators open, this is leaking statements and connections!
2012-08-04 03:04:26,769 WARN [geotools.jdbc] - There is code leaving feature readers/iterators open, this is leaking statements and connections!
2012-08-04 03:04:26,773 INFO [catalog.rest] - Using existing datastore: postgres_imports
2012-08-04 03:04:27,907 ERROR [geoserver.rest] -
java.lang.RuntimeException: java.io.IOException: Error occured on rollback
        at org.geoserver.catalog.rest.DataStoreFileResource.handlePut(DataStoreFileResource.java:452)

The warning says there is code leaving feature readers/iterators open leaking statements and connections, let's look above in the logs to try to find out what the first exception looked like:

2012-08-04 02:29:09,576 INFO [org.geoserver] - GeoServer configuration lock is enabled
2012-08-04 02:35:38,074 INFO [org.geoserver] - Loaded store 'postgres_imports', enabled
2012-08-04 02:35:38,546 INFO [catalog.rest] - POST data store postgres_imports
2012-08-04 02:35:39,310 INFO [catalog.rest] - PUT file, mimetype: application/zip
2012-08-04 02:35:39,792 INFO [catalog.rest] - Using existing datastore: postgres_imports
2012-08-04 02:35:40,301 INFO [catalog.rest] - hti_biota_landcover_spot_cnigs_041998_polygon does not exist in data store postgres_imports. Attempting to create it
2012-08-04 02:36:10,989 WARN [geotools.jdbc] - Failed to use ST_Estimated_Extent, falling back on envelope aggregation
org.postgresql.util.PSQLException: ERROR: LWGEOM_estimated_extent: couldn't locate table within current schema
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:252)
        at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
        at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
        at org.geotools.data.postgis.PostGISDialect.getOptimizedBounds(PostGISDialect.java:247)
        at org.geotools.jdbc.JDBCDataStore.getBounds(JDBCDataStore.java:1109)
        at org.geotools.jdbc.JDBCFeatureSource.getBoundsInternal(JDBCFeatureSource.java:478)
        at org.geotools.jdbc.JDBCFeatureStore.getBoundsInternal(JDBCFeatureStore.java:179)
        at org.geotools.data.store.ContentFeatureSource.getBounds(ContentFeatureSource.java:370)
        at org.geotools.data.store.ContentFeatureSource.getBounds(ContentFeatureSource.java:342)

Since there is code leaking connections, let's try restarting tomcat to see if layers can be uploaded again:

ubuntu@ip-10-84-250-65:~$ sudo service tomcat6 restart
 * Stopping Tomcat servlet engine tomcat6
   ...done.
 * Starting Tomcat servlet engine tomcat6
   ...done.

It made me wait one minute, and then it failed:

ubuntu@ip-10-84-250-65:~$ geonode importlayers data/hti_hazardgeology_landslidesdetail_point_022010.shp
Error deleting PostGIS table hti_hazardgeology_landslidesdetail_point_022010:table "hti_hazardgeology_landslidesdetail_point_022010" does not exist
CONTEXT:  SQL statement "DROP TABLE public.hti_hazardgeology_landslidesdetail_point_022010"
PL/pgSQL function "dropgeometrytable" line 23 at EXECUTE statement

Could not save the layer hti_hazardgeology_landslidesdetail_point_022010, there was an upload error: java.io.IOException: Schema 'http://geonode.org/:hti_hazardhydrology_frequentflood_westdepartment_nathat_polygon_052010' does not exist.
[failed] Layer for 'data/hti_hazardgeology_landslidesdetail_point_022010.shp' (1/1)


Finished processing 1 layers in 69.0 seconds.

0 Created layers
0 Updated layers
0 Skipped layers
1 Failed layers
69.000000 seconds per layer

Datasets used can be downloaded from http://haitidata.org

The datasets used can be downloaded from http://haitidata.org

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions