Skip to content

Latest commit

 

History

History
64 lines (45 loc) · 2.1 KB

README.md

File metadata and controls

64 lines (45 loc) · 2.1 KB
Raw

FIAS-GAR-CONVERTER

Converts FIAS-GAR xml representation to SQL-dump. Конвертирует FIAS-GAR xml представление в SQL-дамп.

The utility requires XSLT 3.0 transformer, because it used the xsl:iterate instruction. This feature implemented in Saxon-PE and Saxon-EE since Saxon 9.6. For using EE and PE version you will need to get trial license key.

CREATE TABLES

# create sql-dump to create all tables from xsd files in directory
sh ./create-all-tables.sh /opt/saxon/saxon-ee-10.5.jar ./gar_schemas ./gar_schemas_sql

# second way
java -jar /opt/saxon/saxon-ee-10.5.jar -xsl:./create-all-tables.xslt -it:main -o:./all-tables.sql

CONVERTING DATA

# convert data from all files in directory (insert mode)
sh ./convert.sh /opt/saxon/saxon-ee-10.5.jar ./gar_xml ./gar_xml_sql [DUMP_VERSION]

# convert data from all files in directory (delta)
# DELTA_VERSION - Must be a number that increments for each new delta file, for example 20210803
sh ./convert.sh /opt/saxon/saxon-ee-10.5.jar ./gar_xml ./gar_xml_sql [DELTA_VERSION]

Examples

sh ./convert.sh /opt/saxon/saxon-ee-10.5.jar /storage/fias-2023-09-19/16 /storage/fias-2023-09-19_sql/16 20230919

IMPORT DUMP

It is very important to use nohup to prevent accidental stopping of the importing process.

nohup sh -c 'psql -h localhost -d your_database -U your_user -f your_file.sql' &

AGREEMENTS

  • table's name is as the main noun in file name
  • column's name is lower-cased attr name without any change
  • boolean => boolean migrations
  • string with any restriction => varchar
  • int (even number enumeration), long <= 10 => int
  • long > 10 => bigint
  • guid => uuid

PROBLEMS

  • missing "AS_ADDHOUSE_TYPES.xsd" in gar_schemas however existing "AS_ADDHOUSE_TYPES_*.xml" with data
  • the delta-changes in the root folder has unchanged data
  • binary dump should be much faster

SEE