Running all setup.py flags and tracking results

[jisantuc]

Setup Flag Tracking

Summary

This issue exists to track the results of running each setup.py flag that we think ought to work. The flags are:

• -g0: Import the first (0 based index) GeoLevel from the configuration file
• -gX: Import the Xth* (0 based index) GeoLevel from the configuration file.
• -nX: Renest the geometry and data values from the Xth* (0 based index) GeoLevel.
• -G: Configure a running Geoserver with the loaded spatial data. (requires a running geoserver)
• -a: Import adjacency data
• -V: Create database views for the spatial layers in Geoserver.
• -t: Create the template plans that are configured. This step may happen any time after the nesting step.
• -s: Collect all the static files used by the web application into a location served by apache for performance.
• -l: Create and compile a message file for each language defined.

Absence of a checkbox indicates that a flag's run should succeed, pending merge of #473. All flags with a checkbox (TODOs, basically) will have an explanation below

Notes

General

Language amibitions

Somewhere, django is getting the impression that it should be building translation files for a truly ambitious list of languages. Here's the list django thinks it should aim for:

[(u'af', u'Afrikaans'), (u'ar', u'Arabic'), (u'ast', u'Asturian'), (u'az', u'Azerbaijani'), (u'bg', u'Bulgarian'), (u'be', u'Belarusian'), (u'bn', u'Bengali'), (u'br', u'Breton'), (u'bs', u'Bosnian'), (u'ca', u'Catalan'), (u'cs', u'Czech'), (u'cy', u'Welsh'), (u'da', u'Danish'), (u'de', u'German'), (u'dsb', u'Lower Sorbian'), (u'el', u'Greek'), (u'en', u'English'), (u'en-au', u'Australian English'), (u'en-gb', u'British English'), (u'eo', u'Esperanto'), (u'es', u'Spanish'), (u'es-ar', u'Argentinian Spanish'), (u'es-co', u'Colombian Spanish'), (u'es-mx', u'Mexican Spanish'), (u'es-ni', u'Nicaraguan Spanish'), (u'es-ve', u'Venezuelan Spanish'), (u'et', u'Estonian'), (u'eu', u'Basque'), (u'fa', u'Persian'), (u'fi', u'Finnish'), (u'fr', u'French'), (u'fy', u'Frisian'), (u'ga', u'Irish'), (u'gd', u'Scottish Gaelic'), (u'gl', u'Galician'), (u'he', u'Hebrew'), (u'hi', u'Hindi'), (u'hr', u'Croatian'), (u'hsb', u'Upper Sorbian'), (u'hu', u'Hungarian'), (u'ia', u'Interlingua'), (u'id', u'Indonesian'), (u'io', u'Ido'), (u'is', u'Icelandic'), (u'it', u'Italian'), (u'ja', u'Japanese'), (u'ka', u'Georgian'), (u'kk', u'Kazakh'), (u'km', u'Khmer'), (u'kn', u'Kannada'), (u'ko', u'Korean'), (u'lb', u'Luxembourgish'), (u'lt', u'Lithuanian'), (u'lv', u'Latvian'), (u'mk', u'Macedonian'), (u'ml', u'Malayalam'), (u'mn', u'Mongolian'), (u'mr', u'Marathi'), (u'my', u'Burmese'), (u'nb', u'Norwegian Bokm\xe5l'), (u'ne', u'Nepali'), (u'nl', u'Dutch'), (u'nn', u'Norwegian Nynorsk'), (u'os', u'Ossetic'), (u'pa', u'Punjabi'), (u'pl', u'Polish'), (u'pt', u'Portuguese'), (u'pt-br', u'Brazilian Portuguese'), (u'ro', u'Romanian'), (u'ru', u'Russian'), (u'sk', u'Slovak'), (u'sl', u'Slovenian'), (u'sq', u'Albanian'), (u'sr', u'Serbian'), (u'sr-latn', u'Serbian Latin'), (u'sv', u'Swedish'), (u'sw', u'Swahili'), (u'ta', u'Tamil'), (u'te', u'Telugu'), (u'th', u'Thai'), (u'tr', u'Turkish'), (u'tt', u'Tatar'), (u'udm', u'Udmurt'), (u'uk', u'Ukrainian'), (u'ur', u'Urdu'), (u'vi', u'Vietnamese'), (u'zh-hans', u'Simplified Chinese'), (u'zh-hant', u'Traditional Chinese')]

Meanwhile, the list we aimed for previously was ['es', 'en', 'fr', 'ja']. That's a pretty big difference! I created an ENABLED_LANGUAGES setting and told django to chill out.

-gX (load geolevel X)

• should warn the user if they ask for a geolevel that isn't present in the source data
• looks for a bunch of attributes that don't appear to be in the sample data (at least, it did this on 10/19, but doesn't seem to be doing it on 10/20? very odd)
• is extremely slow for loading all three geolevels from the config. This could be improved pretty easily. Inserts should be batched to avoid making a database roundtrip for each of several thousand geounits. I'm getting about 5 geounits / second, and there are at least 9,000 of them. Captured as setup.py -gX is very slow #471

-nX (nest geolevel X)

• uses old geos interface and tries to do a unionagg, which is no longer supported

-G (geoserver setup)

• should fail louder -- appears to succeed but doesn't actually do anything

-a (adjacencies)

• blocked by absence of sample data -- the example config refers to a processed_15tiemposecciones.txt, but I have no idea where that is or what process might produce a file in an appropriate format

-s (staticfiles)

• uninstalled compressor during setup of dev environment, and now this needs it, so it needs to come back

-l (languages)

• unterminated string warnings in bg1251 coming from jqGrid -- these are annoying but might not hurt anything

[mcdisc]

If you didn't do this, can you default to the previous implemented list of languages? Btw, given updates to the program since we created ja and fr, I'm uncertain if the software will now fully support translation of these languages. es has been more recently updated for a previous Mexico deployment. ============ Dr. Michael P. McDonald Associate Professor, University of Florida 352-273-2371 www.electproject.org @ElectProject

…

On Fri, Oct 20, 2017 at 1:34 PM, James Santucci ***@***.***> wrote: Setup Flag Tracking Summary This issue exists to track the results of running each setup.py flag that we think ought to work. The flags are: - -g0: Import the first (0 based index) GeoLevel from the configuration file - -gX: Import the Xth* (0 based index) GeoLevel from the configuration file. - -nX: Renest the geometry and data values from the Xth* (0 based index) GeoLevel. - -G: Configure a running Geoserver with the loaded spatial data. (requires a running geoserver) - -a: Import adjacency data - -V: Create database views for the spatial layers in Geoserver. - -t: Create the template plans that are configured. This step may happen any time after the nesting step. - -s: Collect all the static files used by the web application into a location served by apache for performance. - -l: Create and compile a message file for each language defined. Absence of a checkbox indicates that a flag's run should succeed, pending merge of a PR that will exist in a few minutes. All flags with a checkbox (TODOs, basically) will have an explanation below Notes General Language amibitions Somewhere, django is getting the impression that it should be building translation files for a truly ambitious list of languages. Here's the list django thinks it should aim for: [(u'af', u'Afrikaans'), (u'ar', u'Arabic'), (u'ast', u'Asturian'), (u'az', u'Azerbaijani'), (u'bg', u'Bulgarian'), (u'be', u'Belarusian'), (u'bn', u'Bengali'), (u'br', u'Breton'), (u'bs', u'Bosnian'), (u'ca', u'Catalan'), (u'cs', u'Czech'), (u'cy', u'Welsh'), (u'da', u'Danish'), (u'de', u'German'), (u'dsb', u'Lower Sorbian'), (u'el', u'Greek'), (u'en', u'English'), (u'en-au', u'Australian English'), (u'en-gb', u'British English'), (u'eo', u'Esperanto'), (u'es', u'Spanish'), (u'es-ar', u'Argentinian Spanish'), (u'es-co', u'Colombian Spanish'), (u'es-mx', u'Mexican Spanish'), (u'es-ni', u'Nicaraguan Spanish'), (u'es-ve', u'Venezuelan Spanish'), (u'et', u'Estonian'), (u'eu', u'Basque'), (u'fa', u'Persian'), (u'fi', u'Finnish'), (u'fr', u'French'), (u'fy', u'Frisian'), (u'ga', u'Irish'), (u'gd', u'Scottish Gaelic'), (u'gl', u'Galician'), (u'he', u'Hebrew'), (u'hi', u'Hindi'), (u'hr', u'Croatian'), (u'hsb', u'Upper Sorbian'), (u'hu', u'Hungarian'), (u'ia', u'Interlingua'), (u'id', u'Indonesian'), (u'io', u'Ido'), (u'is', u'Icelandic'), (u'it', u'Italian'), (u'ja', u'Japanese'), (u'ka', u'Georgian'), (u'kk', u'Kazakh'), (u'km', u'Khmer'), (u'kn', u'Kannada'), (u'ko', u'Korean'), (u'lb', u'Luxembourgish'), (u'lt', u'Lithuanian'), (u'lv', u'Latvian'), (u'mk', u'Macedonian'), (u'ml', u'Malayalam'), (u'mn', u'Mongolian'), (u'mr', u'Marathi'), (u'my', u'Burmese'), (u'nb', u'Norwegian Bokm\xe5l'), (u'ne', u'Nepali'), (u'nl', u'Dutch'), (u'nn', u'Norwegian Nynorsk'), (u'os', u'Ossetic'), (u'pa', u'Punjabi'), (u'pl', u'Polish'), (u'pt', u'Portuguese'), (u'pt-br', u'Brazilian Portuguese'), (u'ro', u'Romanian'), (u'ru', u'Russian'), (u'sk', u'Slovak'), (u'sl', u'Slovenian'), (u'sq', u'Albanian'), (u'sr', u'Serbian'), (u'sr-latn', u'Serbian Latin'), (u'sv', u'Swedish'), (u'sw', u'Swahili'), (u'ta', u'Tamil'), (u'te', u'Telugu'), (u'th', u'Thai'), (u'tr', u'Turkish'), (u'tt', u'Tatar'), (u'udm', u'Udmurt'), (u'uk', u'Ukrainian'), (u'ur', u'Urdu'), (u'vi', u'Vietnamese'), (u'zh-hans', u'Simplified Chinese'), (u'zh-hant', u'Traditional Chinese')] Meanwhile, the list we aimed for previously was ['es', 'en', 'fr', 'ja']. That's a pretty big difference! I created an ENABLED_LANGUAGES setting and told django to chill out. -gX (load geolevel X) - should warn the user if they ask for a geolevel that isn't present in the source data - looks for a bunch of attributes that don't appear to be in the sample data (at least, it did this on 10/19, but doesn't seem to be doing it on 10/20? very odd) - is *extremely* slow for loading all three geolevels from the config. This could be improved pretty easily. Inserts should be batched to avoid making a database roundtrip for each of several thousand geounits. I'm getting about 5 geounits / second, and there are at least 9,000 of them. Captured as #471 <PublicMapping/districtbuilder#471> -nX (nest geolevel X) - uses old geos interface and tries to do a unionagg, which is no longer supported -G (geoserver setup) - should fail louder -- appears to succeed but doesn't actually do anything -a (adjacencies) - blocked by absence of sample data -- the example config refers to a processed_15tiemposecciones.txt, but I have no idea where that is or what process might produce a file in an appropriate format -s (staticfiles) - uninstalled compressor during setup of dev environment, and now this needs it, so it needs to come back -l (languages) - unterminated string warnings in bg1251 coming from jqGrid -- these are annoying but might not hurt anything — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <PublicMapping/districtbuilder#472>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBMSVSjDHJFII-w8mpFZps_ArivPVupks5suNmygaJpZM4QA_Q6> .

[jisantuc]

Thanks for letting us know about updated Spanish translations. The ENABLED_LANGUAGES flag I created is set to ['es', 'en', 'fr', 'ja'] for now, so those four will be supported. We're not yet working on filling in the specific translation strings.

The application is in a state of flux right now as we try to get it updated.