Merge relax-md-req into master

[ElDeveloper]

No description provided.

[ElDeveloper]

DO NOT MERGE, PLEASE TEST IN YOUR SYSTEM BEFORE ... I'm finding an error, but I'm not sure if it's legitimate or something else is wrong.

[antgonza]

@josenavas quick question: when patching the db I'm getting a lot of QiitaDBWarning: Some columns required to generate a QIIME-compliant mapping file are not present in the template. A placeholder value (XXQIITAXX) has been used to populate these columns. Missing columns: LinkerPrimerSequence, BarcodeSequence QiitaDBWarning), is that expected ? I'm asking cause my guess is that these are from EMP

[antgonza]

Also, getting this error:

Traceback (most recent call last):
  File "/Users/antoniog/svn_programs/qiita/scripts/qiita_env", line 157, in <module>
    env()
  File "/Users/antoniog/.virtualenvs/qiita/lib/python2.7/site-packages/click/core.py", line 610, in __call__
    return self.main(*args, **kwargs)
  File "/Users/antoniog/.virtualenvs/qiita/lib/python2.7/site-packages/click/core.py", line 590, in main
    rv = self.invoke(ctx)
  File "/Users/antoniog/.virtualenvs/qiita/lib/python2.7/site-packages/click/core.py", line 936, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/antoniog/.virtualenvs/qiita/lib/python2.7/site-packages/click/core.py", line 782, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/antoniog/.virtualenvs/qiita/lib/python2.7/site-packages/click/core.py", line 416, in invoke
    return callback(*args, **kwargs)
  File "/Users/antoniog/svn_programs/qiita/scripts/qiita_env", line 108, in patch
    _patch()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/environment_manager.py", line 377, in patch
    execfile(py_patch_fp)
  File "/Users/antoniog/svn_programs/qiita/qiita_db/support_files/patches/python_patches/23.py", line 14, in <module>
    SampleTemplate(s_id).generate_files()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/sample_template.py", line 201, in generate_files
    self.to_file(fp)
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/base_metadata_template.py", line 994, in to_file
    df = self.to_dataframe()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/base_metadata_template.py", line 1024, in to_dataframe
    meta = conn_handler.execute_fetchall(sql, (self._id,))
  File "/Users/antoniog/svn_programs/qiita/qiita_db/sql_connection.py", line 454, in execute_fetchall
    with self._sql_executor(sql, sql_args) as pgcursor:
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/sql_connection.py", line 303, in _sql_executor
    (sql, str(sql_args), e)))
qiita_db.exceptions.QiitaDBExecutionError: 
Error running SQL query: SELECT 
                 FROM qiita.study_sample
                 JOIN qiita.sample_10021 USING (sample_id)
                 WHERE study_id = %s
ARGS: (10021L,)
Error: syntax error at or near "FROM"
LINE 2:                  FROM qiita.study_sample
                         ^

[antgonza]

The last reported errors are due to inconsistencies in my local DB due to tests while writing the delete functions.

Anyway, now I'm getting this and I'm almost certain that is an actual bug.

Traceback (most recent call last):
  File "qiita_db/support_files/patches/python_patches/23.py", line 14, in <module>
    SampleTemplate(s_id).generate_files()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/sample_template.py", line 201, in generate_files
    self.to_file(fp)
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/base_metadata_template.py", line 994, in to_file
    df = self.to_dataframe()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/base_metadata_template.py", line 1024, in to_dataframe
    meta = conn_handler.execute_fetchall(sql, (self._id,))
  File "/Users/antoniog/svn_programs/qiita/qiita_db/sql_connection.py", line 454, in execute_fetchall
    with self._sql_executor(sql, sql_args) as pgcursor:
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/sql_connection.py", line 303, in _sql_executor
    (sql, str(sql_args), e)))
qiita_db.exceptions.QiitaDBExecutionError: 
Error running SQL query: SELECT abx_past_6_mos, age, age_in_years, age_unit, altitude, anatomical_sample_site, anonymized_name, assigned_from_geo, atherosclerosis, barcode_read_group_tag, barcodesequence, bmi_185_249, bmi_25_299, bmi_30_349, bmi_35_399, bmi_greater_40, bmi_less_184, body_habitat, body_mass_index, body_product, body_site, chronic_condition, clinical_condition, collectday, collection_date, collection_timestamp, common_name, common_sample_site, contaminated, country, delivery_method, depth, description, description_duplicate, dna_extracted, elevation, env_biome, env_feature, env_matter, experiment_alias, experiment_center, experiment_design_description, family_relationship, flxdata_published, gender, hiv, hmpbodysubsite, hmpbodysupersite, host_common_name, host_individual, host_subject_id, host_taxid, hypertension, includes_timeseries, intph, investigation_type, key_seq, lab_person, lab_person_contact, latitude, library_construction_protocol, linkerprimersequence, longitude, miens_compliant, mislabeled, months_without_antibiotics, myocardinfarct, newilluminadatagenerated, nucl_acid_amp, nucl_acid_ext, obesity, original_sample_site, pcr_cond, pcr_primers, pfph, physical_specimen_location, physical_specimen_remaining, platform, pmid, pool_member_accession, pool_member_name, pool_proportion, primer_read_group_tag, principal_investigator, project_name, psn, region, required_sample_info_status, reverseprimer, run, run_alias, run_center, run_date, run_lane, run_prefix, samp_collect_device, samp_size, sample_center, sample_id, sample_type, sequencing_meth, sex, smoker, specific_body_site, srs, study_abstract, study_id, study_ref, url, visitno
                 FROM qiita.study_sample
                 JOIN qiita.sample_850 USING (sample_id)
                 WHERE study_id = %s
ARGS: (850L,)
Error: column reference "study_id" is ambiguous
LINE 1: ... smoker, specific_body_site, srs, study_abstract, study_id, ...
                                                             ^

[ElDeveloper]

@antgonza, when are you getting this error.

On (Apr-29-15| 7:49), Antonio Gonzalez wrote:

The last reported errors are due to inconsistencies in my local DB due to tests while writing the delete functions.

Anyway, now I'm getting this and I'm almost certain that is an actual bug.

Traceback (most recent call last):
  File "qiita_db/support_files/patches/python_patches/23.py", line 14, in <module>
    SampleTemplate(s_id).generate_files()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/sample_template.py", line 201, in generate_files
    self.to_file(fp)
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/base_metadata_template.py", line 994, in to_file
    df = self.to_dataframe()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/metadata_template/base_metadata_template.py", line 1024, in to_dataframe
    meta = conn_handler.execute_fetchall(sql, (self._id,))
  File "/Users/antoniog/svn_programs/qiita/qiita_db/sql_connection.py", line 454, in execute_fetchall
    with self._sql_executor(sql, sql_args) as pgcursor:
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/antoniog/svn_programs/qiita/qiita_db/sql_connection.py", line 303, in _sql_executor
    (sql, str(sql_args), e)))
qiita_db.exceptions.QiitaDBExecutionError: 
Error running SQL query: SELECT abx_past_6_mos, age, age_in_years, age_unit, altitude, anatomical_sample_site, anonymized_name, assigned_from_geo, atherosclerosis, barcode_read_group_tag, barcodesequence, bmi_185_249, bmi_25_299, bmi_30_349, bmi_35_399, bmi_greater_40, bmi_less_184, body_habitat, body_mass_index, body_product, body_site, chronic_condition, clinical_condition, collectday, collection_date, collection_timestamp, common_name, common_sample_site, contaminated, country, delivery_method, depth, description, description_duplicate, dna_extracted, elevation, env_biome, env_feature, env_matter, experiment_alias, experiment_center, experiment_design_description, family_relationship, flxdata_published, gender, hiv, hmpbodysubsite, hmpbodysupersite, host_common_name, host_individual, host_subject_id, host_taxid, hypertension, includes_timeseries, intph, investigation_type, key_seq, lab_person, lab_person_contact, latitude, library_construction_protocol, linkerprimersequenc
 e, longi
 tude, miens_compliant, mislabeled, months_without_antibiotics, myocardinfarct, newilluminadatagenerated, nucl_acid_amp, nucl_acid_ext, obesity, original_sample_site, pcr_cond, pcr_primers, pfph, physical_specimen_location, physical_specimen_remaining, platform, pmid, pool_member_accession, pool_member_name, pool_proportion, primer_read_group_tag, principal_investigator, project_name, psn, region, required_sample_info_status, reverseprimer, run, run_alias, run_center, run_date, run_lane, run_prefix, samp_collect_device, samp_size, sample_center, sample_id, sample_type, sequencing_meth, sex, smoker, specific_body_site, srs, study_abstract, study_id, study_ref, url, visitno
                 FROM qiita.study_sample
                 JOIN qiita.sample_850 USING (sample_id)
                 WHERE study_id = %s
ARGS: (850L,)
Error: column reference "study_id" is ambiguous
LINE 1: ... smoker, specific_body_site, srs, study_abstract, study_id, ...
                                                             ^

---

Reply to this email directly or view it on GitHub:
#1122 (comment)

[antgonza]

When patching, as you can tell from the log. We are working on a fix.

[ElDeveloper]

Ah, got it, when I patched my DB it worked just fine (I have ~40-60 EMP
studies).

On (Apr-29-15| 9:38), Antonio Gonzalez wrote:

When patching, as you can tell from the log. We are working on a fix.

---

Reply to this email directly or view it on GitHub:
#1122 (comment)

[antgonza]

The issue is on those studies that come from the old db that have a field called study_id ....