Skip to content

Conversation

@antgonza
Copy link
Member

Before this patch we didn't keep track of deleting jobs and this was the behavior:
2902_before

The main issues were:

  • The artifact id was not being parse as string so JS couldn't be updated
  • The internal jobs were not accounted as part of the artifacts, which mean that they weren't able to be track
  • We didn't have a method to query if an artifact was being deleted; because it was not easy to track
  • Once we have those things we can use in the GUI to reflect the delete status

2902

@codecov-io
Copy link

codecov-io commented Apr 27, 2020

Codecov Report

Merging #2994 into dev will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##              dev    #2994   +/-   ##
=======================================
  Coverage   94.97%   94.98%           
=======================================
  Files          74       74           
  Lines       14124    14151   +27     
=======================================
+ Hits        13414    13441   +27     
  Misses        710      710           
Impacted Files Coverage Δ
qiita_db/artifact.py 98.28% <100.00%> (+0.01%) ⬆️
qiita_db/processing_job.py 72.02% <100.00%> (+0.07%) ⬆️
qiita_db/test/test_artifact.py 99.85% <100.00%> (+<0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a23e49...dfdf13b. Read the comment docs.

@antgonza antgonza changed the title [WIP] fix #2902 fix #2902 Apr 27, 2020
@antgonza antgonza requested a review from ElDeveloper April 27, 2020 14:12
@antgonza antgonza mentioned this pull request Apr 28, 2020
2 tasks
Copy link
Contributor

@ElDeveloper ElDeveloper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few questions/changes. To confirm, is this going to work with the analysis UI as well?

TTRN.add(sql, [artifact_info, job_id])
else:
pending[artifact_info[0]][pname] = artifact_info[1]
elif pname == 'artifact':
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind describing what's going on here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, the create_command method in patch 58.py adds the delete_artifact and other Qiita internal commands; however, the artifact_id is in the parameter "artifact" vs. the parameter description (*); thus, if we want to add these commands to the database we need this special case.

(*) all regular jobs, except internal Qiita, have their parameters in this format: parameter_name: [parameter_type, value]; so normally, we just check that parameter_type is artifact but for internal jobs we need to check the actual parameter.

job.submit()
# let's wait for job
while job.status in ('queued', 'running'):
sleep(1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking around the repo and found qiita_core.testing.wait_for_processing_job. I think that would be able to do this.

# n[1] is the object
for n in graph.nodes():
if n[0] == 'job':
# ignoring internal Jobs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need a new test case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if is needed as a consequence of now adding the internal jobs to the database; my guess is that originally we didn't have it cause we didn't store them. In other words, the test for these method would need to change if we didn't have this if (as now the jobs will be returned here) AKA it's being tested here.

Copy link
Member Author

@antgonza antgonza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ElDeveloper !

TTRN.add(sql, [artifact_info, job_id])
else:
pending[artifact_info[0]][pname] = artifact_info[1]
elif pname == 'artifact':
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, the create_command method in patch 58.py adds the delete_artifact and other Qiita internal commands; however, the artifact_id is in the parameter "artifact" vs. the parameter description (*); thus, if we want to add these commands to the database we need this special case.

(*) all regular jobs, except internal Qiita, have their parameters in this format: parameter_name: [parameter_type, value]; so normally, we just check that parameter_type is artifact but for internal jobs we need to check the actual parameter.

# n[1] is the object
for n in graph.nodes():
if n[0] == 'job':
# ignoring internal Jobs
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if is needed as a consequence of now adding the internal jobs to the database; my guess is that originally we didn't have it cause we didn't store them. In other words, the test for these method would need to change if we didn't have this if (as now the jobs will be returned here) AKA it's being tested here.

Copy link
Contributor

@ElDeveloper ElDeveloper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @antgonza!

@ElDeveloper ElDeveloper merged commit 13f12db into qiita-spots:dev Apr 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants