Skip to content

FIX: Set default result in DistributedPluginBase._clean_queue #2596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 26, 2018

Conversation

achetverikov
Copy link
Contributor

Fixing this: #2570

Changes proposed in this pull request

  • calls to _clean_queue in plugins/base.py now always have the information about exceptions when it's available

@effigies
Copy link
Member

Please remove the unrelated auto test. That is being added elsewhere, and will currently cause this PR to fail.

@effigies
Copy link
Member

Also, I think a cleaner solution would be to set this default in _clean_queue, as it should not crash when its default argument is set.

def _clean_queue(self, jobid, graph, result=None):
logger.debug('Clearing %d from queue', jobid)
if self._status_callback:
self._status_callback(self.procs[jobid], 'exception')
if str2bool(self._config['execution']['stop_on_first_crash']):
raise RuntimeError("".join(result['traceback']))

I would suggest adding at line 225:

if result is None:
    result={'result': None,
            'traceback': '\n'.join(format_exception(*sys.exc_info()))}

@effigies effigies added this to the 1.0.4 milestone May 25, 2018
@effigies
Copy link
Member

@achetverikov Are you going to submit an updated PR? I'd like to get all fixes in today for the next release, and this seems like a quick one.

@achetverikov
Copy link
Contributor Author

yes, wait a sec, I just did a reset on my fork and apparently that closed the PR as well

@achetverikov achetverikov reopened this May 25, 2018
@achetverikov
Copy link
Contributor Author

check the updated commit

@codecov-io
Copy link

codecov-io commented May 25, 2018

Codecov Report

Merging #2596 into master will decrease coverage by 0.01%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #2596      +/-   ##
=========================================
- Coverage   67.62%   67.6%   -0.02%     
=========================================
  Files         336     336              
  Lines       42711   42713       +2     
  Branches     5278    5279       +1     
=========================================
- Hits        28883   28877       -6     
- Misses      13151   13158       +7     
- Partials      677     678       +1
Flag Coverage Δ
#smoketests 50.74% <0%> (-0.01%) ⬇️
#unittests 65.07% <0%> (-0.03%) ⬇️
Impacted Files Coverage Δ
nipype/pipeline/plugins/base.py 59.55% <0%> (-0.34%) ⬇️
nipype/pipeline/plugins/multiproc.py 79.76% <0%> (-2.39%) ⬇️
nipype/utils/profiler.py 43.1% <0%> (-1.15%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9eaa2a3...1e0ac27. Read the comment docs.

Copy link
Member

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thanks. Just a couple style issues and I think this will be good to go.

'result': None,
'traceback': '\n'.join(format_exception(*sys.exc_info()))
}))
graph))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe let's pull all this back onto one line:

self._clean_queue(jobid, graph)

@@ -222,6 +218,9 @@ def _clean_queue(self, jobid, graph, result=None):

if self._status_callback:
self._status_callback(self.procs[jobid], 'exception')
if result is None:
result={'result': None,
'traceback': '\n'.join(format_exception(*sys.exc_info()))}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you're using tabs, not spaces. Do you think you could update this to use 4-space indentation? Also, there should be spaces around the =, and 'traceback' should start on the same column as 'result'.

(Sorry if this is a pain in your editor.)

@achetverikov
Copy link
Contributor Author

done, I think

Copy link
Member

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. Thanks for this.

@achetverikov
Copy link
Contributor Author

you're doing great work guys, I'm glad to be of help

@effigies effigies changed the title [Fix] always add exception info to _clean_queue FIX: Set default result in DistributedPluginBase._clean_queue May 26, 2018
@effigies effigies merged commit 1fcc5d1 into nipy:master May 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants