-
Notifications
You must be signed in to change notification settings - Fork 532
FIX: Copy num_threads to MapNode-generated Nodes #2019
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
lgtm - seems like a simple fix to incorporate in |
Actually, I should probably copy the memory estimate, too. For a test, I can do something like: mapnode = pe.MapNode(...)
nodes = list(mapnode._make_nodes())
for node in nodes:
for attr in attrs_we_care_about:
assert getattr(node, attr) == getattr(mapnode, attr) Any other candidates for |
5787a98
to
edfcf84
Compare
nipype/pipeline/engine/nodes.py
Outdated
@@ -1125,6 +1125,8 @@ def _make_nodes(self, cwd=None): | |||
fieldvals = filename_to_list(getattr(self.inputs, field)) | |||
logger.debug('setting input %d %s %s', i, field, fieldvals[i]) | |||
setattr(node.inputs, field, fieldvals[i]) | |||
node._interface.num_threads = self._interface.num_threads |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
quick question, why doesn't the deepcopy a few lines (1115) back copy num_threads?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have no idea. But you can run _make_nodes
and check for yourself.
I could instead add a BaseInterface.__deepcopy__
function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In [6]: node = MapNode(BET(), iterfield=['in_file'], name='test')
In [7]: node._interface.num_threads
Out[7]: 1
In [8]: node._interface.num_threads = 5
In [9]: node._interface.num_threads
Out[9]: 5
In [10]: node.interface.num_threads
Out[10]: 5
In [11]: node.interface.num_threads = 4
In [12]: node._interface.num_threads
Out[12]: 4
In [13]: from copy import deepcopy
In [14]: deepcopy(node._interface).num_threads
Out[14]: 4
will check _make_nodes
shortly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Try:
>>> pe.Node(deepcopy(node._interface), name='copynode')._interface.num_threads
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I see the problem: In Node.__init__
, self._interface.num_threads = n_procs
, which is 1 by default...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so the issue is with this line:
https://github.com/nipy/nipype/blob/master/nipype/pipeline/engine/nodes.py#L173
therefore in MapNodes, the deepcopy statement should include n_procs=self._interface.num_threads
Node(deepcopy(node._interface),
name='copynode',
n_procs=node._interface.num_threads)._interface.num_threads
and also any other attribute that's being set there.
edfcf84
to
6dd54b2
Compare
Codecov Report
@@ Coverage Diff @@
## master #2019 +/- ##
=========================================
+ Coverage 72.19% 72.2% +<.01%
=========================================
Files 1132 1132
Lines 57012 57023 +11
Branches 8162 8165 +3
=========================================
+ Hits 41161 41173 +12
+ Misses 14567 14566 -1
Partials 1284 1284
Continue to review full report at Codecov.
|
@satra I moved all the parameters that could be moved into the |
6dd54b2
to
f4aab61
Compare
This PR copies the
num_threads
attribute from sourceMapNode
's interface to those of the generated nodes.Fixes #2016