Skip to content

Commit a13768c

Browse files
author
bpinsard
committed
Merge branch 'master' into spm/gzip
* master: (47 commits) fixed applymask output fixed test Fixed some details for the PR nipy#530 Removed accidentally added files (nipy#530) Updated changelog Updated dMRI and fMRI pre-processing Added motion correction pipeline Modified FSL FLIRT interface to provide log support Polished code buildtemplateparallel.sh - issue with input files Update legacy.py - buildtemplateparallel.sh Minor fixes Added MBIS to sub-package configuration list Standardized import to add all utilities at once First implementation for MBIS New EPI dewarp workflow (nipy#525) New EPI dewarping using a workflow api: force users to explicitly define whether to sort files BF: Fix SplitNifti to generate unique filenames for outputs. added mask input for bias field correction ...
2 parents 04d1305 + ea909c7 commit a13768c

34 files changed

+1203
-6562
lines changed

CHANGES

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
11
Next release
22
============
3+
* ENH: New interfaces: nipy.Trim
34

5+
* ENH: Allow control over terminal output for commandline interfaces
6+
* ENH: New workflows for dMRI and fMRI pre-processing: added motion artifact correction
7+
with rotation of the B-matrix, and susceptibility correction for EPI imaging using
8+
fieldmaps. Updated eddy_correct pipeline to support both dMRI and fMRI, and new parameters.
9+
* ENH: Minor improvements to FSL's FUGUE and FLIRT interfaces
410

511
Release 0.7.0 (Dec 18, 2012)
612
============================

doc/users/plugins.rst

Lines changed: 57 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ available plugins allow local and distributed execution of workflows and
99
debugging. Each available plugin is described below.
1010

1111
Current plugins are available for Linear, Multiprocessing, IPython_ distributed
12-
processing platforms and for direct processing on SGE_, PBS_, Condor_, and LSF_. We
12+
processing platforms and for direct processing on SGE_, PBS_, HTCondor_, and LSF_. We
1313
anticipate future plugins for the Soma_ workflow.
1414

1515
.. note::
@@ -34,7 +34,7 @@ Optional arguments::
3434
.. note::
3535

3636
Except for the status_callback, the remaining arguments only apply to the
37-
distributed plugins: MultiProc/IPython(X)/SGE/PBS/Condor/LSF
37+
distributed plugins: MultiProc/IPython(X)/SGE/PBS/HTCondor/HTCondorDAGMan/LSF
3838

3939
For example:
4040

@@ -148,27 +148,71 @@ Optional arguments::
148148
template: custom template file to use
149149
bsub_args: any other command line args to be passed to bsub.
150150

151-
Condor
152-
------
151+
HTCondor
152+
--------
153+
154+
DAGMan
155+
~~~~~~
156+
157+
With its DAGMan_ component HTCondor_ (previously Condor) allows for submitting
158+
entire graphs of dependent jobs at once. With the ``CondorDAGMan`` plug-in
159+
Nipype can utilize this functionality to submit complete workflows directly and
160+
in a single step. Consequently, and in contrast to other plug-ins, workflow
161+
execution returns almost instantaneously -- Nipype is only used to generate the
162+
workflow graph, while job scheduling and dependency resolution are entirely
163+
managed by HTCondor_.
164+
165+
Please note that although DAGMan_ supports specification of data dependencies
166+
as well as data provisioning on compute nodes this functionality is currently
167+
not supported by this plug-in. As with all other batch systems supported by
168+
Nipype, only HTCondor pools with a shared file system can be used to process
169+
Nipype workflows.
170+
171+
Workflow execution with HTCondor DAGMan is done by calling::
172+
173+
workflow.run(plugin='CondorDAGMan')
174+
175+
Job execution behavior can be tweaked with the following optional plug-in
176+
arguments::
177+
178+
template : submit spec template to use for job submission. The template
179+
all generated submit specs are appended to this template. This
180+
can be a str or a filename.
181+
submit_specs : additional submit specs that are appended to the generated
182+
submit specs to allow for overriding or extending the defaults.
183+
This can be a str or a filename.
184+
dagman_args : arguments to be prepended to the job execution script in the
185+
dagman call
186+
187+
Please see the `HTCondor documentation`_ for details on possible configuration
188+
options and command line arguments.
189+
190+
``qsub`` emulation
191+
~~~~~~~~~~~~~~~~~~
192+
193+
.. note::
194+
195+
This plug-in is deprecated and users should migrate to the more robust and
196+
more versatile ``CondorDAGMan`` plug-in.
153197

154-
Despite the differences between Condor and SGE-like batch systems the plugin
155-
usage (incl. supported arguments) is almost identical. The Condor plugin relies
156-
on a ``qsub`` emulation script for Condor, called ``condor_qsub`` that can be
198+
Despite the differences between HTCondor and SGE-like batch systems the plugin
199+
usage (incl. supported arguments) is almost identical. The HTCondor plugin relies
200+
on a ``qsub`` emulation script for HTCondor, called ``condor_qsub`` that can be
157201
obtained from a `Git repository on git.debian.org`_. This script is currently
158-
not shipped with a standard Condor distribution, but is included in the Condor
202+
not shipped with a standard HTCondor distribution, but is included in the HTCondor
159203
package from http://neuro.debian.net. It is sufficient to download this script
160204
and install it in any location on a system that is included in the ``PATH``
161205
configuration.
162206

163207
.. _Git repository on git.debian.org: http://anonscm.debian.org/gitweb/?p=pkg-exppsy/condor.git;a=blob_plain;f=debian/condor_qsub;hb=HEAD
164208

165-
Running a workflow in a Condor pool is done by calling::
209+
Running a workflow in a HTCondor pool is done by calling::
166210

167211
workflow.run(plugin='Condor')
168212

169213
The plugin supports a limited set of qsub arguments (``qsub_args``) that cover
170214
the most common use cases. The ``condor_qsub`` emulation script translates qsub
171-
arguments into the corresponding Condor terminology and handles the actual job
215+
arguments into the corresponding HTCondor terminology and handles the actual job
172216
submission. For details on supported options see the manpage of ``condor_qsub``.
173217

174218
Optional arguments::
@@ -182,4 +226,6 @@ Optional arguments::
182226
.. _Soma: http://brainvisa.info/soma/soma-workflow/
183227
.. _PBS: http://www.clusterresources.com/products/torque-resource-manager.php
184228
.. _LSF: http://www.platform.com/Products/platform-lsf
185-
.. _Condor: http://www.cs.wisc.edu/condor/
229+
.. _HTCondor: http://www.cs.wisc.edu/htcondor/
230+
.. _DAGMan: http://research.cs.wisc.edu/htcondor/dagman/dagman.html
231+
.. _HTCondor documentation: http://research.cs.wisc.edu/htcondor/manual

0 commit comments

Comments
 (0)