Skip to content

Bpipe Version 0.9.11

Compare
Choose a tag to compare
@ssadedin ssadedin released this 04 Sep 07:39
· 351 commits to master since this release
  • FEATURE: support for required attribute on notification
    channels to ensure pipeline failure if channel config fails

  • FEATURE: experimental support for virtual file extensions / file type
    mappings , defined using: filetype <virtual ext> : [ 'ext.one', 'ext.two',...]

  • FEATURE: agent will wait for commands to finish if it is asked to run
    multiple in the same directory at the same time

  • FEATURE: support for "queue" command : schedules pipeline to run after
    another one finishes in a directory

  • FIX: only match process owned by user when checking for running bpipe process

  • FEATURE(experimental): from clause can now be added in pipeline definition to
    cause inputs to prioritise from a given stage, with branch matching

  • FEATURE: update aws sdk to support latest instance types

  • FEATURE: EC2 commands now show name of stage via name tag, eg. in aws console

  • FEAUTURE: accept aws creds from environment if not defined in bpipe.config
    using standard AWS env variables for token / secret

  • FEATURE: support direct transfer of files via scp for AWSEC2 executor.
    If storage config setting is not defined and 'transfer' setting is true
    then Bpipe will utilise scp to transfer input/output files to EC2 instances

  • FEATURE: allow queue to be overridden within pipeline when sending
    to JMS channel via explicit 'queue' property

  • FEATURE: support replyTo as property for replying to JMS messages
    (rather than JMSReplyTo)

  • FEATURE: improved support for archiving: autoarchive and ability to run
    archive command from outside bpipe dir

  • FEATURE: implement support for 'required' and 'unique' as options on
    gitlab messages : ensures pipeline reports error status if
    gitlab calls fail

  • FEATURE: support for experimental "when" feature to make stages conditional

  • FEATURE: mask 'token' config property as sensitive info from log files

  • FEATURE: support for setting R_LIBS_USER with R { r_libs="..." }
    setting in bpipe.config

  • FEATURE: add checks to result message returned in reply-to on
    JMS messages

  • FEATURE: allow specification of gitlab projects by id when
    sending to gitlab

  • FEATURE: show custom title in bpipe startup when configured
    using title : '...' or title '...'

  • FEATURE: support to set env variables for commands via bpipe.config,
    by added env = : , ... to command config

  • FIX: http send not retried on bpipe retry if initial attempt failed

  • FIX: errors printed by trying to send JSON via SMTP

  • FIX: adjust bpipe agent jvm options to reduce footprint on large servers

  • FIX: fix transform of wildcard (eg: *.txt) to multiple outputs
    resulting in regex error

  • FIX: undefined pipeline variable somtimes shows bpipe internal stack trace

  • FIX: possible concurrentmodification errors when user-specificed config
    referenced concurrently in stages / reports

  • FIX: fix multilevel branch names not reflected in file names

  • FIX: error thrown when creating diagram using 'bpipe diagram ...'

  • Support for reservation of gpus with torque (set gpus=N in command config)

  • FIX: body of message relayed in JMS message header (causing parse fail in
    some clients / protocols)

  • FIX: fix regex in transform matching multi inputs only resolving single

  • FIX: fix error when literal regex used as transform input specifier

  • FIX: previous dirty state carrying over causing unexpected
    cleanup of good files

  • FIX: put retry around output dir watcher to handle race conditions
    / file system hiccups better

  • FIX: Explicitly set pwd for singularity as this seems not always
    to occur by default

  • FIX:: fix downstream reference to output created by check failing
    on retry

  • FIX: inability to create gitlab issue with empty content

  • FIX: json output double encoded in .bpipe/sent files

  • FIX: some commands not running with set -e by default

  • FIX:: fix warning printed for dirty files even though
    they don't exist, already notified

  • FIX: remove duplicates from printing of test outputs

  • FIX: remove printout of shebang stripped statement