Skip to content

Release notes

Jump to bottom
alberto edited this page Sep 14, 2023 · 18 revisions

Release 3.23.07 - Morpheus

Xmipp Programs

  • New programs
    • image_peak_high_contrast(for detecting high contrast regions in tomographic reconstruction)
    • misaligment_detection (to detect misalignment in tomographic reconstructions from high-contrast regions)
    • deep_global_assignment
    • deep_center
  • Programs updated
    • xmipp_angular_distance: new features
    • tomo_extract_particles: new features
    • subtract_projection: parallelization with mpi
    • tomo_extract_subtomograms: allow downsampling of features
    • angular_resolution_alignment: Detect misalignment with resolution
  • Programs fixed
    • align_volume_and_particles: Fixed error
    • angular_project_library: Removed deterministic behaviour (mpi)
    • volumen_subtraction: fixed bug
  • Deprecated programs (For more details visit)
    • classify_kmeans_2D
    • rotational_spectra
    • particle_boxsize

Installation and user guide

  • New clearer, more intuitive and informative installer. It also creates a file to facilitate user support.
  • Refactored the deep learning toolkit, more robust and new environment with updated tensorflow.
  • Updated requirement.

More Xmipp

  • Added half precission suport to numpy
  • Added the ability to read and write CIF files
  • Modular design of winner filter
  • Decoupling XmippTomo from XmippSPA
  • Fix Zernike equation
  • Removed all occurrences of non base-class default destructors
  • Improved MultidimArray performance
  • Added support for half precision floating point numbers in MRCs
  • Assign tiff to gain files
  • adding half maps labels

Release 3.23.03 - Kratos

Xmipp Programs

  • New programs: tomo_confidence_map, tomo_extract_particlestacks, tomo_extract_subtomograms, tomo_tiltseries_dose_filter, psd_estimatator

  • Deprecated programs (for more details visit the wiki): angular_distribution_show, apropos ctf_correct_idr, ctf_create_ctfdat , ctf_show , idr_xray_tomo , image_common_lines , metadata_convert_to_spider , metadata_selfile_create , mlf_refine_3d, ml_refine_3d, ml_tomo , mrc_create_metadata , pdb_construct_dictionary, pdb_restore_with_dictionary , reconstruct_admn , reconstruct_art_pseudo , resolution_ibw , resolution_ssnr , score_micrograph , reconstruct_fourier_starpu , tomo_align_tilt_series, tomo_align_dual_tilt_series, tomo_align_refinement, tomo_align_refinement, tomo_extract_subvolume, tomo_project_main, tomo_remove_fluctuations , tomo_align_tilt_series,transform_range_adjust , validation_tilt_pairs , volume_pca , volume_validate_pca , work_test , 6f4d983 , evaulate_coordinates , extract_subset , image_separate_objects , volume_enhance_contrast , volume_reslice , xray_import , xray_project , xray_psf_create , xray_reconstruct_art , gpu_correlation, gpu_utils, classify_significant, deepAlign.

  • volume_from_pdb: fixing input pdb file being overwritten when '-centerPDB' flag was set

  • xmipp_phantom_movie: adding support for fixed step shift & gain and dark image generation

  • CTF simulation allows astigmatism

  • xmipp_metadata_utility: Now join operations with an empty set will return a new empty set (previously no output file was generated).

  • xmipp_matrix_dimred: Program help improved. Exception is now thrown when the number of output dimensions is larger than the input dimensions

  • xmipp_angular_distance: Added itemId column to the output

Installation and user guide

  • Refactor and simplified Readme page.
  • Updating CUDA version compatibility
  • Updating gcc version availables
  • Fixed Matlab installation
  • Added missing array include to fix compilation error with g++12
  • Alert and not block compilation if gcc - CUDA are not compatible
  • Avoid compilation warnings
  • Required pyworkflow==3.0.31

Others

  • Maintenance: Recovered python binding tests
  • Maintenance: fixing dangling pointer in xmipp_error
  • Maintenance: Cleaned includes in xmipp_image_base
  • PSD estimation: templating function, improving performance
  • Flag cleanDeprecate in the installation; clean all deprecated executables programs
  • python binding: fixed bug when Numpy arrays created by slicing were badly interpretted
  • Removed "seed" library
  • Fixed memory pinning CUDA bug
  • Fixed compilation errors on CUDA 9

Release 3.22.11 - Iris

Xmipp Programs

  • Speeding up iterations in some xmipp programs (xmipp_ctf_group, xmipp_image_histogram, xmipp_mpi_angular_class_average, xmipp_angular_distance, xmipp_angular_estimate_tilt_axis, xmipp_ctf_create_ctfdat, xmipp_resolution_ssnr)
  • New Zernike3D programs
  • angular_project_library: Reported some error if there are no images in the range
  • angular_discrete_assign.cpp: Removed memory leak and uninitialized values
  • angular_distance: Fixing condition to avoid iteration behind the end of the MD in cases when input data have different sizes. Optimized performance
  • Pdb_reduce_pseudoatoms: Produced pdb is one-based indexed
  • xmipp_micrograph_automatic_picking: Fixing memory leak
  • subtract_projection: Fixed several bugs (improved results), added circular mask to avoid edge artifacts, added option to boost particles instead of subtract

Protocols scipion-em-xmipp

  • Protocol_cl2d_align: The input can now be a set of averages or a set of 2D classes

  • Protocol_local_ctf: Default value are now changed for maxDefocusChange

  • Protocol_apply_zernike3d: Now accepts either a Volume or SetOfVolumes and applies the coefficients in a loop in the deform step

  • Protocol_postProcessing_deepPostProcessing: Managed GPU memory to avoid errors

  • Protocol_resolution_deepres: Mandatory mask

  • Protocol center particles and Gl2d (all options): Fix streaming

  • Protocol_create_3d_mask: Allows volume Null=True

  • Protocol_reconstruct_fourier: Set pixel size

  • GL2D static: Bug fixing

  • Protocol_trigger_data: Bug fixing

  • Protocol_crop_resize: Set sampling rate of mrc files when cropping resizing volumes or particles

  • subtract_projection: New protocol for boosting particles. Add protocol to wizard XmippParticleMaskRadiusWizard as now the protocol uses it

  • New tests: deep_hand, pick_noise, screen_deep_learning, resolution_B_factor

  • Fixed TestHighres test

Installation and user guide

  • Various bug fixing
  • More information about hdf5 library
  • Updating CUDA - GCC compatibility. Added CUDA 11.7 (not tested)
  • Updating Readme

Others

  • Performance optimization (metadata binding)
  • Python binding: adding methods to directly set / get entire MD row
  • g++ >= 8 required
  • In viewers used pwutils
  • The pdb data library now has all the right fields and should write the record type ("ATOM " or "HETATM") correctly at the beginning of the line and the atomType (element) and charge (if applicable) correctly at the end of the line.
  • Removal of an artifact of symmetrization related to the z pitch (symmetries.cpp)
  • Using the same identical Deprecated param from pyworkflow.

Release 3.22.07 - Helios

Scripts Xmipp

  • xmipp_image_operate: taked into account non existing files
  • angular_continuous_assign2: Bug fixed
  • volume_consensus: Bug fixed
  • ctf.h and angular_continuous_assign_2: Changes for local defocus estimation #578

Protocols scipion-em-xmipp

  • rotate_volume: New protocol

  • subtract_projection: New implementation based on adjustment by regression instead of POCS and improved performance

  • local_ctf: Add new sameDefocus option + formatting

  • compare_reprojections & protocol_align_volume: Fast Fourier by default

  • crop_resize: Allows input pointers

  • resolution_deepres: Resize output to original size

  • denoise_particles: Added setOfAverages as input option

  • process: Change output from stk (spider) to mrcs (mrc)

  • trigger_data: Bug fixed

  • screen_deeplearning: Added descriptive help"

  • center_particles: Added summary info

  • align_volume_and_particles: Summary error fixed

  • cl2d: Summary errors solved

  • New tests: test_protocol_reconstruct_fourier, test_protocols_local_defocus, test_protocols_local_defocus, TestXmippAlignVolumeAndParticles, TestXmippRotateVolume

  • Improved tests: test_protocols_deepVolPostprocessing, test_protocols_xmipp_3d, Test ProjSubtracion

  • Excluded tests: test_protocols_zernike3d, test_protocols_metaprotocol_heterogeneity

Installation and user guide

  • Version info printed at the end of the installation
  • Removed empty folder with cleanBin command
  • Clarifing linking to Scipion and removed a bug with the build link
  • New flag (OPENCV_VERSION) in xmipp.config
  • Updated Readme (explain OpenCV-CUDA support)

Others

  • Validation server: Merged what remains
  • Replaced sincos to sin and cos
  • Handling of pointers in MPI programs
  • "nullptr" used to denote the null pointer not "NULL"
  • Check if nvidiaDriverVer is None

Release 3.22.04 - Gaia

Installation and user guide

  • Updated readme
    • Updated hdf5 info troubleshoting
    • Updated Standalone installation
    • Updated Scons installation
  • xmipp get_models: fixing the run and download path
  • Updating xmipp links for Scipion on installation
  • Removed fatal message in installation
  • Reported error if happen on installation - runjob
  • Ensuring that target directory for the libraries exists

Protocols scipion-em-xmipp

  • protocol_core_analysis: New protocol
  • protocol_compare_angles: Bug fix in compare angles under some conditions
  • protocol_center_particles: protocol simplified (removed setofCoordinates as output)
  • protocol_CTF_consensus: concurrency error fixed
  • protocol_convert_pdb: remove size if deactivated
  • protocol_resolution_deepres: binary masked not stored in Extra folder and avoiding memory problems on GPUs
  • protocol_add_noise: fixes
  • protocol_compare_reprojections: improve computation of residuals + tests + fix + formatting
  • protocol_screen_deepConsensus: multiple fixes in batch processing, trainging and streaming mode
  • protocol_shift_particles: apply transform is now optional

Others

  • New XMIPP logo
  • subtract_projection: adding new flag + fix
  • Add intersection size metadata (bindings/python)
  • Fixed unitialized unique pointers (bindings/python)
  • Bug fixing: Resolution directional and anisotropic filtering fixing the test
  • Removed SonarCloud issues
    • Replaced defines with constexpr
    • Removing Unused funtion parameters
    • Division by zero
    • Memory management
    • Removed field shadowing
    • Destructors should not throw exceptions

Release 3.22.01 - Eris

  • Updating to C++17
  • Support newer versions of CUDA and gcc
  • Zernike programs compatible with Cuda 8.x
  • Fixed Sonar Cloud issues and bugs
  • Matlab compilation Fixed
  • Fixed importing pwem.metadata
  • nma_alignment: Fixed arguments for the xmipp_angular_projection_matching invocation
  • Fixed test fails: ResolutionSsnr, ReconstructArtMpi, ReconstructArt, MlfRefine3dMpi, MlfRefine3d, MlRefine3dMpi, MlRefine3d, xmipp_test_pocs_main & volume_subtraction
  • xmipp_micrograph_automatic_picking: Fixed tests, avoid possible memory corruption
  • resolution_pdb_bfactor: bug fixed - error with multiple chains
  • FlexAlign: Fixed crash when binning > 1
  • Bug fixed and allowed controlling high sampling rate
  • Volume consensus: Fixed number of levels in the wavelet transform
  • Compilation: Fixed compilation of starpu programs
  • xmipp_transform_dimred: Fixed output metadata in append mode, adding MDL_DIMRED label
  • Config file generation: Fixed config version detection outside of the git repo, refactored check_CUDA and managed gcc compiler if it is installed out of /usr/bin/, check and exit if xmipp.conf does not exist
  • Compilation: Fixed detection of the last commit changed the config script
  • Resolution_fso: Bingham test implemented
  • Opencv not detected. Added include to user/include/opencv4 folder on config file
  • Compilation: asking whether to continue with compilation even though the config file is outdated
  • XMIPP install: Linked libsvm to scipion
  • Installation: Referenced 'global' xmipp.conf instead of using local copy of it
  • Multiple MPI programs: replaced CREATE_MPI_METADATA_PROGRAM macro by templated class
  • python_constants: add defocus labels
  • Metadata: added new nmaEigenval label
  • Python binding: added new function - correlationAfterAlignment, MDL_RESOLUTION_ANISOTROPY, MDL_RESOLUTION_ANISOTROPY
  • Matlab binding dependencies: set XMIPP as a hard dependency
  • Projections subtraction: new program
  • FFTwT: added mutex for plan handling
  • Multiple programs: Added a common implementation of the rerun
  • Phantom_create: update info link
  • Multiple programs: Added a common implementation of the rerun
  • Transform Geometry: save new shifted coordinates in option "shift to" + enterOfMass to python binding
  • Readme info: add virtual machine info
  • Removal of the SVM from inside the XMIPP repository and downloading it as an external dependence
  • Solved a configuration problem with CUDA
  • ml_tomo: Using .mrc instead of .vol ; volume_align: Addded wrapping during alignment

Release 3.21.06 - Caerus

  • CUDA-11 support
  • New protocol: Deep align
  • ChimeraX support
  • Improvements of streaming process
  • Several performance optimizations
  • Build time optimization
  • Multiple bug fixes
  • Improved documentation

Release 3.20.07 -

  • Fast CTF estimation
  • CTF includes phase shifts now
  • Selection of alpha helices or beta sheets from a PDB (xmipp_pdb_select)
  • Centering a PDB (xmipp_pdb_center)
  • New Protocol: MicrographCleaner is a new algorithm that removes coordinates picked from carbon edges, aggregations, ice crystals and other contaminations
  • New functionality: The protocol compare reprojections can now compute the residuals after alignment
  • New protocol: Split frames divide input movies into odd and even movies so that they can be processed independently
  • New protocol: Continuous heterogeneity analysis using spherical harmonics (not ready to be used)
  • Bug fixing when some micrograph has no coordinates in the consensus-picking.
  • New functionalities: Different architectures and training modes
  • Normal Mode Analysis protocols have been moved to the plugin ContinuousFlex
  • Fixing MPI version of the Fourier Reconstruction
  • New protocol: local CTF integration and consensus protocol for local ctf (also the viewers)
  • Local CTF analysis tools: Not yet ready for general public
  • New functionallity: Introducing the posibility of automatic estimation of the gain orientation.
  • Bugs fixings regarding stability on streaming processing
  • Support of heterogeneous movie sets
  • New protocol: Clustering of subtomogram coordinates into connected components that can be processed independently
  • New Protocol: Removing duplicated coordinates
  • New protocol: Subtomograms can be projected in several ways to 2D images so that 2D clustering tools can be used
  • New protocol: Regions of Interest can be defined in tomograms (e.g., membranes)
  • Bug fixing in mask3d protocol
  • Bug fix: in helical search symmetry protocol
  • Enhanced precision of the FlexAlign program
  • Now, deepLearningToolkit is under its own conda environment
  • Multiple protocols accelerated using GPU
  • New functionality: Xmipp CTF estimation can now take a previous defocus and do not change it
  • New functionallity: CTF-consensus is able to take the primary main values or an average of the two.
  • New functionallity: CTF-consensus is able to append metadata from the secondary input
  • New functionality: Xmipp Highres can now work with non-phase flipped images
  • New functionality: Xmipp Preprocess particles can now phase flip the images
  • New protocol: Tool to evaluate the quality of a map-model fitting
  • Allowing multi-GPU processing using FlexAlign
  • Improvement in monores and localdeblur
  • Randomize phases also available for images
  • Change the plugin to the new Scipion structure
  • Migrating the code to python3

Release 3.19.04 - Boreas

  • New protocols:

    • New protocol: 3D bionotes
    • New protocol: Align volume and particles
    • New protocol: Center particles
    • New protocols: GL2D, GL2D streaming and GL2D static
    • New protocol: 2D kmeans clustering
    • New protocol: compare angles
    • New protocol: consensus 3D classes
    • New protocol: CTF consensus
    • New protocol: deep denoising
    • New protocols: Eliminate empty particles and eliminate empty classes
    • New protocol: Extract unit cell
    • New protocol: Generate reprojections
    • New protocol: metaprotocol heterogenety output, metaprotocol heterogeneity subset and metaprotocol heterogeneity
    • New protocol: Movie Max Shift
    • New protocol: particle boxsize
    • New protocol: pick noise
    • New protocol: significant heterogeneity
    • New protocol: swarm consensus intial volumes
    • New protocol: directional ResDir
    • New protocol: local monoTomo
    • New protocol: deep consensus picking
    • New protocol: screen deep learning
    • New protocol: split volume hierarchical
    • New protocol: trigger data

  • New features:

    • Highres can now take a global alignment performed by any other method
    • A short presentation with some of the new algorithms can be seen here.

  • New code structure:

This release is the first one after a big re-factorization of the code in several repositories. The previous Xmipp version was inside the Scipion's repository. However, we have now splited the code in 4 Xmipp repositories:

  • Xmipp Core: This repository contains the most basic software for cryo-EM (coded in C++), such us multidimensional matrices, image definition, Fast Fourier Transform, metadata management, symmetry groups, Xmipp programs...
  • Xmipp Software: This repository contains image processing methods for cryo-EM, data management... (mostly coded in C++)
  • Xmipp Viz: This repository contains visualizations programs for cryo-EM data, such us Movies, Micrographs, Particles, Coordinates, stack of images... (coded in java).
  • Xmipp plugin: This is the plugin to be installed in Scipion.

Developed by xmipp team

Clone this wiki locally