Skip to content

mattergen-generate has no seeding option — sampling is non-reproducible (training CLI already supports --seed) #256

Description

Summary

mattergen-generate exposes no seeding option, so sampling is non-reproducible even on the same hardware/checkpoint. The training CLI already supports a seed, so this is an inconsistency in the inference API.

Code

  • mattergen/scripts/generate.py main() has no seed parameter.
  • The number-of-atoms prior is drawn with an unseeded global RNG: np.random.choice(...) in NumAtomsCrystalDataset.from_num_atoms_distribution (mattergen/common/data/dataset.py:330-334).
  • The condition loader constructs DataLoader(..., shuffle=True) with the global default RNG (mattergen/common/data/condition_factory.py:49-54).
  • The reverse-diffusion sampler consumes the global torch RNG state.

By contrast, training has a --seed argument wired into torch.manual_seed/np.random.seed/random.seed (mattergen/diffusion/run.py:167-175, 89-107).

Reproduction

mattergen-generate run1 --pretrained-name=mattergen_base --batch_size=16 --num_batches 1
mattergen-generate run2 --pretrained-name=mattergen_base --batch_size=16 --num_batches 1
diff run1/generated_crystals.extxyz run2/generated_crystals.extxyz   # always differs

There is no --seed=42 that would make run1 and run2 agree.

Expected vs actual

Expected: mattergen-generate ... --seed=42 reproduces the same structures (same num-atoms sequence and initial noises) for a fixed checkpoint and hardware. Actual: every run differs, which complicates debugging, benchmarking, and regression testing.

Proposed fix

Add a seed: int | None = None argument to generate.py/CrystalGenerator, and before building the condition loader/sampler call:

torch.manual_seed(seed)
np.random.seed(seed)
random.seed(seed)
torch.cuda.manual_seed_all(seed)

Optionally thread the seed into NumAtomsCrystalDataset.from_num_atoms_distribution and DataLoader (via torch.Generator) so the num-atoms draw and the batch shuffle are reproducible too.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions