Summary
mattergen-generate exposes no seeding option, so sampling is non-reproducible even on the same hardware/checkpoint. The training CLI already supports a seed, so this is an inconsistency in the inference API.
Code
mattergen/scripts/generate.py main() has no seed parameter.
- The number-of-atoms prior is drawn with an unseeded global RNG:
np.random.choice(...) in NumAtomsCrystalDataset.from_num_atoms_distribution (mattergen/common/data/dataset.py:330-334).
- The condition loader constructs
DataLoader(..., shuffle=True) with the global default RNG (mattergen/common/data/condition_factory.py:49-54).
- The reverse-diffusion sampler consumes the global
torch RNG state.
By contrast, training has a --seed argument wired into torch.manual_seed/np.random.seed/random.seed (mattergen/diffusion/run.py:167-175, 89-107).
Reproduction
mattergen-generate run1 --pretrained-name=mattergen_base --batch_size=16 --num_batches 1
mattergen-generate run2 --pretrained-name=mattergen_base --batch_size=16 --num_batches 1
diff run1/generated_crystals.extxyz run2/generated_crystals.extxyz # always differs
There is no --seed=42 that would make run1 and run2 agree.
Expected vs actual
Expected: mattergen-generate ... --seed=42 reproduces the same structures (same num-atoms sequence and initial noises) for a fixed checkpoint and hardware. Actual: every run differs, which complicates debugging, benchmarking, and regression testing.
Proposed fix
Add a seed: int | None = None argument to generate.py/CrystalGenerator, and before building the condition loader/sampler call:
torch.manual_seed(seed)
np.random.seed(seed)
random.seed(seed)
torch.cuda.manual_seed_all(seed)
Optionally thread the seed into NumAtomsCrystalDataset.from_num_atoms_distribution and DataLoader (via torch.Generator) so the num-atoms draw and the batch shuffle are reproducible too.
Summary
mattergen-generateexposes no seeding option, so sampling is non-reproducible even on the same hardware/checkpoint. The training CLI already supports a seed, so this is an inconsistency in the inference API.Code
mattergen/scripts/generate.pymain()has noseedparameter.np.random.choice(...)inNumAtomsCrystalDataset.from_num_atoms_distribution(mattergen/common/data/dataset.py:330-334).DataLoader(..., shuffle=True)with the global default RNG (mattergen/common/data/condition_factory.py:49-54).torchRNG state.By contrast, training has a
--seedargument wired intotorch.manual_seed/np.random.seed/random.seed(mattergen/diffusion/run.py:167-175, 89-107).Reproduction
mattergen-generate run1 --pretrained-name=mattergen_base --batch_size=16 --num_batches 1 mattergen-generate run2 --pretrained-name=mattergen_base --batch_size=16 --num_batches 1 diff run1/generated_crystals.extxyz run2/generated_crystals.extxyz # always differsThere is no
--seed=42that would makerun1andrun2agree.Expected vs actual
Expected:
mattergen-generate ... --seed=42reproduces the same structures (same num-atoms sequence and initial noises) for a fixed checkpoint and hardware. Actual: every run differs, which complicates debugging, benchmarking, and regression testing.Proposed fix
Add a
seed: int | None = Noneargument togenerate.py/CrystalGenerator, and before building the condition loader/sampler call:Optionally thread the seed into
NumAtomsCrystalDataset.from_num_atoms_distributionandDataLoader(viatorch.Generator) so the num-atoms draw and the batch shuffle are reproducible too.