Summary
The regular guideline pipeline passes trajectories to segment_trajectory with no minimum-length guard. A single-step (or otherwise too-short) trajectory can be "segmented" into ≥2 subtasks that all slice into the same one step, producing multiple near-identical per-segment guidelines from a single step.
The consistency pipeline already guards against this (PR #289 added n_scorable_steps >= 2 and _can_segment_trajectory(...) before segmenting), so the two paths are now inconsistent.
Where
generate_guidelines() in altk_evolve/llm/guidelines/guidelines.py (~L191):
subtasks = []
if evolve_config.segmentation_enabled:
from altk_evolve.llm.guidelines.segmentation import segment_trajectory
try:
subtasks = segment_trajectory(messages) # <-- no n_steps guard
except Exception as e:
...
There is no check on n_steps / len(steps_list) before calling segment_trajectory.
Why it fans out
For a single-step trajectory (n_steps == 1), if the segmentation LLM returns two subtasks both spanning [1, 1]:
start = min(max(0, 1 - 1), 1) = 0, end = min(max(0, 1), 1) = 1 → start < end → the slice passes the range filter.
- Both subtasks produce
steps_list[0:1] — the same single step.
len(valid_slices) >= 2 is satisfied, so the pipeline emits one guideline-generation call per identical slice.
Net effect (reported by @evduester): a single-step smolagent trajectory in both mode yields ~13 guidelines — ~10 from the regular pipeline (multiple identical segments) + 3 consistency. The regular-pipeline inflation is the segmentation-without-a-length-guard behavior described here.
Suggested fix
Mirror the consistency path's guard in the regular path — skip segmentation when the trajectory is too short to segment meaningfully, e.g.:
if evolve_config.segmentation_enabled and n_steps >= 2:
subtasks = segment_trajectory(messages)
Optionally, also short-circuit inside segment_trajectory itself so every caller benefits, and/or de-duplicate identical [start, end] slices before generating (defense in depth against an LLM returning overlapping/duplicate ranges even on longer trajectories).
Scope note
Surfaced during review of #289, but intentionally out of scope for that PR — #289 fixes the consistency path; this is the parallel gap in the regular path.
Summary
The regular guideline pipeline passes trajectories to
segment_trajectorywith no minimum-length guard. A single-step (or otherwise too-short) trajectory can be "segmented" into ≥2 subtasks that all slice into the same one step, producing multiple near-identical per-segment guidelines from a single step.The consistency pipeline already guards against this (PR #289 added
n_scorable_steps >= 2 and _can_segment_trajectory(...)before segmenting), so the two paths are now inconsistent.Where
generate_guidelines()inaltk_evolve/llm/guidelines/guidelines.py(~L191):There is no check on
n_steps/len(steps_list)before callingsegment_trajectory.Why it fans out
For a single-step trajectory (
n_steps == 1), if the segmentation LLM returns two subtasks both spanning[1, 1]:start = min(max(0, 1 - 1), 1) = 0,end = min(max(0, 1), 1) = 1→start < end→ the slice passes the range filter.steps_list[0:1]— the same single step.len(valid_slices) >= 2is satisfied, so the pipeline emits one guideline-generation call per identical slice.Net effect (reported by @evduester): a single-step smolagent trajectory in
bothmode yields ~13 guidelines — ~10 from the regular pipeline (multiple identical segments) + 3 consistency. The regular-pipeline inflation is the segmentation-without-a-length-guard behavior described here.Suggested fix
Mirror the consistency path's guard in the regular path — skip segmentation when the trajectory is too short to segment meaningfully, e.g.:
Optionally, also short-circuit inside
segment_trajectoryitself so every caller benefits, and/or de-duplicate identical[start, end]slices before generating (defense in depth against an LLM returning overlapping/duplicate ranges even on longer trajectories).Scope note
Surfaced during review of #289, but intentionally out of scope for that PR — #289 fixes the consistency path; this is the parallel gap in the regular path.