Skip to content

Regular guideline pipeline lacks a minimal-trajectory-length guard before segmentation #293

Description

@jayaramkr

Summary

The regular guideline pipeline passes trajectories to segment_trajectory with no minimum-length guard. A single-step (or otherwise too-short) trajectory can be "segmented" into ≥2 subtasks that all slice into the same one step, producing multiple near-identical per-segment guidelines from a single step.

The consistency pipeline already guards against this (PR #289 added n_scorable_steps >= 2 and _can_segment_trajectory(...) before segmenting), so the two paths are now inconsistent.

Where

generate_guidelines() in altk_evolve/llm/guidelines/guidelines.py (~L191):

subtasks = []
if evolve_config.segmentation_enabled:
    from altk_evolve.llm.guidelines.segmentation import segment_trajectory
    try:
        subtasks = segment_trajectory(messages)   # <-- no n_steps guard
    except Exception as e:
        ...

There is no check on n_steps / len(steps_list) before calling segment_trajectory.

Why it fans out

For a single-step trajectory (n_steps == 1), if the segmentation LLM returns two subtasks both spanning [1, 1]:

  • start = min(max(0, 1 - 1), 1) = 0, end = min(max(0, 1), 1) = 1start < end → the slice passes the range filter.
  • Both subtasks produce steps_list[0:1] — the same single step.
  • len(valid_slices) >= 2 is satisfied, so the pipeline emits one guideline-generation call per identical slice.

Net effect (reported by @evduester): a single-step smolagent trajectory in both mode yields ~13 guidelines — ~10 from the regular pipeline (multiple identical segments) + 3 consistency. The regular-pipeline inflation is the segmentation-without-a-length-guard behavior described here.

Suggested fix

Mirror the consistency path's guard in the regular path — skip segmentation when the trajectory is too short to segment meaningfully, e.g.:

if evolve_config.segmentation_enabled and n_steps >= 2:
    subtasks = segment_trajectory(messages)

Optionally, also short-circuit inside segment_trajectory itself so every caller benefits, and/or de-duplicate identical [start, end] slices before generating (defense in depth against an LLM returning overlapping/duplicate ranges even on longer trajectories).

Scope note

Surfaced during review of #289, but intentionally out of scope for that PR — #289 fixes the consistency path; this is the parallel gap in the regular path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions