Skip to content

Commit fa76747

Browse files
committed
feat(timeline): show what an audio pill crops, and let it slip
An audio pill is the only timeline object that edits media you cannot see. Every other pill holds a value over a span, and a clip's crop produces a clip that is right there on screen; resizing an audio pill crops an invisible file. The edges already stop at the content — `lowerLeft` and `maxEnd` in the lane drag, with the loop exception that lets a repeating track outrun its source. What was missing is any way to SEE where that content is before you hit the stop. - **Ghost extent.** The file's waveform is drawn dimmed around the pill, at the same row and height, so the pill reads as a window onto a tape. Clamped to the programme, so a four-minute bed under a five-second view asks for a bounded element rather than one tens of screens wide. - **Readout.** While an edge is pulled — or the media slipped — a chip pinned to the pointer shows `in → out / length`. `0:00.0` and `out = length` are the boundary states, self-evident without copy, which is why this adds no locale key. - **Slip.** `Alt` + drag inside the pill slides the media under a span that does not move. This is what the ghost makes necessary rather than optional: an edge drag sets the in-point at TIMELINE scale, so reaching 3:00 inside a four-minute bed on a five-second view meant dragging three minutes of ruler. The rate comes from the FILE instead — one viewport width traverses all of it — floored at the timeline's own scale so a slip is never slower than moving the pill would be. `navSpan` is in the callback's dependencies for that reason: a zoom that leaves `total` alone still changes the rate. `slipAudioOffsetMs` clamps to `[0, duration - span]` and rounds, because `offsetMs` is `z.number().int()` and a fractional slip would fail the parse on the next save rather than at the gesture. It and `audioGhostExtent` both refuse an unknown duration rather than inventing a bound — the same rule the edge stops already follow, so a failed probe never freezes a pill.
1 parent 49023c0 commit fa76747

4 files changed

Lines changed: 368 additions & 60 deletions

File tree

‎src/components/ai-edition/v4/EditorShellV4.module.css‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1514,6 +1514,44 @@
15141514
/* Imported audio tracks (issue #350). A distinct teal so a BGM/voiceover track
15151515
reads apart from the effect pills, and taller than a lane pill so the waveform
15161516
inside it is legible. */
1517+
/* The rest of an audio pill's tape (audioGhostExtent): where its file's content still
1518+
sits around the pill. Dimmed and unclickable, BELOW the pill, so the pill reads as a
1519+
window onto it and an edge drag shows what is still available on each side.
1520+
1521+
Height and row are set inline to match the pill exactly. They have to: a ghost that
1522+
does not line up with the pill reads as a separate object sitting behind it rather
1523+
than as the rest of the same strip. */
1524+
.lanePillGhost {
1525+
position: absolute;
1526+
min-width: 1px;
1527+
overflow: hidden;
1528+
border-radius: 6px;
1529+
/* Faint enough to read as "not the pill". Louder than this and it draws the eye to
1530+
the part you are NOT editing — and on a long file it spans the whole ruler. */
1531+
border: 1px dashed color-mix(in oklch, var(--accent) 18%, transparent);
1532+
pointer-events: none;
1533+
z-index: 1;
1534+
}
1535+
.lanePillGhost .tlWave {
1536+
opacity: 0.18;
1537+
}
1538+
/* Crop readout pinned to the pointer while an audio pill's edge is pulled, or while
1539+
Alt slips the media under it — in -> out over the file's length. Rendered at the
1540+
component root (see the JSX note about the canvas transform). */
1541+
.tlDragTip {
1542+
position: fixed;
1543+
z-index: 1200;
1544+
transform: translate(-50%, calc(-100% - 18px));
1545+
padding: 3px 8px;
1546+
border-radius: 6px;
1547+
border: 1px solid var(--border-hi);
1548+
background: var(--bg);
1549+
color: var(--fg);
1550+
font-size: 11px;
1551+
font-variant-numeric: tabular-nums;
1552+
white-space: nowrap;
1553+
pointer-events: none;
1554+
}
15171555
.laneAudio {
15181556
border-color: #14b8a6;
15191557
background: rgba(20, 184, 166, 0.16);

‎src/components/ai-edition/v4/V4Timeline.tsx‎

Lines changed: 195 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,12 @@ import { ZOOM_DEPTH_SCALES } from "@/components/video-editor/types";
3333
import { useScopedT } from "@/contexts/I18nContext";
3434
import { useShortcuts } from "@/contexts/ShortcutsContext";
3535
import { useAudioPeaks } from "@/hooks/useAudioPeaks";
36-
import { collapseTracksToPills, packAudioTrackRows } from "@/lib/ai-edition/document/audioTracks";
36+
import {
37+
audioGhostExtent,
38+
collapseTracksToPills,
39+
packAudioTrackRows,
40+
slipAudioOffsetMs,
41+
} from "@/lib/ai-edition/document/audioTracks";
3742
import { createId } from "@/lib/ai-edition/document/ids";
3843
import { setUiProbeScrubbing } from "@/lib/ai-edition/perf/uiFrameProbe";
3944
import type { AxcutAudioTrack, AxcutClip } from "@/lib/ai-edition/schema";
@@ -376,6 +381,7 @@ const AudioLanePill = memo(function AudioLanePill({
376381
onSelect,
377382
label,
378383
outputGain,
384+
ghost,
379385
}: {
380386
track: AxcutAudioTrack;
381387
url: string | undefined;
@@ -401,69 +407,106 @@ const AudioLanePill = memo(function AudioLanePill({
401407
/** Linear project output gain, applied on top of the track gain — the mixer
402408
* applies both, so the bars must too or they under-read the exported level. */
403409
outputGain: number;
410+
/** Where the rest of the file sits around the pill, as percentages of the canvas
411+
* and the source window it covers. Absent when there is nothing to show. */
412+
ghost?: {
413+
leftPct: number;
414+
widthPct: number;
415+
sourceStartSec: number;
416+
sourceEndSec: number;
417+
} | null;
404418
}) {
405419
const duration = assetDurationSec ?? track.durationSec;
406420
return (
407-
<div
408-
role="button"
409-
tabIndex={0}
410-
className={`${styles.lanePill} ${styles.laneAudio}${
411-
selected ? ` ${styles.lanePillSel}` : ""
412-
}`}
413-
style={{
414-
left: `${leftPct}%`,
415-
width: `${widthPct}%`,
416-
minWidth: 3,
417-
top: AUDIO_LANE_PAD_PX + row * rowHeight,
418-
height: AUDIO_ROW_HEIGHT_PX,
419-
}}
420-
// Body drag moves the track; it also selects and stops the .tlTracks scrub.
421-
onPointerDown={(e) => onStartDrag(e, track, "move")}
422-
onKeyDown={(e) => {
423-
if (e.key === "Enter" || e.key === " ") {
424-
e.preventDefault();
425-
onSelect(track.id);
426-
}
427-
}}
428-
title={label}
429-
>
430-
<span
431-
className={styles.lanePillHandle}
432-
style={{ left: 0 }}
433-
onPointerDown={(e) => onStartDrag(e, track, "l")}
434-
/>
435-
<ClipWaveform
436-
videoUrl={url}
437-
assetDurationSec={duration}
438-
sourceStartSec={sourceStartSec}
439-
sourceEndSec={sourceEndSec}
440-
// Track gain AND the project output gain — `finish_audio` applies both and
441-
// clamps, so scaling by the track gain alone under-read a boosted output.
442-
gain={audioGainScalar(track.gainDb) * outputGain}
443-
/>
444-
{/* Where the file starts over, so a looping bed reads as one deliberate
421+
<>
422+
{/* The rest of the tape, dimmed and unclickable, behind the pill — so the pill
423+
reads as a window onto it and an edge drag shows what is still available on
424+
each side before it hits the stop. Same height and row as the pill: a ghost
425+
that does not line up reads as a separate object sitting behind it. */}
426+
{ghost ? (
427+
<div
428+
aria-hidden
429+
className={styles.lanePillGhost}
430+
style={{
431+
left: `${ghost.leftPct}%`,
432+
width: `${ghost.widthPct}%`,
433+
top: AUDIO_LANE_PAD_PX + row * rowHeight,
434+
height: AUDIO_ROW_HEIGHT_PX,
435+
}}
436+
>
437+
<ClipWaveform
438+
videoUrl={url}
439+
assetDurationSec={duration}
440+
sourceStartSec={ghost.sourceStartSec}
441+
sourceEndSec={ghost.sourceEndSec}
442+
gain={audioGainScalar(track.gainDb) * outputGain}
443+
/>
444+
</div>
445+
) : null}
446+
<div
447+
role="button"
448+
tabIndex={0}
449+
className={`${styles.lanePill} ${styles.laneAudio}${
450+
selected ? ` ${styles.lanePillSel}` : ""
451+
}`}
452+
style={{
453+
left: `${leftPct}%`,
454+
width: `${widthPct}%`,
455+
minWidth: 3,
456+
top: AUDIO_LANE_PAD_PX + row * rowHeight,
457+
height: AUDIO_ROW_HEIGHT_PX,
458+
}}
459+
// Body drag moves the track; it also selects and stops the .tlTracks scrub.
460+
onPointerDown={(e) => onStartDrag(e, track, "move")}
461+
onKeyDown={(e) => {
462+
if (e.key === "Enter" || e.key === " ") {
463+
e.preventDefault();
464+
onSelect(track.id);
465+
}
466+
}}
467+
title={label}
468+
>
469+
<span
470+
className={styles.lanePillHandle}
471+
style={{ left: 0 }}
472+
onPointerDown={(e) => onStartDrag(e, track, "l")}
473+
/>
474+
<ClipWaveform
475+
videoUrl={url}
476+
assetDurationSec={duration}
477+
sourceStartSec={sourceStartSec}
478+
sourceEndSec={sourceEndSec}
479+
// Track gain AND the project output gain — `finish_audio` applies both and
480+
// clamps, so scaling by the track gain alone under-read a boosted output.
481+
gain={audioGainScalar(track.gainDb) * outputGain}
482+
/>
483+
{/* Where the file starts over, so a looping bed reads as one deliberate
445484
repeat rather than a mystery. Only drawn when the pill actually
446485
outruns its source — otherwise there is nothing to repeat. */}
447-
{track.loop && loopWindowSec > 0
448-
? Array.from({ length: Math.min(200, Math.ceil(spanSec / loopWindowSec) - 1) }, (_, i) => (
449-
<span
450-
key={`loop-${i + 1}`}
451-
data-testid="audio-loop-mark"
452-
className={styles.laneLoopMark}
453-
style={{ left: `${(((i + 1) * loopWindowSec) / spanSec) * 100}%` }}
454-
/>
455-
))
456-
: null}
457-
<span className={styles.laneAudioLabel}>
458-
<Music size={11} />
459-
{label}
460-
</span>
461-
<span
462-
className={styles.lanePillHandle}
463-
style={{ right: 0 }}
464-
onPointerDown={(e) => onStartDrag(e, track, "r")}
465-
/>
466-
</div>
486+
{track.loop && loopWindowSec > 0
487+
? Array.from(
488+
{ length: Math.min(200, Math.ceil(spanSec / loopWindowSec) - 1) },
489+
(_, i) => (
490+
<span
491+
key={`loop-${i + 1}`}
492+
data-testid="audio-loop-mark"
493+
className={styles.laneLoopMark}
494+
style={{ left: `${(((i + 1) * loopWindowSec) / spanSec) * 100}%` }}
495+
/>
496+
),
497+
)
498+
: null}
499+
<span className={styles.laneAudioLabel}>
500+
<Music size={11} />
501+
{label}
502+
</span>
503+
<span
504+
className={styles.lanePillHandle}
505+
style={{ right: 0 }}
506+
onPointerDown={(e) => onStartDrag(e, track, "r")}
507+
/>
508+
</div>
509+
</>
467510
);
468511
});
469512

@@ -936,6 +979,16 @@ export function V4Timeline({
936979
trimEnd: number;
937980
} | null>(null);
938981
const audioDragRef = useRef<typeof audioDrag>(null);
982+
/** `in -> out / length` pinned to the pointer while an audio edge is pulled or the
983+
* media is slipped under the pill. Rendered at the component root: the lane sits
984+
* inside the zoomed canvas transform, which would scale a chip placed in it. */
985+
const [audioDragTip, setAudioDragTip] = useState<{
986+
x: number;
987+
y: number;
988+
inSec: number;
989+
outSec: number;
990+
durationSec: number;
991+
} | null>(null);
939992
// The user-visible tracks and their lane rows. Packed from the STORED spans,
940993
// not the live drag geometry: a pill that changed rows halfway through a drag
941994
// would jump out from under the pointer.
@@ -969,6 +1022,17 @@ export function V4Timeline({
9691022
const origStart = track.startMs / 1000;
9701023
const origTrimStart = track.offsetMs / 1000;
9711024
const origTrimEnd = origTrimStart + spanSec;
1025+
// Alt inside the pill slips it: the span stays put and the media slides under
1026+
// it. On the BODY only — the edges keep their crop semantics.
1027+
//
1028+
// An edge drag sets the in-point at TIMELINE scale, which is unusable once the
1029+
// file is much longer than the pill: reaching 3:00 inside a four-minute bed on
1030+
// a five-second view means dragging three minutes of ruler. So the slip rate is
1031+
// derived from the FILE — one viewport width traverses all of it — floored at
1032+
// the timeline's own scale so a slip is never slower than moving the pill,
1033+
// which would be its own surprise on a file shorter than the view.
1034+
const slipping = mode === "move" && e.altKey && sourceLen > 0;
1035+
const slipSecPerPx = Math.max(total / r.width, sourceLen / Math.max(1, r.width * navSpan));
9721036
// A looping track may be pulled out PAST the end of its file — that is
9731037
// the whole point of looping, and capping at the source length is what
9741038
// made the loop toggle do nothing: the span could never exceed the
@@ -1002,6 +1066,33 @@ export function V4Timeline({
10021066
return best;
10031067
};
10041068
const move = (ev: PointerEvent) => {
1069+
if (slipping) {
1070+
const nextOffsetMs = slipAudioOffsetMs(
1071+
track.offsetMs,
1072+
track.endMs - track.startMs,
1073+
sourceLen,
1074+
(ev.clientX - startX) * slipSecPerPx * 1000,
1075+
);
1076+
if (nextOffsetMs == null) return;
1077+
const nextTrimStart = nextOffsetMs / 1000;
1078+
setAudioDragTip({
1079+
x: ev.clientX,
1080+
y: ev.clientY,
1081+
inSec: nextTrimStart,
1082+
outSec: nextTrimStart + spanSec,
1083+
durationSec: sourceLen,
1084+
});
1085+
// The span does not move; only the window onto the file does.
1086+
const slipState = {
1087+
id: track.id,
1088+
start: origStart,
1089+
trimStart: nextTrimStart,
1090+
trimEnd: nextTrimStart + spanSec,
1091+
};
1092+
audioDragRef.current = slipState;
1093+
setAudioDrag(slipState);
1094+
return;
1095+
}
10051096
const dxSec = ((ev.clientX - startX) / r.width) * total;
10061097
let ns = origStart;
10071098
let nts = origTrimStart;
@@ -1034,12 +1125,25 @@ export function V4Timeline({
10341125
origTrimStart + Math.max(0, total - origStart),
10351126
);
10361127
}
1128+
// The readout answers "where am I in the file", which is the one thing the
1129+
// pill cannot show: its edges stop at the content, but nothing said where
1130+
// that content was.
1131+
if (mode !== "move") {
1132+
setAudioDragTip({
1133+
x: ev.clientX,
1134+
y: ev.clientY,
1135+
inSec: nts,
1136+
outSec: nte,
1137+
durationSec: sourceLen,
1138+
});
1139+
}
10371140
const next = { id: track.id, start: ns, trimStart: nts, trimEnd: nte };
10381141
audioDragRef.current = next;
10391142
setAudioDrag(next);
10401143
};
10411144
const up = () => {
10421145
setSnapPct(null);
1146+
setAudioDragTip(null);
10431147
window.removeEventListener("pointermove", move);
10441148
window.removeEventListener("pointerup", up);
10451149
const fin = audioDragRef.current;
@@ -1067,7 +1171,10 @@ export function V4Timeline({
10671171
window.addEventListener("pointermove", move);
10681172
window.addEventListener("pointerup", up);
10691173
},
1070-
[tl, total, clips, pxPerSec],
1174+
// navSpan: the slip rate is derived from the VISIBLE width, so a zoom that
1175+
// leaves `total` alone still changes it. Left out, the rate froze at whatever
1176+
// the zoom was when the callback was last built.
1177+
[tl, total, clips, pxPerSec, navSpan],
10711178
);
10721179

10731180
const startNavDrag = useCallback(
@@ -1946,6 +2053,24 @@ export function V4Timeline({
19462053
onSelect={tl.selectAudioTrack}
19472054
label={track.label || asset?.label || ts("audioTrack.defaultLabel")}
19482055
outputGain={audioGainScalar(settings.audioGainDb)}
2056+
ghost={((g) =>
2057+
g
2058+
? {
2059+
leftPct: pctOf(g.startT),
2060+
widthPct: pctOf(g.endT - g.startT),
2061+
sourceStartSec: g.sourceStartSec,
2062+
sourceEndSec: g.sourceEndSec,
2063+
}
2064+
: null)(
2065+
audioGhostExtent(
2066+
trimStart,
2067+
widthSec,
2068+
duration,
2069+
start,
2070+
start + widthSec,
2071+
total,
2072+
),
2073+
)}
19492074
/>
19502075
);
19512076
})
@@ -2119,6 +2244,16 @@ export function V4Timeline({
21192244
</div>
21202245
</div>
21212246
) : null}
2247+
{/* The crop readout, at the component ROOT rather than in the lane: the lane
2248+
sits inside the zoomed canvas transform, which would scale a chip placed
2249+
there. `in -> out / length` — 0:00.0 and out = length are the boundary
2250+
states, self-evident without copy, which is why this adds no locale key. */}
2251+
{audioDragTip ? (
2252+
<div className={styles.tlDragTip} style={{ left: audioDragTip.x, top: audioDragTip.y }}>
2253+
{formatSec(audioDragTip.inSec)} → {formatSec(audioDragTip.outSec)}
2254+
{audioDragTip.durationSec > 0 ? ` / ${formatSec(audioDragTip.durationSec)}` : ""}
2255+
</div>
2256+
) : null}
21222257
</div>
21232258
);
21242259
}

0 commit comments

Comments
 (0)