Skip to content

docs: Add SyncPoints for alphaTex #136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Danielku15 merged 2 commits into from
May 30, 2025
Merged

docs: Add SyncPoints for alphaTex #136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b35d8f4
feat: Add alphaTex generation to sync point editor
Danielku15 May 30, 2025
cde82d5
docs: Add SyncPoints for alphaTex
Danielku15 May 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions docs/alphatex/introduction.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import { AlphaTexSample } from '@site/src/components/AlphaTexSample';

In this section you find all details about how to write music notation using AlphaTex.
AlphaTex is a text format for writing music notation for AlphaTab. AlphaTex loading
can be enabled by specifying `data-tex="true"` on the container element.
AlphaTab will load the tex code from the element contents and parse it.
can be enabled by setting the [`tex`](/docs/reference/settings/core/tex) option or loading it via [`tex()`](/docs/reference/api/tex) method on the API.
AlphaTab will load the tex code from the element contents and parse it. You can also load it from a file like other formats.
AlphaTex supports most of the features alphaTab supports overall.
If you find anything missing you would like to see, feel free to
[initiate a Discussion on GitHub](https://github.com/CoderLine/alphaTab/discussions/new) so we can find a good solution together.
Expand All @@ -30,3 +30,23 @@ Here is an example score fully rendered using alphaTex.
15.1.8 :16 14.1{tu 3} 15.1{tu 3} 14.1{tu 3} :8 17.2 15.1 14.1 :16 12.1{tu 3} 14.1{tu 3} 12.1{tu 3} :8 15.2 14.2 |
12.2 14.3 12.3 15.2 :32 14.2{h} 15.2{h} 14.2{h} 15.2{h} 14.2{h} 15.2{h} 14.2{h} 15.2{h} 14.2{h} 15.2{h} 14.2{h} 15.2{h} 14.2{h} 15.2{h} 14.2{h} 15.2{h}
`}</AlphaTexSample>


## General Song Structure

alphaTex has the following structure variations. Comments are supported in C-style comments via `// Single Line` and `/* Multi Line */`.


```title=General File Structure
/* Song Metadata */
.
/* Song Contents */
.
/* Sync Points */
```

The Song Metadata and Sync Points are optional but the dots are mandatory to separate the sections in case there is any content filled.

* Song Metadata: This section contains all information generally about the song like title.
* Song Contents: This section contains defines the whole song contents with all the tracks, staves, bars, beats, notes that alphaTab supports. Bars are separated by `|` symbols.
* Sync Points: alphaTab can be synchronized with external media like audio backing tracks or videos. To have the correct cursor display and highlighting, songs have to be synchronized. This section defines such markers.
34 changes: 34 additions & 0 deletions docs/alphatex/sync-points.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: Sync Points
---


alphaTex support specifying sync points for the [synchronization with external media](/docs/guides/audio-video-sync).

The related sync points are specified as flat list at the end of the song contents separated by a dot `.`.
As we consider it unlikely that authors write this information manually, we separated the sync points from the other song.
This way tools like our [Media Sync Editor](/docs/playground/) on the Playground can be used to synchronize songs and
the sync info can be copy-pasted after the main song.

The supported formats of sync points are:

* `\sync BarIndex Occurence MillisecondOffset`
* `\sync BarIndex Occurence MillisecondOffset RatioPosition`

Where:

* `BarIndex` is the numeric (0-based) index of the bar for which the sync point applies.
* `Occurence` is the numeric (0-based) index of bar repetitions. e.g. on Repeats or Jumps bars might be played multiple times. This value allows specifying points on subsequent plays of a bar.
* `MillisecondOffset` is the numeric timestamp in milliseconds in the external audio.
* `RatioPosition` is the relative offset within the bar at which the sync point is placed (0 if not provided).

The `BarIndex`, `Occurence`, `RatioPosition` values define the absolute position within the music sheet.
The `MillisecondOffset` defines the absolute position within the external media.

With this information known, alphaTab can synchronize the external media with the music sheet.

The sample below uses an audio backing track with inconsistent tempos. The sync points correct the tempo differences and the cursor is placed correctly.

import { AlphaTexSyncPointSample } from '@site/src/components/AlphaTexSyncPointSample';

<AlphaTexSyncPointSample />
1 change: 1 addition & 0 deletions sidebars.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ const sidebars: SidebarsConfig = {
"alphatex/note-effects",
"alphatex/percussion",
"alphatex/lyrics",
"alphatex/sync-points",
],
},
showcase: [
Expand Down
15 changes: 13 additions & 2 deletions src/components/AlphaTabPlayground/media-sync-editor.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ import {
buildSyncPointInfoFromSynth,
syncPointsToTypeScriptCode,
syncPointsToCSharpCode,
syncPointsToKotlinCode
syncPointsToKotlinCode,
syncPointsToAlphaTex
} from './sync-point-info';
import { WaveformCanvas } from './waveform-canvas';
import { SyncPointMarkerPanel } from './sync-point-marker-panel';
Expand Down Expand Up @@ -374,7 +375,8 @@ export const MediaSyncEditor: React.FC<MediaSyncEditorProps> = ({
values={[
{ label: 'TypeScript', value: 'ts' },
{ label: 'C#', value: 'cs' },
{ label: 'Kotlin', value: 'kt' }
{ label: 'Kotlin', value: 'kt' },
{ label: 'alphaTex', value: 'at' }
]}>
<TabItem value="ts">
<CodeBlock language="typescript" title="SyncPoints">
Expand Down Expand Up @@ -462,6 +464,15 @@ export const MediaSyncEditor: React.FC<MediaSyncEditorProps> = ({
].join('\n')}
</CodeBlock>
</TabItem>
<TabItem value="at">
Place the following alphaTex at the end of your alphaTex song for applying the sync points on load.
<CodeBlock title="alphaTex Sync Points">
{[
'.',
syncPointsToAlphaTex(syncPointInfo),
].join('\n')}
</CodeBlock>
</TabItem>
</Tabs>
<div className={styles['modal-actions']}>
<button
Expand Down
207 changes: 20 additions & 187 deletions src/components/AlphaTabPlayground/sync-point-info.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,124 +303,13 @@ function buildSyncPointMarkers(api: alphaTab.AlphaTabApi): SyncPointMarker[] {
syncTime: syncTime,
synthTime: synthTime,
synthBpm: synthBpm,
//syncBpm: syncBpm,

markerType: SyncPointMarkerType.EndMarker,
});
} else {
//lastSyncPoint.syncBpm = syncBpm;
lastSyncPoint.markerType = SyncPointMarkerType.EndMarker;
}



// for (const masterBar of api.tickCache!.masterBars) {
// const occurence = occurences.get(masterBar.masterBar.index) ?? 0;
// occurences.set(masterBar.masterBar.index, occurence + 1);

// const duration = masterBar.end - masterBar.start;

// if (masterBar.masterBar.syncPoints) {
// // if we have sync points we have to correctly walk through the points and tempo changes
// // and place the markers accordingly

// let tempoChangeIndex = 0;
// for (const syncPoint of masterBar.masterBar.syncPoints) {
// if (syncPoint.syncPointValue!.barOccurence !== occurence) {
// continue;
// }

// const syncPointTick = masterBar.start + syncPoint.ratioPosition * duration;

// // first process all tempo change until this sync point
// while (
// tempoChangeIndex < masterBar.tempoChanges.length &&
// masterBar.tempoChanges[tempoChangeIndex].tick <= syncPointTick
// ) {
// const tempoChange = masterBar.tempoChanges[tempoChangeIndex];
// const absoluteTick = tempoChange.tick;
// const tickOffset = absoluteTick - synthTickPosition;
// if (tickOffset > 0) {
// const timeOffset = ticksToMilliseconds(tickOffset, synthBpm);
// synthTickPosition = absoluteTick;
// synthTimePosition += timeOffset;
// }

// synthBpm = tempoChange.tempo;
// tempoChangeIndex++;
// }

// // process time until sync point
// const tickOffset = syncPointTick - synthTickPosition;
// if (tickOffset > 0) {
// synthTickPosition = syncPointTick;
// const timeOffset = ticksToMilliseconds(tickOffset, synthBpm);
// synthTimePosition += timeOffset;
// }

// // create sync point marker
// const newMarker: SyncPointMarker = {
// masterBarIndex: masterBar.masterBar.index,
// occurence: occurence,
// syncTime: syncPoint.syncPointValue!.millisecondOffset,
// synthTime: synthTimePosition,
// synthBpm: masterBar.tempoChanges[0].tempo,
// modifiedTempo: 0 /* calculated by next marker */,
// markerType:
// syncPoint.ratioPosition === 0
// ? SyncPointMarkerType.MasterBar
// : SyncPointMarkerType.Intermediate,
// ratioPosition: syncPoint.ratioPosition,
// synthTick: synthTickPosition
// };
// if (syncPointTick === 0) {
// newMarker.markerType = SyncPointMarkerType.StartMarker;
// }
// markers.push(newMarker);

// if (markers.length > 0) {
// updateModifiedTempo(markers.at(-1)!, newMarker.synthTime, newMarker.syncTime);
// }
// }

// // process remaining tempo changes after all sync points
// while (tempoChangeIndex < masterBar.tempoChanges.length) {
// const tempoChange = masterBar.tempoChanges[tempoChangeIndex];
// const absoluteTick = tempoChange.tick;
// const tickOffset = absoluteTick - synthTickPosition;
// if (tickOffset > 0) {
// const timeOffset = ticksToMilliseconds(tickOffset, synthBpm);
// synthTickPosition = absoluteTick;
// synthTimePosition += timeOffset;
// }

// synthBpm = tempoChange.tempo;
// tempoChangeIndex++;
// }
// }
// }

// // at the very end we create the end marker
// const lastMasterBar = api.tickCache!.masterBars.at(-1)!;
// const endSyncPoint = lastMasterBar.masterBar.syncPoints?.find(m => m.ratioPosition === 1);

// const tickOffset = lastMasterBar.end - syncLastTick;
// const endSyncPointTime = endSyncPoint
// ? endSyncPoint.syncPointValue!.millisecondOffset
// : syncLastMillisecondOffset + ticksToMilliseconds(tickOffset, syncBpm);

// markers.push({
// masterBarIndex: lastMasterBar.masterBar.index,
// occurence: occurences.get(lastMasterBar.masterBar.index)! - 1,
// syncTime: endSyncPointTime,
// synthTime: synthTimePosition,
// synthBpm,
// modifiedTempo: endSyncPoint?.syncPointValue?.modifiedTempo ?? synthBpm,
// markerType: SyncPointMarkerType.EndMarker,
// ratioPosition: 1,
// synthTick: synthTickPosition
// });

return markers;
}

Expand Down Expand Up @@ -524,99 +413,31 @@ export function autoSync(oldState: SyncPointInfo, api: alphaTab.AlphaTabApi, pad

// create initial sync points for all tempo changes to ensure the song and the
// backing track roughly align
let synthBpm = api.tickCache!.masterBars[0].tempoChanges[0].tempo;
let synthTimePosition = 0;
let synthTickPosition = 0;

const syncPoints: SyncPointMarker[] = [];

// first create all changes not respecting the song start and end
const occurences = new Map<number, number>();
for (const masterBar of api.tickCache!.masterBars) {
const occurence = occurences.get(masterBar.masterBar.index) ?? 0;
occurences.set(masterBar.masterBar.index, occurence + 1);

// we are guaranteed to have a tempo change per master bar indicating its own tempo
// (even though its not a change)
for (const changes of masterBar.tempoChanges) {
const absoluteTick = changes.tick;
const tickOffset = absoluteTick - synthTickPosition;
if (tickOffset > 0) {
const timeOffset = ticksToMilliseconds(tickOffset, synthBpm);
synthTickPosition = absoluteTick;
synthTimePosition += timeOffset;
}

const marker: SyncPointMarker = {
uniqueId: uid(),
markerType: SyncPointMarkerType.MasterBar,
masterBarIndex: masterBar.masterBar.index,
masterBarStart: masterBar.start,
masterBarEnd: masterBar.end,
occurence,
syncTime: synthTimePosition,
synthBpm,
synthTime: synthTimePosition,
syncBpm: undefined,
synthTick: synthTickPosition
};

if (masterBar.start === 0) {
marker.markerType = SyncPointMarkerType.StartMarker;
} else if (changes.tick > masterBar.start) {
marker.markerType = SyncPointMarkerType.Intermediate;
}

if (changes.tempo !== synthBpm || marker.markerType === SyncPointMarkerType.StartMarker) {
syncPoints.push(marker);
marker.syncBpm = changes.tempo;
}

synthBpm = changes.tempo;

state.syncPointMarkers.push(marker);
}

const tickOffset = masterBar.end - synthTickPosition;
const timeOffset = ticksToMilliseconds(tickOffset, synthBpm);
synthTickPosition += tickOffset;
synthTimePosition += timeOffset;
}

// end marker
const lastMasterBar = api.tickCache!.masterBars.at(-1)!;
state.syncPointMarkers.push({
uniqueId: uid(),
masterBarIndex: lastMasterBar.masterBar.index,
masterBarStart: lastMasterBar.start,
masterBarEnd: lastMasterBar.end,
occurence: occurences.get(lastMasterBar.masterBar.index)! - 1,
syncTime: synthTimePosition,
synthTime: synthTimePosition,
synthBpm,
syncBpm: synthBpm,
markerType: SyncPointMarkerType.EndMarker,
synthTick: synthTickPosition
});

state.syncPointMarkers = buildSyncPointMarkers(api);

// with the final durations known, we can "squeeze" together the song
// from start and end (keeping the relative positions)
// and the other bars will be adjusted accordingly
if (padToAudio) {
const [songStart, songEnd] = findAudioStartAndEnd(state);

const synthDuration = synthTimePosition;
const synthDuration = state.syncPointMarkers.at(-1)!.synthTime;
const realDuration = songEnd - songStart;
const scaleFactor = realDuration / synthDuration;

state.syncPointMarkers.at(0)!.syncBpm = state.syncPointMarkers.at(0)!.synthBpm;
state.syncPointMarkers.at(-1)!.syncBpm = state.syncPointMarkers.at(-1)!.synthBpm;

// 1st Pass: shift all tempo change markers relatively and calculate BPM
const syncPoints = state.syncPointMarkers.filter(m => m.syncBpm !== undefined);
let syncTime = songStart;
for (let i = 0; i < syncPoints.length; i++) {
const syncPoint = syncPoints[i];

syncPoint.syncTime = syncTime;

if (i < 0) {
if (i > 0) {
const previousMarker = syncPoints[i - 1];
const synthDuration = syncPoint.synthTime - previousMarker.synthTime;
const syncedDuration = syncPoint.syncTime - previousMarker.syncTime;
Expand Down Expand Up @@ -898,3 +719,15 @@ export function syncPointsToKotlinCode(info: SyncPointInfo, indent: string): str

return lines.join(',\n');
}

export function syncPointsToAlphaTex(info: SyncPointInfo): string {
const lines: string[] = [];

const flat = toFlatSyncPoints(info);
for (const m of flat) {
const barPosition = m.barPosition > 0 ? ` ${Number(m.barPosition.toFixed(3))}` : '';
lines.push(`\\sync ${m.barIndex} ${m.barOccurence} ${m.millisecondOffset}${barPosition}`)
}

return lines.join('\n');
}
Loading