Reintroduce JSON export from 0cc

[nstbayless]

This pull request reintroduces JSON export (#197), which is a handy feature in 0CC-Famitracker that makes it easy to read famitracker data from scripts (say, for exporting to a custom engine).

The feature works and is basically identical to 0CC's. However, here are some missing features:

• Subindex is no longer a field channels have, and I'm not sure how to get this data. I could use some advice. (Line.)
• I couldn't really figure out the "bookmarks" field. This probably isn't a very important thing for scripts to have though.
• Any other new fields added by Dn-Famitracker since 0CC are not exported
• There is no JSON import (nor was there in 0CC, but it looked like it was intended eventually)
• I haven't added a help entry yet (0CC didn't seem to have a .chm at all?).

[nstbayless]

Oh yeah, there are also a couple mysteries I encountered.

• When iterating through the dpcm keys in a note, there's a weird off-by-one. The original 0CC version iterates through all notes from 0-95, whereas here I am only iterating through 1-95 and writing the note index as though it were one lower. This approach is required to get identical output compared to 0CC on the file I'm testing on.
• For note effects, 0CC wrote column indices 14, 13, 12, 11, 10, etc. instead of 0, 1, 2, 3. (Also, it looks to me like only 4 effect columns are supported now? Is that true?)

[Gumball2415]

This PR addresses #200.

[Gumball2415]

• Subindex is no longer a field channels have, and I'm not sure how to get this data. I could use some advice. (Line.)

Subindex appears to be a per-channel indexing of all the chips in 0CC's code (line). seems like a major refactoring of CSoundGen's channel structure/CTrackerChannel in general? here's the closest thing i can find about subindexes in Dn-FT

[Gumball2415]

• I haven't added a help entry yet (0CC didn't seem to have a .chm at all?).

for .chm manual entries, feel free to submit a corresponding PR at https://github.com/Dn-Programming-Core-Management/Dn-help

[Gumball2415]

• When iterating through the dpcm keys in a note, there's a weird off-by-one. The original 0CC version iterates through all notes from 0-95, whereas here I am only iterating through 1-95 and writing the note index as though it were one lower. This approach is required to get identical output compared to 0CC on the file I'm testing on.

the MIDI note helper functions use equations that are off-by-one, perhaps Hertz fixed this in later versions, with all other code using these functions following suit?

[Gumball2415]

• For note effects, 0CC wrote column indices 14, 13, 12, 11, 10, etc. instead of 0, 1, 2, 3. (Also, it looks to me like only 4 effect columns are supported now? Is that true?)

FamiTracker has always supported 4 effect columns, all the way to 0CC, and today in Dn.

it seems that fx is used incorrectly? its values (like 14, 13, 12, 11, 10, etc.) is an index into EFF_CHAR[]

for (const auto &[fx, param] : note.Effects)
	if (fx != effect_t::none)
		j["effects"].push_back(json {
			{"column", fx},		// fx is treated as a column value, when it's really an index into EFF_CHAR[]
			{"name", std::string {EFF_CHAR[value_cast(fx)]}},		// as shown here
			{"param", param},
		});

as for chip, channel, and note structuring, it seems that Hertz has overhauled/refactored everything, so i sadly wouldn't expect 100% compatibility

[Gumball2415]

can you try this fix for subindex compatibility?

// hack to get the equivalent 0CC-exclusive channel subindex
// TODO: implement whatever 0CC's doing for better compatibility
auto chip_type = ch->GetChip();
uint8_t subindex = ch->GetID();

switch (chip_type) {
	case SNDCHIP_VRC6: subindex -= 5; break;
	case SNDCHIP_VRC7: subindex -= 20; break;
	case SNDCHIP_FDS: subindex -= 19; break;
	case SNDCHIP_MMC5: subindex -= 8; break;
	case SNDCHIP_S5B: subindex -= 26; break;
	default: break;
}

channels.push_back({
	{ "chip", GetChannelChipName(chip_type)},
	{ "subindex", subindex},
	});

[nstbayless]

the MIDI note helper functions use equations that are off-by-one, perhaps Hertz fixed this in later versions, with all other code using these functions following suit?

It's possible. My only concern is that 0CC iterates through 96 values, and my code only iterates through 95, but I'm not sure what note I'm missing exactly.

as for chip, channel, and note structuring, it seems that Hertz has overhauled/refactored everything, so i sadly wouldn't expect 100% compatibility.

I feel like the JSON export is already like 95% of the way there, so even if the internals are different, I think we can get 100% compatibility (except maybe for the bookmarks field, unless I'm missing something).

can you try this fix for subindex compatibility?

I really appreciate the detailed response. I will try this out when I have access to a windows computer again, which could be a while.

[Gumball2415]

It's possible. My only concern is that 0CC iterates through 96 values, and my code only iterates through 95, but I'm not sure what note I'm missing exactly.

judging purely by the code shown, i think you're missing note "95". 0CC indexes through values 0 until 95, while the code so far indexes through 0 until 94

if the for loop is offset by +1, and the MIDI note helper functions are offset by -1, wouldn't they cancel each other out?

for (int n = 0; n < NOTE_COUNT; ++n)
{
	int oct = GET_OCTAVE(n);
	int note = GET_NOTE(n);
	if (auto d_index = inst.GetSampleIndex(oct, note); d_index != 0)
	{
		j["dpcm_map"].push_back(json{
			{"dpcm_index", d_index},
			{"pitch", inst.GetSamplePitch(oct, note) & 0x0Fu},
			{"loop", inst.GetSampleLoop(oct, note)},
			{"delta", inst.GetSampleDeltaValue(oct, note)},
			{"note", n}, 
			});
	}
}

edit: looking deeper into the d_index bit, DPCM sample indexing is weird. nearly all the code that calls CInstrument2A03::GetSampleIndex() always subtracts it by one

perhaps a fix might look like this instead:

for (int n = 0; n < NOTE_COUNT; ++n)
{
	int oct = GET_OCTAVE(n);
	int note = GET_NOTE(n);
	if (auto d_index = (inst.GetSampleIndex(oct, note) - 1); d_index > 0)		// offset d_index by -1
	{
		j["dpcm_map"].push_back(json{
			{"dpcm_index", d_index},
			{"pitch", inst.GetSamplePitch(oct, note) & 0x0Fu},
			{"loop", inst.GetSampleLoop(oct, note)},
			{"delta", inst.GetSampleDeltaValue(oct, note)},
			{"note", n}, 
			});
	}
}

[Gumball2415]

apologies if i can't test this code out myself, i'm not sure what to look for in terms of output

[nstbayless]

Oh it's okay, it's my responsibility to test it out!

If you want to test it, you could compare the output with the output from 0CC-famitracker's JSON export for a simple module with just a couple notes.

[Gumball2415]

i hope you don't mind if i relegate this PR for version 0.5.0.2 (next next release)

[Gumball2415]

rebased to main

[Gumball2415]

initial observations:

• missing "samples": [] entry in a given dpcm sample entry, not sure what it's intended for in 0CC
• "column" has incorrect value on 0CC, instead it writes the effect command byte?
• missing "bookmarks" key
• "highlight" does not output Offset value
• "effect_columns" has incorrect value?
• 0CC exports ghost instrument sequences?? edit: might happen when instrument sequences are not properly initialized.

[Gumball2415]

for info not present in the 0CC json export (such as device mixing, OPLL hardware patches, etc.), i think following the text export formatting would help

[Gumball2415]

@nstbayless i've added the missing Dn-FT data blocks from v0.5.0.0+, please check if i've missed anything with regards to 0CC's data format

[nstbayless]

Thank you! That's amazing. It looks complete to me so far.

[Gumball2415]

as far as i can tell, this PR looks complete. i will merge it soon.
thank you for your pull request