TZX tape format support

[fruit-bat]

Need to decide the scope and feasibility of this issue.

[badrianiulian]

Hmm txz read support sounds amazing.
As I've read somewhere, txz format tends to be upgraded with new features once in a while. But for all the existing tzx files for the ZX, I don't think we need to worry about the new features for now since all that could be dumped for the ZX is already dumped, apart from some few missing apps/games that still need to be recovered witch usually are dumped with the existing txz formats.
Unless the developers update their conversion utilities or the tzx format itself is updated, the latest version of tzx is 1.20 and hasn't been updated since 2006.
On the other hand, if the block isn't recognised, skipping it would bypass future TZX format updates.

In the other issue you mentioned something about tzx write support. What did you mean about that?

[fruit-bat]

Having had a brief look, writing this format on the fly would be very involved... I mentioned it previously as the emulator doesn't really have a way to save to tape... the way the audio output is handled I doubt it is stable enough to be read back.

I'm terms of reading tzx, I'll have a read through the spec and have a think about it. If I build a framework for it, I could implement one block type at a go.

[badrianiulian]

In my opinion, If it takes too much of the system, just reading from tzx or tap files should be enough. After all this format is meant for PC in general... right? (for archiving the old tapes)
If it's about writing to a tzx file from cassette tape on the fly then I don't think that would work very fine anyway... As I've seen the last few days with dumping the tapes I bought, maketzx on PC is way more reliable and works way better since you can tweak it.

[badrianiulian]

I'v just noticed that you've made the first steps into adding TZX support: the selecting part of the process and some header identification. It's looking nice and clean.
I enjoy reading good organised code... it's just easy on the eyes for me :)

I've been off the last couple of days, busy with some tapes... dumping, editing, converting. Found about 14 tapes in a local antiques shop that are still waiting to be checked out.

[fruit-bat]

Yes, just starting. TZX looks like it has quite a lot to it. I am trying to dream up a way to keep the code small and still support lots of the differnt options...
see PulseCpu.h & PulseCpu.cpp
I'm having a try at describing the formats with a 'made up' processor - the Cassette Processing Unit (CPU)
It may end up too complicated but it's a bit different so I thought I would give it a go.

I hope you find some gems in your new tape collection!

[badrianiulian]

Until now I found at least 4 important not dumped programs/games and a few modified games from the Europe's Eastern Block 90's period (the old unregulated days).

I made a pause for now, as I've been tinkering to fix the 'go into protection blinking red light mode' of the Sony stereo system I've been using to dump the tapes. I've been meaning to fix it for a while now (at least 5 years) but since it did not happen once while dumping tapes I just left it as it were.
I think I've fixed it by changing two 100 ohm smd resistors (in the overload protection circuit) that usually come in 1/4W format in these systems. It's been working without problems for at least an hour, so I hope I'm in the clear.
Will have to add some braided wire terminals for the speakers at some point as I think those (loose) wires could also cause these overload states. For now I've bathed the terminals in solder but I'm not very satisfied with the result.

While searching for information on some dumps I did manage to find some new possible additions for the collection, as in programs and games, dumped as tzx from Italian magazines. I'm not fluent in Italian but I now enough to understand them games.

[fruit-bat]

The TZX format does not seem to have been written with memory constraints in mind for decoding. The blocks in the 2X range seem to want to be able hop backwards and forwards through the file structure. The blocks in the 1X range look more achievable in the little memory left to work in... but I don't know what percentage of TZX that would give access to.

[fruit-bat]

Just a quick update. I've finally got a framework which I hope will be good enough to add the various block type decoders to. Currently, it just reads block type 10 (the standard timings) but it works nice and efficiently.

I'm not sure how I am going to get test tzx files to try out the other block types as I implement them. Is this something you know about or could help with?

[badrianiulian]

Hello! Been busy these days and finished parsing those tapes.
In the process I've managed to dump and edit some tzx files using ZX-BlockEditor.
It's a really nice piece of software with lots of block editing abilities. It's part of a larger collection and isn't available for download anymore but after digging through the Internet I managed to find the last working versions.
I can start testing as soon as you say. Also there's lots of tzx files in the TOSEC archive for ZX Spectrum. They're really nice organised and I'm hoping to contribute to it by sorting out some files from different sources and the txz dumps I've been working with these last weeks.

[fruit-bat]

Thanks for the links and glad you still fancy helping.

If I suggest a block type I am going to try and implement would you be able to make a tzx containing that block type? The idea would be something not copyrighted and obvious as to whether it had loaded correctly; in most cases a screen save would be ideal. I could then add these to the project as a test set (or you could directly pull-request them into a test folder). It would be really good to have these with the project so I can tell if future bug 'fixes' have broken something.

I've already implemented block type 10, and will look at 11 next (the turbo loader block).

I've not published any binaries yet, but if you are interested in trying any of it directly let me know and I will start pushing updates to https://github.com/fruit-bat/pico-zxspectrum/tree/feature/tzx

Some of the more esoteric block types may need some thought as to what makes a good test for them... but we could worry about that later - I only get the odd hour here and there to write this stuff anyhow!

[badrianiulian]

Another good place to get tzx files with turbo blocks that can be downloaded for free would be itch.io. I'd have to dig for some files with turbo blocks though.

As an alternative, I would be willing to insert myself into the Z80 assembler in order to build at least a screen that can be loaded as a turbo block. Reverse engineering an already available tzx with turbo blocks would be the way I would go.
And you are right. As copyright is the problem, only changing the content of the turbo block screen would be a good solution. I'll try and find some way to build one.

As for testing, let me just build a test tzx first and then we can proceed to the testing phase.

[fruit-bat]

Excellent... and I now have prototype Block 11 code ready to test :-)

[badrianiulian]

So:
At first I tried this piece of software: Truboloader from "Your Sinclair Magazine - type-in from issue 5 - 1986/May"
It created the turbo block with a program header instead of a bytes header and even if I tried all kinds of fiddling, I couldn't find a way to load it in the emulators. I may have to browse the magazine in order to figure it out.

Then I found this free piece of code from mcleoud-ideafix
I already made the screen so all I had to do now was insert it into the sample text file... I used hexdump to arrange the scr bytes just like in the example and after that converted the text to pzx.
Using ZX Blockeditor I merged the 'demo_turboloader.tzx' with the converted screen and all went well as the new screen turbo block loads fine into the emulators (zxsp, Fuse for Mac and UnrealSpeccyPortable). I tried to add some pokes and colours in the program block with ZX-Blockeditor and inside ZX-Editor and it messed up the TZX file so I left the end file as it was (in working condition).

Pico-ZX.zip

[fruit-bat]

Fantastic, and I love the image. Helped me sort out a handful of silly mistakes. Block type 11 now working :-)

The following block types now have prototype code for them:

ID 12 - Pure Tone
ID 13 - Pulse sequence
ID 14 - Pure Data Block
ID 15 - Direct Recording

If you can come up with some tests for those that would be great.

[badrianiulian]

Unfortunately, after searching far and wide I could not find some info on how to test those blocks... I'll still search a way but in the meanwhile have a look at this.

[badrianiulian]

And since the last comment, I found someone with some info on those blocks. Back in the days there was some company in the region that used a block like the above as a sort of protection from copying those tapes. Will investigate further.

[fruit-bat]

Unfortunately, after searching far and wide I could not find some info on how to test those blocks... I'll still search a way but in the meanwhile have a look at this.

Thanks for investigating. I have been using playtzx.c as a reference. It needs far too much memory to use directly but is a handy guide for at least most of the block types.

[badrianiulian]

Sorry to say, things are on hold for a while.
I'll be MIA for a few days since my dad just died. He's the one who bought me my first ZX Spectrum clone... a HC-91.
I'll be back in action in a few days.

[fruit-bat]

Very sorry about your dad. I never know what to say at such times but for what it is worth I send my best wishes.

Also, will be very glad to have you back on the project when circumstances allow.

[fruit-bat]

There is now an alpha release:
https://github.com/fruit-bat/pico-zxspectrum/blob/feature/tzx/uf2/ZxSpectrumBreadboardHdmi4PinAudio.uf2

I'm proud of the 'Generalized data block', which was quite tricky implement.

It has some notable omissions:

• Block type 18 - CSW support (not spotted any of these in the wild yet)
• Block type 28 - Select block

Also, most of the 'information' blocks are just skipped.

[badrianiulian]

Hello again.
I finished the procedures regarding my dad and "got back in the game".
With the help of a member from the Romanian Home Computer group, I found a way to test the pure tone blocks. There are some protected math apps that have a pure tone signal on the tape right after the screen. If that pure tone isn't loaded at that point, when reaching the end of the loading process for that program, the system resets itself by checking some variables and all loaded data is lost. That pure tone could not be copied with regular copying software but it can be reproduced inside a tzx file.
In the near future we intent to build a website with all the software developed in our country. It's a work in progress since we have to identify and actually find tapes that had either original software or modified software for our systems.

[fruit-bat]

Good to have you back. The idea for the pure tone test sounds interesting. You have some good contacts!
Let me know if you come across any tzx files containing CSW blocks. I'm wondering if I should bother implementing it; it looks like it needs zlib inflate which is complicated.

[badrianiulian]

Well... I'm working on many angles right now... ever since I got into finding and dumping these tapes, I've tried to keep ahead but things are catching up. You know your project was the starter for this, right?! Thanks by the way! I found some new tapes along the way, with old software that can't be found anywhere else. Also it's a very nice way to remind me about my dad. He bought me my first Home Computer back in 1993. Also he loved to learn about history. I got the computer bug ever since back then and now it's mixed in with his history bug.
Will try the new alpha release tomorrow. As for the CSW blocks, I did not encounter such blocks... yet. There could be a first for anything I guess. Let's see how the first implemented blocks work first.

[badrianiulian]

Tested a couple of games with the alpha build.

3-04 Adunarea si scaderea. Joc instructiv.tzx
BASIC-01 Vocabular.tzx
Krakout (Gremlin)[h Nicolas Rodionov].tzx
Psi Chess (1986)(Micro Selection, The)[Alkatraz Protection System][re-release].tzx
Psycho Soldier (1987)(Imagine)(48K-128K)[SpeedLock 5].tzx
Puzznic (1990)(Ocean)(48K-128K)[SpeedLock 7].tzx
Raid (1985)(Americana)[SpeedLock 1][re-release].tzx
Road Blasters (1988)(Kixx)(128K)[SpeedLock 4][re-release].tzx
Road Runner (1987)(U.S. Gold)[a][SpeedLock 4].tzx
Road Runner (1987)(U.S. Gold)[SpeedLock 4].tzx

I had some problems with Road Runner (1987)(U.S. Gold)[SpeedLock 4].tzx witch I can't seem to load it completely and Road Runner (1987)(U.S. Gold)[a][SpeedLock 4].tzx witch loads but doesn't stop at the first level. This ones are bigger than the average dumps. The other ones on the list load fine.

[fruit-bat]

New binary available with support for the TZX 'Select' block. This block is used for selecting 48k/128k or trainer/normal loads.
It opens the menu and allows an option to be selected. Also, a few bugs have been sorted out. Have fun!

P.S. The uf2 still sends lots of debug to the serial port, which may slow it down a bit.

[badrianiulian]

Hy..
Today I downloaded the previous build, but since you compiled the new one with new mods, I'll re-download the new one. I'll have a go at it tomorrow since today I tried to sort out some of the converted tapes.
Also, through the old conversions I had, I found one with "Direct recording" blocks... I didn't made those but I will test those too, along with the previous tzx files and maybe have a go at some new ones.
Somewhere around the living room I have one USB to TTL adapter... maybe I should hook it up at some point.

[fruit-bat]

I've been really struggling to work out why some of the pure tone blocks are not recognized. I wondered if they were down to CPU cycles being miss counted so I have tried a new CPU implementation. I prefer this one as it can be kept separate from the emulator source. It comes with it's own test suite and is being looked after by its author - great!

...but on the down site it made no difference to the tape loading issues (grr)... will keep trying :-)

[badrianiulian]

I've been off for a few days, I found new tapes, had to repair some electronics that got all my time, instructed a friend (with a huge collection of all sorts) on how to dump from cassette to PC, I'm also working on sorting the dumps I already made... busy days and I feel exhausted... I'm not off your project yet... just distracted with other stuff for now. When there's another bin file, I'll dig into it and test things out. I still have to find a working (affordable) ZX Spectrum to test with PlayTZX, but for now, for the dumping process, I found an 5,25 floppy drive. Don't know if it works yet.