mpqapi cleanup

[glebm]

1. Do not rely on stream positions for getting the initial file size.
2. Remove most seek calls that were unnecessary.
3. Replace magic numbers with constants.
4. A class to manage archive lifetime and all associated data.
5. Works around save issue on Amiga

@Cowcat5150, could you please test this?

[glebm]

@AJenbo Do the tests run on Windows in CI?

[AJenbo]

@glebm no, but it probably wouldn't be hard to add.
https://github.com/diasurgical/devilutionX/blob/master/appveyor.yml
https://github.com/diasurgical/devilutionX/blob/master/CMakeSettings.json

[Cowcat5150]

1. Do not rely on stream positions for getting the initial file size.
2. Remove most seek calls that were unnecessary.
3. Replace magic numbers with constants.
4. A class to manage archive lifetime and all associated data.
5. Works around save issue on Amiga

@Cowcat5150, could you please test this?

Mmm....latest changes/fixes/ works for debian, but still not saves for Morphos. Opens old single files. Also there are some problems with std::free not being recognized for a Warpos build (for Morphos is ok).

Basically what an earlier mpqapi log frame said;

Opening PROGDIR:single_1.sv
new std::fstream("PROGDIR:single_1.sv", std::ios::binary | std::ios::in | std::ios::out | std::ios::trunc)
seekp(65640, std::ios::beg): failed with "Illegal seek"
Closing PROGDIR:single_1.sv
seekp(0, std::ios::beg): failed with "Illegal seek"
Apps:Diablo>

[glebm]

I guess #ifdef __AMIGA__ doesn't apply to these platforms? Do you know what libc warpos and morphos use?
If it's bebbo/libnix, the illegal seek bug in it might have been fixed just now: bebbo/libnix#30

[Cowcat5150]

I guess #ifdef __AMIGA__ doesn't apply to these platforms? Do you know what stdlib warpos and morphos use?

AMIGA defines are by default included for warpos/morphos:

stdlib.h for warpos (gcc 6.4 - newlib): void free (void *) _NOTHROW;
stdlib.h for morphos (gcc 9); void free __P((void *));

No bebbo/libnix: This is Amiga-like powerpc ecosystem and nothing to do with 68k new gcc's. Thinking about doing some stream tests by my own before going to look out for the Morphos team developers. Warpos can't test it because of the std::free (it uses basically gcc/g++ from Morphos but not the full SDK). Anyways do your own thing.

@glebm

LOL, adding this:

char_buffer[65640];
cur_archive.stream.write(buffer, pBlk->offset);

before

if(!cur_archive_stream.seekp(pBlk->offset, std::ios::beg))
    goto_on_error;

Works ! :)

But then cannot read if I save later:( Got to fix that,,,,,,

Ok. Seems to work now: A simple bool inside InitDefaultMpqHeader to control if a new stream is created and then a check if I had to write this buffer or not.

[glebm]

@Cowcat5150 I've replaced the uses of malloc/free in that file, please try again.

LOL, adding this:

It's exactly what this block does (but without using 60 KiB of stack):

#ifndef __AMIGA__
		// Amiga currently cannot seekp beyond end-of-file, so we fill up the space.
		// The data is incorrect at this point, it will be overwritten on Close.
		// See https://github.com/bebbo/libnix/issues/30
		if (!cur_archive.exists)
			cur_archive.WriteHeaderAndTables();
#endif

May no longer be necessary on Amiga with the latest bebbo/libnix though.

[Cowcat5150]

@Cowcat5150 I've replaced the uses of malloc/free in that file, please try again.

LOL, adding this:

It's exactly what this block does (but without using 60 KiB of stack):

#ifndef __AMIGA__
		// Amiga currently cannot seekp beyond end-of-file, so we fill up the space.
		// The data is incorrect at this point, it will be overwritten on Close.
		// See https://github.com/bebbo/libnix/issues/30
		if (!cur_archive.exists)
			cur_archive.WriteHeaderAndTables();
#endif

May no longer be necessary on Amiga with the latest bebbo/libnix though.

Previously I enabled it for use with Amiga but got same problems. Will try your new changes today.

Tested. New delete compiles but still the save problem: Using my buffer hack kinda works but now saving sets the system in bad situation that leads to crash sooner or later.

Forget to mention: The use of truncate in file_util.h/ResizeFile function can be changed by ftruncate or is totally different ?. "truncate" not available in Warpos and I use "ftruncate"......;(

[qndel]

Crashes for me when trying to create a character in x64 release/x64 debug builds in visual studio on windows

[glebm]

@qndel Log and stack trace please. I don't have a Windows with Visual Studio at the moment.

Tracked down and fixed the Windows crash. together with @AJenbo. Release vs Debug issue still not fixed (isn't affected by this PR)

[glebm]

@Cowcat5150 OK, I've added some code to ensure that we do not seekp beyond EOF in mpqapi_write_file_contents on __AMIGA__. This should hopefully work now.

Regarding ftruncate, it's fine to use it instead on platforms where truncate is not available. Please send a PR!

[AJenbo]

<3 SourceT/file_util_test.cpp

[Cowcat5150]

Seems to work as expected for Morphos, hope is good for rest of Amiga's . :)

On a previous, previous mpqapi fix, due to a buggy/old MOS sdl library (I think) I ended with a single_0.sv file impossible to open even on debian. Supposedly files are "safe" against problems when game fails to continue but "MPQ free list error" was the message.

Hope that this is also fixed (was source from a couple of days ago). Good work anyways.

[qndel]

Can't create new characters, crashes while saving on windows

crash while making a new char
@glebm

[glebm]

@qndel Is this from the latest version? I thought aec52ae fixed it

[qndel]

well I'm not a git god, but I pulled changes from master to my light branch then even tested with just the master 🤷‍♂