Exceedingly long resource (re)start times

[sacr1ficez]

Describe the bug

After PR #3924 server started to behave slow, often throwing long execution warning & causing network trouble on resource (re)starts. It also causes server console to be lagging, or have some kind of input delay. It has never happened before with this resource, and only recently after server update i've started noticing something isn't alright. Test resource i've used has 100 MB overall, and around 426 .lua scripts, but according to others it also happens on default resources, and custom smaller systems (see additional context). What's worse, it seems to be inconsistent, sometimes you can reproduce it right away, and the other time it will randomly appear. Server running time might be a factor.

Steps to reproduce

local loadTimes = {}

function onResourceStart(startedResource)
	local diffResource = (resource ~= startedResource)

	if (not diffResource) then
		return false
	end

	local resourceName = getResourceName(startedResource)
	local savedTime = loadTimes[resourceName]

	if (not savedTime) then
		return false
	end

	local mtaVer = getVersion()
	local mtaVerSortable = mtaVer.sortable
	local timeNow = getTickCount()
	local timeDiff = (timeNow - savedTime).." ms"
	local startLog = resourceName.." took "..timeDiff.." to load @ "..mtaVerSortable

	outputDebugString(startLog)
	loadTimes[resourceName] = nil
end
addEventHandler("onResourceStart", root, onResourceStart)

function onResourceStop(stoppedResource)
	local diffResource = (resource ~= stoppedResource)

	if (not diffResource) then
		return false
	end

	local resourceName = getResourceName(stoppedResource)
	local timeNow = getTickCount()

	loadTimes[resourceName] = timeNow
end
addEventHandler("onResourceStop", root, onResourceStop)

Version

Server: r22883 and above
Windows 64 bit server

Additional context

See:
https://discord.com/channels/801330706252038164/1289708691830083775/1358180315209797673
https://discord.com/channels/801330706252038164/1289708691830083775/1358181367002959892
(MTA:SA development discord - GNN6PRtTnu)

Relevant log output

[2025-04-08 21:21:35] Stopping dayz
[2025-04-08 21:21:35] Starting dayz
[2025-04-08 21:21:37] INFO: dayz took 2342 ms to load @ 1.6.0-9.22880.0

[2025-04-08 21:26:23] Stopping dayz
[2025-04-08 21:26:23] Starting dayz
[2025-04-08 21:26:30] WARNING: Long execution (dayz)
[2025-04-08 21:26:32] INFO: dayz took 9737 ms to load @ 1.6.0-9.22883.0

Security Policy

• I have read and understood the Security Policy and this issue is not security related.

[AlexTMjugador]

Thanks for providing some performance comparison numbers, they are useful to characterize the problem! I understand that with so many files the filesystem call overhead may start becoming significant.

A month ago, I shared pointer about a potential performance improvement for this situation when asked over Discord:

In theory yes, you could make it slightly more efficient by streaming the file contents in chunks, computing a checksum while copying the file at the same time.

I could give this improvement idea a try, but I'm not familiar enough with C++ to properly decide what the most idiomatic way of doing that in MTA's codebase is, and I don't have that much time to learn at the moment. Any pointers here?

[Dutchman101]

It's difficult to choose between servers that rely on heavy-file-size features like IMG loading (who may now finally be stable), and servers affected in their resource loading times by the "Make server resource file checksumming code memory-efficient" and when i shared this feedback with srslyy and Alex, i heard this interesting idea:

srslyyyy — 23:32
mtaserver.conf setting would be okayish
like a toggle
this way we won't have to punish one or other side, because ultimately it would be server owner choice to select between one approach or another

Even then, to do it the best way, we shouldn't be choosing, but rather investigate the issue and try to fix it.

One side (e.g servers like srslyyy's) is demanding a revert before anything else, and we also need to advocate for the other side that found stability in the change.

The highlighted part from original PR clarifies the perspective of the other side, "servers that rely on heavy-file-size features like IMG loading (who may now finally be stable)":

[Dutchman101]

AlexTMJugador's opinion on the "mtaserver.conf" approach idea:

More specifically, it's about his knownledge of C++ idioms. If anyone is willing to help out (i will link this post to botder and lopsi to begin with), it'll be much welcome.

[q8X]

this is really annoying, every time i want to debug resource, i have to wait 30 secs until the local server respond

for example, starting/restarting editor takes 32 secs

dialogs took 32109 ms to load @ 1.6.0-9.23147.0
tooltip took 32111 ms to load @ 1.6.0-9.23147.0
editor_gui took 32111 ms to load @ 1.6.0-9.23147.0

starting a tool called MSTU takes 32 secs
[10:54:00] INFO: mstu took 32401 ms to load @ 1.6.0-9.23147.0

[q8X]

well, only solution is to wait i guess. Anyway, if anyone tired of waiting 30s every time you change small part in the code
revert to this version and debug locally

so now you have got two versions to test with, latest one and r22880

[sacr1ficez]

The issue still persists on Multi Theft Auto v1.6-release-23219
Every restart of my resource was met with long execution & network trouble, it started happening out of nowhere and went away after restarting server.