Intermixed info in log file when processing multiple files #564

Activity

1. HolterPhylo

   added

   bugAn unexpected and/or unintended program behaviour.An unexpected and/or unintended program behaviour.

   on Aug 15, 2023
2. HolterPhylo commented on Aug 28, 2023

   

   HolterPhylo

   on Aug 28, 2023

   CollaboratorAuthor

   More actions

   Also, this happens: missing 2 spaces in front of lines 18-19 on the left that are normally like the ones on the right. This with what happens above makes parsing for the right filename of the reported ERROR/WARNING very hit and miss.
   
3. cmkushnir commented on Aug 31, 2023

   

   cmkushnir

   on Aug 31, 2023

   Last edited by cmkushnir

   Collaborator

   More actions

   If mbinc is processing mult mbin in parallel then messages written to single log will be mixed. The easiest way to prevent this is to have each mbin instance maintain its own log (list) of messages, then write them all to the log once the mbin finishes processing. The write to the log would be serialized, so although all messages for a given mbin would be together the order that the mbin's are reported would be random (depends when a thread finishes a given mbin). If you want mbin's in order as well then wait until all mbin's finish then write to log, however, then carrying state around for each mbin after it's finished which won't scale well ... though for this case would likely be ok. Other issue is if crash, then won't have messages in log upto crash point.
4. HolterPhylo commented on Aug 31, 2023

   

   HolterPhylo

   on Aug 31, 2023

   CollaboratorAuthor

   More actions

   Thanks for comment.
   Keeping the order of the files processing is not important.
   Keeping the information of each file process together is what counts, not all intermixed with other files output
5. HolterPhylo commented on Jul 20, 2024

   

   HolterPhylo

   on Jul 20, 2024

   CollaboratorAuthor

   More actions

   From Discord:

   • One idea I have would be to assign an index to each read file on input and use that index in all logging for that file. The log could then be all mixed up but it would be easy to related the index to the right file when reading the log.

   • AMUMSS does not run multiple copy of MBINCompiler at the same time, it just ask MBINCompiler to process all MBIN or EXML in a given folder. Almost the same as dropping the folder content on MBINCompiler.

   • The script CompanionPetUnlocker_1.lua.txt
     (remove .txt extension) put in AMUMSS ModScript folder and executing BUILDMOD.bat will produce a MBINCompiler.log
     MBINCompiler_1.log. One MBIN to process to a modded EXML that cannot be compiled (see WARN)

   • The script CompanionPetUnlocker_2.lua.txt
     (remove .txt extension) put in AMUMSS ModScript folder and executing BUILDMOD.bat will produce a MBINCompiler.log
     MBINCompiler_2.log where the WARN is anonymous/not related to the preceding processed EXML file. It is impossible to connect the WARN to the right file.
6. HolterPhylo commented on Jul 20, 2024

   

   HolterPhylo

   on Jul 20, 2024

   CollaboratorAuthor

   More actions

   MBINCompiler is called like this:

     -- action = Compile
     -- sourcePath = where the EXML files are relative to MODBUILDER folder
     -- IsWithThreads = bool, optional
   --
   -- returns: string: success
   function H.MBINCompiler_C(sourcePath,IsWithThreads)
     if IsWithThreads == nil then IsWithThreads = true end
     local threadInfo = "(multi-thread)"
     local withThreads = ""
     if not IsWithThreads then
       withThreads = "--no-threads"
       threadInfo = "(single-thread, due to MBINCompiler buggy log in multi-thread)"
     end
     
     local success = ""
     
      --clear .log
      local cmd = [[del MBINCompiler.log 1>NUL 2>NUL]]
      os.execute(cmd)
    
      local start = os.clock()
      print(H._zBRIGHTGREEN.."     @@@ creating MBIN files "..threadInfo.."..."..H._zDEFAULT)
   
     --  ===========
      local cmd = [[MBINCompiler.exe -q -y -f -iEXML --exclude=";" ]]..withThreads..[[ "]]..sourcePath --..[["]]
     --  ===========
   
      local state,str,num = os.execute(cmd) --fast and same output as batch
      
      local delta = os.clock() - start
      print(H._zBRIGHTGREEN.."      - done in "..H.dClock(delta)..H._zDEFAULT)
      
      if state then
        success = "OK"
      else
        -- print("@@@ MBINCompiler returned: "..str..", "..tostring(num))
        success = "ERROR"
      end 
   
     return success
   end