fixes for building collectives, exe, apps with Max UI and fileusage_*

[diablodale]

Building of collectives/exe in Max with the fileusage_*** apis and the Max UI has missing and errant behavior that make it difficult to support more complex solutions and 3rd party libraries. Max does not support whole directories of files and maintaining hierarchy. Fallback workarounds lead to usage things like _sym_filepath attributes and locatefile_* which have their own problems. All these problems exist in both Max 6 and 7.

It is desired for the fileusage_*** apis and the workflow within Max itself be updated to support the following:

• including dependent files of a compiled mxe, mxe64, mxo
• including dependent directories of files recursively
• maintain hierarchy of directory structures -- don't flatten

Real World Example

Some externals use 3rd party libraries/code. These dependent DLLs and dependent data files may require specific directories relative to the compiled code (aka the external). A dev can not always adapt these locations/directories to be all in the same directory and flattened. 3rd party libraries from companies like Cycling74, Microsoft, Google, etc. are often not open source and impossible to get one-off changes. Therefore, the distribution/install tool needs flexibility to meet the needs of all included libraries/data for both me (the C++ external dev) and the professional patch author.

It is not possible today to create the following as a collective; where everything is bundled into a single mxf.

MyCoolApp\
    MyCoolApp.maxpatch
    foo.mxe64
    thirdParty.dll     <-- used by foo.mxe64
    thirdPartyData/    <-- used by third party dll; must have that folder name, must be alongside DLL
        file1.dat      <-- must have these filenames and be in specifically named subdirectory
        file2.dat
        ...
        file164.dat

In dp.kinect2, I have different workarounds for exes and collective that a professional patch author must use. Why? Because Max behaves differently for file paths, file names, loading directories, etc. https://github.com/diablodale/dp.kinect2/wiki#standalone-application-and-collectives
The underlying C-code to support those workarounds is a incomplete solution. It requires using Windows delayed DLL loading, hooking into that delay load mechanism to rewrite directories/filenames, a workaround for collective detection Cycling74/max-sdk#6, blind searches through the PATH with locatefile_extended() trying to find where on the hard drive "we" exist because object_attr_getsym(patcher, _sym_filepath) in collectives returns no path...only a filename.

When the Max UI is used to build an EXE or a collective, and using a similar approach to today's declaration APIs like fileusage_addfilename, I want Max to "query" my external for needed dependencies. My external will use Max APIs to define those dependencies. I can with those APIs tell Max that I need some DLLs, I need some directories with files in them, etc. Then later when the collective or exe is run, Max will extract those files/directories into known locations with the desired directory structure.

I would also welcome a retirement of this API approach. Describing install and/or configuration with compiled C code has always been odd to me. I can happily imagine the Max UI for building a collective/exe allowing include of any files/directories desired into the MXF. This, to me, is exactly like using a ZIP archive...I can put anything in it I want and on extract it maintains the relative directory hierarchy. I can also imagine a sidecar YAML, JSON, etc.

Notes

I acknowledge this presents significant roadblocks for a small set of your customers using Max for professional work. The existing workarounds either satisfy their needs or they have already switched to another toolset/process. Therefore, I do not consider this a Priority 1 issue.

// works
fileusage_addfilename(hCollective, 0, "Kinect20.Face.x64.dll");

// reports in console it added file (and build seems slower), yet it is not in temp folder for collective or in standalone app folders
fileusage_addfile(hCollective, 0, "NuiDatabase/FaceAlignment.bin", 0);

// reports can't find it
fileusage_addpathname(hCollective, 0, "NuiDatabase/FaceAlignment.bin");

// console reports "Including File FaceAlignment.bin" but I can't find it in the collective or standalone folders
fileusage_addpathname(hCollective, 0, "C:/Users/Dale/Documents/Visual Studio 2010/Projects/kinect/dp.kinect2/bin/NuiDatabase/FaceAlignment.bin");

// always gives max console error: C:/Users/Dale/Documents/Visual Studio 2010/Projects/kinect/dp.kinect2/bin/NuiDatabase: can't find folder to copy
fileusage_copyfolder(hCollective, "C:/Users/Dale/Documents/Visual Studio 2010/Projects/kinect/dp.kinect2/bin/NuiDatabase", 0);

// makes a folder named "DaleFolder" in the support folder of the standalone app. Therefore doesn't put it in the collective.
fileusage_makefolder(hCollective, "DaleFolder");