IDE Development with arm64 Architecture on Apple M1 Machine

[Alnyli07]

I have apple m1 machine, want to develop theia based desktop IDE.

Feature Description:

I have apple m1 machine, want to develop theia based desktop IDE.
I have tested Theia-blueprint IDE , gives (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')) error on ffmpeg.dylıb, drivelist.node ... vs build files.

I searched any document to develop with arm64 but could't find.

I reviewed theia/cli source code especially searched 'rebuild:electron' command. I found electron-rebuild command in https://github.com/eclipse-theia/theia/blob/master/dev-packages/application-manager/src/rebuild.ts#L218.

I added following two piece of code to rebuild.ts file

       //line 38 added "keytar" module to default modules.
       export const DEFAULT_MODULES = [
          'node-pty',
          'nsfw',
          'native-keymap',
          'find-git-repositories',
          'drivelist',
          'keytar'
      ];
       // line 218
        console.log("REBUILD ARCH: ", process.env.ARCH)
        if (process.env.ARCH) {
            command += ` --arch ${process.env.ARCH}`;
        }

Finally run again theia rebuild:electron then yarn start command in applicatios/electron folder.
ARCH=arm64 theia rebuild:electron
It worked properly.

How can contribute this to theia repo?

Thanks,

[colin-grant-work]

@Alnyli07, thanks for the information. If you would like to contribute your fix to the repo, you would need to open a PR and sign the Eclipse Contributor Agreement You can find the basics of the PR process here and information about the ECA can be found here. In principle, I don't see any reason we wouldn't accept your fix. @paul-marechal would know the details of whether this looks like it would have any negative side effects.

[SConaway]

Using node v16 solves the issue. Because node v14 doesn't have an arm build, things get compiled weirdly. Switching to node v16 seems to work fine!

[kittaakos]

@Alnyli07, why did you decide to discontinue your PR?

I have the same problem. I switched to Node.js 16+, but the ffmpeg check failed. Is there a way to bypass it? @paul-marechal or @vince-fugnitto, could you please advise? Do you build electron apps for M1? Thank you!

Details:

~/Desktop/a.kitta/arduino-ide m1 !2                                                  
❯ node
Welcome to Node.js v16.14.2.
Type ".help" for more information.
> console.log(require('os').platform() + '-' + require('os').arch())
darwin-arm64
undefined
> .exit

~/Desktop/a.kitta/arduino-ide m1 !2                                               42s
❯ node -v 
v16.14.2

electron-app: Successfully replaced "/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Libraries/libffmpeg.dylib".
electron-app: Error: dlopen(/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Libraries/libffmpeg.dylib, 0x0002): tried: '/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Libraries/libffmpeg.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libffmpeg.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))
electron-app:     at Object.getFfmpegCodecs (/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/@theia/ffmpeg/lib/ffmpeg.js:80:20)
electron-app:     at Object.checkFfmpeg (/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/@theia/ffmpeg/lib/check-ffmpeg.js:23:27)
electron-app:     at ApplicationPackageManager.prepareElectron (/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/@theia/application-manager/lib/application-package-manager.js:153:22)
electron-app:     at async ApplicationPackageManager.prepare (/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/@theia/application-manager/lib/application-package-manager.js:64:13)
electron-app:     at async ApplicationPackageManager.generate (/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/@theia/application-manager/lib/application-package-manager.js:69:13)
electron-app:     at async ApplicationPackageManager.build (/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/@theia/application-manager/lib/application-package-manager.js:89:9)
electron-app:     at async Object.handler (/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/@theia/cli/lib/theia.js:146:13)

ARCH=arm64 yarn rebuild:browser && yarn rebuild:electron did not help. Starting the app fails with this:

Can't find Node.js binary "/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/.bin/electron": error getting version: /Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/.bin/electron exited with code 1: dyld[96304]: Library not loaded: @rpath/libffmpeg.dylib
Referenced from: /Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework
Reason: tried: '/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libffmpeg.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/MacOS/../Frameworks/libffmpeg.dylib' (no such file), '/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/MacOS/../Frameworks/libffmpeg.dylib' (no such file), '/usr/local/lib/libffmpeg.dylib' (no such file), '/usr/lib/libffmpeg.dylib' (no such file)
/Users/tooling/Desktop/a.kitta/arduino-ide/node_modules/electron/dist/Electron.app/Contents/MacOS/Electron exited with signal SIGABRT
. Make sure Node.js is installed and in your PATH, or set the "runtimeExecutable" in your launch.json

Update
I can build and start the Theia electron app (I tried with 1.30.0 and 1.25.0) from the sources, but for the downstream app, it fails with the libffmpeg.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e') error.

[kittaakos]

Using node v16 solves the issue. Because node v14 doesn't have an arm build, things get compiled weirdly. Switching to node v16 seems to work fine!

I can confirm it.

OK, I have figured it out. I managed to build Theia v1.25.0 tag, HEAD of the master (7e753f4), and any generated Theia apps. All worked fine with Node.js v16.14.2 except my downstream app.

I found that the CLI does cache files here:

theia/dev-packages/ffmpeg/src/replace-ffmpeg.ts

Line 37 in 7e753f4

const ffmpegCachedPath = path.join(os.tmpdir(), `theia-cli/cache/electron-v${electronVersion}`, ffmpegName);

I initially built the downstream app with Node.js 14, where the arm64 arch is unavailable. The x64 build was cached. After wiping the cache folder all works.

---

Update:
Here is a valuable comment with a workaround: eclipse-theia/vscode-builtin-extensions#101 (comment)

[tsmaeder]

@kittaakos since we're now on a much newer version of nodejs, can this be closed?

[kittaakos]

since we're now on a much newer version of nodejs, can this be closed?

@tsmaeder, can you ask the author of the issue? I did not open it. Thanks!

[tsmaeder]

@Alnyli07 ?

[Alnyli07]

@tsmaeder @kittaakos .
Sory for the delay

For development, node 16 or over resolve this issue.

But, For production in the Azure pipeline, we need do some changes to release for M1 ARM-arch.

in dev-packages/application-manager/src/application-package-manager.ts:

 const ffmpeg = await import('@theia/ffmpeg');
        await ffmpeg.replaceFfmpeg();
        try {
            await ffmpeg.checkFfmpeg();
        } catch (error) {
            if ((error.message.includes('(have \'arm64\', need \'x86_64\')') || error.message.includes('(have (arm64), need (x86_64)))')) && process.env.ARCH === 'arm64') {
                console.warn('WARN: ffmpeg checking skipped because mac-o file incompatible in this step');
            } else {
                throw error;
            }
        }

in dev-packages/ffmpeg/src/replace-ffmpeg.ts:

const ffmpegZipPath = await electronGet.downloadArtifact({
            version: electronVersion,
            artifactName: 'ffmpeg',
            arch: process.env.ARCH
        });

set ARCH environment variable as 'arm64' for M1 ARM release

[Alnyli07]

My new issue is related with Azure Pipeline environment. We can close this issue (for development)