Implement macOS signing and notarization

[tmontes]

PLAN:

• Collect and share documentation links about the topic (many of which I've been reading as of recently).
• Get a developer certificate from Apple.
• Explore the process manually / via ad-hoc scripts, confirming that it seems to work.
• Try to validate it in collaboration with someone else, running macOS Catalina or Big Sur.
• Transpose the process into pup code.

NOTE:

• From the investigation so far, it is a very good idea to have a properly layed out, as small as possible macOS Application Bundle to submit for notarization.
• With that in mind, Implement an effective cleanup packaging step #38 becomes a relevant dependency to this issue's "correct" implementation.

[tmontes]

RESOURCES and LINKS:

• This Mu issue, nearly one year old (!!!).
• These articles by @glyph:
  • Mac Python Distribution Post Updated for Catalina and Notarization
  • Tips And Tricks for Shipping a PyGame App on the Mac
• Apple sourced documents:
  • WWDC 2019 "All About Notarization" session
  • Code Signing Tasks
  • Customizing the Notarization Workflow
  • Resolving Common Notarization Issues
  • About Entitlements
• More:
  • Notarize a Command Line Tool
  • How to notarize your software on macOS
  • Free Pascal Wiki | Code Signing for macOS
  • Free Pascal Wiki | Notarization for macOS 10.14.5+
  • Notarizing your Flash/AIR applications for macOS

[tmontes]

CONCEPTS (that need better understanding):

• Hardened runtime (and variations thereof?).
• Secure timestamps.
• Entitlements.

[tmontes]

WORK IN PROGRESS:

signing script

#!/bin/bash
set -e

# Use keychain to allow "codesign" to always access the code signing certificate's private key.

cd ./build/pup > /dev/null 2>&1

BASE_DIR=./mu-editor.app/Contents/Resources/Python/

SIG_ID='<apple signing certificate id -- 10 alphanumeric digits>'
CSOPTS="--deep --force --verify --verbose --sign $SIG_ID --options runtime --timestamp"


# --------------------------------------------------------------------------------
# SIGN all shared libraries

find ${BASE_DIR}/lib -type f -name "*.so" -o -name "*.dylib" | while read SHLIB_TO_SIGN
do
    echo "SIGNING ${SHLIB_TO_SIGN}"
    codesign ${CSOPTS} ${SHLIB_TO_SIGN}
done


# --------------------------------------------------------------------------------
# SIGN binaries

find ${BASE_DIR}/bin -type f | while read BIN_TO_SIGN
do
    echo "SIGNING ${BIN_TO_SIGN}"
    codesign ${CSOPTS} "${BIN_TO_SIGN}"
done


# --------------------------------------------------------------------------------
# SIGN the application bundle itself

echo "SIGNING ./mu-editor.app"
codesign ${CSOPTS} ./mu-editor.app


cd -  > /dev/null 2>&1

[tmontes]

WORK IN PROGRESS:

notarization request script

#!/bin/bash

cd ./build/pup > /dev/null 2>&1


BID='com.github.mu-editor.mu'
USER='<apple-id>'
MUAS='<application-specific-password>'

/usr/bin/ditto -c -k --sequesterRsrc --keepParent ./mu-editor.app/ mu-editor.zip
xcrun altool --notarize-app --primary-bundle-id ${BID} --username ${USER} -p ${MUAS} --file mu-editor.zip  

# will print the Notarization RequestUUID

cd -

notarization status script:

#!/bin/bash

USER='<apple-id>'
MUAS='<application-specific-password>'

if [ -z "${RID}" ]
then
    echo "Set RID to the Notarization RequestUUID"
    exit
fi

xcrun altool --notarization-info ${RID} -u ${USER} -p ${MUAS}

[tmontes]

PROGRESS:

• Preliminary scripts created and shared above.
• All scripts work (up to a point).
• Notarization result is unsuccessful:

2020-09-17 17:00:21.395 altool[78612:1135309] No errors getting notarization info.

   RequestUUID: ...
          Date: 2020-09-17 15:43:31 +0000
        Status: invalid
    LogFileURL: ...
   Status Code: 2
Status Message: Package Invalid

[glyph]

Just dropping in here due to the @-mention… (I am so glad that my writing is helping with this, annoying as it is!)

The status message rarely contains enough information to debug notarization failures; you'll really want to look at the log file in that elided LogFileURL.

How did you generate the .zip? IIRC "Package Invalid" is an error you can get when forgetting to pass the -y option.

[tmontes]

@glyph, thanks for chiming in,

Agree and undestand that the actual log holds details that the status cannot convey - the note was posted with brevity and progress sharing/noting in mind. For completeness, it did note that several files were not signed which, somehow, may point at a .zip file creation issue (is it including said signatures? I will investigate) which, again, may be somewhat consistent with the "Package Invalid" status...

Wondering what you mean by the -y option... The .zip file was produced with /usr/bin/ditto -c -k --sequesterRsrc --keepParent <path-to-app-bundle> <name-of-zip-file> which, per the man page, does not seem to support such option. Maybe you are refering to the zip command's option, WRT to symlinks?

Regardless of that, again, thanks for your articles and this message. I know I'll have to keep reading, undertanding, exploring, until I have a clear enough mental model of how every bit and piece should fit together: then things should work! :)

[glyph]

Agree and undestand that the actual log holds details that the status cannot convey - the note was posted with brevity and progress sharing/noting in mind. For completeness, it did note that several files were not signed which, somehow, may point at a .zip file creation issue (is it including said signatures? I will investigate) which, again, may be somewhat consistent with the "Package Invalid" status...

As I understand it, code signatures are just regular files so they should be included with just about any archive creator...

If you're creating binary files with extensions other than .so and .dylib you might need to modify my simple signing script. (What files is it complaining about?)

Wondering what you mean by the -y option... The .zip file was produced with /usr/bin/ditto -c -k --sequesterRsrc --keepParent <path-to-app-bundle> <name-of-zip-file> which, per the man page, does not seem to support such option. Maybe you are refering to the zip command's option, WRT to symlinks?

Ah, yes, sorry I didn't realize that's what ditto was doing; I've never used it to create zip archives. No idea if it has problems with this process!

Regardless of that, again, thanks for your articles and this message. I know I'll have to keep reading, undertanding, exploring, until I have a clear enough mental model of how every bit and piece should fit together: then things should work! :)

Good luck!

[tmontes]

PROGRESS:

• Read a bit more.
• Brought in yet another article on the subject.

THOUGHTS:

• Previous (failed and, admittedly, somewhat rushed) experiences go along with recent readings, boiling down to "probably the signing I've been doing isn't 100% as it should".
• Need to specify entitlements, which I haven't done up until now.
• Want faster iteration cycles:
  • For that I'll create a minimal GUI app, tkinter-based.
  • No Qt dependencies (or any other, from PyPI) so smaller in size and with much fewer shared libraries.
  • Should also make it easier to digest any notarization failure reports.

[tmontes]

PROGRESS:

• Created minimal GUI app at https://github.com/tmontes/puppy.
• Adjusted hand-hacked scripts:
  • Now signing with entitlements.
  • Now subitting a .zip file.
• Notarization is still failing, but now down to a single, common error, once for each of the 9 files that were signed within puppy.app. It reads: "The binary is not signed with a valid Developer ID certificate.".
• Wonderful. I want my 100 EUR back! :P

[tmontes]

PROGRESS (and preliminary SUCCESS!):

• Figured out I had been using the wrong kind of certificate for signing after reading more and with more attention (thanks again to @glyph, who very carefully pinpoints that in his artice).
• Successfully signed, notarized, and stapled puppy.
• Will share the binary with the Mu development team to gather feedback.
  (published the signed+notarized macOS puppy 1.0.0 release here)