Exit code detection failing on iOS/tvOS 15+

[premun]

With the latest move of dotnet/runtime onto OSX 12.00, apple run and apple just-run commands started failing in 100% of cases:

Metrics
| where EventType == "_MobileDeviceOperation" and MetricName == "ExitCode"
| join kind=inner Jobs on JobId
| where Finished > now()-1d
| extend Dimensions = parse_json(Dimensions)
| project 
    Finished,
    JobName,
    QueueName,
    WorkItemFriendlyName,
    Platform = tostring(Dimensions.platform),
    Command = tostring(Dimensions.command),
    ExitCode = MetricValue,
    Target = tostring(Dimensions.target),
    IsDevice = tobool(Dimensions.isDevice)
| where Platform == "apple"
| summarize SuccessRate=countif(ExitCode < 6)*100/count(), Count=count() by Platform, Command, QueueName
| where SuccessRate != 100
| order by Platform, SuccessRate, Command

[premun]

Some reading for later

• https://developer.apple.com/documentation/corefoundation/cffiledescriptor-ru3#2556086
• https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/kqueue.2.html
• https://stackoverflow.com/a/24690619/6289970
• https://developer.apple.com/library/archive/technotes/tn2050/_index.html

[premun]

Seems like the issue is Xcode 13.2 rather than OSX 12.00.

Also I found out, I might be able to use kqueue and kevent to get notified about a process' exit where the exit code should be in the data of the event:
https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/kevent.2.html

EVFILT_PROC  Takes the process ID to monitor as the	identifier and
			 the events to watch for in fflags, and	returns	when
			 the process performs one or more of the requested
			 events.  If a process can normally see	another
			 process, it can attach	an event to it.	 The events to
			 monitor are:

			 NOTE_EXIT	   The process has exited. The exit status will be stored in data.

[premun]

Oh noez, the sentence "The exit status will be stored in data." actually only appears in FreeBSD man pages and not OSX man pages 😢

I have tested getting the PID (the line with kev.data):

// cc test.c -framework CoreFoundation -O

#include <CoreFoundation/CoreFoundation.h>
#include <unistd.h>
#include <sys/event.h>

// build: cc test.c -framework CoreFoundation -O
// run: ./a.out 57168

static void noteProcDeath(CFFileDescriptorRef fdref, CFOptionFlags callBackTypes, void *info) {
    struct kevent kev;
    int fd = CFFileDescriptorGetNativeDescriptor(fdref);
    kevent(fd, NULL, 0, &kev, 1, NULL);
    // take action on death of process here
    printf("process with pid '%u' died with '%ld'\n", (unsigned int)kev.ident, kev.data);
    CFFileDescriptorInvalidate(fdref);
    CFRelease(fdref); // the CFFileDescriptorRef is no longer of any use in this example
}

// one argument, an integer pid to watch, required
int main(int argc, char *argv[]) {
    if (argc < 2) exit(1);
    int fd = kqueue();
    struct kevent kev;
    EV_SET(&kev, atoi(argv[1]), EVFILT_PROC, EV_ADD|EV_ENABLE, NOTE_EXIT, 0, NULL);
    kevent(fd, &kev, 1, NULL, 0, NULL);
    CFFileDescriptorRef fdref = CFFileDescriptorCreate(kCFAllocatorDefault, fd, true, noteProcDeath, NULL);
    CFFileDescriptorEnableCallBacks(fdref, kCFFileDescriptorReadCallBack);
    CFRunLoopSourceRef source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fdref, 0);
    CFRunLoopAddSource(CFRunLoopGetMain(), source, kCFRunLoopDefaultMode);
    CFRelease(source);
    // run the run loop for 20 seconds
    CFRunLoopRunInMode(kCFRunLoopDefaultMode, 200.0, false);
    return 0;
}

It is always 0 😢

This also supports that: https://lists.apple.com/archives/Darwin-dev/2009/Aug/msg00210.html

[premun]

This is not a problem of Xcode, this is a problem of iOS 15. The system log of the simulator contains a very limited amount of logs compared to 13.5 for example (both running on Xcode 13.1).

[premun]

I was able to get more logs via:

xcrun simctl spawn booted log stream --level=Info | grep [APP NAME]

The exit code is there, so is the stdout. So we should also be able to fix #694

More info:
https://stackoverflow.com/questions/10165641/how-can-i-get-the-console-logs-from-the-ios-simulator

[premun]

Solution for Simulators is here: https://github.com/premun/xharness/tree/prvysoky/simulator-exit-code

MacCatalyst is left