Mac.zip symbol pack doesn't contain all release kernels

[jhilgert]

Hey there,

so I was trying to run Volatility 3 on the Mac memory samples of the Art of Memory Forensics book. For this, I downloaded the mac.zip symbol file, but still run into the following error when running for example the mac.pslist.PsList plugin. Is there anything else that needs to be installed to fulfil the requirements?

Unsatisfied requirement plugins.PsList.primary: Memory layer for the kernel
Unsatisfied requirement plugins.PsList.darwin: Mac kernel symbols

A symbol table requirement was not fulfilled.  Please verify that:
        You have the correct symbol file for the requirement
        The symbol file is under the correct directory or zip file
        The symbol file is named appropriately or contains the correct banner


A translation layer requirement was not fulfilled.  Please verify that:
        A file was provided to create this layer (by -f, --single-location or by config)
        The file exists and is readable
        The necessary symbols are present and identified by volatility
Unable to validate the plugin requirements: ['plugins.PsList.primary', 'plugins.PsList.darwin']

I didn't get any useful information from the debug output, I just saw that it was running the MultiStringScanner and then continued.

Level 9  volatility.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.primary
Level 9  volatility.framework.configuration.requirements: TypeError - SymbolTableRequirement only accepts string labels: None
Level 7  volatility.framework.layers.resources: Available URL handlers: HTTPErrorProcessor, HTTPDefaultErrorHandler, HTTPRedirectHandler, ProxyHandler, HTTPBasicAuthHandler, ProxyBasicAuthHandler, HTTPDigestAuthHandler, ProxyDigestAuthHandler, AbstractHTTPHandler, HTTPHandler, HTTPSHandler, HTTPCookieProcessor, UnknownHandler, FileHandler, FTPHandler, CacheFTPHandler, DataHandler, JarHandler
Level 8  volatility.framework.automagic.stacker: Attempting to stack using Elf64Stacker
Level 6  volatility.framework.layers.elf: Exception: Bad magic 0x80000002 at file offset 0x0
Level 8  volatility.framework.automagic.stacker: Attempting to stack using LimeStacker
Level 8  volatility.framework.automagic.stacker: Attempting to stack using WindowsCrashDump32Stacker
Level 8  volatility.framework.automagic.stacker: Attempting to stack using VmwareStacker
Level 8  volatility.framework.automagic.stacker: Attempting to stack using WintelStacker
DEBUG    volatility.framework.automagic.windows: Self-referential pointer not in well-known location, moving to recent windows heuristic
Level 8  volatility.framework.automagic.stacker: Attempting to stack using MacintelStacker
Level 8  volatility.framework.automagic.stacker: Attempting to stack using LintelStacker
INFO     volatility.framework.automagic.linux: No Linux banners found - if this is a linux plugin, please check your symbol files location
Level 9  volatility.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.primary
Level 9  volatility.framework.configuration.requirements: TypeError - Layer is not the required Architecture: FileLayer
DEBUG    volatility.framework.automagic.stacker: Stacked layers: ['FileLayer']
INFO     volatility.framework.automagic: Running automagic: MacSymbolFinder
Level 9  volatility.framework.configuration.requirements: TypeError - SymbolTableRequirement only accepts string labels: None
Level 9  volatility.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.primary
Level 9  volatility.framework.configuration.requirements: TypeError - SymbolTableRequirement only accepts string labels: None

[ikelos]

Hiya,

Just to verify that the mac.zip file is being put in the correct place, you should be installing it to where-ever volatility is install or you've checked it out to, then under the volatility/symbols directory.

You can verify that the caching of the mac banners has worked by checking the cache file (<user home directory/.cache/volatility/mac_banners.cache). In that file, you should be able to find the Mac kernel string that the images use and the path to them (such as jar:file:/path/to/mac.zip!mac/<kernel-name>.json.xz). If you're not seeing any lines in there then volatility doesn't know about any mac symbol tables and either the mac.zip file didn't download correctly or is in the wrong place. If you can see entries in there, then it's possible that the sample you're testing against hasn't had its symbols properly generated and included in the zip file, in which case you can use dwarf2json along with the Mac KDK to construct your own JSON file.

If you could report back on the contents of the cache file, then we can figure out which it is and how to proceed... 5:)

[jhilgert]

Thanks for your reply! So the cache file has entries and references to the mac.zip file. The AMF samples use a Mavericks_10.9.3_AMD profile, and apparently there is a 10.9.3. json file in the mac.zip file. I can try to generate the symbols myself and see if this helps.

[ikelos]

Yep, you're welcome to give it a go. The shell script we used for unpacking a KDK dmg and creating a JSON file from it are rather scratchily coded into some scripts that might help? Before you head down that path, it might be worth trying to figure out why volatility didn't even select that symbol table (the debug output doesn't indicate it found a match). You can do that by looking in the the JSON file in the zip, and making sure it has a version symbol (it's probably the only one with a constant_data field, which may be easier to search for). For that kernel, it should read:

RGFyd2luIEtlcm5lbCBWZXJzaW9uIDEzLjIuMDogVGh1IEFwciAxNyAyMzowMzozNiBQRFQgMjAxNDsgcm9vdDp4bnVfZGVidWctMjQyMi4xMDAuMTN+MS9ERUJVR19YODZfNjQA

which is the base64 encoding of:

b'Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:36 PDT 2014; root:xnu_debug-2422.100.13~1/DEBUG_X86_64\x00'

If you can search through your memory image for that exact string, it could help us start to figure out what's wrong. Don't forget the null character at the end, the entire string has to be found somewhere in the image. If that string doesn't exist in there, it might be worth finding a similar string (search for Darwin ) and then seeing if perhaps we just haven't included the right architecture or if it's a new kernel we don't know about?

[ikelos]

@jhilgert Did you make any progress on this? I'm keen to get to the bottom of the problem you were having?

[vesche]

Hey there, I'm having this same problem. Not sure if I'm doing something incorrectly. I'm trying to analyze a memory dump from a 10.14.6 Mojave system. I see the KDKs for 10.14.6 in the mac.zip symbols and I have placed the zip in volatility/symbols ...

Here's what I'm getting:

$ python3 vol.py -f ../Memory_Capture/mem.raw mac.psaux
Volatility 3 Framework 1.0.0-beta.1
Progress:   99.91		Scanning FileLayer using MultiStringScanner
Unsatisfied requirement plugins.Psaux.primary: Memory layer for the kernel
Unsatisfied requirement plugins.Psaux.darwin: Mac kernel symbols

A symbol table requirement was not fulfilled.  Please verify that:
	You have the correct symbol file for the requirement
	The symbol file is under the correct directory or zip file
	The symbol file is named appropriately or contains the correct banner


A translation layer requirement was not fulfilled.  Please verify that:
	A file was provided to create this layer (by -f, --single-location or by config)
	The file exists and is readable
	The necessary symbols are present and identified by volatility
Unable to validate the plugin requirements: ['plugins.Psaux.primary', 'plugins.Psaux.darwin']

Like @jhilgert said I see the MultiStringScanner running, but then it just errors out.

I followed a bit of what you posted on Dec 9th @ikelos and so I went into the mac.zip, found the latest 10.14.6 which seems to be Kernel_Debug_Kit_10.14.6_Build_18G84.dmg.json.xz. It did not have the exact same kernel as me (4903.270.47~4 vs 4903.278.19~1) is it really that picky?
Anyways so I used xz tools to crack it open, base64 decoded the string, modified it to fit my kernel, and then base64'd/xz'd/zip'd up/replaced the symbol file. But this also DID NOT WORK.

$ xzcat Kernel_Debug_Kit_10.14.6_Build_18G84.dmg.json.xz  | grep -i constant_data
      "constant_data": "RGFyd2luIEtlcm5lbCBWZXJzaW9uIDE4LjcuMDogVGh1IEp1biAyMCAxODo0MjoyMSBQRFQgMjAxOTsgcm9vdDp4bnUtNDkwMy4yNzAuNDd+NC9SRUxFQVNFX1g4Nl82NAA="
$ echo RGFyd2luIEtlcm5lbCBWZXJzaW9uIDE4LjcuMDogVGh1IEp1biAyMCAxODo0MjoyMSBQRFQgMjAxOTsgcm9vdDp4bnUtNDkwMy4yNzAuNDd+NC9SRUxFQVNFX1g4Nl82NAA= | base64 -D
Darwin Kernel Version 18.7.0: Thu Jun 20 18:42:21 PDT 2019; root:xnu-4903.270.47~4/RELEASE_X86_64⏎
$ uname -a
Darwin hostname 18.7.0 Darwin Kernel Version 18.7.0: Sun Dec  1 18:59:03 PST 2019; root:xnu-4903.278.19~1/RELEASE_X86_64 x86_64

>>> base64.b64encode(base64.b64decode('RGFyd2luIEtlcm5lbCBWZXJzaW9uIDE4LjcuMDogVGh1IEp1biAyMCAxODo0MjoyMSBQRFQgMjAxOTsgcm9vdDp4bnUtNDkwMy4yNzAuNDd+NC9SRUxFQVNFX1g4Nl82NAA=').replace('Darwin Kernel Version 18.7.0: Thu Jun 20 18:42:21 PDT 2019; root:xnu-4903.270.47~4/RELEASE_X86_64', 'Darwin Kernel Version 18.7.0: Sun Dec  1 18:59:03 PST 2019; root:xnu-4903.278.19~1/RELEASE_X86_64'))
'RGFyd2luIEtlcm5lbCBWZXJzaW9uIDE4LjcuMDogVGh1IEp1biAyMCAxODo0MjoyMSBQRFQgMjAxOTsgcm9vdDp4bnUtNDkwMy4yNzguMTl+MS9SRUxFQVNFX1g4Nl82NAA='

So... Now I'm looking now at generating my own symbols using the link you shared: https://github.com/volatilityfoundation/volatility3/tree/master/development/mac-kdk

So i have two questions hope you can help me out:

1. Is there a command to list loaded symbol files? Just want to make sure it can actually see the mac.zip I put in the folder because I have no confirmation of that.
2. How do I use these scripts to gen a symbol file?

Thanks in advance & thanks for the work on this tool- I've been using vol2 for ~8 years.

[ikelos]

Hi @vesche!

I'm afraid it is that picky (it uses the exact string match) which means not just the version number, but also the compilation date. In your example it looks like one is Thu Jun 20 18:42:21 PDT 2019 and the other is Sun Dec 1 18:59:03 PST 2019. I didn't realize Apple rebuilt them quite so often, but yep, you'd need to either change the whole string to exactly what's in the memory, or rebuild the symbols from the KDK... 5:S Just changing the string should work under the assumption that the offset locations for certain structures don't change. If they do, using the right KDK is the best route to go...

As to your questions:

1. No, not at the moment, although that should be somewhat easy to add in to the lowest level of debug logs. There is a (pickled) list of the kernel strings and the files that relate to the kernel strings that volatility has cached. That can be found in ~/.cache/volatility3/mac_banners.cache, but as I say I may add it to the debugging output soon. Do prod me/file a ticket if it doesn't appear by the end of next week...
2. Yes, all the scripts we use for taking a KDK's dmg image and extract the symbols from it can be found in /development/mac-kdk/. They're not what I consider release worthy, but they are what we use and whilst not documented should work.

I hope that helps? I'm glad you've gotten such good use out of it, this one started getting written about when you first started using vol2. Glad to see the other one had such staying power! 5:D

[vesche]

Thanks so much for the fast reply! I confirmed by looking in ~/.cache/volatility3/mac_banners.cache that the symbol files are being loaded so that's good. I tried replacing the entire string from the symbol file, but that didn't work either.

I've now run into two roadblocks... The first is that, it seems that this just simply won't be possible at the moment:

$ sw_vers | grep BuildVersion
BuildVersion:	18G2022

I'm on latest 10.14.6 (18G2022). Apple hasn't released the KDK yet it seems: https://developer.apple.com/download/more/?=debug

Edit: Unless there's a way to gen the symbol file without the KDK from the OS kernel?

However, I am trying to use 18G2016 (Nov 2019) to generate a symbol file and I'm having an issue. I extracted the pkg contents (following your extract_kernel.sh script), and did this:

$ pwd
...../Library/Kernels/kernel.dSYM/Contents/Resources/DWARF
$ file kernel
kernel: Mach-O 64-bit dSYM companion file x86_64
$ dwarf2json mac --macho kernel > Kernel_Debug_Kit_10.14.6_Build_18G2016.dmg.json

Hopefully this is correct? I was feeling good about it, but the "constant_data" string is messed up / doesn't decode correct:

$ cat Kernel_Debug_Kit_10.14.6_Build_18G2016.dmg.json|grep -i constant_data
      "constant_data": "LGMFAA8BAAAA6ZwAgP///0djBQAPDwAA0AfiAID///9gYwUADxEAABiU5wCA////eWMFAA8BAACw55wAgP///5djBQAPAQAAIOecAID///+1YwUADwEAAFDknACA////1GM="
$ echo LGMFAA8BAAAA6ZwAgP///0djBQAPDwAA0AfiAID///9gYwUADxEAABiU5wCA////eWMFAA8BAACw55wAgP///5djBQAPAQAAIOecAID///+1YwUADwEAAFDknACA////1GM= | base64 -D
,c�����Gc������`c������yc�������c ������cP������c⏎

The generated file seems to match outside of that problem. I haven't tried using it yet. About to just try it out, but I don't have much faith. Any advice?

[ikelos]

Hmmmm, that's a tough one. 5:S I'm very surprised making the string match exactly didn't work? I'd have expected some debugging output to say that the string matched if it found it in the file. If you use lots of -vs (I think we now go up to 7 or so), you ought to hit this logging line:
vollog.debug("Identified banner: {}".format(repr(banner)))

If you're not seeing that, it means it can't find the string in the file (which you can check with a simple grep). Also, be aware that if the filename hasn't changed, there'll already be an entry in the cache and it won't have been refreshed, so you might also try removing the mac_banners.cache file to ensure proper rereading of the file to make use of the correct banner string...

[ikelos]

@vesche As of commit db8ba402 there should now be ridiculous levels of debugging (if you include -vvvvvv). I didn't really want to list each banner we might detect, but that might be more useful than the files we're getting that banner from? Let me know if that's helpful, too much, not descriptive enough, etc, etc. It kinda feels like a lot, and doesn't really help people identify the cacheing issue. I guess probably I should include a "clear cache" option in the CLI? Hmmmmm, mebbe I'll mock that up at some point and then ditch the extra debugging...

[ikelos]

Yep, changed my mind (sorry). Instead, there's now a --clear-cache option which will get rid of the existing cached files. The next automagic run for a linux or mac plugin will repopulate the banners, and that will list each banner and file seperately there.

[jhilgert]

Sorry for my late reply! I will also give it a try as soon as I get back to work. --clear-cache option is definitely a good idea.

[vesche]

EDIT 29JUL2021: If you're reading this keep in mind- this is hacky. Read this please.

Ok, I got vol3 to work with my 10.14.6 18G2022 memory dump!!!

Here's what I did in case anyone comes peeking at this:

• Downloaded the latest KDK (closest available to 18G2022 as of writing this): Kernel_Debug_Kit_10.14.6_build_18G2016.dmg
• Extracted the pkg file, KernelDebugKit.pkg
• Dump the pkg: xar -x -f KernelDebugKit.pkg
• Run this script: python parse_pbzx2.py KDK.pkg/Payload
• xz -d KDK.pkg/*.xz
• cat KDK.pkg/*.cpio > Payload\~
• cpio -i < Payload\~
• Download dwarf2json and build it go build
• Use dwarf2json to generate the symbol file: /path/to/dwarf2json mac --macho kernel > Kernel_Debug_Kit_10.14.6_Build_18G2016.dmg.json
• Now you need to match your constant_data key in the JSON file to the "kernel string" in your memory dump... For example: strings Memory_Capture/mem.raw |grep -i "Darwin Kernel Version" -> Darwin Kernel Version 18.7.0: Sun Dec 1 18:59:03 PST 2019; root:xnu-4903.278.19~1/RELEASE_X86_64 is mine
• Now we need to replace this as base64 with a null byte on the end in our json file:

>>> import base64
>>> base64.b64encode('Darwin Kernel Version 18.7.0: Sun Dec  1 18:59:03 PST 2019; root:xnu-4903.278.19~1/RELEASE_X86_64\x00')
'RGFyd2luIEtlcm5lbCBWZXJzaW9uIDE4LjcuMDogU3VuIERlYyAgMSAxODo1OTowMyBQU1QgMjAxOTsgcm9vdDp4bnUtNDkwMy4yNzguMTl+MS9SRUxFQVNFX1g4Nl82NAA='

• Now take that base64 string, and replace it in the constant_data key inside Kernel_Debug_Kit_10.14.6_Build_18G2016.dmg.json
• Now xz it up xz Kernel_Debug_Kit_10.14.6_Build_18G2016.dmg.json
• Download a fresh mac symbols zip file and unzip mac.zip
• Place your new json.xz file there, and the rezip it zip -vr mac.zip mac/
• Place that new mac.zip in volatility3/volatility/symbols/
• Either clear your cache out or just yolo rm -rf ~/.cache/volatility3
• Profit? python3 vol.py -vvv -f ../Memory_Capture/mem.raw mac.psaux :)

Thanks a lot for the help @ikelos !

[vesche]

My next goal is to create a tool to automate: dumping macos memory (using osxpmem), generating the appropriate symbol file, and dumping all mac vol3 plugin outputs. I'll let y'all know how that goes. Cheers.

[ikelos]

No problem, glad you got it working! If it helps, you don't need to put it into mac.zip, you can just put it under volatility/symbols/mac/ instead, the zip is just a nice way of keeping everything together. 5:)

[garanews]

There is someone created symbols for images on https://www.memoryanalysis.net/amf
and would share?

[dvcrn]

I tried creating a profile for 10.15 from the latest KDK that matches my OS build which is 10.15.4. Thanks for the great writeup @vesche, didn't run into any issues with creating it. I installed the KDK on disk first though and just used the path to the kernel inside the Developer folder instead.

However, even with the created symbols for my OS (with or without replacing output_data), I am running into the same issues that OP has. I checked the kernel string of my dwarf2json kdk, and matched it with the debug output and information inside my dump

Output of -vvvvvv:

Level 8  volatility.framework.automagic.symbol_cache: Caching banner b'Darwin Kernel Version 18.0.0: Tue Jul  9 11:12:08 PDT 2019; root:xnu-4903.201.2.100.7~1/RELEASE_X86_64 x86_64\x00' for file file://xxxxxxxx/volatility3/volatility/symbols/mac/Kernel_Debug_Kit_10.15.4_Build_19E287.dmg.json.xz

Then a bit later:

DEBUG    volatility.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 18.0.0: Tue Jul  9 11:12:08 PDT 2019; root:xnu-4903.201.2.100.7~1/RELEASE_X86_64 x86_64\x00'

Dump:

$ strings ../path/do/my/dump |grep -i "Darwin Kernel Version"
Darwin osx310.apple.com 19.0 Darwin Kernel Version 18.0.0: Tue Jul  9 11:12:08 PDT 2019; root:xnu-4903.201.2.100.7~1/RELEASE_X86_64 x86_64

[ikelos]

@dvcrn If volatility 3 is reporting that it identified the banner, that's the point at which it ties the symbol table to the layer that the automagic is checking. As you can see at https://github.com/volatilityfoundation/volatility3/blob/master/volatility/framework/automagic/symbol_finder.py#L104 this should then have been followed by another DEBUG message saying Using symbol library:. If it wasn't, that means that the cached file that that banner points to wasn't accessible for some reason. If it was accessible, then the symbol table will have been tied to the layer, and we'd need to see the rest of the debug output to figure out what was going on.

Sorry, I conflated two separate mac issues. Since your output suggests it cached the file that instance, it's likely that isn't the issue, but as I say, we'd need to see the whole output, so please file a different bug and we can look into it there. 5:)

[ikelos]

Sorry for the spam! 5:S This was issue I was thinking of, but there were two resolutions. One was the ability to clean out old caches, and the second was to regenerate the mac file with all the release kernels and not just (perhaps not even) the debug kernels, so I'll leave this open until we get that second bit sorted.

[vesche]

@dvcrn What's the latest KDK available if you're on 10.15.4? (having troubles logging in to see) If the KDK you're using to gen the symbols file doesn't start with 19E you're probably SOL until they release at least 19E266.

I'm genuinely curious how LE tools conduct macOS memory forensics. It's my understanding that Cellebrite products can conduct memory forensics on latest builds without issue. Either they have a deal with Apple to get KDKs before general release or they have advanced techniques. Time to IDA Pro BlackBag?

[dvcrn]

I am using the latest KDK for my OS version, that's at the time of writing Kernel_Debug_Kit_10.15.4_Build_19E287.

I'm getting this output after the "identified banner" message:

DEBUG    volatility.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 18.0.0: Tue Jul  9 11:12:08 PDT 2019; root:xnu-4903.201.2.100.7~1/RELEASE_X86_64 x86_64\x00'
DEBUG    volatility.framework.automagic.mac: Mac ASLR shift value determined: 0
DEBUG    volatility.framework.automagic.mac: Invalid kalsr_shift found at offset: 2346584159

so I'm guessing my dump might be the problem, hrm.

No need to create an additional issue, I'm pretty certain that it's just something on my end and not actually a bug in volatility

[vesche]

What did you use to take the memory dump? I'm using OSXpmem: https://github.com/google/rekall/releases/tag/v1.5.1

[dvcrn]

You were right, it was because of my dump. It was a lldb coredump and I just realized that volatility doesn't support that. I tried with a pmemdump I had from the same system and yep it works now!

Thanks for the help 😄

[dvcrn]

This is getting offtopic but I wanted to ask anyway: Since the majority of mac plugins haven't been ported from v2 yet, did someone here found a way to generate a profile for v2 with recent KDKs? The scripts aren't working anymore and with focus on v3 right now I don't think they are getting fixed anytime soon.

[ikelos]

@dvcrn That might be a better question for vol2. Since we've now updated the mac pack with the latest release kernels (only) I'm going to close this issue off...

[ikelos]

For any reading this thread, and I hope getting beyond this comment. Whilst this may have worked, it was with an existing KDK, because the banner location will have matched the system as well. Changing just the banner's constant_data alone will not in general work, which is why we don't generally provide instructions on how to achieve this. Volatility uses both the banner and its location to determine the how to read mac memory images, and if either one of those is wrong, things will fail.

[vesche]

@ikelos FWIW, I edited my comment you referenced and linked to your note.

[ikelos]

Thanks very much, it's appreciated. 5:)

If it helps, I'm not sure how others do it, they might do it the way volatility 2 used to, which is to have a rough template and assume that it doesn't change very often in the structures that you need (the most common ones). It was a design decision for volatility 3 to push this in the direction of accuracy and therefore not to use generic symbols made by hand, but instead use the structures that came from the authoritative source...