Blacklisted functions and weird code gaps

[alexrp]

I was cleaning up some functions that were falsely matched by signature scanning earlier, and by the time I was done, I saw some curious warnings in the log:

Auto-analysis skipping blacklisted function: windows-x86_64@0x140ca06a0

(There are multiple locations.)

Navigating to that area of the binary, I see:

There seems to be a weird gap here. Reading some bytes via the Python console shows that there's clearly code in the gap:

>>> bv.read(0x140ca069d, 50)
b'\xcc\xcc\xccH\x83\xec8\xf3\x0f\x10!3\xc0\x0f)t$ \x0f(\xcc\xf3\x0f\x10q\x04\xf3\x0fY\n\x0f(\xc6\xf3\x0fYB\x10\x0f)|$\x10\xf3\x0f\x10y\x14\xf3\x0f'

I don't really know what's happened here or what a blacklisted function is - I can't find anything in the documentation elaborating on it. What can I do to fix up this issue?

[fuzyll]

In the specific case of the image you've provided, those \xcc bytes are being skipped because, according to Binary Ninja, there's no way to get control flow to execute those bytes (because of the return just before them). Other disassemblers might still disassemble them as code because they're in a text section, but many compilers will interleave code and data as an optimization, so we assume those bytes are data or padding.

I assume, because I see a bunch of rax's and things, this is an x64 executable. The \xcc bytes are probably padding and intended to cause a SIGTRAP during execution if they're ever executed? So, without any additional context, I believe it's reasonable for Binary Ninja to not disassemble this as code.

For your other question, I believe a "blacklisted function" is what happens when a user function is removed. I believe we do this so auto-analysis doesn't try to re-create the function on future passes, but I'm not sure. If you (or someone else - the blacklist gets stored in the database) had previously tried to make that a function, but then removed it, I believe that would be the expected result. If not, I'm not too sure what's going on and might need additional context.

[psifertex]

Yes, simply re-creating the function should cause it to be removed from the blacklist.

[WZ-Tong]

Yes, simply re-creating the function should cause it to be removed from the blacklist.

I am regret to undefine a lot of functions, is there any ways such as “directly modding the bndb database (seems like it's SQLite)” or execute some api command so I will not re-create them one-by-one?

[psifertex]

Sure, the API in question is: https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.remove_user_function

For example:

for fn in bv.functions:
  if "bad" in fn.name:
    bv.remove_user_function(fn)

[WZ-Tong]

Sure, the API in question is: https://api.binary.ninja/binaryninja.binaryview-module.html#binaryninja.binaryview.BinaryView.remove_user_function

For example:

for fn in bv.functions:
  if "bad" in fn.name:
    bv.remove_user_function(fn)

This can undefine functions in batch, but what I need is "remove them from blacklist" so linear sweep or other auto process will re-create them back.

[psifertex]

Ahh, sorry, I misunderstood before. No, there is no official way to do that and I would advise caution against poking the BNDB directly as the format is subject to change without warning.

That said -- the undo system does record a log of removed functions and so you can either:

1. save a list of functions removed yourself (potentially in the bv.metadata, eg) and then just re-create them with add_user_function() if needed
2. use bv.undo() if you have not made other changes as that will be able to undo the blacklisting

When #2070 is solved, that might provide another approach for more fine-grained undo-manipulation.