Fix custom write function segfault

[james-clemer-actblue]

This is just a PR-back of this PR in our fork

Below is the original PR description reproduced for convenience.

One caveat is, the git history here isn't as clean as might be hoped for in this repository. If there are any org-wide contribution guidelines or implicit norms that this PR breaks, let me know. I'm happy to edit the PR, or else reproduce it as a PR from a different fork &c.

Addresses this issue.

We call client.py via parse_as_files_custom.

In turn parse_as_files_custom takes an argument, open_function, and passes it in to utils.provide_write_callback, which handles writing to various files and uses open_function to create new file handles for various files.

utils.provide_write_callback uses a type-factory from ctypes and deems it the CUSTOM_WRITE type.

CUSTOM_WRITE represents, on the C side of things, CustomWriteFunction.

CUSTOM_WRITE = CFUNCTYPE(None, c_char_p, c_char_p, POINTER(c_char), c_int)

...

typedef void (*CustomWriteFunction)(char *filename, char *extension, char *contents, int numBytes);

But! Calling any python CustomWriteFunction induces a SEGFAULT!

So, I wrote a CustomWriteFunction in C, and passed that in. This, and some heavy printf-debugging, led me to notice that contents is not a NULL-terminated char*. That made me wonder if somewhere along the way, there was machinery in ctypes that assumed it was, since NULL-terminated char*s are the defacto str type in C-land.

It turns out, in the docs for ctypes.c_char_p, we're told that c_char_p:

Represents the C char* datatype when it points to a zero-terminated string. For a general character pointer that may also point to binary data, POINTER(c_char) must be used. The constructor accepts an integer address, or a bytes object.

Doing so means that calls to:

#! python

import smart_open
from fastfec import *


if __name__ == "__main__":
    headers = {'headers': {'User-Agent': 'Mozilla/5.0'}}
    with smart_open.open(f'http://docquery.fec.gov/dcdev/posted/1606847.fec', 'rb', transport_params=headers) as f:
        with FastFEC() as fastfec:
            fastfec.parse_as_files(f, "some_output_directory", include_filing_id='1606847')

no longer SEGFAULT.

[freedmand]

Thanks for the PR! This is a very important one that will solve some issues others have encountered as well. I just pushed a lot of changes to the release process for FastFEC that should make the GitHub actions checks pass. Could you pull/merge develop back into your branch @james-clemer-actblue? Once the GH actions checks pass, I will give it the go-ahead.

[freedmand]

Works! Thanks again

[james-clemer-actblue]

Awesome! For future reference, is "open an issue, investigate, open a PR" the best way to contribute? This seems like it went smoothly enough but if you have any processes that make it easier on you, I'm happy to respect them.