Skip to content

Conversation

@vlaci
Copy link
Contributor

@vlaci vlaci commented Jul 23, 2024

Unfortunately dissect.cstruct 4.0 types are not pickleable

@vlaci vlaci force-pushed the dissect-cstruct-40-compatibility branch from c5ac2bb to 27dd541 Compare July 23, 2024 18:09
Copy link
Contributor

@e3krisztian e3krisztian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not fully understand the original problem it solves.

We are passing the process boundary when finishing tasks and returning reports. So we have some reports that contain references to dissect.cstruct.Instance/Structure instead of raw Python atoms?
We probably should not, and in that case a more specific fix would be more appropriate?

An example/stack trace for what it fixes exactly may be enlightening ;)

@vlaci
Copy link
Contributor Author

vlaci commented Jul 24, 2024

Spectacularly, even primitive types are runtime-generated:

2024-07-24 09:30.07 [debug    ] creating special file          pid=1766981 special_path=nuts/hazelnut
2024-07-24 09:30.07 [warning  ] Root privileges are required to create block and char devices. Skipped. device=0x1f00 mode=0x21a4 path=nuts/hazelnut pid=1766981
Process Process-1:
Traceback (most recent call last):
  File "/nix/store/hyppxdhgqx7nbg394j97yhmi6vizl074-python3-3.11.9/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/nix/store/hyppxdhgqx7nbg394j97yhmi6vizl074-python3-3.11.9/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/vlaci/devel/git/github.com/onekey-sec/unblob/unblob/pool.py", line 62, in _worker_process
    output.put(result)
  File "/nix/store/hyppxdhgqx7nbg394j97yhmi6vizl074-python3-3.11.9/lib/python3.11/multiprocessing/queues.py", line 371, in put
    obj = _ForkingPickler.dumps(obj)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/hyppxdhgqx7nbg394j97yhmi6vizl074-python3-3.11.9/lib/python3.11/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'types.uint32'>: attribute lookup uint32 on types failed

@vlaci vlaci force-pushed the dissect-cstruct-40-compatibility branch from 27dd541 to 9de891b Compare July 24, 2024 11:38
@vlaci vlaci force-pushed the dissect-cstruct-40-compatibility branch from 9de891b to d01c726 Compare August 2, 2024 14:00
Unfortunately `dissect.cstruct` 4.0 types are not pickleable
@kissgyorgy kissgyorgy force-pushed the dissect-cstruct-40-compatibility branch from d01c726 to 3653e5c Compare August 7, 2024 12:10
@vlaci vlaci merged commit 5e775ac into main Aug 7, 2024
@vlaci vlaci deleted the dissect-cstruct-40-compatibility branch August 7, 2024 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants