Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--include option should work with pipe #59

Open
estahn opened this issue Feb 24, 2019 · 1 comment
Open

--include option should work with pipe #59

estahn opened this issue Feb 24, 2019 · 1 comment
Milestone

Comments

@estahn
Copy link
Collaborator

estahn commented Feb 24, 2019

It would be great to just read a JSON from STDIN instead of reading from different file descriptors.

@corvus-ch corvus-ch added this to the Version 3 milestone Apr 22, 2019
@corvus-ch
Copy link
Owner

corvus-ch commented Apr 23, 2019

I have starte working on a new major version. One of my goals is to simplify a lot. One thing I am taking into consideration is to remove support for passing the payload by argument. If I proceed proceed with that idea, I would like to also reject that request. This is due to the fact that a message body can be arbitrary binary data. In order to place it into a JSON data structure, the body would have to be encoded with the consumer and decoded within the worker script. This reduces the throughput this consumer can handle.

So the options are:

  1. Read from two file descriptors and run json decode on one of them. (less memory usage, less computing required)
  2. Read from one file descriptor and run json decode. Extract one field and decode it again (e.g base64). (more computing and memory required)

@estahn What would be your argument to have both attributes and body within the same data structure?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants