Skip to content

Add support for json '-e' fields #455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Add support for json '-e' fields #455

wants to merge 5 commits into from

Conversation

IrPgFKS0
Copy link

@IrPgFKS0 IrPgFKS0 commented Feb 2, 2021 •
edited
Loading

This request adds support that standardizes the json output (using '-e ' flags) by removing multidimensional json field nesting, and in turn will increase parsing speeds (see added support flags below)...

custom_parameters = [
    '-e', 'ip.src',
    '-e', 'ip.dst',
    '-e', 'http.file_data']

Output generated by tshark with above '-e <field>' flags:
{
    "_index": "packets-2021-02-02",
    "_type": "pcap_file",
    "_score": null,
    "_source": {
      "layers": {
        "ip.src": [
          "216.58.195.78"
        ],
        "ip.dst": [
          "10.0.0.10"
        ],
        "http.file_data": [
          "<HTML><HEAD><meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<TITLE>301 Moved<\/TITLE><\/HEAD><BODY>\n<H1>301 Moved<\/H1>\nThe document has moved\n<A HREF=\"http://www.google.com/\">here<\/A>.\r\n<\/BODY><\/HTML>\r\n"
        ]
      }
    }
  }

@BMWE
Copy link

BMWE commented Feb 9, 2021

@IrPgFKS0, Can you please do the same for other pdml?

@IrPgFKS0
Copy link
Author

IrPgFKS0 commented Feb 15, 2021 •
edited
Loading

What do you want from the pdml output that you cannot get from json? On another note if you are looking for the fastest way to read and parse JSON tshark output "-T ek" is really the best, the below project is a pretty good example of this.

https://github.com/vichargrave/espcap

He is using a synchronized process which is fine if the post-processing is minimal, but if you are running CPU bound tasks on the same proc I recommend "multiprocessing Pipe" to dedicate a processor to reading the "tshark" output and writing to a pipe, and a dedicated processor for reading from the pipe and any post-processing tasks (I am using that for CPU bound entropy calculations on the payloads).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@IrPgFKS0 @BMWE