| Version | name | meaning |
|---|---|---|
1.00 |
Itamar Raviv (itraviv) |
first version |
2.00 |
Anton Kiselev (kisel) |
Add description for updated compatibilities of packet creation and modification |
3.00 |
Vyacheslav Ogai (hedjuo) |
Add description for supporting Filed Engine and predefined packet templates |
Scapy Server is implemented following the JSON-RPC 2.0 specification,
Therefore, requests and replies follow the JSON-RPC 2.0 spec.
The server operates on a Request-Response basis over ZMQ, and does not support batched commands handling.
Read more about ZMQ here
Error codes are given according to this table: [also follows the JSON-RPC spec, with added error codes]
| Error Code | Message | Meaning |
|---|---|---|
-32700 |
Parse Error |
Invalid JSON was received by the server. An error occurred on the server while parsing the JSON input. |
-32600 |
Invalid Request |
The JSON sent is not a valid Request object. |
-32601 |
Method not found |
The method does not exist / is not available |
-32603 |
Invalid params |
Invalid method parameter(s) |
-32097 |
Syntax Error |
Syntax Error in input |
-32098 |
Scapy Server: message |
Scapy Server had an error while executing your command, described in the message given |
-32096 |
Scapy Server: Unknown Error |
Scapy Server encountered an error that cannot be described |
Following JSON represents a Scapy structure, which can be used to build packet from scratch(build_pkt) or to modify particular fields in the prococol(reconstruct_pkt). Most fields can be omitted, in this case default or calculated values will be used. For reconstruct_pkt default values will be taken from the original packet.
Exaples of JSON payloads and their scapy expression alternatives
Ether(src="de:ad:be:ef:de:ad")/Dot1Q()/Dot1Q(vtype=1)/IP(src="127.0.0.1", chksum="0x312")/TCP(sport=443)[
{ "id": "Ether", "fields": [{"id": "src", "value": "de:ad:be:ef:de:ad"}] },
{ "id": "Dot1Q"},
{ "id": "Dot1Q", "fields": [{"id": "vtype", "value": "1"}] },
{ "id": "IP", "fields": [{"id": "src", "value": "127.0.0.1"}, {"id": "chksum", "value": "0x312"}] },
{ "id": "TCP", "fields": [{"id": "sport", "value": "443"}] }
]Field Engine VM instruction model
For more reference see Field Engine modules spec
{
"field_engine":{
"instructions":[
{
"id":"STLVmFlowVar",
"parameters": {
"name": "Ether_dst",
"init_value": "00:00:00:01:00:00", #can be a string or an integer
"max_value": "0", #can be a string or an integer
"min_value": "0", #can be a string or an integer
"step": "1",
"size": "4",
"op": "inc"
}
},
...
],
"global_parameters":{
"cache_size":"1000"
}
}
}Most values can be passed as strings(including decimal numbers, hex numbers, enums, values), but for binary payload, value object should be used
- int/long/str - they can de specified directly as a value of a field
- {"vtype": "BYTES", "base64": "my_payload_base64"} - binary payload passed as base64
- {"vtype": "EXPRESSION", "expr": "TCPOptions()"} - python expression(normally, should be avoided)
- {"vtype": "UNDEFINED"} - unset field value, and let it be assigned automatically
- {"vtype": "RANDOM"} - assign a random value to a fieldExample of object value usage(to specify binary payload)
Ether()/IP()/TCP()/Raw(load=my_payload)
[
{ "id": "Ether"},
{ "id": "IP"},
{ "id": "TCP"},
{ "id": "Raw", "fields": [
{
"id": "load",
"value": {"vtype": "BYTES", "base64": "my_payload_base64"}
}
]}
]build_pkt, build_pkt_ex and reconstruct pkt take packet model and produce result JSON, with the binary payload and field values and offsets defined
{
"binary": "AAAAAQAAAAAAAgAACABFAAAoAAEAAEAGOs4QAAABMAAAAQAUAFAAAAAAAAAAAFACIABPfQAA", // base64 encoded binary payload
"data": [
{
"id": "Ether", # scapy class
"name": "Ethernet", # human-readable protocol name
"offset": 0, # global offset for all fields
"fields": [
{
"id": "dst", # scapy field id
"hvalue": "00:00:00:01:00:00", # human readable value
"length": 6, # 6 bytes
"offset": 0, # 0 bytes offset from
"value": "00:00:00:01:00:00" # internal value, which for this type is the same as hvalue
},
{
"id": "src",
... # same as for dst
},
{
"hvalue": "IPv4", # human-readable value
"id": "type",
"length": 2,
"offset": 12, #
"value": 2048 # integer value for IPv4(0x800)
}
]
},
{
"id": "IP",
"name": "IP",
"offset": 14,
"fields": [
{
"hvalue": "4",
"id": "version",
"length": 0, # the length is 0, which means it is a bitfield. mask should be used to show location
"offset": 0, # offset from the IP.offset. it needs to be added to all fields of IP
"value": 4
},
{
"hvalue": "5",
"id": "ihl",
"length": 0, # again length is 0. that's other part of the first byte of IP
"offset": 0,
"value": 5
},
{
"hvalue": "0x0",
"id": "tos",
"length": 1,
"offset": 1,
"value": 0
},
{
"hvalue": "40",
"id": "len",
"length": 2,
"offset": 2,
"value": 40
},
{
"hvalue": "1",
"id": "id",
"length": 2,
"offset": 4,
"value": 1
},
{
"hvalue": "", # no flags are specified here. but this field can contain "US" for URG+SYN flags
"id": "flags",
"length": 0,
"offset": 6,
"value": 0
},
{
"hvalue": "0",
"id": "frag",
"length": 0,
"offset": 6,
"value": 0
},
{
"hvalue": "64",
"id": "ttl",
"length": 1,
"offset": 8,
"value": 64
},
{
"hvalue": "tcp", # this field is enum. enum dictionary can be obtained as a medatata for IP fields.
"id": "proto",
"length": 1,
"offset": 9,
"value": 6
},
{
"hvalue": "0x3ace",
"id": "chksum",
"length": 2,
"offset": 10,
"value": 15054
},
{
"hvalue": "[]",
"id": "options",
"length": 2,
"offset": 20,
"value": { # options can not be representted as a human string, so they are passed as an expression
"expr": "[]",
"vtype": "EXPRESSION"
}
}
]
},
{
"id": "TCP",
"name": "TCP",
"offset": 34
"fields": [
{
"hvalue": "20",
"id": "sport",
"length": 2,
"offset": 0,
"value": 20
},
# .. some more TCP fields here
{
"hvalue": "{}",
"id": "options",
"ignored": true,
"length": 2,
"offset": 20,
"value": { # TCPOptions are represented as a python expression with tuple and binary buffers
"expr": "[('MSS', 1460), ('NOP', None), ('NOP', None), ('SAckOK', b'')]",
"vtype": "EXPRESSION"
}
}
]
}
]
}{
"binary": #same as in build_pkt
"data": #same as in build_pkt
"field_engine": {
"error": "",
"instructions": {
"cache": 1000,
"instructions": [
...
#FlowVar definition instruction
{
"name": "Ether_dst",
"max_value": 0,
"min_value": 0,
"init_value": 65536,
"step": 1,
"size": 4,
"type": "flow_var",
"op": "inc"
},
#FlowVar write instruction
{
"name": "Ether_dst",
"is_big_endian": true,
"pkt_offset": 6,
"type": "write_flow_var",
"add_value": 0
}
...
]
}
}
}Scapy server can return metadata object, describing protocols and fields. Most values, including field types are optional in the definition. If field type is missing, it can be treated as a STRING.
"protocols": [
{
"id": "Ether", # scapy class
"name": "Ethernet", # name of the protocol
"fields": [
{
"id": "dst",
"name": "Destination", # GUI will display Destination instead of dst
"type": "STRING",
"regex": "^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$"
},
{
"id": "src",
"name": "Source",
"type": "STRING",
"regex": "^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$"
},
{
"values_dict": {
"ATMMPOA": 34892,
"RAW_FR": 25945,
"DNA_DL": 24577,
"ATMFATE": 34948,
"ATALK": 32923,
"BPQ": 2303,
"X25": 2053,
"PPP_DISC": 34915,
"DEC": 24576,
"n_802_1Q": 33024,
"PPP_SES": 34916,
"TEB": 25944,
"SCA": 24583,
"PPP": 34827,
"FR_ARP": 2056,
"CUST": 24582,
"ARP": 2054,
"DNA_RC": 24578,
"NetBEUI": 33169,
"AARP": 33011,
"DIAG": 24581,
"IPv4": 2048,
"DNA_RT": 24579,
"IPv6": 34525,
"LAT": 24580,
"IPX": 33079,
"LOOP": 36864
},
"id": "type",
"name": "Type"
"type": "ENUM"
}
]
},
{
"id": "TCP",
"name": "TCP",
"fields": [
{
"id": "sport",
"name": "Source port",
"type": "NUMBER",
"min": 0, # optional min value
"max": 65535 # optional max value
},
{
"id": "dport",
"name": "Destination port",
"type": "NUMBER",
"min": 0,
"max": 65535
},
{
"id": "seq",
"name": "Sequence number",
"type": "NUMBER"
},
{
"id": "ack",
"name": "Acknowledgment number",
"type": "NUMBER"
},
{
"id": "dataofs",
"name": "Data offset",
"type": "NUMBER"
},
{
"id": "reserved",
"name": "Reserved",
"type": "NUMBER"
},
{
"id": "flags",
"name": "Flags",
"auto": false,
"type": "BITMASK",
"bits": [ # fields definition for the UI
{"name": "URG", "mask": 32, "values":[{"name":"Not Set", "value": 0}, {"name":"Set", "value": 32}]},
{"name": "ACK", "mask": 16, "values":[{"name":"Not Set", "value": 0}, {"name":"Set", "value": 16}]},
{"name": "PSH", "mask": 8, "values":[{"name":"Not Set", "value": 0}, {"name":"Set", "value": 8}]},
{"name": "RST", "mask": 4, "values":[{"name":"Not Set", "value": 0}, {"name":"Set", "value": 4}]},
{"name": "SYN", "mask": 2, "values":[{"name":"Not Set", "value": 0}, {"name":"Set", "value": 2}]},
{"name": "FIN", "mask": 1, "values":[{"name":"Not Set", "value": 0}, {"name":"Set", "value": 1}]}
]
},
{
"id": "window",
"name": "Window size",
"type": "NUMBER"
},
{
"id": "chksum",
"name": "Checksum",
"auto": true,
"type": "NUMBER"
},
{
"id": "urgptr",
"name": "Urgent pointer",
"type": "NUMBER"
},
{
"id": "options",
"name": "Options",
"type": "EXPRESSION"
}
]
},
{
"id": "IP",
"name": "Internet Protocol Version 4",
"fields": [
{
"id": "version", # only renaming
"name": "Version"
},
{
"id": "ihl",
"name": "IHL",
"type": "NUMBER",
"auto": true # calculate IHL automatically
},
{
"id": "tos",
"name": "TOS",
"type": "NUMBER"
},
{
"id": "len",
"name": "Total Length",
"type": "NUMBER",
"auto": true
},
{
"id": "id",
"name": "Identification",
"type": "NUMBER"
},
{
"id": "flags",
"name": "Flags",
"type": "BITMASK",
"min": 0,
"max": 8,
"bits": [ # bitmask definition for IP.flags
{"name": "Reserved", "mask": 4, "values":[{"name":"Not Set", "value": 0}, {"name":"Set", "value": 4}]},
{"name": "Fragment", "mask": 2, "values":[{"name":"May fragment (0)", "value": 0}, {"name":"Don't fragment (1)", "value": 2}]},
{"name": "More Fragments(MF)", "mask": 1, "values":[{"name":"Not Set", "value": 0}, {"name":"Set", "value": 1}]}
]
},
{
"id": "frag",
"name": "Fragment offset",
"type": "NUMBER"
},
{
"id": "ttl",
"name": "TTL",
"type": "NUMBER",
"min": 1,
"max": 255
},
{
"id": "proto",
"name": "Protocol"
},
{
"id": "chksum",
"name": "Checksum",
"type": "STRING",
"auto": true
},
{
"id": "src",
"name": "Source address",
"type": "STRING",
"regexp": "regexp-to-check-this-field"
},
{
"id": "dst",
"name": "Destination address",
"regexp": "regexp-to-check-this-field"
},
{
"id": "options",
"name": "Options",
"type": "EXPRESSION"
}
]
},
{
"id": "Dot1Q",
"name": "802.1Q",
"fields": [
{
"id": "prio",
"name": "prio"
"type": "NUMBER",
},
{
"id": "id",
"type": "NUMBER",
"name": "id"
},
{
"id": "vlan",
"type": "NUMBER",
"name": "vlan"
},
{
"values_dict": {
"ATMMPOA": 34892,
"RAW_FR": 25945,
"DNA_DL": 24577,
"ATMFATE": 34948,
"ATALK": 32923,
"BPQ": 2303,
"X25": 2053,
"PPP_DISC": 34915,
"DEC": 24576,
"n_802_1Q": 33024,
"PPP_SES": 34916,
"TEB": 25944,
"SCA": 24583,
"PPP": 34827,
"FR_ARP": 2056,
"CUST": 24582,
"ARP": 2054,
"DNA_RC": 24578,
"NetBEUI": 33169,
"AARP": 33011,
"DIAG": 24581,
"IPv4": 2048,
"DNA_RT": 24579,
"IPv6": 34525,
"LAT": 24580,
"IPX": 33079,
"LOOP": 36864
},
"id": "type",
"name": "type",
"type": "ENUM"
}
]
},
{
"id": "Raw",
"name": "Raw",
"fields": [
{
"id": "load",
"name": "Payload",
"type": "BYTES"
}
]
}
]
]The following RPC commands are supported. Please refer to databases section for elaboration for each database.
-
Name - supported_methods
-
Description - returns the list of all supported methods by Scapy Server and their parameters
-
Parameters - the parameter (all) will return ALL supported methods.
other string delivered as parameter will return True/False if the string matches a supported method name -
Result - according to input: all string will return list of supported methods, otherwise will return True/False as mentioned.
The returned dictionary describes for each method it’s number of parameters followed by a list of their names.
Example:
'Request':
{
"jsonrpc": "2.0",
"id": "1",
"method": "supported_methods",
"params": ["all"]
}
'Result':
{'id': '1',
'jsonrpc': '2.0',
'result': { .
.
.
.
'build_pkt': [1, [u'pkt_descriptor']],
'check_update': [2, [u'db_md5', u'field_md5']],
'get_all': [0, []],
'get_tree': [0, []],
'get_version': [0, []],
'supported_methods': [1, [u'method_name']]
}
}-
Name - get_all
-
Description - Returns the supported protocols library (DB) and Field-to-RegEx mapping library, and their MD5
-
Paramters - None
-
Result [object] - JSON format of dictionary. see table below
| Key | Key Type | Value | Value Type |
|---|---|---|---|
db |
string |
supported protocols dictionary |
protocol dictionary |
fields |
string |
Field-to-RegEx dictionary |
Field-to-RegEx dictionary |
db_md5 |
string |
MD5 of DB |
encoded in base64 |
fields_md5 |
string |
MD5 of fields |
encoded in base64 |
Example:
'Request':
{
"jsonrpc": "2.0",
"id": 1,
"method": "get_all",
"params": []
}
'Response':
{
"jsonrpc" : "2.0",
"id" : 1,
"result" : {'db': {'ARP': [('hwtype', 'XShortField', '(1)'),
('ptype', 'XShortEnumField', '(2048)'),
('hwlen', 'ByteField', '(6)'),
('plen', 'ByteField', '(4)'),
('op', 'ShortEnumField', '(1)'),
('hwsrc', 'ARPSourceMACField', '(None)'),
('psrc', 'SourceIPField', '(None)'),
('hwdst', 'MACField', "('00:00:00:00:00:00')"),
('pdst', 'IPField', "('0.0.0.0')")],
.
.
.
'db_md5': 'Z+gRt88y7SC0bDu496/DQg==\n',
'fields': {'ARPSourceMACField': 'empty',
'BCDFloatField': 'empty',
'BitEnumField': 'empty',
.
.
.
}-
Name - check_update
-
Description - checks if both protocol database and fields database are up to date according to md5 comparison
-
Parameters - md5 of database, md5 of fields
-
Result - upon failure: error code -32098 (see RPC server error codes)
followed by a message: "Fields DB is not up to date" or "Protocol DB is not up to date"
upon success: return true as result (see below)
Example:
'Request':
{
"jsonrpc": "2.0",
"id": "1",
"method": "check_update",
"params": ["md5_of_protocol_db", "md5_of_fields"]
}
'Response': #on failure
{
"jsonrpc": "2.0",
"id": "1",
"error": {
"code": -32098,
"message:": "Scapy Server: Fields DB is not up to date"
}
}
'Response': #on success
{
"jsonrpc": "2.0",
"id": "1",
"result": true
}-
Name - get_version
-
Description - Queries the server for version information
-
Paramters - None
-
Result [object] - See table below
| Field | Type | Description |
|---|---|---|
version |
string |
Scapy Server version |
built_by |
string |
who built this version |
Example:
'Request':
{
"jsonrpc": "2.0",
"id": "1",
"method": "get_version",
"params": []
}
'Response':
{
"jsonrpc": "2.0",
"id": "1",
"result": {
"version": "v1.0",
"built_by": "itraviv"
}
}-
Name - build_pkt
-
Description - Builds a new packet from the definition and returns binary data and json structure
-
Return Value - Returns Scapy packet result payload.
-
Paramters - JSON packet definition model.
-
Name - build_pkt_ex
-
Description - Builds a new packet with VM instructions from the definition and returns binary data and json structure
-
Return Value - Returns Scapy packet result with VM instructions payload.
-
Paramters - JSON Packet definition model.
-
Name - reconstruct_pkt
-
Description - Builds a new packet from the binary data and returns binary data and json structure
-
Return Value - Returns Scapy packet result payload.
-
Paramters - base64-encoded packet bytes, optional JSON packet definition model with fields to override.
-
Name - get_definitions
-
Description - Returns definitions for protocols and fields
-
Return Value - array of protocol definitions in a "result.protocols" json. Output model
-
Paramters - array of protocol class names to define or null to fetch metadata for all protocols. ex. ["Ether", "TCP"]
-
Name - get_tree
-
Description - returns a suggested dictionary of protocols ordered in a hierarchy tree.
User can still create non valid hierarchies. (such as Ether()/DNS()/IP()) -
Parameters - none
-
Result [dictionary] - Example for packet layers that can be used to build a packet. Ordered in an hierarchy tree.
Example:
'Request':
{
"id": "1",
"jsonrpc": "2.0",
"method": "get_tree",
"params": []
}
'Response':
{'id': '1',
'jsonrpc': '2.0',
'result': {'ALL': {
'Ether': {'ARP': {},
'IP': { 'TCP': {'RAW': 'payload'},
'UDP': {'RAW': 'payload'}
}
}
}
}
}-
Name - get_templates
-
Description - returns a list of predefined templates.
-
Parameters - none
-
Result [array] - A list of predefined templates.
Example:
[
{
"id": "TCP-SYN",
"meta": {
"name": "TCP-SYN",
"description": ""
},
}
]Notice the existence of the following files:
-
scapy_service.py
-
scapy_zmq_server.py
-
scapy_zmq_client.py
In this section we will see how to bring up the Scapy ZMQ server. There are 2 ways to run this server:
-
Through command line
-
Through Python interpreter
Run the file scapy_zmq_server.py with the argument -s to declare the port that the server will listen to.
Running the file without the "-s" argument will use port 4507 by default.
Notice:
-
The Server’s IP will be the IP address of the local host.
-
The Server will accept requests from any IP address on that port.
user$ python scapy_zmq_server.py -s 5555
***Scapy Server Started***
Listening on port: 5555
Server IP address: 10.0.0.1-
Run the Python Interpreter (Scapy Server currently supports Python2)
-
Import the scapy_zmq_server.py file
-
Create a Scapy_server Object with argument as port number. default argument is port 4507
-
Invoke method activate(). (This method is blocking because the server is listening on the port).
user$ python
>>> from scapy_zmq_server import *
>>> s = Scapy_server() // starts with port 4507
>>> s = Scapy_server(5555) //starts with port 5555
>>> s.activate()
***Scapy Server Started***
Listening on port: 5555
Server IP address: 10.0.0.1There are 2 ways to shut down the server:
-
The server can be shut down using the keyboard interrupt Ctrl+C
-
The server can be shut down remotely with the method "shut_down" with no arguments
//Sending Request: {"params": [], "jsonrpc": "2.0", "method": "shut_down", "id": "1"}
//Will result in this print by the server:
Server: Shut down by remote user