Skip to content

Latest commit

 

History

History
143 lines (129 loc) · 4.46 KB

ANNOTATION-COOKBOOK.md

File metadata and controls

143 lines (129 loc) · 4.46 KB
Raw

Annotation Cookbook

Writing an annotation in ZTag is easy, and consists of writing a short Python class. You define what protocols the annotation should operate on and implement a single process function, and ZTag handles the rest.

Terminology

ZTag has two main functions: transforms and annotations. Annotations run on the output of a transformation, and annotate the transformed data with any combination of device information, software and hardware details, and any other relevant information.

For example, one annotation might annotate any hosts that support export RSA ciphers in TLS with the tag freakattack-vulnerable, whereas another annotation might only annotate ASUS home routers with their make and model.

There are three types of annotated data that can be generated by an annotation: local metadata, global metadata, and tags.

Local Metadata Key-value pairs added to host data at the given (port, protocol, subprotocol)tuple. Follows a predefined schema. local_metadata.manufacturer = Manufacturer.MICROSOFT
Global Metadata Key-value pairs to host data at the top level. This will be merged across all (port, protocol, subprotocol) tuples. Follows a predefined schema. global_metadata.device_type = Type.SOHO_ROUTER
Tags Set of strings added to host data at the top level. Does not follow a specific schema, but try not to needlessly create new values. tags.add("heartbleed-vulnerable")

Creating an Annotation

In the root of the repository, run ./new_tag. This will copy example/annotation.tmpl to ztag/annotations/$TAG_NAME.py and populate it with values based on your answers to a few questions.

To finish your tag, implement the method def process(self, obj, meta) function. The argument obj is a Python dictionary that matches the JSON from Censys for the specified (port, protocol, subprotocol) tuple. The meta argument is a Python object containing the annotated data, and matches the schema described below. Populate meta based on the values in obj.

Your implementation of process should modify and return meta, but should not modify obj. Only set the properties on meta that your annotation can determine. Optionally, your function can also return None to signify it cannot annotate obj. This is functionally identical to returning meta unmodified.

Schema

Layout

metadata
  .local_metadata
    .manufacturer
    .product
    .version
    .revision
  .global_metadata
    .manufacturer
    .product
    .version
    .revision
    .device_type
    .os
    .os_version
    .os_revision
  .tags

Types

metadata.local_metadata.manufacturer String Constant Manufacturer.MICROSOFT
metadata.local_metadata.product String "IIS"
metadata.local_metadata.version String "2012"
metadata.local_metadata.revision String "R2"
metadata.global_metadata.manufacturer String Constant Manufacturer.LINKSYS
metadata.global_metadata.product String "WRT54G-L"
metadata.global_metadata.version String "3.1.0"
metadata.global_metadata.revision String "r2236"
metadata.global_metadata.device_type String Constant Type.SOHO_ROUTER
metadata.global_metadata.os String Constant OperatingSystem.WINDOWS
metadata.global_metadata.os_version String "XP"
metadata.global_metadata.os_revision String "SP2"
metadata.tags Set of Strings "heartbleed-vulnerable"