API Documentation

[kauffj]

This is the epic issue to collect all changes required to have API documentation work inside of lbry.tech.

After some discussion, there is now some uncertainty as to whether OpenAPI/Swagger is the right solution.

Required properties:

• Web-based display of all endpoints/methods, including parameters, description, and example response signature. Methods should be directly linkable.
• Modern, clean, friendly, usable UI (Stripe is some of the best API documentation I've ever seen / used)
• Method documentation in-code. The documentation tooling should be able to take this documentation and generate the interactive page(s).
• Build-integrable tooling (not sure how this one wouldn't be possible though :)

Ideal properties:

• Identical or similar API documentation standard across repositories.
• Method documentation located in-line with method entries, rather than a separate YML/markup file.
• Handles both CLI and JSON-RPC ways of method calling
• Auto code-gen, live testing, and any other benefits from Swagger.

Other considerations:

• API versioning?

[eukreign]

Consider https://github.com/lord/slate, it produces similar looking docs to stripe. Slate appears to be a pretty popular solution: https://github.com/lord/slate/wiki/Slate-in-the-Wild

[eukreign]

{
        'name': 'Actual Method or Command Name,
        'description': 'Long description of what the method does.',
        'arguments': [
              {
                'name': 'name of the argument, e.g. key in the JSON request to daemon',
                'type': 'type of value, e.g. str, int',
                'description': 'Long explanation of the argument.',
                'is_required': 'True/False if the argument is required.',
                'example': 'An example value, will also be in code examples.'
            }
        ],
        'examples': {
            'python': { ... TBD ... },
            'curl': { ... TBD ... }
        },
        'returns': 'Annotated data structure returned.'
}

[lyoshenka]

suggesting a more complete format (example below):

Structure

{
  "name": "command name",
  "short_desc": "One sentence about what the thing does",
  "long_desc": "Longer, multi-paragraph description if you want",
  "args": [
    {
      "name": "name of arg",
      "type": "type (str, dict, int)",
      "description": "english-language description",
      "is_required": true/false,
      "example": "an example value",
      "fields": [ // for dicts. leave empty for other types
        {
          "name": "same as above",
          "type": "...",
          "description": "...",
          "is_required": false,
          "example": "...",
          "fields": [] // nested dicts. same structure as top-level "fields"
        },
        ...
      ]
    }
  ],
  "examples": [
    {
      "language": "name of language",
      "description": "not sure we need this, but might be nice as help text at the top or bottom?",
      "example": "code goes here"
    },
  ],
  "return": {
    "type": "type (str, dict, etc)",
    "example": "sample return value. leave empty for dicts, as you can construct an example from the 'fields' data for a dict ",
    "fields": [ // for dicts. leave empty (or just leave it out) for other types
      {
          "name": "name of field",
          "type": "the type",
          "description": "what this is",
          "example": "sample return value",
          "fields": [] // nested dicts. same structure as top-level "fields"
      },
      ...
    ]
  }
}

Example

{
  "name": "publish",
  "short_desc": "Make a new name claim and publish associated data to lbrynet, update over existing claim if user already has a claim for name.",
  "long_desc": "Fields required in the final Metadata are:\n            'title'\n            'description'\n            'author'\n            'language'\n            'license'\n            'nsfw'\n\n        Metadata can be set by either using the metadata argument or by setting individual arguments\n        fee, title, description, author, language, license, license_url, thumbnail, preview, nsfw,\n        or sources. Individual arguments will overwrite the fields specified in metadata argument.",
  "args": [
    {
      "name": "name",
      "type": "str",
      "description": "name fo the content",
      "is_required": true,
      "example": "lbry-rocks"
    },
    {
      "name": "bid",
      "type": "decimal",
      "description": "amount to back the claim",
      "is_required": true,
      "example": 123.45
    },
    {
      "name": "file_path",
      "type": "str",
      "description": "path to file to be published",
      "is_required": true,
      "example": "/home/lbry/lbryrocks.mp4"
    },
    {
      "name": "title",
      "type": "str",
      "description": "title of the publication",
      "is_required": false,
      "example": "\\m/ LBRY rocks \\m/"
    },
    {
      "name": "description",
      "type": "str",
      "description": "description of the publication",
      "is_required": false,
      "example": "So lit right now"
    },
    {
      "name": "nsfw",
      "type": "bool",
      "description": "is the content safe for work",
      "is_required": false,
      "example": true
    },
    {
      "name": "metadata",
      "type": "dict",
      "description": "other metadata associated with the claim",
      "is_required": false,
      "example": "So lit right now",
      "fields": [ // for dicts. leave empty for other types
        {
          "name": "license",
          "type": "str",
          "description": "license for the content",
          "is_required": false,
          "example": "CC0"
        },
        {
          "name": "thumbnail",
          "type": "str",
          "description": "preview thumbnail",
          "is_required": false,
          "example": "https://spee.ch/lbryrocks.jpg"
        }
      ]
    }
  ],
  "examples": [
    {
      "language": "python",
      "description": "not sure we need this, but might be nice as help text at the top or bottom?",
      "example": "code goes here"
    },
    {
      "language": "curl",
      "description": "not sure we need this, but might be nice as help text at the top or bottom?",
      "example": "actual example goes here"
    }
  ],
  "return": {
    "type": "dict",
    "fields": [ // for dicts. leave empty (or just leave it out) for other types
      {
          "name": "tx",
          "type": "str",
          "description": "hex encoded transaction",
          "example": "0100000008fa4e01e57294cd72e...d4e1c557033b26256b0c1a96286488ac00000000",
          "fields": [] // nested dicts. same structure as top-level "fields"
      },
      {
          "name": "txid",
          "type": "str",
          "description": "transaction id of resulting claim",
          "example": "e3ca2057767a7fbae36ea51af977ae86c67833c0634cbd0975ac536cd77f43c7"
      },
      {
          "name": "nout",
          "type": "int",
          "description": "nout of the resulting claim",
          "example": 1
      },
      {
          "name": "fee",
          "type": "decimal",
          "description": "fee paid for the claim transaction",
          "example": 0.001
      },
      {
          "name": "claim_id",
          "type": "str",
          "description": "claim ID of the resulting claim",
          "example": "304402202d0aaf5ad2f67a331ae453e2cc3ff"
      }
    ]
  }
}

[rynomad]

I know I'm an outsider but I've begun taking a crack at a JS api binding so this convo is relevant for me.

Might want to take a look at jrgen; there's a decent amount of prior work there (.md, .html, gitbook support, plus code-gen), the schema is not dissimilar from what you've got above, and it doesn't barf on adding extra properties (examples). I just did a quick sanity check porting the example given by @lyoshenka and ran into no trouble. Here's what it looks like:

{
  "$schema":
    "https://rawgit.com/mzernetsch/jrgen/master/jrgen-spec.schema.json",
  "jrgen": "1.1",
  "jsonrpc": "2.0",

  "info": {
    "title": "ExampleAPI",
    "description": [
      "An example api which handles various rpc requests.",
      "This api follows the json-rpc 2.0 specification. More information available at http://www.jsonrpc.org/specification."
    ],
    "version": "1.0"
  },

  "definitions": {
    // this section can be used to define reusable schemas internally; claims, transactions...
  },

  "methods": {
    "publish": {
      "summary": "Make a new name claim and publish associated data to lbrynet, update over existing claim if user already has a claim for name.",
      "description":
          "Fields required in the final Metadata are:\n            'title'\n            'description'\n            'author'\n            'language'\n            'license'\n            'nsfw'\n\n        Metadata can be set by either using the metadata argument or by setting individual arguments\n        fee, title, description, author, language, license, license_url, thumbnail, preview, nsfw,\n        or sources. Individual arguments will overwrite the fields specified in metadata argument.",
  
      "tags": ["content", "fee"],

      "params": {
        "type": "object",
        "properties": {
          "name": {
            "description": "name for the content",
            "default": "admin",
            "type": "string",
            "minLength": 1,
            "example": "lbry-rocks"
          },
          "bid": {
            "description": "Password of the user to create a session for.",
            "default": "123456",
            "type": "number",
            "minimum": 0.0001
          },
          "file_path" : {
            "type": "string",
            "description": "path to file to be published",
            "example": "/home/lbry/lbryrocks.mp4"
          },
          "title": {
            "type": "string",
            "description": "title of the publication",
            "example": "\\m/ LBRY rocks \\m/"
          },
          "description": {
            "type": "string",
            "description": "description of the publication",
            "example": "So lit right now"
          },
          "nsfw":{
            "type": "boolean",
            "description": "is the content safe for work",
            "example": true
          },
          "metadata": {
            "type" : "object",
            "description": "other metadata associated with the claim",
            "properties": {
              "license" : {
                "type": "string",
                "description": "license for the content",
                "example": "CC0"

              },
              "thumbnail" : {
                "type": "string",
                "description": "preview thumbnail",
                "example": "https://spee.ch/lbryrocks.jpg"        
              }
            }
          }
        },

        "required": ["name", "bid", "file_path"]
      },

      "examples": [
        {
          "language": "python",
          "description": "not sure we need this, but might be nice as help text at the top or bottom?",
          "example": "code goes here"
        },
        {
          "language": "curl",
          "description": "not sure we need this, but might be nice as help text at the top or bottom?",
          "example": "actual example goes here"
        }
      ],

      "result": {
        "type" : "object",
        "properties": {
          "tx" : {
            "type" : "string",
            "description": "hex encoded transaction",
            "example": "0100000008fa4e01e57294cd72e...d4e1c557033b26256b0c1a96286488ac00000000"  
          },
          "txid": {
            "type": "string",
            "description": "transaction id of resulting claim",
            "example": "e3ca2057767a7fbae36ea51af977ae86c67833c0634cbd0975ac536cd77f43c7" 
          },
          "nout": {
            "type": "number",
            "description": "nout of the resulting claim",
            "example": 1
          },
          "fee" : {
            "type": "number",
            "description": "fee paid for the claim transaction",
            "example": 0.001
          },
          "claim_id": {
            "type": "string",
            "description": "claim ID of the resulting claim",
            "example": "304402202d0aaf5ad2f67a331ae453e2cc3ff"
          }
        }
      },

      "errors": [
        {
          "description": "missing required arguments",
          "code": 1,
          "message": "InvalidArgumants"
        }
      ]
    }
  }
}

I'm probably going to use it for code-gen on my side to get started with a Javascript binding; which means I'll end up writing a schema file as part of work on that client. Happy to be advised as to the most useful thing to do with it :)

[tzarebczan]

@rynomad thanks for the suggestion! @lyoshenka / @eukreign any thoughts on the above?

[eukreign]

I don't have additional thoughts. @NetOperatorWibby is most affected by this (he has to convert the JSON into a new HTML document and/or use jrgen as suggested by @rynomad) so it's up to him.

[NetOpWibby]

You’re creating the parser @eukreign, all I’m doing is parsing a JSON file. So, it’s up to you.

[eukreign]

@NetOperatorWibby Okay, this is easy: are you planning to use jrgen?

If no, then I'll stick to a combination of my original JSON and what @lyoshenka suggested.
If yes, then I'll make it compatible with jrgen.

[NetOpWibby]

@eukreign Nope, all I should be doing is parsing a file. The generator should take care of everything else.