JSON custom specs

Generate a json file.

Code snippet to get started

import json
from openai import OpenAI

# set client
client = OpenAI(
  base_url="https://api.recursal.com/v1",
  api_key="rx_xxx",
)

# create template
tocreate = {
  "Colours": {
    "cat": "$string",
    "mat": "$string",
    "house": "$string"
  }
}

#perform completion
response = client.completions.create(
  model='EagleX-V2',
  prompt="The blue cat sat on the red mat in the yellow house. ",
  max_tokens=500,
  temperature=0.0,
  extra_body={
    "response_format":{
        "type": json.dumps(tocreate)
    }
  },
  top_p=0.5
  stream=True
)

generated = ""
for message in response:

  # get output from message
  output = message.choices[0].text
  
  # append output to generated
  generated += output if output else ""
  
  # print output to console
  print(output, end="", flush=True)

object = json.loads(generated)

Example

Level 1:

Primitive types.

These values are to be directly replaced with content generated by the LLM

“$boolean” ”$float” ”$int” ”$string”

Any values other than these will be passed through without being replaced.

INPUT

{
"on-or-off": "$boolean", // (true)
"cost": "$float",
"amount": "$int",
"description": "$string",
"constbool": true,
"constnumber": 5.8,
"constinteger": 5,
"conststring": "this is a string"
}

OUTPUT

{
 "on-or-off": true,
 "cost": 123.456,
 "amount": 123,
 "description": "Hello World",
 "constbool": true,
 "constnumber": 5.8,
 "constinteger": 5,
 "conststring": "this is a string"
}

Level 2

This extends to all naive structures, such as sub objects and arrays.

INPUT

{
"this is a dictionary": {
		"this is a key": "$string",
		"this is another key": "$float"
		},
		
"this is a list": [
		"$string",
		"$float"
		]
}

OUTPUT

{
  "this is a dictionary":{
    "this is a key":"value",
    "this is another key":123.456
  },
  "this is a list":[
    "apple",
    123.456
  ]
}

Level 3

Shortcuts

Lets start with an annoying to write schema:

Schema:

{
  "names": [
    {
      "gender": "$string",
      "name": "$string"
    },
    {
      "gender": "$string",
      "name": "$string"
    },
    {
      "gender": "$string",
      "name": "$string"
    },
    {
      "gender": "$string",
      "name": "$string"
    },
    {
      "gender": "$string",
      "name": "$string"
    }
  ],
}

This is annoying and repetitive, and if you need like, 15 of these, or the object is expected to be larger, it would become annoying to debug an write.

Here is how you can use the ‘$arrayLength’ and the ‘$schema’ control keys to accomplish the exact same thing

Simplified Schema

{
  "names": {
	  "$arrayLength": 5,
    "$schema":{
      "gender": "$string",
      "name": "$string"
    }
  },
}

These objects will have the exact same output.

To further simplify, another shorthand is that you can infer the control structure by including the control keys in the object:

Note: this only works when the “$schema” value is a dictionary

Simplified Schema v2:

{
  "names":{
      "$arrayLength": 5,
      "gender": "$string",
      "name": "$string"
   }
}

Note: These last three snippets, have the exact same output.

Level 4

Async

If for example, you are extracting details from some text,

Prompt: the blue cat sat on the red mat in the pink house

schema:

{
	"colors":
		{
			"cat": "$string",
			"mat": "$string",
			"house": "$string"
		}
}

The keys are independent on each other: it would not matter in what order you generate them.

to take advantage of the throughput and prompt caching abilities of our model, you can use the “$async” control key to tell the model to generate these simultaneously, instead of in serial.

Async schema:

{
	"colors":
		{
			"$async": true,
			"cat": "$string",
			"mat": "$string",
			"house": "$string"
		}
}

This also works for arrays, here is the previous examples, all giving the same output:

{
  "names": [
	  "$async",
    {
      "gender": "$string",
      "name": "$string"
    },
    {
      "gender": "$string",
      "name": "$string"
    },
    {
      "gender": "$string",
      "name": "$string"
    },
    {
      "gender": "$string",
      "name": "$string"
    },
    {
      "gender": "$string",
      "name": "$string"
    }
  ],
}

{
  "names": {
	  "$arrayLength": 5,
	  "$async": true,
    "$schema":{
      "gender": "$string",
      "name": "$string"
    }
  },
}

{
  "names":{
      "$arrayLength": 5,
      "$async": true,
      "gender": "$string",
      "name": "$string"
   }
}

IMPORTANT NOTE, if you are doing the last one, and you want both the repetitions, and the keys, to be async, you will need to do the following, as the arrayLength shortcut takes precedence over the async:

Repetition and async:

{
  "names": {
	  "$arrayLength": 5,
	  "$async": true,
    "$schema":{
	    "$async": true,
      "gender": "$string",
      "name": "$string"
    }
  },
}