[Feature request]: Programmatic use and API

[ebceu4]

I would like to request a feature that allows for the saving and loading of pipelines as JSON. This will enable users to create complex and advanced pipelines using the graph/nodes/flowchart based interface and then leverage the visually built pipelines programmatically or via API through a runner

[comfyanonymous]

See: https://github.com/comfyanonymous/ComfyUI/blob/master/script_examples/basic_api_example.py

[IcelandicIcecream]

Is there a guide on how the API works

[lunaticer]

I also want to use workflow struct my special API,because I can't process batch images now.It will make the workflow more powerful.

[icelandno1]

That's a good question, and I'd like to implement this functionality by calling the ComfyUI workflow directly from an external application, which eliminates the hassle of using the stable diffusion api.

[Razorbob]

I would also love to have an API. If possible also an OpenApi/Swagger definition would be awesome, so you could generate a client in different languages.

[misza222]

+1

[comfyanonymous]

if you update to the latest you can "enable dev mode options" in the settings (gear beside the "Queue Size: ") this will enable a button on the UI to save workflows in api format.

You can then use these api jsons in these script examples: https://github.com/comfyanonymous/ComfyUI/tree/master/script_examples

[tomermesser]

Hi, Thanks for adding the "API Format" option it is super helpful !

[fidecastro]

I took a look at how the websocket service works, and created a simple class to use the already existing API. It facilitated my workflow a lot and maybe some of you can find it useful.

https://github.com/fidecastro/ComfyUI-serverless/

[p0mad]

I'm Stucking in here! #1083 (comment)

Can you give me a tought?

[kiriri]

/I could not find an api documentation, so here's what I gleamed from the source code and everyone's comments :

GET

/history
Last queued renders, including all nodes used in the prompt, and all output images.

/history/{UID}
A specific history entry by uid. When calling prompt, the result will contain this uid.

/view?filename={FILENAME}
Access the images from history via this.

/extensions
paths to the javascript files for each extension.

/embeddings
Presumably a list of all embeddings. These aren't available as inputs in any Node, unlike models (see OTHERS).

/object_info
List of all types of nodes that are available. (Not the actual node instances, just the node types.)
This exposes a lot of important enumerable values in the form of input types.

/queue
What's happening or queued at the moment. Surprisingly comprehensive.

/system_stats
os, python version, torch devices (like gpus) and vram used.

POST

/prompt
https://github.com/comfyanonymous/ComfyUI/blob/master/script_examples/basic_api_example.py

The json data payload must be stored under the name "prompt". In JS, it would look like so :

const res = await axios.post('http://127.0.0.1:8188/prompt',{
	prompt:json_data_object	
});

You can get an example of the json_data_object by enabling Dev Mode in the ComfyUI settings, and then clicking the newly added export button.

/interrupt
Interrupts the execution of the running prompt and starts the next one in the queue.

/queue
/history
In POST, these routes should accept the payloads clear:true and delete:number[] .

/upload/image
/upload/mask
This uploads an image and returns a short path under which the image can be referred to in comfyui.
Takes payload 'image' in body.

SYNTAX

The following typescript definitions are incomplete and may be wrong entirely.
You can find an update version of these types here .

// This is what apis like "prompt" expect.
export type JSON_ComfyGraph = {
    [uid: string|number]: JSON_ComfyNode
}

// A node instance.
export type JSON_ComfyNode = {
    // The type of the node as found in 'GET /object_info' by key
    class_type: string;
    // All incoming connections
    inputs: {
        [input_name: string]: JSON_ValueRef;
    }
}

// Any value that can is passed into a ComfyNode
// This can be either an atomic value, like a string or a number, or a reference to another node's output.
export type JSON_ValueRef = [target_node_uid: string, output_socket_index: number] | string | number | boolean


// All possible input/output types. This type is incomplete.
// string[] represents an enum dropdown.
export type JSON_ComfyPrimitive = "MODEL" | "CLIP" | "STYLE_MODEL" | "CLIP_VISION_OUTPUT" | "CONTROL_NET" | "UPSCALE_MODEL" | "CLIP_VISION" | "VAE" | "LATENT" | "INT" | "FLOAT" | "IMAGE" | "TEXT" | "CONDITIONING" | "EXTRA_PNGINFO" | "MASK" | string[];

// Some types accept options like restrictions or special ui operations. So they are wrapped in an array of type + options
export type JSON_ComfyPrimitiveDeclaration = 
// An empty array means the name of the output/input is the same as the type. Case insensitive. Eg an output named 'latent' defaults to type "LATENT".
[] 
| [JSON_ComfyPrimitive]
| [
    "INT" | "FLOAT",
    {
        default?: number,
        min?: number,
        max?: number,
        step?: number,
        round?: number
    }
]
| [
    "TEXT"|string[],
    {
        multiline?:boolean,
        default?:string
    }
]
| [
    "IMAGE"|string[],
    {
        image_upload?:boolean,
        default?:string
    }
]

// This is what apis like "object_info" return.
export type JSON_ComfyNodeTypes = {
    [key: string]: {
        // This is what creates the submenus in the context menu.
        category: string,
        // Verbal description of the node for use in the ui.
        description: string,
        // The name which should be displayed in the editor.
        display_name: string,
        input: {
            // List of all input types
            required: {
                [name: string]: JSON_ComfyPrimitiveDeclaration
            },
            hidden: {
                [name: string]: JSON_ComfyPrimitiveDeclaration
            }
        },
        // Unique name
        name: string,
        // Unique names of all the outputs
        output: string[],
        // Is the output a list?
        output_is_list: boolean[],
        // Labels of all the outputs
        output_name: string[],
        // Is this node an output? Useful for evaluating if a graph is valid.
        output_node: boolean;
    }
}

// This is what apis like "history" return.
export type JSON_History = {
    [uid: string]: JSON_HistoryEntry;
}

export type JSON_HistoryEntry = {
    prompt: [id_maybe: number, uid:string, list_of_used_nodes:JSON_ComfyGraph, no_idea:any, no_idea2:[string]];
    outputs: {
        images: JSON_ImageReference[];
    }[];
}

export type JSON_ImageReference = {
    filename: string;
    subfolder: string;
    type: "output";
}

OTHER

You can retrieve the list of all available models from the CheckpointLoader object.
GET /object_info
Then CheckpointLoader->input->required->ckpt_name->0 .

[darkcurrent]

I took a look at how the websocket service works, and created a simple class to use the already existing API. It facilitated my workflow a lot and maybe some of you can find it useful.

https://github.com/fidecastro/ComfyUI-serverless/

It's a great work

[zephirusgit]

Could anyone give me any functional example of the use of the API?
What is complete and that has an example embedded as those of the Save in Api Format?
I can't run the Basic Example:/

[zephirusgit]

si actualizas a lo último puedes "solar las opciones de modo dev" en la configuración (envejecer junto al "Queue Size: ") esto permitirá un botón en la interfaz de usuario para guardar flujos de trabajo en formato api.

A continuación puede utilizar estos api jsons en estos ejemplos de script: https://github.com/comfyanonymous/ComfyU/tree/master/script-examples

Could anyone give me any functional example of the use of the API?
What is complete and that has an example embedded as those of the Save in Api Format?
I can't run the Basic Example:/