add support to read pyproject.toml from custom node

[jtydhr88]

Inspired by @christian-byrne comments jtydhr88/ComfyUI_frontend_vue_basic#8 and Robin's question, I believe the fundamental solution to this problem is to provide ComfyUI's Core with the capability to process pyproject.toml files in custom nodes, thereby obtaining attributes such as project name.

In fact, we already have corresponding pyproject.toml definitions in comfy-cli https://github.com/Comfy-Org/comfy-cli/blob/main/comfy_cli/registry/types.py. What we need to do is synchronize these definitions in the core and have the capability to process these attributes in custom nodes.

Most of the logic in this PR comes from comfy-cli, while making certain modifications to meet runtime parsing requirements (for example, using the custom node's folder name as the project name when there's no pyproject.toml file).

This PR can also serve as a prerequisite for the following two requirements:

1. As @robinjhuang asked, how to automatically register custom nodes' web folders. After this change is merged, we can add a new attribute to the pyproject definition, such as "web" or "web_dir", and detect and automatically process it when starting the ComfyUI main application.
2. As Blake's issue proposal here Add abstraction layer between pinia stores and extensions ComfyUI_frontend#4020 (adding namespace for custom nodes in the frontend). Since we already have corresponding definitions in pyproject, after this change is merged, we can have the backend return relevant information to the frontend and use it in the frontend.

Based on these considerations, for better feature support, I copied the entire types over instead of just copying the project name that I currently need.

However, it should be noted that this solution currently has a requirement: when we update the definitions in types in comfy-cli, we need to immediately synchronize to the core for adaptation.

At the same time, because we need to parse toml files, I need to introduce new dependency packages - tomlkit (same as comfy-cli) in the core's requirements.

As a test, I used my vue-basic jtydhr88/ComfyUI_frontend_vue_basic#12 as a demonstration.

[guill]

Now that we have pydantic as a dependency in ComfyUI anyway, it might be worth using that for loading the toml data rather than having the custom parsing logic that we need to keep in sync between the main repo and comfy-cli. We'd still have to keep the structure definitions in sync to some degree, but that would at least decrease the chance of divergence.

(I'll also add that reading the pyproject.toml is going to be necessary for some of the other node work that's in progress, so we do want to get this in one way or another.)

[comfyanonymous]

Yeah, this doesn't need a new dependency.

[jtydhr88]

@guill I switched pydantic and pydantic_settings to handle toml file.
Please note, for any alias, such as
homepage: str = Field(default="", alias="Homepage")
this is because we already use these name in cli, see https://github.com/Comfy-Org/comfy-cli/blob/2e36f33dd39ef43b5acf7d1fc5acc5e01be92360/comfy_cli/registry/config_parser.py#L188
still keep definition same as types in comfy-cli

[guill]

Some more feedback. One potential bug, the rest is fairly minor.

[jtydhr88]

@guill just fixed your feedback, please double check, thanks

[guill]

Looks good to me!

[christian-byrne]

Follow-up task might be to add something to HTTP API to get list of all installed node ids. WDYT @jtydhr88?

[jtydhr88]

@christian-byrne yes! I think so!
Another possible follow up is to add “web” in pyproject.toml, then automatically register it while start server

[christian-byrne]

Another possible follow up is to add “web” in pyproject.toml, then automatically register it while start server

Corresponding 1-to-1 with WEB_DIRECTORY python value?

[jtydhr88]

@comfyanonymous yes
And from my knowledge, one project should have only one web folder, I am not sure we should support one project has multi-folders for web

[webfiltered]

I am not sure we should support one project has multi-folders for web

Strongly agree! Sounds like it would just be confusing at best.

[mcmonkey4eva]

It looks like pydantic_settings is a separate pip package ( https://pypi.org/project/pydantic-settings/ ) from pydantic itself?

ie it would actually require a new dependency

There are error reports on discord from when this was momentarily triggered by nodes.py of pydantic_settings failing to import

[jtydhr88]

@mcmonkey4eva yes, see this #8478, I will fix soom

[jtydhr88]

@mcmonkey4eva sent a new PR to fix it #8505

[ltdrdata]

We should not assume that all custom nodes have a valid pyproject.toml file.

#8514 (comment)

[jtydhr88]

one more fix #8526