Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

自动编译Docker镜像并上传到ghcr #662

Merged
merged 10 commits into from
May 3, 2023
Merged

自动编译Docker镜像并上传到ghcr #662

merged 10 commits into from
May 3, 2023

Conversation

reonokiy
Copy link
Contributor

主要完成了两个方面的工作:

由于Docker通常使用环境变量或者env_file作为配置项目的方法,在使用Github Actions自动编译镜像之后还修改了config.py来从环境变量读取配置。不过读取环境变量作为配置的部分写的比较粗糙,可能需要修改来匹配项目。

现在使用Docker Compose部署服务的一个示例:

# docker-compose.yml
version: '3'
services:
  gpt_academic:
    image: ghcr.io/sperjar/gpt_academic:master
    container_name: gpt_academic
    environment:
      GPT_ACADEMIC_API_KEY: your-api-key-here
      GPT_ACADEMIC_WEB_PORT: 10054
      # more config
    ports:
      - 10054:10054

@binary-husky
Copy link
Owner

binary-husky commented Apr 30, 2023

你好,可以把config.py中的变更清除,然后放到get_conf中吗,我们希望config.py保持简单、易拓展的状态

https://github.com/binary-husky/gpt_academic/blob/9d3b01af754a0a950bde4c58ecb91a044e32b889/toolbox.py#LL520C1-L526C60

比如

    try:
        r = getattr(importlib.import_module('config_private'), arg)
    except:
        r = getattr(importlib.import_module('config'), arg)

修改成

try:
    r = os.environ[arg] # 获取环境变量(arg)
except KeyError:
    try:
        r = getattr(importlib.import_module('config_private'), arg)
    except:
        r = getattr(importlib.import_module('config'), arg)

@reonokiy
Copy link
Contributor Author

我需要直接在read_single_conf_with_lru_cache函数中对于环境变量进行格式的处理吗?这样感觉很多非str的类型都需要重新做一次判断,感觉会导致函数比较臃肿。
或者是新建一个config_env.py将环境变量转换成config.py的格式再在这里导入

@binary-husky
Copy link
Owner

binary-husky commented Apr 30, 2023

我需要直接在read_single_conf_with_lru_cache函数中对于环境变量进行格式的处理吗?这样感觉很多非str的类型都需要重新做一次判断,感觉会导致函数比较臃肿。 或者是新建一个config_env.py将环境变量转换成config.py的格式再在这里导入

字符串可以根据默认值的数据类型,进行转换,下面这个函数是我在另一个项目中用的代码,这里不需要的数据类型可以直接raise error也行

def auto_convertion():
    replace_item = # 环境变量值(字符串)
    original_item = # config.py中的默认值
    if isinstance(original_item, float):
        replace_item = float(replace_item)
    elif isinstance(original_item, bool):
        if replace_item == 'True':
            replace_item = True
        elif replace_item == 'False':
            replace_item = False
        elif isinstance(replace_item, bool):
            replace_item = replace_item
        else:
            assert False, ('enter True or False, but have:', replace_item)
    elif isinstance(original_item, int):
        assert int(replace_item) == float(replace_item), ("warning, this var **%s** has an int default, but given a float override!")
        replace_item = int(replace_item)
    elif isinstance(original_item, str):
        replace_item = replace_item
    elif isinstance(original_item, list):
        assert isinstance(replace_item, list)
    elif isinstance(original_item, dict):
        assert isinstance(replace_item, dict)
    else:
        assert False, ('not support this type')
    return replace_item

@reonokiy
Copy link
Contributor Author

reonokiy commented Apr 30, 2023

应该修改完毕了,主要的逻辑在这里:e5e3e0a#diff-71e5cf52ccb5b80caa00b86ae70d9e941a3e5cdb8279d7cb5c197c636cd79ee4R521
代码中给环境变量加了一个前缀是为了防止和其他已有项目之间造成冲突。

简单跑了一下这个测试是正常的,但是没有具体测试每个值的情况。

# docker-compose.yml
version: '3'
services:
  gpt_academic:
    image: ghcr.io/sperjar/gpt_academic:master
    container_name: gpt_academic
    environment:
      GPT_ACADEMIC_API_KEY: your-api-key-here
      GPT_ACADEMIC_WEB_PORT: 10054
      # more config
    ports:
      - 10054:10054

后续应该还要补充相关的README和文档,环境变量处理的部分和直接配置config.py有些不一致。

@binary-husky
Copy link
Owner

应该修改完毕了,主要的逻辑在这里:e5e3e0a#diff-71e5cf52ccb5b80caa00b86ae70d9e941a3e5cdb8279d7cb5c197c636cd79ee4R521 代码中给环境变量加了一个前缀是为了防止和其他已有项目之间造成冲突。

简单跑了一下这个测试是正常的,但是没有具体测试每个值的情况。

# docker-compose.yml
version: '3'
services:
  gpt_academic:
    image: ghcr.io/sperjar/gpt_academic:master
    container_name: gpt_academic
    environment:
      GPT_ACADEMIC_API_KEY: your-api-key-here
      GPT_ACADEMIC_WEB_PORT: 10054
      # more config
    ports:
      - 10054:10054

后续应该还要补充相关的README和文档,环境变量处理的部分和直接配置config.py有些不一致。

感谢,等我稍稍调整一下注释

toolbox.py Outdated
# 对于API_URL_REDIRECT的环境变量配置,我们允许用户使用;分隔转发地址
# 格式为:url1:redirect1;url2:redirect2
for item in os.environ[env_arg].split(";"):
k, v = item.split(":")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

使用:分隔原url和重定向url会和http://中的:冲突

@binary-husky
Copy link
Owner

binary-husky commented May 2, 2023

这样写更简单明了一点,我在windows cmd中进行了测试没问题,请您试一下docker compose是否一样work

    环境变量可以是 `GPT_ACADEMIC_CONFIG`(优先),也可以直接是`CONFIG`
    例如在windows cmd中,既可以写:
        set USE_PROXY=True
        set API_KEY=sk-j7caBpkRoxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        set proxies={"http":"http://127.0.0.1:10085", "https":"http://127.0.0.1:10085",}
        set AVAIL_LLM_MODELS=["gpt-3.5-turbo", "chatglm"]
        set AUTHENTICATION=[("username", "password"), ("username2", "password2")]
    也可以写:
        set GPT_ACADEMIC_USE_PROXY=True
        set GPT_ACADEMIC_API_KEY=sk-j7caBpkRoxxxxxxxxxxxxxxxxxxxxxxxxxxxx
        set GPT_ACADEMIC_proxies={"http":"http://127.0.0.1:10085", "https":"http://127.0.0.1:10085",}
        set GPT_ACADEMIC_AVAIL_LLM_MODELS=["gpt-3.5-turbo", "chatglm"]
        set GPT_ACADEMIC_AUTHENTICATION=[("username", "password"), ("username2", "password2")]

@binary-husky
Copy link
Owner

binary-husky commented May 2, 2023

以前从来没用过docker-compose,是这样写的吗?(不好看, 但是似乎管用)

version: '3'
services:
  gpt_academic:
    image: fuqingxu/gpt_academic:no-local-llms
    container_name: gpt_academic
    environment:
      # see `config.py` all configuration options !
      API_KEY:                  '     your-api-key-here                                                      '
      WEB_PORT:                 '     10054                                                                  '
      proxies:                  '     {"http":"http://127.0.0.1:10085", "https":"http://127.0.0.1:10085",}   '
      AVAIL_LLM_MODELS:         '     ["gpt-3.5-turbo", "chatglm"]                                           '
      AUTHENTICATION:           '     [("username", "password"), ("username2", "password2")]                 '

    ports:
      - 10054:10054

@reonokiy
Copy link
Contributor Author

reonokiy commented May 2, 2023

yaml格式没有特殊字符字符串不打引号也是可以的,
但是你的配置里API_KEY用引号了就不要多加空格了: ' your-api-key-here '

version: '3'
services:
  gpt_academic:
    image: fuqingxu/gpt_academic:no-local-llms
    container_name: gpt_academic
    environment:
      # see `config.py` all configuration options !
      API_KEY: 'sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' # your-api-key-here
      WEB_PORT: '10054'
      proxies: '{"http":"http://198.18.0.1:7890", "https":"http://198.18.0.1:7890",}'
      AVAIL_LLM_MODELS: '["gpt-3.5-turbo", "chatglm"]'
      AUTHENTICATION: '[("username", "password"), ("username2", "password2")]'

    ports:
      - 10054:10054

也可以使用env_file导入文件中的变量,如果配置比较多可以分开写:

# test.env
API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
WEB_PORT=10054
proxies='{"http":"http://198.18.0.1:7890", "https":"http://198.18.0.1:7890",}'
AVAIL_LLM_MODELS='["gpt-3.5-turbo", "chatglm"]'
AUTHENTICATION='[("username", "password"), ("username2", "password2")]'
version: '3'
services:
  gpt_academic:
    image: fuqingxu/gpt_academic:no-local-llms
    container_name: gpt_academic
    env_file:
      - test.env
    ports:
      - 10054:10054

还有一个需要注意的点是:如果不是使用容器的host模式的话,需要使用局域网的代理地址而不是localhost本机地址。

@binary-husky binary-husky merged commit 30d1698 into binary-husky:master May 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants