error on uploading a file

[xubairp]

hi,

I am facing issue whenever i try to upload the markdown file. can you please guide if i am missing anything.
i am attaching the images of the console log.

screenshot of the page if i enter the url : http://localhost:5173/api/upload

[3Alan]

There are three things you need to confirm:

1. Have you cloned my latest code?
2. Have you run the backend Python code?
3. It looks like you are using the production environment. I see that the UI is calling http://localhost:5173/ instead of http://127.0.0.1:5000/. If you are using the production environment, have you correctly set VITE_SERVICES_URL?

[xubairp]

Thanks Alan,

After running the latest source, and executing the python command (which i missed previously) the app is running and i can see the preloaded files. however, i face the following issue now:

1. when i upload new file, i get the error of LIST INDEX OUT OF RANGE

INFO:werkzeug:127.0.0.1 - - [17/Apr/2023 21:07:48] "POST /api/upload HTTP/1.1" 500 -
INFO:werkzeug:127.0.0.1 - - [17/Apr/2023 21:08:07] "OPTIONS /api/upload HTTP/1.1" 200 -
C:\markdown-chat\chat-markdown-main\api-src C:\markdown-chat\chat-markdown-main\api-src\app.py
list index out of range upload error

2. I also tried to query the existing document/markdown files that comes with the application, whenever i type something, the app will load for few seconds and then reload to blank page. sharing the video i recorded.

Screenshare.-.2023-04-17.9_03_46.PM.1.mp4

INFO:werkzeug:127.0.0.1 - - [17/Apr/2023 21:30:56] "GET /userData/html/clean-code-javascript.html HTTP/1.1" 304 -
INFO:werkzeug:127.0.0.1 - - [17/Apr/2023 21:30:56] "GET /userData/html/clean-code-javascript.html HTTP/1.1" 200 -
INFO:llama_index.token_counter.token_counter:> [query] Total LLM token usage: 683 tokens
INFO:llama_index.token_counter.token_counter:> [query] Total embedding token usage: 4 tokens
some error: RetryError[<Future at 0x22cc2083d90 state=finished raised APIConnectionError>]
ERROR:app:<Response 93 bytes [500 INTERNAL SERVER ERROR]>
INFO:werkzeug:127.0.0.1 - - [17/Apr/2023 21:31:47] "GET /api/query?index=clean-code-javascript&openAiKey=-----hidden-----&query=example%20of%20bad%20function HTTP/1.1" 500 -

[3Alan]

It appears that OpenAI does not support the region you are in.

some error: RetryError[<Future at 0x22cc2083d90 state=finished raised APIConnectionError>]

You can check if your region is supported for access at https://platform.openai.com/docs/supported-countries.
Or you can test it using the following command:

curl https://api.openai.com

If it returns a response, it means you can access it normally:

{
  "error": {
    "message": "Invalid URL (GET /)",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

If you cannot access it, it will return:

curl: (28) Failed to connect to api.openai.com port 80 after 75249 ms: Operation timed out

If you are unable to access it in your region, you will need to configure the proxy environment variable (OPENAI_PROXY).

Finally, If you can access it normally in your region, please check the real-time output log of your Python application during runtime or check the error log in api-src/app.log.

[xubairp]

I do have access to the Openai, and i am using the API in other applications. So i dont think its the regional limitation.
the app.log file only have the following lines:

<Response 150 bytes [500 INTERNAL SERVER ERROR]>
<Response 93 bytes [500 INTERNAL SERVER ERROR]>
<Response 87 bytes [500 INTERNAL SERVER ERROR]>

[3Alan]

I tried to fix this network issue, but due to my region restrictions, I need to set up a proxy or VPN to access OpenAI directly. Therefore, I need your help to check if the problem has been fixed (please pull my latest code).

[xubairp]

I tried the latest code, here is what i found:

1. previously reported issue on UPLOAD (list of index out of range) seems to be resolved. the upload takes time but then throws error for APICONNECTION. however, when i open the application i can see the newly uploaded file in the dropdown selection. also i noticed the file seems to be uploaded to the USERDATA/HTML directory but not the USERDATA/INDEX directory

2. I tried to chat/query your existing files and it took couple seconds and then returned with PLEASE RETRY message. i am attaching the log here :

2023-04-21 18:21:24 ERROR: RetryError[<Future at 0x20b65da1010 state=finished raised APIConnectionError>]
Traceback (most recent call last):
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\openai\api_requestor.py", line 516, in request_raw
    result = _thread_context.session.request(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\requests\sessions.py", line 573, in request
    prep = self.prepare_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\requests\sessions.py", line 484, in prepare_request
    p.prepare(
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\requests\models.py", line 368, in prepare
    self.prepare_url(url, params)
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\requests\models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL '/engines/text-embedding-ada-002/embeddings': No scheme supplied. Perhaps you meant https:///engines/text-embedding-ada-002/embeddings?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\tenacity\__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\embeddings\openai.py", line 106, in get_embedding
    return openai.Embedding.create(input=[text], engine=engine)["data"][0]["embedding"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\openai\api_resources\embedding.py", line 33, in create
    response = super().create(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
                           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\openai\api_requestor.py", line 216, in request
    result = self.request_raw(
             ^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\openai\api_requestor.py", line 528, in request_raw
    raise error.APIConnectionError(
openai.error.APIConnectionError: Error communicating with OpenAI: Invalid URL '/engines/text-embedding-ada-002/embeddings': No scheme supplied. Perhaps you meant https:///engines/text-embedding-ada-002/embeddings?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\flask\app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\flask\app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\app.py", line 149, in query_index
    res = index.query(query_text, streaming=True)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\indices\base.py", line 244, in query
    return query_runner.query(query_str)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\indices\query\query_runner.py", line 341, in query
    return query_combiner.run(query_bundle, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\indices\query\query_combiner\base.py", line 66, in run
    return self._query_runner.query_transformed(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\indices\query\query_runner.py", line 201, in query_transformed
    return query_obj.query(query_bundle)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\token_counter\token_counter.py", line 78, in wrapped_llm_predict
    f_return_val = f(_self, *args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\indices\query\base.py", line 393, in query
    return self._query(query_bundle)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\indices\query\base.py", line 379, in _query
    nodes = self.retrieve(query_bundle)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\indices\query\base.py", line 249, in retrieve
    nodes = self._retrieve(query_bundle, similarity_tracker=similarity_tracker)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\indices\query\vector_store\base.py", line 54, in _retrieve
    self._service_context.embed_model.get_agg_embedding_from_queries(
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\embeddings\base.py", line 79, in get_agg_embedding_from_queries
    query_embeddings = [self.get_query_embedding(query) for query in queries]
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\embeddings\base.py", line 79, in <listcomp>
    query_embeddings = [self.get_query_embedding(query) for query in queries]
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\embeddings\base.py", line 68, in get_query_embedding
    query_embedding = self._get_query_embedding(query)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\llama_index\embeddings\openai.py", line 223, in _get_query_embedding
    return get_embedding(query, engine=engine)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\tenacity\__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\tenacity\__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\chat-markdown\api-src\.venv\Lib\site-packages\tenacity\__init__.py", line 326, in iter
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x20b65da1010 state=finished raised APIConnectionError>]

[3Alan]

Does your .env file contain OPENAI_PROXY=""? If so, deleting it may solve the problem. I think there may be some ambiguity in my .env.example file.

[xubairp]

Finally, i was able to upload and run the app. here are the changes i made:

1. removed the OPENAI_PROXY variable from the .env file. after removing it it was throwing error for the openai_api_key variable declaration in app.py.

2. then i passed the openai key to the variable :

    if open_ai_key:
        os.environ["OPENAI_API_KEY"] = "-------api key-----------"

thanks for your help.