I found a flaw: "Bot is cooking for too long."

[yoobaring]

Hi @n4ze3m I have encountered an issue. The bot is cooking for too long. I've found this problem in documents with multiple pages, but for a small number of pages, I haven't encountered this issue. I hope this issue will be resolved. Please note that I'm running tests on a railway.

[chalitbkb]

Hi @n4ze3m I have encountered an issue. The bot is cooking for too long. I've found this problem in documents with multiple pages, but for a small number of pages, I haven't encountered this issue. I hope this issue will be resolved. Please note that I'm running tests on a railway.

This is the same issue I've encountered as well. I've opened a ticket to report this problem before. I believe it won't be long before it gets fixed. Let's wait for the next update...

[n4ze3m]

which data source is causing the issue, the docx or the pdf? I know this issue occurs with the railway and works fine locally. I will be looking into a solution

[yoobaring]

which data source is causing the issue, the docx or the pdf? I know this issue occurs with the railway and works fine locally. I will be looking into a solution

Both

[n4ze3m]

I have updated the railway template, which may fix the file processing issues.

[yoobaring]

@n4ze3m I waited 3 hours and still got the same problem. Nothing has changed. The problem has not been completely resolved. My file has approximately 500-1000 pages. How many pages did you test the document for? Please try 500-1000 or more pages and you will encounter this problem.

[n4ze3m]

Is this issue related to the railway or is it local?

For Railway,I think you need to reinstall the railway template. The old one doesn't have a Docker mount, which may be causing the issue.

[yoobaring]

Is this issue related to the railway or is it local?

For Railway,I think you need to reinstall the railway template. The old one doesn't have a Docker mount, which may be causing the issue.

railway

I tried it and your latest version is 1.4.1.

[n4ze3m]

Hello, can you reinstall your railway app? The latest update has mounted an upload folder, preventing the deletion of uploaded files.

Railway template: https://railway.app/template/TXdjD7

I have tested a 758-page PDF, approximately 17 MB, using Cohere embedding, and it's working without any issue.

PDF I tested: https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf

[yoobaring]

Hello, can you reinstall your railway app? The latest update has mounted an upload folder, preventing the deletion of uploaded files.

Railway template: https://railway.app/template/TXdjD7

I have tested a 758-page PDF, approximately 17 MB, using Cohere embedding, and it's working without any issue.

PDF I tested: https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf

@n4ze3m

Please carefully watch the video. Do not fast forward or skip, as there are explanations that you need to read.

https://streamable.com/afx0tc

I have tested on the "Railway" again, and it seems that I am still encountering the same issues. Here are my observations:

1. When testing with the "Cohere API," there are no issues when using files with the .pdf extension. However, problems arise when working with files in the .docx format.

2. Testing with the "OpenAI API" reveals problems with files that have multiple pages, including the file you provided me for testing.

[n4ze3m]

When testing with the "Cohere API," there are no issues when using files with the .pdf extension. However, problems arise when working with files in the .docx format.

I will look into it. I think the issue is with the DOCX loader.

Testing with the "OpenAI API" reveals problems with files that have multiple pages, including the file you provided me for testing.

I will test with the OpenAI API, as I think the issue may be caused by a rate limit. I will look into it

Currently, you cannot delete a data source while it is processing. I will update the error label.

[n4ze3m]

Hello,

I have released a new update which addresses the issue with the docx loader. This update has been tested on a 700+ page docx document on railways using the text-embedding-ada-002 model.

The processing time for the file is approximately 2-3 minutes.

teste docx link: https://docs.google.com/document/d/18-ETRBO4yRpRl3nF68P8vTbunlBgdy_t/edit?usp=sharing&ouid=108531690400573042017&rtpof=true&sd=true

demo.mp4