| title | authors | |
|---|---|---|
Task queue management and monitoring |
|
The task queue for the ETL workers is managed by Celery in a RabbitMQ queue.
So you can use the queue management tools of RabbitMQ and queue management tools of Celery.
If this feature is not disabled by config, the status of file imports and OCR tasks is shown in the search user interface:
While the file import is running, you can yet search by filenames and prioritize the import of certain files by click on "Prioritize import".
You can monitor and manage the ETL task queue for document processing by Celery Flower web user interface (UI):
A link to the Flower instance on your host (f.e. http://localhost/flower) is available in the search user interface in the menu "Datasources" as "Show running and open imports and analysis tasks (ETL tasks)".
By the config file /etc/opensemanticsearch/task_priorities you can set priorities by file extension types like .pdf.
As default this option is set to process documents like PDF or DOC before images like JPG and files like .exe last.