Skip to content

Latest commit

 

History

History
81 lines (53 loc) · 3.8 KB

README.md

File metadata and controls

81 lines (53 loc) · 3.8 KB
Raw

Bothub NLP - Natural Language Processing services

Build Status Coverage Status version 3.0.1 python 3.6 license AGPL-3.0

Services

bothub-nlp-nlu-worker

bothub-nlp-api

Packages

bothub-backend (python 3.6)

bothub-nlp-celery (python 3.6)

Requirements

  • Python (3.6)
  • Docker
  • Docker-Compose

Development

Use make commands

Command Description
make init_development_env Init file .env with variables environment
make start_development Start build docker
make install_development_requirements Install some default models
make start_celery Run celery application

Environment Variables

You can set environment variables in your OS, write on .env file or pass via Docker config.

bothub-backend

Variable Type Default Description
BOTHUB_ENGINE_URL str https://api.bothub.it Web service url

nlp-nlu-worker / nlp-ai-platform

You can set environment variables in your OS, write on .env file or pass via Docker config.

Variable Type Default Description
WORKER_CACHE_CLEANING_PERIOD float 3*3600 Period of time (seconds) the worker will look for idle interpreters to clean cache
INTERPRETER_CACHE_IDLE_LIMIT float 24*3600 Idle limit of time (seconds) the interpreter cache will keep cache
DYNAMIC_EPOCHS_THRESHOLD int 10000 Minimum number of sentences to start decreasing training number of epochs
BOTHUB_NLP_AWS_ACCESS_KEY_ID str AWS bucket access to save trained models and evaluation results
BOTHUB_NLP_AWS_SECRET_ACCESS_KEY str AWS bucket access to save trained models and evaluation results
BOTHUB_NLP_AWS_S3_BUCKET_NAME str AWS bucket access to save trained models and evaluation results
BOTHUB_NLP_AWS_REGION_NAME str AWS bucket access to save trained models and evaluation results

bothub-celery

Variable Type Default Description
BOTHUB_NLP_CELERY_BROKER_URL string redis://localhost:6379/0 Celery Broker URL, check usage instructions in Celery Docs
BOTHUB_NLP_CELERY_BACKEND_URL string BOTHUB_NLP_CELERY_BROKER_URL value Celery Backend URL, check usage instructions in Celery Docs
BOTHUB_NLP_CELERY_SENTRY_CLIENT bool False Enable Sentry
BOTHUB_NLP_CELERY_SENTRY str None Set URL Sentry Server
BOTHUB_NLP_LANGUAGE_QUEUE string en Set language of model that will be loaded in celery and will define its queue
BOTHUB_LANGUAGE_MODEL string None Set type of model (BERT/SPACY/NONE)
TASK_GENERAL_TIME_LIMIT int 120 Time limit of celery tasks
TASK_PARSE_TIME_LIMIT int 10 Time limit of parse task

Docker Arguments

You need to set --build-arg when you are building docker-compose

Argument Type Default Description
DOWNLOAD_MODELS string en-BERT Set language and model in build time. Following the format: [LANGUAGE_CODE]-[LANGUAGE_MODEL].