Skip to content

Latest commit

 

History

History
108 lines (86 loc) · 5.34 KB

usage.md

File metadata and controls

108 lines (86 loc) · 5.34 KB
Raw

Basic usage

Running with job script

Carry out local job (local training / local inference / local interaction) with a configuration file. You can specify GPU by export CUDA_VISIBLE_DEVICES=XXX in ./scripts/local/job.sh. You can also specify other environment variables in the script.

./scripts/local/job.sh ${JOB_CONF}
  • An example of training configuration files is ./package/dialog_en/24L_train.conf. It contains three sections: job, task and training.
  • An example of evaluation configuration files is ./package/dialog_en/24L_evaluate.conf. It contains three sections: job, task and evaluation.
  • An example of inference configuration files is ./package/dialog_en/24L_infer.conf. It contains three sections: job, task and inference.
  • An example of interaction configuration files is ./package/dialog_en/24L_interact.conf. It contains three sections: job, task and interaction.
  • An example of self-chat configuration files is ./package/dialog_en/self_chat.conf. It contains three sections: job, task and self-chat.

job

This section defines:

  • job_script: the main script of this task.
    • ./scripts/single_gpu/train.sh: single GPU training
    • ./scripts/distributed/train.sh: distributed GPU training (NCCL required).
    • ./scripts/single_gpu/infer.sh: single GPU inference
    • ./scripts/distributed/infer.sh: distributed GPU inference on single machine, and it will merge different inference results on each GPU.
    • ./scripts/single_gpu/interact.sh: running interaction with a dialogue model

task

This section defines:

  • model: the model used in specify task, such as
    • UnifiedTransformer: used in the DialogGeneration task
    • Plato: used in the DialogGeneration task
    • NSPModel: used in the NextSentencePrediction task
  • task: task name, such as
    • DialogGeneration: generating a response for the given context.
    • NextSentencePrediction: judging whether the sentence is the next sentence follows given context.
  • vocab_path: vocabulary path
  • tokenizer related: spm_model_file for SentencePieces Tokenizer, and so on.
  • config_path: the model configuration file.
  • dataset files related: train_file / valid_file (for training), infer_file (for inference), data_format and file_format.
  • Choices of data_format:
    • raw: untokenized data tsv file, example: ./data/example/train.tsv, each column is a field.
    • tokenized: tokenized data tsv, example: ./data/example/train_tokenized.tsv which is generated by ./knover/tools/pre_tokenized.sh.
    • numerical: each line contains numerical data (token_ids, type_ids and pos_ids) , example: ./data/example/train.numerical.tsv which is generated by ./knover/tools/pre_numericalize.sh.
  • Choices of file_format:
    • file: a file only.
    • filelist: containing multiple files, where each line is a file, example: ./data/example/train_filelist.
  • It also supports the file with .gz suffix which is compressed by gzip command.

training

This section defines training related settings:

  • init_params: initialized parameters.
  • init_checkpoint: initialized checkpoints (containing not only the parameters of the model, but also the persitables of the optimizer) . You can also set train_args="--start_step 1000" for better display of log (when you continue the training from step 1000) , but this is not necessary.
  • batch_size, lr, num_epochs and so on.
  • log_dir: the output path of training logs, including the log file (${log_dir}/workerlog.${DEV_ID}) of each GPU trainer.
  • save_path: the output path of saved parameters.
  • You can define other arguments in training configuration, such as:
train_args="--max_src_len 384 --max_seq_len 512"
  • You can find more arguments in knover/tasks/${TASK_NAME}.py and knover/models/${MODEL_NAME}.py.

inference

This section defines inference related settings:

  • init_params: initialized parameters.
  • nsp_init_params: initialized NSP model parameters to re-rank candidate responses.
  • batch_size, in_tokens and so on.
  • output_name: the name of output field.
  • save_path: the output path of inference result.
  • You can define other arguments in inference configuration, such as:

rerank by NSP score:

infer_args="--ranking_score nsp_score" # this will re-rank candidate responses by scores given by NSP model.

top-k sampling and rerank:

infer_args="--decoding_strategy topk_sampling --num_samples 20 --topk 10 --length_average true"

top-p sampling and rerank:

infer_args="--decoding_strategy topp_sampling --num_samples 20 --topp 0.9 --length_average true"

beam search:

infer_args="--decoding_strategy beam_search --beam_size 10 --length_average true"
  • You can find more arguments in knover/tasks/${TASK_NAME}.py and knover/models/${MODEL_NAME}.py.

interaction

This section defines interaction related settings:

  • init_params: initialized parameters.
  • nsp_init_params: initialized NSP model parameters to re-rank candidate responses.
  • You can define other arguments in interaction configuration, such as:
infer_args="--ranking_score nsp_score" # this will re-rank candidate responses by scores given by NSP model.
  • You can find more arguments in knover/tasks/${TASK_NAME}.py and knover/models/${MODEL_NAME}.py.