Neural chat api

neural_chat/README.md

@@ -0,0 +1,155 @@
+**NeuralChat** is a powerful and versatile chatbot designed to facilitate textual or voice conversations. By providing NeuralChat with textual or voice instructions, users can receive accurate and relevant responses. We provide a comprehensive API for building a highly customizable end-to-end chatbot service, covering model pre-training, model fine-tuning, model compression, prompt engineering, knowledge base retrieval, talkingbot and quick deployment.
+
+# Installation
+
+NeuralChat is seamlessly integrated into the Intel Extension for Transformers. Getting started is quick and simple, just simply install 'intel-extension-for-transformers'.
+
+## Install from Pypi
+```bash
+pip install intel-extension-for-transformers
+```
+> For more installation method, please refer to [Installation Page](../docs/installation.md)
+
+<a name="quickstart"></a>
+# Quick Start
+
+Users can have a try of NeuralChat with [NeuralChat Command Line](./cli/README.md) or Python API.
+
+
+## Text Chat
+
+Giving NeuralChat the textual instruction, it will respond with the textual response.
+
+**command line experience**
+
+```shell
+neuralchat textchat --prompt "Tell me about Intel Xeon Scalable Processors."
+```
+
+**Python API experience**
+
+```python
+>>> from neural_chat.config import NeuralChatConfig
+>>> from neural_chat.chatbot import NeuralChatBot
+>>> config = NeuralChatConfig()
+>>> chatbot = NeuralChatBot(config)
+>>> chatbot.build_chatbot()
+>>> response = chatbot.predict("Tell me about Intel Xeon Scalable Processors.")
+```
+
+
+## Voice Chat
+
+In the context of voice chat, users have the option to engage in various modes: utilizing input audio and receiving output audio, employing input audio and receiving textual output, or providing input in textual form and receiving audio output.
+
+**command line experience**
+
+- audio in and audio output
+    ```shell
+    neuralchat voicechat --input assets/audio/say_hello.wav --output response.wav
+    ```
+
+- audio in and text output
+    ```shell
+    neuralchat voicechat --input assets/audio/say_hello.wav
+    ```
+
+- text in and audio output
+    ```shell
+    neuralchat voicechat --input "Tell me about Intel Xeon Scalable Processors." --output response.wav
+    ```
+
+
+**Python API experience**
+
+For the Python API code, users have the option to enable different voice chat modes by setting audio_input to True for input or audio_output to True for output.
+
+```python
+>>> from neural_chat.config import NeuralChatConfig
+>>> from neural_chat.chatbot import NeuralChatBot
+>>> config = NeuralChatConfig(audio_input=True, audio_output=True)
+>>> chatbot = NeuralChatBot(config)
+>>> chatbot.build_chatbot()
+>>> result = chatbot.predict("Tell me about Intel Xeon Scalable Processors.")
+```
+
+## Finetuning
+
+Finetune the pretrained large language model (LLM) with the instruction-following dataset for creating the customized chatbot is very easy for NeuralChat.
+
+**command line experience**
+
+```shell
+neuralchat finetune --base_model "meta-llama/Llama-2-7b-chat-hf" --config finetuning/config/finetuning.config
+```
+
+
+**Python API experience**
+
+```python
+>>> from neural_chat.config import FinetuningConfig, NeuralChatConfig
+>>> from neural_chat.chatbot import NeuralChatBot
+>>> finetuneCfg = FinetuningConfig()
+>>> config = NeuralChatConfig(finetuneConfig=finetuneCfg)
+>>> chatbot = NeuralChatBot(config)
+>>> chatbot.build_chatbot()
+>>> chatbot.finetune()
+>>> response = chatbot.predict("Tell me about Intel Xeon Scalable Processors.")
+```
+
+
+<a name="quickstartserver"></a>
+# Quick Start Server
+
+Users can have a try of NeuralChat server with [NeuralChat Server Command Line](./server/README.md).
+
+
+**Start Server**
+- Command Line (Recommended)
+    ```shell
+    neuralchat_server start --config_file ./conf/neuralchat.yaml
+    ```
+
+- Python API
+    ```python
+    from neuralchat.server.neuralchat_server import NeuralChatServerExecutor
+    server_executor = NeuralChatServerExecutor()
+    server_executor(
+        config_file="./conf/neuralchat.yaml", 
+        log_file="./log/neuralchat.log")
+    ```
+
+**Access Text Chat Service**
+
+- Command Line
+    ```shell
+    neuralchat_client textchat --server_ip 127.0.0.1 --port 8000 --prompt "Tell me about Intel Xeon Scalable Processors."
+    ```
+
+- Python API
+    ```python
+    from neuralchat.server.neuralchat_client import TextChatClientExecutor
+
+    executor = TextChatClientExecutor()
+    executor(
+        prompt="Tell me about Intel Xeon Scalable Processors.",
+        server_ip="127.0.0.1",
+        port=8000)
+    ```
+
+- Curl
+    ```
+    curl -X POST -H "Content-Type: application/json" -d '{"prompt": "Tell me about Intel Xeon Scalable Processors."}' http://127.0.0.1:80/v1/chat/completions
+    ```
+
+**Access Voice Chat Service**
+
+```shell
+neuralchat_client voicechat --server_ip 127.0.0.1 --port 8000 --input say_hello.wav --output response.wav
+```
+
+**Access Retrieval Service**
+```shell
+neuralchat_client retrieval --server_ip 127.0.0.1 --port 8000 --input ./docs/
+```
+

neural_chat/chatbot.py

@@ -0,0 +1,56 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2023 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Neural Chat Chatbot API."""
+
+from .config import NeuralChatConfig, FinetuningConfig, OptimizationConfig
+from .pipeline.finetuning.finetuning import Finetuning
+
+class NeuralChatBot:
+    def __init__(self, config: NeuralChatConfig):
+        self.config = config
+        # Initialize other attributes here
+
+    def build_chatbot(self):
+        # Implement the logic to build the chatbot
+        pass
+
+    def finetune_model(self, config: FinetuningConfig=None):
+        # Implement the logic to finetune the model
+        config = config if config is not None else self.config.finetune_config
+        assert config is not None, "FinetuningConfig is needed for finetuning."
+        self.finetuning = Finetuning(config)
+        self.finetuning.finetune()
+
+    def optimize_model(self, config: OptimizationConfig):
+        # Implement the logic to optimize the model
+        pass
+
+    def register_model(self, model):
+        # Implement the logic to register the model
+        pass
+
+    def predict(self, prompt, temperature, top_p, top_k, repetition_penalty,
+                max_new_tokens, do_sample, num_beams, num_return_sequences,
+                bad_words_ids, force_words_ids, use_hpu_graphs, use_cache):
+        # Implement the logic to generate predictions
+        pass
+
+    def predict_stream(self, prompt, temperature, top_p, top_k, repetition_penalty,
+                       max_new_tokens, do_sample, num_beams, num_return_sequences,
+                       bad_words_ids, force_words_ids, use_hpu_graphs, use_cache):
+        # Implement the logic to generate predictions in a streaming manner
+        pass

neural_chat/cli/log.py

@@ -0,0 +1,63 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2023 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import functools
+import logging
+
+__all__ = [
+    'logger',
+]
+
+
+class Logger(object):
+    def __init__(self, name: str=None):
+        name = 'NeuralChat' if not name else name
+        self.logger = logging.getLogger(name)
+
+        log_config = {
+            'DEBUG': 10,
+            'INFO': 20,
+            'TRAIN': 21,
+            'EVAL': 22,
+            'WARNING': 30,
+            'ERROR': 40,
+            'CRITICAL': 50,
+            'EXCEPTION': 100,
+        }
+        for key, level in log_config.items():
+            logging.addLevelName(level, key)
+            if key == 'EXCEPTION':
+                self.__dict__[key.lower()] = self.logger.exception
+            else:
+                self.__dict__[key.lower()] = functools.partial(self.__call__,
+                                                               level)
+
+        self.format = logging.Formatter(
+            fmt='[%(asctime)-15s] [%(levelname)8s] - %(message)s')
+
+        self.handler = logging.StreamHandler()
+        self.handler.setFormatter(self.format)
+
+        self.logger.addHandler(self.handler)
+        self.logger.setLevel(logging.INFO)
+        self.logger.propagate = False
+
+    def __call__(self, log_level: str, msg: str):
+        self.logger.log(log_level, msg)
+
+
+logger = Logger()