fine-turning on vicuna-13b-1.1

[apachemycat]

demo fine-turning json file not find

[bupticybee]

Raw data can be fine here: https://huggingface.co/datasets/icybee/share_gpt_90k_v1

Use the data cleaning script here: https://github.com/lm-sys/FastChat/blob/main/docs/commands/data_cleaning.md#steps

I wrote all those in readme here: https://github.com/bupticybee/FastLoRAChat#dataset-release ,please check it out.

[apachemycat]

thanks I try it

[apachemycat]

Traceback (most recent call last):
File "finetune_fastchat.py", line 534, in
fire.Fire(train)
File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "finetune_fastchat.py", line 524, in train
trainer.train(resume_from_checkpoint=resume_from_checkpoint)
File "/usr/local/lib/python3.8/dist-packages/transformers/trainer.py", line 1662, in train
return inner_training_loop(
File "/usr/local/lib/python3.8/dist-packages/transformers/trainer.py", line 1899, in _inner_training_loop
for step, inputs in enumerate(epoch_iterator):
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 633, in next
data = self._next_data()
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 677, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 51, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "finetune_fastchat.py", line 215, in getitem
data_dict = preprocess(
File "finetune_fastchat.py", line 150, in preprocess
header = f"{conversation_lib.default_conversation.system}\n\n"
AttributeError: module 'fastchat.conversation' has no attribute 'default_conversation'

——————————————
data file from share_gpt_90k_v1 cleaned ,fowlling is part of
{
"from": "human",
"value": "isn't jesus's "the way to attain a close relationship with God is through love and obedience to God's commands" can be the description for the same process as the kabbala: "process of purification and illumination, where the individual removes obstacles and limitations in their consciousness" ?"
},
{
"from": "gpt",
"value": "There is a general similarity between the two concepts you described. Both the teachings of Jesus and Kabbalah view the process of spiritual development as one of purifying the self and removing obstacles in order to attain a closer relationship with God. In both traditions, love and obedience to God's commands are seen as important aspects of this process.\n\nHowever, the specific practices, concepts, and terminology used in each tradition are distinct and not directly comparable. For example, in Christianity, obedience to God's commands is seen as a way to demonstrate love for God, while in Kabbalah, the purification of the soul is seen as a way to allow the light of the divine to enter. Additionally, the concepts of sin and righteousness are understood differently in each tradition, with Christianity emphasizing the importance of forgiveness and salvation through faith in Jesus, while Kabbalah focuses on the purification of the soul and union with the divine."
},

---

fastchat
fschat 0.2.10

[bupticybee]

Traceback (most recent call last): File "finetune_fastchat.py", line 534, in fire.Fire(train) File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "finetune_fastchat.py", line 524, in train trainer.train(resume_from_checkpoint=resume_from_checkpoint) File "/usr/local/lib/python3.8/dist-packages/transformers/trainer.py", line 1662, in train return inner_training_loop( File "/usr/local/lib/python3.8/dist-packages/transformers/trainer.py", line 1899, in _inner_training_loop for step, inputs in enumerate(epoch_iterator): File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 633, in next data = self._next_data() File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 677, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 51, in data = [self.dataset[idx] for idx in possibly_batched_index] File "finetune_fastchat.py", line 215, in getitem data_dict = preprocess( File "finetune_fastchat.py", line 150, in preprocess header = f"{conversation_lib.default_conversation.system}\n\n" AttributeError: module 'fastchat.conversation' has no attribute 'default_conversation'

—————————————— data file from share_gpt_90k_v1 cleaned ,fowlling is part of { "from": "human", "value": "isn't jesus's "the way to attain a close relationship with God is through love and obedience to God's commands" can be the description for the same process as the kabbala: "process of purification and illumination, where the individual removes obstacles and limitations in their consciousness" ?" }, { "from": "gpt", "value": "There is a general similarity between the two concepts you described. Both the teachings of Jesus and Kabbalah view the process of spiritual development as one of purifying the self and removing obstacles in order to attain a closer relationship with God. In both traditions, love and obedience to God's commands are seen as important aspects of this process.\n\nHowever, the specific practices, concepts, and terminology used in each tradition are distinct and not directly comparable. For example, in Christianity, obedience to God's commands is seen as a way to demonstrate love for God, while in Kabbalah, the purification of the soul is seen as a way to allow the light of the divine to enter. Additionally, the concepts of sin and righteousness are understood differently in each tradition, with Christianity emphasizing the importance of forgiveness and salvation through faith in Jesus, while Kabbalah focuses on the purification of the soul and union with the divine." },

fastchat fschat 0.2.10

You are using wrong version of fastchat, check https://github.com/bupticybee/FastLoRAChat#inference-setup