[BUG] Trained data not being used

[mrSingh007]

Description

I have performed some train iteration and found that still when running 'crewai run' trained data not being considered. Doesn't matter which LLM model is being used the result is not being changed according to trained data.

Used commands:
To train crewai train -n 2 -f lorem-train.pkl
To run `crewai run``

Steps to Reproduce

How to reproduce ?

1. in new very new project. (using any llm provider. I tested with llama and gemini)
2. in reporting_task under add "A well-organized written report." as expected_output

Now start at least 2 iterations of training and for reporting task give feedback to not include 'conclusion' section and to use numbering for all sections.

After training start now normal run with 'crewai run' on same default topic. You will find that now report is being generated as trained during training. (I just provided simple example here, you could play around on feedback).

After trying many attempts found that result is not being changed.

Expected behavior

According to provided reproduce steps:
The generated report should contain numbering and should not include 'conclusion' section

Screenshots/Code snippets

NA

Operating System

Other (specify in additional context)

Python Version

3.12

crewAI Version

0.117.1

crewAI Tools Version

0.117.1,<1.0.0

Virtual Environment

Venv

Evidence

NA

Possible Solution

NA

Additional context

[mouramax]

Correct me if I'm wrong, but I think your issue might be the same one covered in this forum thread?

If that's the case, it's the same problem: your training file needs to be named exactly trained_agents_data.pkl, because that value is hardcoded in crewai/utilities/constants.py.

[mrSingh007]

@mouramax You're right — the file is expected to be named trained_agents_data.pkl, but it still behaves inconsistently. Sometimes it works, and sometimes it doesn't.

I trained the data and renamed the file, and it worked — it produced the expected output using the trained data.
Then I deleted the .pkl file, retrained the LLM, and renamed the new file, but now it's not recognizing the trained data.

When I print data.get(self.role), it has data the first time, but in the second test, it returns None.

if data := CrewTrainingHandler(TRAINED_AGENTS_DATA_FILE).load():
    print(data.get(self.role))
    if trained_data_output := data.get(self.role):

Sometimes it generates two .pkl files, and sometimes only one. It's really flaky!
@joaomdmoura