feat: Rejection sampling data generation pipeline with SelfImprovingCoT pipeline #1646

JoyceXu02 · 2025-02-24T00:06:50Z

Description

Describe your changes in detail (optional if the linked issue already contains a detailed description of the changes).

Checklist

Go over all the following points, and put an x in all the boxes that apply.

I have read the CONTRIBUTION guide (required)
I have linked this PR to an issue using the Development section on the right sidebar or by adding Fixes #issue-number in the PR description (required)
I have checked if any dependencies need to be added or updated in pyproject.toml and poetry.lock
I have updated the tests accordingly (required for a bug fix or a new feature)
I have updated the documentation if needed:
I have added examples if this is a new feature
Fixes [Feature Request] Rejection sampling data generation pipeline with SelfImprovingCoT pipeline #1504
If you are unsure about any of these, don't hesitate to ask. We are here to help!

review-notebook-app · 2025-02-24T00:06:54Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

Wendong-Fan

thanks @JoyceXu02 , seems this PR also includes the change to cookbook, could you clean this PR only including the change to self_improving_cot.py file?

Wendong-Fan

thanks @JoyceXu02 , left some comments below

Wendong-Fan · 2025-02-24T04:43:26Z

camel/datagen/self_improving_cot.py

+        rejection_sampling: Optional[bool] = False,
+        rejection_sampling_n: int = 5,


we can simpify the interface, if rejection_sampling_n is not None, then we enables the pipeline to generate multiple candidate traces

Wendong-Fan · 2025-02-24T04:47:13Z

camel/datagen/self_improving_cot.py

+        for _i in range(self.rejection_sampling_n):
+            trace = self.generate_reasoning_trace(problem)
+            candidate_traces.append(trace)


some models support n parameter that can generate multi output with one request, could we leverage this for better efficiency? refer: https://platform.openai.com/docs/api-reference/chat/create#chat-create-n

Wendong-Fan · 2025-02-24T04:55:05Z

camel/datagen/self_improving_cot.py

+
+        Args:
+            problem (str): The problem text for generating a reasoning trace.
+            max_attempts (int): The number of candidate traces to generate.


no max_attempts passed to arg

Wendong-Fan · 2025-02-24T04:56:06Z

camel/datagen/self_improving_cot.py

+            first candidate if none qualify.
+        """
+        candidate_traces = []
+        for _i in range(self.rejection_sampling_n):


self.rejection_sampling_n should not be hardcoded to 5

Wendong-Fan · 2025-02-24T04:58:00Z

camel/datagen/self_improving_cot.py

+            candidate_traces.append(trace)
+
+        best_trace = None
+        best_avg_score = -1.0


I think we need to define initial value of best_avg_score >0

Thanks @Wendong-Fan , will fix it.

…1595) Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Xiaotian Jin <jinxiaotian_sal@outlook.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

…1627) Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Wendong-Fan

Thanks @JoyceXu02 ! Left some comments below

Wendong-Fan · 2025-02-26T06:17:51Z

camel/datagen/self_improving_cot.py

+        r"""
+        Generate multiple candidate reasoning traces for a problem and


docstring format

Suggested change

r"""

Generate multiple candidate reasoning traces for a problem and

r"""Generate multiple candidate reasoning traces for a problem and

Wendong-Fan · 2025-02-26T06:18:04Z

camel/datagen/self_improving_cot.py

+            str: The best candidate trace that meets quality criteria, or the
+            first candidate if none qualify.


Suggested change

str: The best candidate trace that meets quality criteria, or the

first candidate if none qualify.

str: The best candidate trace that meets quality criteria, or the

first candidate if none qualify.

Wendong-Fan · 2025-02-26T06:19:09Z

camel/datagen/self_improving_cot.py

+        self.reason_agent.model_backend.model_config_dict['n'] = (
+            self.rejection_sampling_n
+        )


not all models support n parameter, for those doesn't support n we still need to use loop to generate multiple content

make sense. I will update on this soon.

Wendong-Fan · 2025-02-26T06:21:12Z

camel/datagen/self_improving_cot.py

+                best_trace = trace
+                best_avg_score = avg_score
+        if best_trace is None:
+            best_trace = candidate_traces[0]


should we return the one with highest score even it didn't meet the threshold instead of hardcode to the first candidate?

yes. I will make a change on this one too.

Wendong-Fan

thanks @JoyceXu02 !

…oT pipeline (camel-ai#1646) Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Xiaotian Jin <jinxiaotian_sal@outlook.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Asher-hss <101127070+Asher-hss@users.noreply.github.com> Co-authored-by: Zoe Yan <73959962+zoezyn@users.noreply.github.com> Co-authored-by: Sarthak Bhardwaj <7sarthakbhardwaj@gmail.com> Co-authored-by: Isaac Jin <whale3ye@gmail.com> Co-authored-by: TTS <50868301+TOGOTOO@users.noreply.github.com> Co-authored-by: Lei Zhang <zhanglei@apache.org> Co-authored-by: Yifeng Wang（正经人王同学） <86822589+zjrwtx@users.noreply.github.com>

JoyceXu02 and others added 10 commits February 15, 2025 11:12

unify cookbooks of basic concepts

00dc3b5

fix incorrect url in self_instruct_data_generation

7391e99

Merge branch 'master' into docs/unify-cookbooks

9d7c329

exhibit the cell output

281c8cc

Merge branch 'master' into docs/unify-cookbooks

98ba898

added colab notebook links

c65f4d0

Merge branch 'master' into docs/unify-cookbooks

0ce6d73

update the cell output

9fa337f

unify data processing cookbooks

8756f04

add rejection sampling to selfimprovingCOT

85db6b7

JoyceXu02 requested a review from Wendong-Fan February 24, 2025 00:08

Wendong-Fan changed the title ~~Added Rejection sampling data generation pipeline with SelfImprovingCoT pipeline~~ feat: Rejection sampling data generation pipeline with SelfImprovingCoT pipeline Feb 24, 2025

Wendong-Fan reviewed Feb 24, 2025

View reviewed changes

Wendong-Fan assigned JoyceXu02 Feb 24, 2025

Wendong-Fan added the New Feature label Feb 24, 2025

Wendong-Fan added this to the Sprint 23 milestone Feb 24, 2025

MuggleJinx and others added 12 commits February 24, 2025 15:14

feat: new implementation of timeout functionality for all toolkits (#…

9bf3d30

…1595) Co-authored-by: Wendong <w3ndong.fan@gmail.com>

feat: Integrate Zapier AI tool (#1594)

0f1ceaf

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

feat: add warning (#1591)

9c61eb4

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Xiaotian Jin <jinxiaotian_sal@outlook.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

fix: Typo in base message

2802a30

docs: Update documentation for models (#1567)

304eae0

fix: Backend model tool call (#1624)

fa3d9f3

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

docs: Unify cookbooks of basic concepts (#1615)

4025f91

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

feat: Add api key link in error info (#1630)

f88c1f5

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

feat(datagen): add stop_on_first_failure option to InstructionFilter (#…

7911a22

…1627) Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

fix: Url error of STar (#1634)

2774178

fix: Base model handle image type (#1633)

2d73eff

feat: KnowNo implementation (#1629)

9a60d93

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

JoyceXu02 and others added 2 commits February 24, 2025 15:22

updated rejection sampling according to comments

737ae52

Merge branch 'master' into features/cot-rejection-sampling

87328f9

Wendong-Fan reviewed Feb 26, 2025

View reviewed changes

cot rejection sampling update2

4517742

Wendong-Fan approved these changes Feb 27, 2025

View reviewed changes

Wendong-Fan merged commit 4bb539b into master Feb 27, 2025
6 checks passed

Wendong-Fan deleted the features/cot-rejection-sampling branch February 27, 2025 15:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Rejection sampling data generation pipeline with SelfImprovingCoT pipeline #1646

feat: Rejection sampling data generation pipeline with SelfImprovingCoT pipeline #1646

JoyceXu02 commented Feb 24, 2025

review-notebook-app bot commented Feb 24, 2025

Wendong-Fan left a comment

Wendong-Fan left a comment

Wendong-Fan Feb 24, 2025

Wendong-Fan Feb 24, 2025

Wendong-Fan Feb 24, 2025

Wendong-Fan Feb 24, 2025

Wendong-Fan Feb 24, 2025

JoyceXu02 Feb 24, 2025

Wendong-Fan left a comment

Wendong-Fan Feb 26, 2025

Wendong-Fan Feb 26, 2025

Wendong-Fan Feb 26, 2025

JoyceXu02 Feb 26, 2025

Wendong-Fan Feb 26, 2025

JoyceXu02 Feb 26, 2025

Wendong-Fan left a comment

		rejection_sampling: Optional[bool] = False,
		rejection_sampling_n: int = 5,

		r"""
		Generate multiple candidate reasoning traces for a problem and

	r"""
	Generate multiple candidate reasoning traces for a problem and
	r"""Generate multiple candidate reasoning traces for a problem and

		str: The best candidate trace that meets quality criteria, or the
		first candidate if none qualify.

feat: Rejection sampling data generation pipeline with SelfImprovingCoT pipeline #1646

feat: Rejection sampling data generation pipeline with SelfImprovingCoT pipeline #1646

Conversation

JoyceXu02 commented Feb 24, 2025

Description

Checklist

review-notebook-app bot commented Feb 24, 2025

Wendong-Fan left a comment

Choose a reason for hiding this comment

Wendong-Fan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Wendong-Fan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Wendong-Fan left a comment

Choose a reason for hiding this comment