Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add notebook on fine tuning for function calling #810

Merged
merged 20 commits into from
Nov 10, 2023
Merged

Conversation

jhills20
Copy link
Contributor

@jhills20 jhills20 commented Oct 27, 2023

Summary

Adding fine tuning for function calling recipe. This generates synthetic data to improve the performance of function calling for a 'drone co-pilot ai'. The notebook shows how an untuned 3.5 Turbo fails to reject certain requests that the drone cannot do, and improperly calls functions. Then, the notebook creates invocations for every function, generates synthetic prompts that would lead to that invocation, and fine-tunes the model. The result is an improved model that properly rejects those questions.

Motivation

There is no notebook now on fine tuning for function calling, and this is a big customer need and ask.


For new content

When contributing new content, read through our contribution guidelines, and mark the following action items as completed:

  • I have added a new entry in registry.yaml so that my content renders on the cookbook website.
  • I have conducted a self-review of my content based on the c:
    • Relevance: This content is related to building with OpenAI technlogies and is useful to others.
    • Uniqueness: I have searched for related examples in the OpenAI Cookbook, and verified that my content offers new insights or unique information compared to existing documentation.
    • Spelling and Grammar: I have checked for spelling or grammatical mistakes.
    • Clarity: I have done a final read-through and verified that my submission is well-organized and easy to understand.
    • Correctness: The information I include is correct and all of my code executes successfully.
    • Completeness: I have explained everything fully, including all necessary references and citations.

We will rate each of these areas on a scale from 1 to 4, and will only accept contributions that score 3 or higher on all areas. Refer to our contributions guidelines for more details.

Copy link
Collaborator

@ibigio ibigio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to be such an awesome notebook to share with people – love it!

examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
Copy link
Collaborator

@simonpfish simonpfish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great notebook! People will find this very valuable. There is room for improvement in clarity, correctness, and grammar. I left some comments throughout the notebook.

Criteria Description Score Explanation
Relevance Is the content related to building with OpenAI technologies? Is it useful to others? 4 Your submission is highly relevant and provides useful information for individuals working with OpenAI tech.
Uniqueness Does the content offer new insights or unique information compared to existing documentation? 4 The content presents completely unique perspectives and insights not covered in existing documentation.
Clarity Is the language easy to understand? Are things well-explained? Is the title clear? 3 The language is clear, but there may be some sections where the structure or explanations could be improved.
Correctness Are the facts, code snippets, and examples correct and reliable? Does everything execute correctly? 3 The factual information is accurate; however, there might be minor errors or improvements needed for code snippets.
Completeness Is the content thorough and detailed? Are there things that weren’t explained fully? 4 The content is comprehensive and detailed, covering all necessary aspects of the topic.
Grammar Are there grammatical or spelling errors present? 3 The submission has correct grammar with a few typos present.

examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
examples/Fine_tuning_for_function_calling.ipynb Outdated Show resolved Hide resolved
jhills20 and others added 15 commits October 31, 2023 13:43
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Copy link
Contributor Author

@jhills20 jhills20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All changes should be made

@simonpfish
Copy link
Collaborator

Looks great! 🎉
Merging

@simonpfish simonpfish changed the title fine tuning for function calling recipe Add notebook on fine tuning for function calling Nov 10, 2023
@simonpfish simonpfish merged commit ac4cb3b into main Nov 10, 2023
@simonpfish simonpfish deleted the james/ft_for_fc branch November 10, 2023 01:36
ibigio pushed a commit that referenced this pull request Nov 10, 2023
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
colin-openai pushed a commit that referenced this pull request Nov 12, 2023
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
katia-openai pushed a commit that referenced this pull request Feb 29, 2024
Co-authored-by: Simón Fishman <simonpfish@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants