Skip to content

[pull] main from openai:main #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 28, 2022
Merged

[pull] main from openai:main #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 28 additions & 16 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ print(completion.choices[0].text)

### Microsoft Azure Endpoints

In order to use the library with Microsoft Azure endpoints, you need to set the api_type, api_base and api_version in addition to the api_key. The api_type must be set to 'azure' and the others correspond to the properites of your endpoint.
In order to use the library with Microsoft Azure endpoints, you need to set the api_type, api_base and api_version in addition to the api_key. The api_type must be set to 'azure' and the others correspond to the properties of your endpoint.
In addition, the deployment name must be passed as the engine parameter.

```python
Expand All @@ -78,8 +78,9 @@ print(search)
```

Please note that for the moment, the Microsoft Azure endpoints can only be used for completion, search and fine-tuning operations.
For a detailed example on how to use fine-tuning and other operations using Azure endpoints, please check out the following Jupyter notebook:
[Using Azure fine-tuning](https://github.com/openai/openai-python/blob/main/examples/azure/finetuning.ipynb)
For a detailed example on how to use fine-tuning and other operations using Azure endpoints, please check out the following Jupyter notebooks:
* [Using Azure fine-tuning](https://github.com/openai/openai-cookbook/tree/main/examples/azure/finetuning.ipynb)
* [Using Azure embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/azure/embeddings.ipynb)

### Microsoft Azure Active Directory Authentication

Expand Down Expand Up @@ -118,7 +119,18 @@ openai api completions.create -e ada -p "Hello world"

## Example code

Examples of how to use [embeddings](https://github.com/openai/openai-python/tree/main/examples/embeddings), [fine tuning](https://github.com/openai/openai-python/tree/main/examples/finetuning), [semantic search](https://github.com/openai/openai-python/tree/main/examples/semanticsearch), and [codex](https://github.com/openai/openai-python/tree/main/examples/codex) can be found in the [examples folder](https://github.com/openai/openai-python/tree/main/examples).
Examples of how to use this Python library to accomplish various tasks can be found in the [OpenAI Cookbook](https://github.com/openai/openai-cookbook/). It contains code examples for:

* Classification using fine-tuning
* Clustering
* Code search
* Customizing embeddings
* Question answering from a corpus of documents
* Recommendations
* Visualization of embeddings
* And more

Prior to July 2022, this OpenAI Python library hosted code examples in its examples folder, but since then all examples have been migrated to the [OpenAI Cookbook](https://github.com/openai/openai-cookbook/).

### Embeddings

Expand All @@ -140,17 +152,17 @@ model_id = "text-similarity-davinci-001"
embedding = openai.Embedding.create(input=text_string, engine=model_id)['data'][0]['embedding']
```

An example of how to call the embeddings method is shown in the [get embeddings notebook](https://github.com/openai/openai-python/blob/main/examples/embeddings/Get_embeddings.ipynb).
An example of how to call the embeddings method is shown in this [get embeddings notebook](https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings.ipynb).

Examples of how to use embeddings are shared in the following Jupyter notebooks:

- [Classification using embeddings](https://github.com/openai/openai-python/blob/main/examples/embeddings/Classification.ipynb)
- [Clustering using embeddings](https://github.com/openai/openai-python/blob/main/examples/embeddings/Clustering.ipynb)
- [Code search using embeddings](https://github.com/openai/openai-python/blob/main/examples/embeddings/Code_search.ipynb)
- [Semantic text search using embeddings](https://github.com/openai/openai-python/blob/main/examples/embeddings/Semantic_text_search_using_embeddings.ipynb)
- [User and product embeddings](https://github.com/openai/openai-python/blob/main/examples/embeddings/User_and_product_embeddings.ipynb)
- [Zero-shot classification using embeddings](https://github.com/openai/openai-python/blob/main/examples/embeddings/Zero-shot_classification.ipynb)
- [Recommendation using embeddings](https://github.com/openai/openai-python/blob/main/examples/embeddings/Recommendation.ipynb)
- [Classification using embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/Classification_using_embeddings.ipynb)
- [Clustering using embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/Clustering.ipynb)
- [Code search using embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/Code_search.ipynb)
- [Semantic text search using embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/Semantic_text_search_using_embeddings.ipynb)
- [User and product embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/User_and_product_embeddings.ipynb)
- [Zero-shot classification using embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/Zero-shot_classification_with_embeddings.ipynb)
- [Recommendation using embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/Recommendation_using_embeddings.ipynb)

For more information on embeddings and the types of embeddings OpenAI offers, read the [embeddings guide](https://beta.openai.com/docs/guides/embeddings) in the OpenAI documentation.

Expand All @@ -160,11 +172,11 @@ Fine tuning a model on training data can both improve the results (by giving the

Examples of fine tuning are shared in the following Jupyter notebooks:

- [Classification with fine tuning](https://github.com/openai/openai-python/blob/main/examples/finetuning/finetuning-classification.ipynb) (a simple notebook that shows the steps required for fine tuning)
- [Classification with fine tuning](https://github.com/openai/openai-cookbook/blob/main/examples/Fine-tuned_classification.ipynb) (a simple notebook that shows the steps required for fine tuning)
- Fine tuning a model that answers questions about the 2020 Olympics
- [Step 1: Collecting data](https://github.com/openai/openai-python/blob/main/examples/finetuning/olympics-1-collect-data.ipynb)
- [Step 2: Creating a synthetic Q&A dataset](https://github.com/openai/openai-python/blob/main/examples/finetuning/olympics-2-create-qa.ipynb)
- [Step 3: Train a fine-tuning model specialized for Q&A](https://github.com/openai/openai-python/blob/main/examples/finetuning/olympics-3-train-qa.ipynb)
- [Step 1: Collecting data](https://github.com/openai/openai-cookbook/blob/main/examples/fine-tuned_qa/olympics-1-collect-data.ipynb)
- [Step 2: Creating a synthetic Q&A dataset](https://github.com/openai/openai-cookbook/blob/main/examples/fine-tuned_qa/olympics-2-create-qa.ipynb)
- [Step 3: Train a fine-tuning model specialized for Q&A](https://github.com/openai/openai-cookbook/blob/main/examples/fine-tuned_qa/olympics-3-train-qa.ipynb)

Sync your fine-tunes to [Weights & Biases](https://wandb.me/openai-docs) to track experiments, models, and datasets in your central dashboard with:

Expand Down
7 changes: 7 additions & 0 deletions examples/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Examples have moved to the [OpenAI Cookbook](https://github.com/openai/openai-cookbook/)

Looking for code examples? Visit the [OpenAI Cookbook](https://github.com/openai/openai-cookbook/), which shares examples of how to use the OpenAI Python library to accomplish common tasks.

Prior to July 2022, code examples were hosted in this examples folder; going forward, code examples will be hosted in the [OpenAI Cookbook](https://github.com/openai/openai-cookbook/).

This separation will help keep the [OpenAI Python library](https://github.com/openai/openai-python) simple and small, without extra files or dependencies.
174 changes: 9 additions & 165 deletions examples/azure/embeddings.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,174 +4,13 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Azure embeddings example\n",
"In this example we'll try to go over all operations for embeddings that can be done using the Azure endpoints. \\\n",
"This example focuses on finetuning but touches on the majority of operations that are also available using the API. This example is meant to be a quick way of showing simple operations and is not meant as a tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import openai\n",
"from openai import cli"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"In the following section the endpoint and key need to be set up of the next sections to work. \\\n",
"Please go to https://portal.azure.com, find your resource and then under \"Resource Management\" -> \"Keys and Endpoints\" look for the \"Endpoint\" value and one of the Keys. They will act as api_base and api_key in the code below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"openai.api_key = '' # Please add your api key here\n",
"openai.api_base = '' # Please add your endpoint here\n",
"\n",
"openai.api_type = 'azure'\n",
"openai.api_version = '2022-03-01-preview' # this may change in the future"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deployments\n",
"In this section we are going to create a deployment using the finetune model that we just adapted and then used the deployment to create a simple completion operation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Deployments: Create Manually\n",
"Let's create a deployment using the text-similarity-curie-001 engine. You can create a new deployment by going to your Resource in your portal under \"Resource Management\" -> \"Deployments\"."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### (Optional) Deployments: Create Programatically\n",
"We can also create a deployment using code:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model = \"text-similarity-curie-001\"\n",
"\n",
"# Now let's create the deployment\n",
"print(f'Creating a new deployment with model: {model}')\n",
"result = openai.Deployment.create(model=model, scale_settings={\"scale_type\":\"manual\", \"capacity\": 1})\n",
"deployment_id = result[\"id\"]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### (Optional) Deployments: Retrieving\n",
"Now let's check the status of the newly created deployment"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(f'Checking for deployment status.')\n",
"resp = openai.Deployment.retrieve(id=deployment_id)\n",
"status = resp[\"status\"]\n",
"print(f'Deployment {deployment_id} is with status: {status}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Deployments: Listing\n",
"Now because creating a new deployment takes a long time, let's look in the subscription for an already finished deployment that succeeded."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print('While deployment running, selecting a completed one.')\n",
"deployment_id = None\n",
"result = openai.Deployment.list()\n",
"for deployment in result.data:\n",
" if deployment[\"status\"] == \"succeeded\":\n",
" deployment_id = deployment[\"id\"]\n",
" break\n",
"\n",
"if not deployment_id:\n",
" print('No deployment with status: succeeded found.')\n",
"else:\n",
" print(f'Found a successful deployment with id: {deployment_id}.')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Embeddings\n",
"Now let's send a sample embedding to the deployment."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"embeddings = openai.Embedding.create(deployment_id=deployment_id,\n",
" input=\"The food was delicious and the waiter...\")\n",
" \n",
"print(embeddings)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### (Optional) Deployments: Delete\n",
"Finally let's delete the deployment"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(f'Deleting deployment: {deployment_id}')\n",
"openai.Deployment.delete(sid=deployment_id)"
"This code example has moved. You can now find it in the [OpenAI Cookbook](https://github.com/openai/openai-cookbook) at [examples/azure/embeddings.ipynb](https://github.com/openai/openai-cookbook/tree/main/examples/azure/embeddings.ipynb)."
]
}
],
"metadata": {
"interpreter": {
"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
},
"kernelspec": {
"display_name": "Python 3.8.10 64-bit",
"display_name": "Python 3.9.9 ('openai')",
"language": "python",
"name": "python3"
},
Expand All @@ -185,9 +24,14 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
"version": "3.9.9"
},
"orig_nbformat": 4
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "365536dcbde60510dc9073d6b991cd35db2d9bac356a11f5b64279a5e6708b97"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
Expand Down
Loading