Skip to content
This repository has been archived by the owner on Feb 22, 2023. It is now read-only.

Clean up API codebase by using idiomatic DRF and removing boilerplate #194

Merged
merged 37 commits into from
Sep 21, 2021
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
cc19670
Replace `str` with `uuid` to narrow matches
dhruvkb Aug 30, 2021
022ad42
Define a helper function to raise `APIExceptions` from views
dhruvkb Aug 30, 2021
064ccbe
Define a serializer for the audio waveform endpoint
dhruvkb Aug 30, 2021
40a4eb4
Add cURL example for the OEmbed endpoint
dhruvkb Aug 31, 2021
c45d395
Create serializer for `ContentProvider`
dhruvkb Aug 31, 2021
a2f700e
Move pagination attributes from search query serializer to paginator …
dhruvkb Aug 31, 2021
0e03571
Cleanup OEmbed input and output serializers
dhruvkb Aug 31, 2021
870908a
Create parent viewset for all media types
dhruvkb Aug 31, 2021
3c9d2b6
Add examples for reporting audio files
dhruvkb Aug 31, 2021
e0c0ec3
Replace `AboutMediaSerializer` superseded by `ProviderSerializer`
dhruvkb Aug 31, 2021
8d8d226
Add attribute 'page' to search results
dhruvkb Aug 31, 2021
6ec373b
Infer reported content identifier from the endpoint URL
dhruvkb Aug 31, 2021
cc009f3
Migrate endpoint documentation to a docs directory
dhruvkb Aug 31, 2021
a9f92f4
Consolidate endpoints into a DRF viewset
dhruvkb Aug 31, 2021
0acf192
Use DRF router to automatically define endpoints from the viewset
dhruvkb Aug 31, 2021
d5e050f
Update tests based on the new code structure
dhruvkb Aug 31, 2021
c152280
Fix code style violations
dhruvkb Aug 31, 2021
796cd10
Remove redundant import
dhruvkb Aug 31, 2021
a6fa43a
Make serializer nomenclature consistent
dhruvkb Aug 31, 2021
136af05
Programmatically generate list of fields
dhruvkb Aug 31, 2021
d0c8b33
Remove unused serializer
dhruvkb Aug 31, 2021
bfb25a0
Use cleaner validation for deprecated fields, making post-error redun…
dhruvkb Aug 31, 2021
b828ed6
Update error serializers
dhruvkb Aug 31, 2021
c19a0d9
Fix broken reference in audio report endpoint
dhruvkb Sep 1, 2021
ceafca2
Remove field 'id'
dhruvkb Sep 1, 2021
9d120d9
Use `HyperlinkedIdentityField` to automatically generate related URLs
dhruvkb Sep 1, 2021
4862f95
Fix nomenclature in example requests and responses
dhruvkb Sep 1, 2021
a001243
Define request response mappings and use them to run tests on the end…
dhruvkb Sep 1, 2021
545916a
Type hint `SerializerMethodField`s for Swagger
dhruvkb Sep 1, 2021
b263238
Move MD-in-Python out to its separate file
dhruvkb Sep 2, 2021
5d9d667
Add testing for the report endpoint
dhruvkb Sep 2, 2021
b745c30
Unskip thumbnail checks for audio ref. #171
dhruvkb Sep 2, 2021
a405b79
Remove unnecessary scheme replacement ref. 9d120d9
dhruvkb Sep 2, 2021
14914f7
Reformat code
dhruvkb Sep 2, 2021
f15b736
Make operation ID for OEmbed consistent
dhruvkb Sep 2, 2021
683eb17
Disable pagination on stats endpoint
dhruvkb Sep 2, 2021
b93edeb
Add examples in the help text for provider serializer
dhruvkb Sep 21, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions openverse-api/catalog/api/docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Introduction
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these docs go to the make handbook, just like the Catalog and Frontend docs?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, because they kind of have to live here: https://api.openverse.engineering/v1/ and I don't see a strong reason to duplicate the information. We should however add a link and reference to the API docs in the handbook.


The Openverse API ('openverse-api') is a system that allows programmatic access
to public domain digital media. It is our ambition to index and catalog billions
of openly-licensed works, including articles, songs, videos, photographs,
paintings, and more. Using this API, developers will be able to access the
digital commons in their own applications.

Please note that there is a rate limit of 5000 requests per day and 60 requests
per minute rate limit in place for anonymous users. This is fine for introducing
yourself to the API, but we strongly recommend that you obtain an API key as
soon as possible. Authorized clients have a higher rate limit of 10000 requests
per day and 100 requests per minute. Additionally, Openverse can give your key
an even higher limit that fits your application's needs. See the
[Register and Authenticate section](#section/Register-and-Authenticate) for
instructions on obtaining an API key.

# Register and Authenticate

## Register for a key
Before using the Openverse API, you need to register access via OAuth2. This can
be done using the `/v1/auth_tokens/register` endpoint.

Example on how to register for a key:
```bash
$ curl \
-X POST \
-H "Content-Type: application/json" \
-d '{"name": "My amazing project", "description": "To access Openverse API", "email": "zack.krida@automattic.com"}' \
"https://api.openverse.engineering/v1/auth_tokens/register"
```
If your request is successful, you will get a `client_id` and `client_secret`.

Example of successful request:
```json
{
"client_secret" : "YhVjvIBc7TuRJSvO2wIi344ez5SEreXLksV7GjalLiKDpxfbiM8qfUb5sNvcwFOhBUVzGNdzmmHvfyt6yU3aGrN6TAbMW8EOkRMOwhyXkN1iDetmzMMcxLVELf00BR2e",
"client_id" : "pm8GMaIXIhkjQ4iDfXLOvVUUcIKGYRnMlZYApbda",
"name" : "My amazing project"
}
```

## Authenticate
In order to use the Openverse API endpoints, you need to include access token in
the header. This can be done by exchanging your client credentials for a token
using the `/v1/auth_tokens/token/` endpoint.

Example on how to authenticate using OAuth2:
```bash
$ curl \
-X POST \
-d "client_id=pm8GMaIXIhkjQ4iDfXLOvVUUcIKGYRnMlZYApbda&client_secret=YhVjvIBc7TuRJSvO2wIi344ez5SEreXLksV7GjalLiKDpxfbiM8qfUb5sNvcwFOhBUVzGNdzmmHvfyt6yU3aGrN6TAbMW8EOkRMOwhyXkN1iDetmzMMcxLVELf00BR2e&grant_type=client_credentials" \
"https://api.openverse.engineering/v1/auth_tokens/token/"
```
If your request is successful, you will get an access token.

Example of successful request:
```json
{
"access_token" : "DLBYIcfnKfolaXKcmMC8RIDCavc2hW",
"scope" : "read write groups",
"expires_in" : 36000,
"token_type" : "Bearer"
}
```

Check your email for a verification link. After you have followed the link, your
API key will be activated.

## Using Access Token
Include the `access_token` in the authorization header to use your key in your
future API requests.

Example on how to make an authenticated request:
```bash
$ curl \
-H "Authorization: Bearer DLBYIcfnKfolaXKcmMC8RIDCavc2hW" \
"https://api.openverse.engineering/v1/images?q=test"
```

> **NOTE:** Your token will be throttled like an anonymous user until the email
> address has been verified.

# Glossary

| Term | Definition |
|-------------------|---|
| API | an abbreviation for Application Programming Interface |
| OAuth2 | an authorization framework that enables a third party application to get access to an HTTP service |
| access token | a private string that authorizes an application to make API requests |
| client ID | a publicly exposed string used by Openverse API to identify the application |
| client secret | a private string that authenticates the identity of the application to the Openverse API |
| CC | an abbreviation for Creative Commons |
| copyright | a type of intellectual property that gives the owner an exclusive right to reproduce, publish, sell or distribute content |
| mature content | any content that requires the audience to be 18 and older |
| sensitive content | any content that depicts graphic violence, adult content, and hostility or malice against others based on their race, religion, disability, sexual orientation, ethnicity and national origin |

# Contribute

We love pull requests! If you’re interested in
[contributing on Github](https://github.com/wordpress/openverse-api), here’s a
todo list to get started.

- Read up about [Django REST Framework](https://www.django-rest-framework.org/),
which is the framework used to build Openverse API
- Read up about [drf-yasg](https://drf-yasg.readthedocs.io/en/stable/), which is
a tool used to generate real Swagger/OpenAPI 2.0 specifications
- Read up about Documentation Guidelines, which provides guidelines on how to
contribute to documentation, documentation styles and cheat sheet for drf-yasg
- Run the server locally by following this
[link](https://github.com/wordpress/openverse-api#running-the-server-locally)
- Update documentation or codebase
- Make sure the updates passed the automated tests in this
[file](https://github.com/wordpress/openverse-api/blob/master/.github/workflows/integration-tests.yml)
- Commit and push
- Create pull request
Empty file.
214 changes: 214 additions & 0 deletions openverse-api/catalog/api/docs/audio_docs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
from drf_yasg import openapi

from catalog.api.docs.media_docs import (
fields_to_md,
MediaSearch,
MediaStats,
MediaDetail,
MediaRelated,
MediaComplain,
)
from catalog.api.examples import (
audio_search_list_curl,
audio_search_200_example,
audio_search_400_example,
audio_stats_curl,
audio_stats_200_example,
audio_detail_curl,
audio_detail_200_example,
audio_detail_404_example,
audio_related_curl,
audio_related_200_example,
audio_related_404_example,
audio_complain_curl,
audio_complain_201_example,
)
from catalog.api.serializers.audio_serializers import (
AudioSearchRequestSerializer,
AudioSearchSerializer,
AudioSerializer,
AudioReportSerializer,
)
from catalog.api.serializers.error_serializers import (
InputErrorSerializer,
NotFoundErrorSerializer,
)
from catalog.api.serializers.provider_serializers import ProviderSerializer


class AudioSearch(MediaSearch):
desc = f"""
audio_search is an API endpoint to search audio files using a query string.

By using this endpoint, you can obtain search results based on specified
query and optionally filter results by
{fields_to_md(AudioSearchRequestSerializer.fields_names)}.

{MediaSearch.desc}""" # noqa

responses = {
"200": openapi.Response(
description="OK",
examples=audio_search_200_example,
schema=AudioSearchSerializer(many=True)
),
"400": openapi.Response(
description="Bad Request",
examples=audio_search_400_example,
schema=InputErrorSerializer
),
}

code_examples = [
{
'lang': 'Bash',
'source': audio_search_list_curl,
},
]

swagger_setup = {
'operation_id': 'audio_search',
'operation_description': desc,
'query_serializer': AudioSearchRequestSerializer,
'responses': responses,
'code_examples': code_examples
}


class AudioStats(MediaStats):
desc = f"""
audio_stats is an API endpoint to get a list of all content providers and their
respective number of audio files in the Openverse catalog.

{MediaStats.desc}""" # noqa

responses = {
"200": openapi.Response(
description="OK",
examples=audio_stats_200_example,
schema=ProviderSerializer(many=True)
)
}

code_examples = [
{
'lang': 'Bash',
'source': audio_stats_curl,
},
]

swagger_setup = {
'operation_id': 'audio_stats',
'operation_description': desc,
'responses': responses,
'code_examples': code_examples,
}


class AudioDetail(MediaDetail):
desc = f"""
audio_detail is an API endpoint to get the details of a specified audio ID.

By using this endpoint, you can get audio details such as
{fields_to_md(AudioSerializer.fields_names)}.

{MediaDetail.desc}""" # noqa

responses = {
"200": openapi.Response(
description="OK",
examples=audio_detail_200_example,
schema=AudioSerializer
),
"404": openapi.Response(
description="OK",
examples=audio_detail_404_example,
schema=NotFoundErrorSerializer
)
}

code_examples = [
{
'lang': 'Bash',
'source': audio_detail_curl,
},
]

swagger_setup = {
'operation_id': 'audio_detail',
'operation_description': desc,
'responses': responses,
'code_examples': code_examples,
}


class AudioRelated(MediaRelated):
desc = f"""
recommendations_audio_read is an API endpoint to get related audio files
for a specified audio ID.

By using this endpoint, you can get the details of related audio such as
{fields_to_md(AudioSerializer.fields_names)}.

{MediaRelated.desc}""" # noqa

responses = {
"200": openapi.Response(
description="OK",
examples=audio_related_200_example,
schema=AudioSerializer
),
"404": openapi.Response(
description="Not Found",
examples=audio_related_404_example,
schema=NotFoundErrorSerializer
)
}

code_examples = [
{
'lang': 'Bash',
'source': audio_related_curl,
},
]

swagger_setup = {
'operation_id': 'audio_related',
'operation_description': desc,
'responses': responses,
'code_examples': code_examples
}


class AudioComplain(MediaComplain):
desc = f"""
audio_report_create is an API endpoint to report an issue about a specified
audio ID to Openverse.

By using this endpoint, you can report an audio file if it infringes copyright,
contains mature or sensitive content and others.

{MediaComplain.desc}""" # noqa

responses = {
"201": openapi.Response(
description="OK",
examples=audio_complain_201_example,
schema=AudioReportSerializer
)
}

code_examples = [
{
'lang': 'Bash',
'source': audio_complain_curl,
}
]

swagger_setup = {
'operation_id': 'audio_report',
'operation_description': desc,
'query_serializer': AudioReportSerializer,
'responses': responses,
'code_examples': code_examples,
}
Loading