This repository has been archived by the owner on Feb 22, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 51
Clean up API codebase by using idiomatic DRF and removing boilerplate #194
Merged
Merged
Changes from all commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
cc19670
Replace `str` with `uuid` to narrow matches
dhruvkb 022ad42
Define a helper function to raise `APIExceptions` from views
dhruvkb 064ccbe
Define a serializer for the audio waveform endpoint
dhruvkb 40a4eb4
Add cURL example for the OEmbed endpoint
dhruvkb c45d395
Create serializer for `ContentProvider`
dhruvkb a2f700e
Move pagination attributes from search query serializer to paginator …
dhruvkb 0e03571
Cleanup OEmbed input and output serializers
dhruvkb 870908a
Create parent viewset for all media types
dhruvkb 3c9d2b6
Add examples for reporting audio files
dhruvkb e0c0ec3
Replace `AboutMediaSerializer` superseded by `ProviderSerializer`
dhruvkb 8d8d226
Add attribute 'page' to search results
dhruvkb 6ec373b
Infer reported content identifier from the endpoint URL
dhruvkb cc009f3
Migrate endpoint documentation to a docs directory
dhruvkb a9f92f4
Consolidate endpoints into a DRF viewset
dhruvkb 0acf192
Use DRF router to automatically define endpoints from the viewset
dhruvkb d5e050f
Update tests based on the new code structure
dhruvkb c152280
Fix code style violations
dhruvkb 796cd10
Remove redundant import
dhruvkb a6fa43a
Make serializer nomenclature consistent
dhruvkb 136af05
Programmatically generate list of fields
dhruvkb d0c8b33
Remove unused serializer
dhruvkb bfb25a0
Use cleaner validation for deprecated fields, making post-error redun…
dhruvkb b828ed6
Update error serializers
dhruvkb c19a0d9
Fix broken reference in audio report endpoint
dhruvkb ceafca2
Remove field 'id'
dhruvkb 9d120d9
Use `HyperlinkedIdentityField` to automatically generate related URLs
dhruvkb 4862f95
Fix nomenclature in example requests and responses
dhruvkb a001243
Define request response mappings and use them to run tests on the end…
dhruvkb 545916a
Type hint `SerializerMethodField`s for Swagger
dhruvkb b263238
Move MD-in-Python out to its separate file
dhruvkb 5d9d667
Add testing for the report endpoint
dhruvkb b745c30
Unskip thumbnail checks for audio ref. #171
dhruvkb a405b79
Remove unnecessary scheme replacement ref. 9d120d9
dhruvkb 14914f7
Reformat code
dhruvkb f15b736
Make operation ID for OEmbed consistent
dhruvkb 683eb17
Disable pagination on stats endpoint
dhruvkb b93edeb
Add examples in the help text for provider serializer
dhruvkb File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,116 @@ | ||
# Introduction | ||
|
||
The Openverse API ('openverse-api') is a system that allows programmatic access | ||
to public domain digital media. It is our ambition to index and catalog billions | ||
of openly-licensed works, including articles, songs, videos, photographs, | ||
paintings, and more. Using this API, developers will be able to access the | ||
digital commons in their own applications. | ||
|
||
Please note that there is a rate limit of 5000 requests per day and 60 requests | ||
per minute rate limit in place for anonymous users. This is fine for introducing | ||
yourself to the API, but we strongly recommend that you obtain an API key as | ||
soon as possible. Authorized clients have a higher rate limit of 10000 requests | ||
per day and 100 requests per minute. Additionally, Openverse can give your key | ||
an even higher limit that fits your application's needs. See the | ||
[Register and Authenticate section](#section/Register-and-Authenticate) for | ||
instructions on obtaining an API key. | ||
|
||
# Register and Authenticate | ||
|
||
## Register for a key | ||
Before using the Openverse API, you need to register access via OAuth2. This can | ||
be done using the `/v1/auth_tokens/register` endpoint. | ||
|
||
Example on how to register for a key: | ||
```bash | ||
$ curl \ | ||
-X POST \ | ||
-H "Content-Type: application/json" \ | ||
-d '{"name": "My amazing project", "description": "To access Openverse API", "email": "zack.krida@automattic.com"}' \ | ||
"https://api.openverse.engineering/v1/auth_tokens/register" | ||
``` | ||
If your request is successful, you will get a `client_id` and `client_secret`. | ||
|
||
Example of successful request: | ||
```json | ||
{ | ||
"client_secret" : "YhVjvIBc7TuRJSvO2wIi344ez5SEreXLksV7GjalLiKDpxfbiM8qfUb5sNvcwFOhBUVzGNdzmmHvfyt6yU3aGrN6TAbMW8EOkRMOwhyXkN1iDetmzMMcxLVELf00BR2e", | ||
"client_id" : "pm8GMaIXIhkjQ4iDfXLOvVUUcIKGYRnMlZYApbda", | ||
"name" : "My amazing project" | ||
} | ||
``` | ||
|
||
## Authenticate | ||
In order to use the Openverse API endpoints, you need to include access token in | ||
the header. This can be done by exchanging your client credentials for a token | ||
using the `/v1/auth_tokens/token/` endpoint. | ||
|
||
Example on how to authenticate using OAuth2: | ||
```bash | ||
$ curl \ | ||
-X POST \ | ||
-d "client_id=pm8GMaIXIhkjQ4iDfXLOvVUUcIKGYRnMlZYApbda&client_secret=YhVjvIBc7TuRJSvO2wIi344ez5SEreXLksV7GjalLiKDpxfbiM8qfUb5sNvcwFOhBUVzGNdzmmHvfyt6yU3aGrN6TAbMW8EOkRMOwhyXkN1iDetmzMMcxLVELf00BR2e&grant_type=client_credentials" \ | ||
"https://api.openverse.engineering/v1/auth_tokens/token/" | ||
``` | ||
If your request is successful, you will get an access token. | ||
|
||
Example of successful request: | ||
```json | ||
{ | ||
"access_token" : "DLBYIcfnKfolaXKcmMC8RIDCavc2hW", | ||
"scope" : "read write groups", | ||
"expires_in" : 36000, | ||
"token_type" : "Bearer" | ||
} | ||
``` | ||
|
||
Check your email for a verification link. After you have followed the link, your | ||
API key will be activated. | ||
|
||
## Using Access Token | ||
Include the `access_token` in the authorization header to use your key in your | ||
future API requests. | ||
|
||
Example on how to make an authenticated request: | ||
```bash | ||
$ curl \ | ||
-H "Authorization: Bearer DLBYIcfnKfolaXKcmMC8RIDCavc2hW" \ | ||
"https://api.openverse.engineering/v1/images?q=test" | ||
``` | ||
|
||
> **NOTE:** Your token will be throttled like an anonymous user until the email | ||
> address has been verified. | ||
|
||
# Glossary | ||
|
||
| Term | Definition | | ||
|-------------------|---| | ||
| API | an abbreviation for Application Programming Interface | | ||
| OAuth2 | an authorization framework that enables a third party application to get access to an HTTP service | | ||
| access token | a private string that authorizes an application to make API requests | | ||
| client ID | a publicly exposed string used by Openverse API to identify the application | | ||
| client secret | a private string that authenticates the identity of the application to the Openverse API | | ||
| CC | an abbreviation for Creative Commons | | ||
| copyright | a type of intellectual property that gives the owner an exclusive right to reproduce, publish, sell or distribute content | | ||
| mature content | any content that requires the audience to be 18 and older | | ||
| sensitive content | any content that depicts graphic violence, adult content, and hostility or malice against others based on their race, religion, disability, sexual orientation, ethnicity and national origin | | ||
|
||
# Contribute | ||
|
||
We love pull requests! If you’re interested in | ||
[contributing on Github](https://github.com/wordpress/openverse-api), here’s a | ||
todo list to get started. | ||
|
||
- Read up about [Django REST Framework](https://www.django-rest-framework.org/), | ||
which is the framework used to build Openverse API | ||
- Read up about [drf-yasg](https://drf-yasg.readthedocs.io/en/stable/), which is | ||
a tool used to generate real Swagger/OpenAPI 2.0 specifications | ||
- Read up about Documentation Guidelines, which provides guidelines on how to | ||
contribute to documentation, documentation styles and cheat sheet for drf-yasg | ||
- Run the server locally by following this | ||
[link](https://github.com/wordpress/openverse-api#running-the-server-locally) | ||
- Update documentation or codebase | ||
- Make sure the updates passed the automated tests in this | ||
[file](https://github.com/wordpress/openverse-api/blob/master/.github/workflows/integration-tests.yml) | ||
- Commit and push | ||
- Create pull request |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,214 @@ | ||
from drf_yasg import openapi | ||
|
||
from catalog.api.docs.media_docs import ( | ||
fields_to_md, | ||
MediaSearch, | ||
MediaStats, | ||
MediaDetail, | ||
MediaRelated, | ||
MediaComplain, | ||
) | ||
from catalog.api.examples import ( | ||
audio_search_list_curl, | ||
audio_search_200_example, | ||
audio_search_400_example, | ||
audio_stats_curl, | ||
audio_stats_200_example, | ||
audio_detail_curl, | ||
audio_detail_200_example, | ||
audio_detail_404_example, | ||
audio_related_curl, | ||
audio_related_200_example, | ||
audio_related_404_example, | ||
audio_complain_curl, | ||
audio_complain_201_example, | ||
) | ||
from catalog.api.serializers.audio_serializers import ( | ||
AudioSearchRequestSerializer, | ||
AudioSearchSerializer, | ||
AudioSerializer, | ||
AudioReportSerializer, | ||
) | ||
from catalog.api.serializers.error_serializers import ( | ||
InputErrorSerializer, | ||
NotFoundErrorSerializer, | ||
) | ||
from catalog.api.serializers.provider_serializers import ProviderSerializer | ||
|
||
|
||
class AudioSearch(MediaSearch): | ||
desc = f""" | ||
audio_search is an API endpoint to search audio files using a query string. | ||
|
||
By using this endpoint, you can obtain search results based on specified | ||
query and optionally filter results by | ||
{fields_to_md(AudioSearchRequestSerializer.fields_names)}. | ||
|
||
{MediaSearch.desc}""" # noqa | ||
|
||
responses = { | ||
"200": openapi.Response( | ||
description="OK", | ||
examples=audio_search_200_example, | ||
schema=AudioSearchSerializer(many=True) | ||
), | ||
"400": openapi.Response( | ||
description="Bad Request", | ||
examples=audio_search_400_example, | ||
schema=InputErrorSerializer | ||
), | ||
} | ||
|
||
code_examples = [ | ||
{ | ||
'lang': 'Bash', | ||
'source': audio_search_list_curl, | ||
}, | ||
] | ||
|
||
swagger_setup = { | ||
'operation_id': 'audio_search', | ||
'operation_description': desc, | ||
'query_serializer': AudioSearchRequestSerializer, | ||
'responses': responses, | ||
'code_examples': code_examples | ||
} | ||
|
||
|
||
class AudioStats(MediaStats): | ||
desc = f""" | ||
audio_stats is an API endpoint to get a list of all content providers and their | ||
respective number of audio files in the Openverse catalog. | ||
|
||
{MediaStats.desc}""" # noqa | ||
|
||
responses = { | ||
"200": openapi.Response( | ||
description="OK", | ||
examples=audio_stats_200_example, | ||
schema=ProviderSerializer(many=True) | ||
) | ||
} | ||
|
||
code_examples = [ | ||
{ | ||
'lang': 'Bash', | ||
'source': audio_stats_curl, | ||
}, | ||
] | ||
|
||
swagger_setup = { | ||
'operation_id': 'audio_stats', | ||
'operation_description': desc, | ||
'responses': responses, | ||
'code_examples': code_examples, | ||
} | ||
|
||
|
||
class AudioDetail(MediaDetail): | ||
desc = f""" | ||
audio_detail is an API endpoint to get the details of a specified audio ID. | ||
|
||
By using this endpoint, you can get audio details such as | ||
{fields_to_md(AudioSerializer.fields_names)}. | ||
|
||
{MediaDetail.desc}""" # noqa | ||
|
||
responses = { | ||
"200": openapi.Response( | ||
description="OK", | ||
examples=audio_detail_200_example, | ||
schema=AudioSerializer | ||
), | ||
"404": openapi.Response( | ||
description="OK", | ||
examples=audio_detail_404_example, | ||
schema=NotFoundErrorSerializer | ||
) | ||
} | ||
|
||
code_examples = [ | ||
{ | ||
'lang': 'Bash', | ||
'source': audio_detail_curl, | ||
}, | ||
] | ||
|
||
swagger_setup = { | ||
'operation_id': 'audio_detail', | ||
'operation_description': desc, | ||
'responses': responses, | ||
'code_examples': code_examples, | ||
} | ||
|
||
|
||
class AudioRelated(MediaRelated): | ||
desc = f""" | ||
recommendations_audio_read is an API endpoint to get related audio files | ||
for a specified audio ID. | ||
|
||
By using this endpoint, you can get the details of related audio such as | ||
{fields_to_md(AudioSerializer.fields_names)}. | ||
|
||
{MediaRelated.desc}""" # noqa | ||
|
||
responses = { | ||
"200": openapi.Response( | ||
description="OK", | ||
examples=audio_related_200_example, | ||
schema=AudioSerializer | ||
), | ||
"404": openapi.Response( | ||
description="Not Found", | ||
examples=audio_related_404_example, | ||
schema=NotFoundErrorSerializer | ||
) | ||
} | ||
|
||
code_examples = [ | ||
{ | ||
'lang': 'Bash', | ||
'source': audio_related_curl, | ||
}, | ||
] | ||
|
||
swagger_setup = { | ||
'operation_id': 'audio_related', | ||
'operation_description': desc, | ||
'responses': responses, | ||
'code_examples': code_examples | ||
} | ||
|
||
|
||
class AudioComplain(MediaComplain): | ||
desc = f""" | ||
audio_report_create is an API endpoint to report an issue about a specified | ||
audio ID to Openverse. | ||
|
||
By using this endpoint, you can report an audio file if it infringes copyright, | ||
contains mature or sensitive content and others. | ||
|
||
{MediaComplain.desc}""" # noqa | ||
|
||
responses = { | ||
"201": openapi.Response( | ||
description="OK", | ||
examples=audio_complain_201_example, | ||
schema=AudioReportSerializer | ||
) | ||
} | ||
|
||
code_examples = [ | ||
{ | ||
'lang': 'Bash', | ||
'source': audio_complain_curl, | ||
} | ||
] | ||
|
||
swagger_setup = { | ||
'operation_id': 'audio_report', | ||
'operation_description': desc, | ||
'query_serializer': AudioReportSerializer, | ||
'responses': responses, | ||
'code_examples': code_examples, | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should these docs go to the make handbook, just like the Catalog and Frontend docs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think so, because they kind of have to live here: https://api.openverse.engineering/v1/ and I don't see a strong reason to duplicate the information. We should however add a link and reference to the API docs in the handbook.