drf-spectacular: replace django-rest-swagger #1248

MyPyDavid · 2025-02-21T08:27:02Z

copied from #1050, rebased onto 2.3.0 > dependency-updates > drf-spectacular

Description

Duplicate of PR: #1050
Related issue: #698

Tasks

decide about NOTICE
remove swagger.scss
decide about the schema
decide about the purpose of the static schema in https://github.com/rdmorganiser/rdmo-openapi, which is 6 years old
handle warning and errors in schema generation
add tests
delete temporary django-rest-swagger.yaml schema file
remove ignore warnings from pytest config
squash commits

`drf-spectacular` warnings to be fixed

1. `/rdmo/conditions/`

Serializers: `serializers/v1.py`

ConditionViewSet > ConditionSerializer: Unable to resolve type hint for function get_model. Consider using a type hint or @extend_schema_field. Defaulting to string.

Viewsets: `viewsets.py`

ConditionViewSet: Could not derive type of path parameter "var" because model rdmo.conditions.models.Condition contained no such field. Consider annotating parameter with @extend_schema. Defaulting to "string".

2. `/rdmo/core/`

Serializers: `serializers.py`

RelationViewSet > ChoicesSerializer: Unable to resolve type hint for functions get_id and get_text.
GroupViewSet > GroupSerializer: Identical component names "Group" found in rdmo.core.serializers.GroupSerializer and rdmo.accounts.serializers.v1.GroupSerializer. Consider renaming one.
SitesViewSet > SiteSerializer: Identical component names "Site" found in rdmo.core.serializers.SiteSerializer and rdmo.accounts.serializers.v1.SiteSerializer. Consider renaming one.

Viewsets: `viewsets.py`

SettingsViewSet: Exception raised while getting serializer. Ensure get_serializer_class() is not returning None and get_queryset() works.
TemplatesViewSet: Similar issue as above.

3. `/rdmo/domain/`

Serializers: `serializers/v1.py`

AttributeViewSet > AttributeListSerializer: Unable to resolve type hints for get_model and is_leaf_node.
AttributeViewSet > AttributeSerializer: Unable to resolve type hints for get_model, get_tasks, and get_attributes.

Viewsets: `viewsets.py`

AttributeViewSet: Could not derive type of path parameter "var" due to missing field in rdmo.domain.models.Attribute.

4. `/rdmo/management/`

Viewsets: `viewsets.py`

ImportViewSet, MetaViewSet, UploadViewSet: Unable to guess serializer. Consider using GenericAPIView or explicitly adding serializer_class.

5. `/rdmo/options/`

Serializers: `serializers/v1/option.py`

OptionViewSet > OptionSerializer: Unable to resolve type hints for get_model, text, help, view_text, label, and get_warning.

Serializers: `serializers/v1/optionset.py`

OptionSetViewSet > OptionSetSerializer: Unable to resolve type hints for get_model and get_condition_uris.

Viewsets: `viewsets.py`

OptionViewSet, OptionSetViewSet: Path parameter "var" could not be derived due to missing fields in rdmo.options.models.Option and rdmo.options.models.OptionSet.

6. `/rdmo/projects/`

Serializers: `serializers/v1/init.py`

MembershipViewSet > MembershipSerializer: Identical component names "Membership" found in rdmo.projects.serializers.v1.MembershipSerializer and rdmo.accounts.serializers.v1.MembershipSerializer. Consider renaming one.
ProjectViewSet > ProjectSerializer: Unable to resolve type hint for catalog_uri.
ProjectViewSet > ProjectSerializer > UserSerializer: Identical component names "User" found in rdmo.projects.serializers.v1.UserSerializer and rdmo.accounts.serializers.v1.UserSerializer. Consider renaming one.

Viewsets: `viewsets.py`

MembershipViewSet: Failed to obtain model due to missing queryset. Fix by setting queryset = Model.objects.none() or using @extend_schema.

7. `/rdmo/questions/`

Serializers: `serializers/v1/catalog.py`

CatalogViewSet > CatalogSerializer: Identical component names "Catalog" found in rdmo.questions.serializers.v1.catalog.CatalogSerializer and rdmo.projects.serializers.v1.overview.CatalogSerializer. Consider renaming one.
Unable to resolve type hints for get_model, title, help, and get_warning.

Viewsets: `viewsets.py`

CatalogViewSet, PageViewSet, QuestionViewSet, QuestionSetViewSet, SectionViewSet: Path parameter "var" could not be derived due to missing fields.

8. `/rdmo/tasks/`

Serializers: `serializers/v1.py`

TaskViewSet > TaskSerializer: Unable to resolve type hints for get_model, title, text, get_warning, and get_condition_uris.

Viewsets: `viewsets.py`

TaskViewSet: Path parameter "var" could not be derived due to missing field.

9. `/rdmo/views/`

Serializers: `serializers/v1.py`

ViewViewSet > ViewSerializer: Unable to resolve type hints for get_model, title, help, and get_warning.

Viewsets: `viewsets.py`

ViewViewSet: Path parameter "var" could not be derived due to missing field.

10. General API Operation ID Collisions

Fix operation ID collisions in endpoints related to:
- conditions_conditions_export_retrieve
- domain_attributes_export_retrieve
- options_options_export_retrieve
- options_optionsets_export_retrieve
- questions_catalogs_export_retrieve
- questions_pages_export_retrieve
- questions_questions_export_retrieve
- questions_questionsets_export_retrieve
- questions_sections_export_retrieve
- tasks_tasks_export_retrieve
- views_views_export_retrieve

Types of Changes

Mix between bug fix (deprecated library is replaced), new feature (schema is enhanced) and maybe breaking change (schema is different from before).

Checklist

I have read the contributor guide.
My code follows the code style of this project.
I have added tests to cover my changes.
All new and existing tests passed.

Refs

MyPyDavid · 2025-02-21T08:33:37Z

Something was discussed about the urlpatterns:
#1050 (comment)

jochenklar · 2025-02-22T13:32:41Z

About the url patterns: I think they are fine, I would move the schema generation to /api/v1/openapi/ and leave the root /api/v1 empty or add a minimal template. With the 3 links.

jochenklar · 2025-02-22T13:36:09Z

I managed to fix a lot of warnings by adding return value type hints to (a) model properties and (b) SerializerMethodField methods. I am still not a fan, but I can live with those. I think its better than to inlcude the decorator from drf-spectacular. It could even be an optional dependency. For the remaining problems this could be the way: https://drf-spectacular.readthedocs.io/en/latest/customization.html#step-5-extensions

I think all of this should be done after at least the release candidate and we live with the warnings for now.

jochenklar · 2025-03-21T17:16:45Z

Ok, Update:

I moved the whole thing to an optional dependencies, to work you need to add

INSTALLED_APPS += [
    'drf_spectacular'
]

REST_FRAMEWORK.update({
    'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema',
    'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.URLPathVersioning'
})

to the settings.

I added type hints to remove some of the warnings.
I fixed the regexes for the export and detail_export actions
I added an API landing page:
I renamed some serializers and used ref_name on others (to remove some warnings)
I added a schema.py with some overrides to fix the other things drf-spectacular cannot figuer out on its own.

Besides ref_name (and the hints) nothing is in the regular rdmo source code, which was important to me.

jochenklar · 2025-03-25T20:44:03Z

@MyPyDavid I disabled the tests for the now optional openapi endpoint for now. We need to discuss if and how we include this in the tests.

jochenklar · 2025-03-25T20:45:03Z

The settings for the app are now:

INSTALLED_APPS += [
    'drf_spectacular'
]

REST_FRAMEWORK.update({
    'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema',
    'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.URLPathVersioning',
    'DEFAULT_VERSION': 'v1',
    'ALLOWED_VERSIONS': ('v1', ),
})

MyPyDavid · 2025-03-26T11:29:57Z

rdmo/core/urls/v1/openapi.py

+    path('schema/', login_required(SpectacularAPIView.as_view()), name='schema'),
+    path('swagger/', login_required(SpectacularSwaggerView.as_view(url_name='v1-openapi:schema')), name='swagger'),
+    path('redoc/', login_required(SpectacularRedocView.as_view(url_name='v1-openapi:schema')), name='redoc'),
+    path('', RedirectView.as_view(pattern_name='api', permanent=False))


When I include in my rdmo-app

urlpatterns += [ path('api/v1/', include('rdmo.core.urls.v1.openapi', namespace='v1')), ]

I get an error for http://localhost:8000/api/v1/

Reverse for 'api' not found. 'api' is not a valid view function or pattern name.

Where is this landing page supposed to be?

from django.contrib import admin from django.urls import include, path from rdmo.core.views import about, api, home urlpatterns = [ path('', home, name='home'), path('about/', about, name='about'), path('api/', api, name='api'), path('', include('rdmo.core.urls')), path('api/v1/', include('rdmo.core.urls.v1')), path('api/v1/', include('rdmo.core.urls.v1.openapi')), path('admin/', admin.site.urls) ]

rdmo/core/schema.py

MyPyDavid · 2025-03-27T16:26:40Z

is ready but will be merged after (or at the end of) 2.3.0-more-fixes (still needs to be rebased)

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>

afuetterer · 2025-03-30T13:59:26Z

The schema file django-rest-swagger.yaml was generated for testing purposes only, it was meant to be deleted after demonstrating drf-spectular's capacity. It has no purpose being in the root dir anymore.

afuetterer · 2025-03-30T13:59:53Z

What about these tests? https://github.com/rdmorganiser/rdmo/blob/2.3.0/rdmo/core/tests/test_openapi.py

Delete?

afuetterer · 2025-03-30T14:00:51Z

I think this can really be deleted, archived or otherwise stated as outdated: https://github.com/rdmorganiser/rdmo-openapi.

jochenklar · 2025-04-05T11:02:14Z

What about these tests? https://github.com/rdmorganiser/rdmo/blob/2.3.0/rdmo/core/tests/test_openapi.py

Delete?

I addressed this in #1298.

MyPyDavid changed the base branch from main to dependency-updates February 21, 2025 08:27

MyPyDavid mentioned this pull request Feb 21, 2025

feat: replace django-rest-swagger with drf-spectacular #1050

Closed

13 tasks

MyPyDavid added type:maintenance status:work-in-progress dependencies Pull requests that update a dependency file labels Feb 21, 2025

jochenklar added this to the RDMO 2.3.0 milestone Feb 21, 2025

MyPyDavid force-pushed the dependency-updates branch from 855e167 to 65608be Compare February 24, 2025 08:28

MyPyDavid force-pushed the drf-spectacular branch from 8abe6ad to c0615a9 Compare February 24, 2025 08:31

MyPyDavid changed the base branch from dependency-updates to 2.3.0 February 24, 2025 09:59

MyPyDavid force-pushed the drf-spectacular branch from c0615a9 to dd852d2 Compare February 24, 2025 12:25

MyPyDavid removed this from the RDMO 2.3.0 milestone Feb 26, 2025

MyPyDavid changed the title ~~feat: replace django-rest-swagger with drf-spectacular~~ drf-spectacular: replace django-rest-swagger with drf-spectacular Mar 11, 2025

MyPyDavid changed the title ~~drf-spectacular: replace django-rest-swagger with drf-spectacular~~ drf-spectacular: replace django-rest-swagger Mar 11, 2025

MyPyDavid added this to the RDMO 2.3.0 milestone Mar 18, 2025

jochenklar force-pushed the drf-spectacular branch from dd852d2 to df39fb0 Compare March 21, 2025 17:11

jochenklar marked this pull request as ready for review March 25, 2025 20:44

MyPyDavid commented Mar 26, 2025

View reviewed changes

MyPyDavid commented Mar 27, 2025

View reviewed changes

rdmo/core/schema.py Outdated Show resolved Hide resolved

MyPyDavid commented Mar 27, 2025

View reviewed changes

rdmo/core/schema.py Outdated Show resolved Hide resolved

MyPyDavid commented Mar 27, 2025

View reviewed changes

rdmo/core/schema.py Show resolved Hide resolved

jochenklar approved these changes Mar 27, 2025

View reviewed changes

MyPyDavid added status:hold and removed status:work-in-progress labels Mar 27, 2025

afuetterer and others added 16 commits March 28, 2025 11:09

temp add OpenAPIRenderer

dfef222

feat: replace django-rest-swagger with drf-spectacular

24df76b

chore: remove swagger.scss

e20d75b

Move drf-spectacular to optional dependencies

2382248

Add api view and reorganize core v1 urls

4e4bf3b

Fix export action regexes

d5c5250

Add type hints for model properties and serializer method fields

6bf6f78

Rename serializers or add ref_name

482d2a7

Add schema extensions and update settings

40b1868

Fix/disable tests for openapi

10b4cc0

core(schema): solve warnings for MembershipViewSet with introspection

23f6b2f

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>

core(schema): update patch get_queryset methods dynamically in schema

7eee76a

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>

projects(permissions): remove swagger related exceptions

00e9825

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>

projects(permissions): revert has object permission

dedba3b

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>

core(schema): simplify patch get_queryset

4440f25

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>

style: update simple type hints

0baebd9

Signed-off-by: David Wallace <david.wallace@tu-darmstadt.de>

jochenklar force-pushed the drf-spectacular branch from 6d4e7cc to 0baebd9 Compare March 28, 2025 10:09

jochenklar merged commit 9ecb250 into 2.3.0 Mar 28, 2025
8 of 12 checks passed

jochenklar deleted the drf-spectacular branch March 28, 2025 10:11

afuetterer removed the status:hold label Mar 30, 2025

afuetterer mentioned this pull request Apr 9, 2025

chore: remove obsolete warnings filter #1305

Closed

drf-spectacular: replace django-rest-swagger #1248

drf-spectacular: replace django-rest-swagger #1248

Uh oh!

Conversation

MyPyDavid commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Tasks

drf-spectacular warnings to be fixed

1. /rdmo/conditions/

Serializers: serializers/v1.py

Viewsets: viewsets.py

2. /rdmo/core/

Serializers: serializers.py

Viewsets: viewsets.py

3. /rdmo/domain/

Serializers: serializers/v1.py

Viewsets: viewsets.py

4. /rdmo/management/

Viewsets: viewsets.py

5. /rdmo/options/

Serializers: serializers/v1/option.py

Serializers: serializers/v1/optionset.py

Viewsets: viewsets.py

6. /rdmo/projects/

Serializers: serializers/v1/__init__.py

Viewsets: viewsets.py

7. /rdmo/questions/

Serializers: serializers/v1/catalog.py

Viewsets: viewsets.py

8. /rdmo/tasks/

Serializers: serializers/v1.py

Viewsets: viewsets.py

9. /rdmo/views/

Serializers: serializers/v1.py

Viewsets: viewsets.py

10. General API Operation ID Collisions

Types of Changes

Checklist

Refs

Uh oh!

MyPyDavid commented Feb 21, 2025

Uh oh!

jochenklar commented Feb 22, 2025

Uh oh!

jochenklar commented Feb 22, 2025

Uh oh!

jochenklar commented Mar 21, 2025

Uh oh!

jochenklar commented Mar 25, 2025

Uh oh!

jochenklar commented Mar 25, 2025

Uh oh!

MyPyDavid Mar 26, 2025

Choose a reason for hiding this comment

Uh oh!

jochenklar Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MyPyDavid commented Mar 27, 2025

Uh oh!

Uh oh!

afuetterer commented Mar 30, 2025

Uh oh!

afuetterer commented Mar 30, 2025

Uh oh!

afuetterer commented Mar 30, 2025

Uh oh!

jochenklar commented Apr 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

MyPyDavid commented Feb 21, 2025 •

edited

Loading

`drf-spectacular` warnings to be fixed

1. `/rdmo/conditions/`

Serializers: `serializers/v1.py`

Viewsets: `viewsets.py`

2. `/rdmo/core/`

Serializers: `serializers.py`

Viewsets: `viewsets.py`

3. `/rdmo/domain/`

Serializers: `serializers/v1.py`

Viewsets: `viewsets.py`

4. `/rdmo/management/`

Viewsets: `viewsets.py`

5. `/rdmo/options/`

Serializers: `serializers/v1/option.py`

Serializers: `serializers/v1/optionset.py`

Viewsets: `viewsets.py`

6. `/rdmo/projects/`

Serializers: `serializers/v1/init.py`

Viewsets: `viewsets.py`

7. `/rdmo/questions/`

Serializers: `serializers/v1/catalog.py`

Viewsets: `viewsets.py`

8. `/rdmo/tasks/`

Serializers: `serializers/v1.py`

Viewsets: `viewsets.py`

9. `/rdmo/views/`

Serializers: `serializers/v1.py`

Viewsets: `viewsets.py`