Swagger not updating when new API endpoints added in plugin

[tbotnz]

Deployment Type

Self-hosted

Triage priority

N/A

NetBox Version

v4.1.3

Python Version

3.11

Steps to Reproduce

1. add an endpoint to an API plugin
2. reload netbox

Expected Behavior

swagger should update showing the new endpoint

Observed Behavior

Problem is caused by the caching of the swagger in redis, this needs to be cleared every time in order for the swagger to update. It does appear to load after the cache expires but this takes 24 hours

Same problem with v4.1.6

[bctiemann]

Can this be addressed with documentation, e.g. a note that if you add an API endpoint in your plugin you need to flush your redis cache (with accompanying instructions)?

[tbotnz]

In my mind what should happen is the cached swagger should clear on container/app startup if its deemed out of date

[tbotnz]

here a simple management command that fixes it once called python manage.py $command_file_name

$command_file_name.py

from django.core.management.commands.makemigrations import Command as _Command

from django.core.cache import cache


class Command(_Command):

    def handle(self, *args, **kwargs):
        """ Temporary command to clear cache keys that are not cleared by the cache clear command. """
        _KEYS_PREFIX_REQUIRING_CLEARING = [
            "*api_schema_*",
        ]
        for key_prefix in _KEYS_PREFIX_REQUIRING_CLEARING:
            matched_keys_to_delete = cache.keys(key_prefix)
            for matched_key in matched_keys_to_delete:
                cache.delete(matched_key)

[github-actions]

This is a reminder that additional information is needed in order to further triage this issue. If the requested details are not provided, the issue will soon be closed automatically.

[prryplatypus]

@bctiemann what information is missing here?

[bctiemann]

Seems like the ideal solution would be for NetBox to invalidate the cache automatically on startup, but what I'm not clear on is what the mechanism should be for it to determine that the Swagger schema is out of date. How should it "know" that anything has changed?

Alternatively, the manual cache-clearing script above would do the job, but I wonder if it would be more useful for it to just be a general "clear the entire cache" command, with an optional CLI arg to specify only the matching keys to delete (e.g. *api_schema_*), or else CLI args to target specific classes of keys, such as --api_schema. But this seems more like feature territory than a bug fix.

[prryplatypus]

Seems like the ideal solution would be for NetBox to invalidate the cache automatically on startup, but what I'm not clear on is what the mechanism should be for it to determine that the Swagger schema is out of date. How should it "know" that anything has changed?

I'm not a Django power user by any means, but what if instead of caching it in Redis across all instances, it was cached in-memory, per instance, during application startup? This would eliminate the need to bust the cache at all, would keep responses to requests quick and would presumably also clear during app restarts (like when using autoreload in development or when deploying new things).