Remove old assets inclusion/generation at build process

[humitos]

We are generating some .js files and we are including some other static files. Mainly in MkDocs.

We should remove them from the build process once we are using the new addons approach. See the code to remove at https://github.com/readthedocs/readthedocs.org/blob/6f394fd89a34e636de30fc23268bb51b7c85028b/readthedocs/doc_builder/backends/mkdocs.py#L170-L180

Note there are a bunch of tests and other related things we can remove as well.

Issues:

• Closes MkDocs 1.5 and the !relative YAML tag readthedocs.org#10803
• Closes https://github.com/readthedocs/meta/issues/85
• Related Help ReadTheDocs to better integrate with MkDocs mkdocs/mkdocs#3413

[humitos]

Take a look at this comment in https://github.com/readthedocs/meta/issues/85#issuecomment-1616690964

[ericholscher]

We are generating some .js files and we are including some other static files. Mainly in MkDocs.

We should remove them from the build process once we are using the new addons approach. See the code to remove at https://github.com/readthedocs/readthedocs.org/blob/6f394fd89a34e636de30fc23268bb51b7c85028b/readthedocs/doc_builder/backends/mkdocs.py#L170-L180

Note there are a bunch of tests and other related things we can remove as well.

This is also being included in Sphinx via our extension:

https://github.com/readthedocs/readthedocs-sphinx-ext/blob/7c60d1646c12ac0b83d61abfbdd5bcd77d324124/readthedocs_ext/_templates/readthedocs-insert.html.tmpl#L21-L33

[pawamoy]

Hello! Coming from readthedocs/readthedocs.org#7844.

I opened mkdocs/mkdocs#3413 to start a discussion about readthedocs/readthedocs.org#10803. Do I understand correctly that the addons approach will remove the need to safely load and re-dump MkDocs config file to inject your own configuration? Meaning that any custom tag MkDocs might implement in the future would be immediately supported by ReadTheDocs?

[stsewd]

@pawamoy yeah, overriding the mkdocs config isn't required with this approach.

[humitos]

I took a look at the conf.py.tmpl that we append on Sphinx projects and I was thinking on a way to detect what projects are using what variables from the context. Is there any way we can achieve this?

[ericholscher]

I think we can probably wrap the function in some hacky way so that it logs a message or something? It would definitely be a hack or a monky patch though 🙈

[humitos]

I was thinking something similar yesterday, but I wasn't sure how to implement it and I ended up reading about metaclasses (https://stackoverflow.com/questions/11349183/how-to-wrap-every-method-of-a-class) which I wanted to avoid.

Today, I arrived at a lot simpler solution: wrap the context dictionary variable we create into a class that inherit from dict that we control and log something when any of the injected keys are accessed:

READTHEDOCS_INJECTED_KEYS = [
    'html_theme',
    'current_version',
    'version_slug',
    'MEDIA_URL',
    'STATIC_URL',
    'PRODUCTION_DOMAIN',
    'proxied_static_path',
    'versions',
    'downloads',
    'subprojects',
    'slug',
    'name',
    'rtd_language',
    'programming_language',
    'canonical_url',
    'analytics_code',
    'single_version',
    'conf_py_path',
    'api_host',
    'github_user',
    'proxied_api_host',
    'github_repo',
    'github_version',
    'display_github',
    'bitbucket_user',
    'bitbucket_repo',
    'bitbucket_version',
    'display_bitbucket',
    'gitlab_user',
    'gitlab_repo',
    'gitlab_version',
    'display_gitlab',
    'READTHEDOCS',
    'using_theme',
    'new_theme',
    'source_suffix',
    'ad_free',
    'docsearch_disabled',
    'user_analytics_code',
    'global_analytics_code',
    'commit',
]

class ReadTheDocsSphinxContext(dict):

    def __getitem__(self, name):
        if name in READTHEDOCS_INJECTED_KEYS:
            print("Read the Docs value from injected context called.")
        return super().__getitem__(name)

    def __getattribute__(self, name):
        if name in READTHEDOCS_INJECTED_KEYS:
            print("Read the Docs value from injected context called.")
        return super().__getattribute__(name)

Then, when the user access any of these keys we will get an extra output in the console (we can perform an internal API call or even call NR to log the line somehow). The example shows just an output in the console:

In [14]: context = MyClass({"name": "Read the Docs", "slug": "read-the-docs", "html_theme": "sphinx_rtd_theme", "non-RTD-injected": "
    ...: a users's value"})

In [15]: context["name"]
Read the Docs value from injected context called.
Out[15]: 'Read the Docs'

In [16]: context["html_theme"]
Read the Docs value from injected context called.
Out[16]: 'sphinx_rtd_theme'

In [17]: context["non-RTD-injected"]
Out[17]: "a users's value"

In [18]:

I think something like this could work. We will need to define this class in the conf.py.tmpl and wrap the context variable we define with this class. Honestly, it doesn't sound terrible to me and I don't think it will affect people's build in a bad way. This solution will allow us to know exactly what projects are using what injected keys and be able to talk to them about next steps. @ericholscher What do you think about this idea?

[humitos]

I tested this locally and it fails when pickling the Sphinx environment at https://github.com/sphinx-doc/sphinx/blob/master/sphinx/builders/__init__.py#L330

 _pickle.PicklingError: Can't pickle : attribute lookup ReadTheDocsSphinxContext on builtins failed 

I think there is some modifications to the code we can make to make this object picklable. I tried to disable pickling the environment on Sphinx but I didn't find a configuration for that. I'm not sure to understand why it fails when pickling this object inside of Sphinx but it doesn't when I pickle it directly from the terminal.

[ericholscher]

This is what I was imagining when I said a hack or monkey patch :) It's a reasonable way to get that data. 💯

Could perhaps use this solution which sounds similar: https://stackoverflow.com/a/21145155