[plugins] Load plugins on demand #11305

coletdjnz · 2024-10-21T19:41:02Z

Load plugins when requested, rather than automatically on import.
- By default, YoutubeDL will request for plugins to be loaded if they have not already been
Add --no-plugin-dirs to disable plugin loading
Improved --plugin-dirs to support post processors; ability to disable looking in external locations.

You can specify --plugin-dirs after --no-plugin-dirs to restrict plugin dirs

Added python APIs to interact with plugin loading logic:

register_plugin_spec(plugin_spec: PluginSpec) - add a new plugin specification (e.g. something other than extractors/post-processors)
clear_plugins()
load_all_plugins()
load_plugins(plugin_spec: PluginSpec)

Template

Before submitting a pull request make sure you have:

At least skimmed through contributing guidelines including yt-dlp coding conventions
Searched the bugtracker for similar pull requests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

I am the original author of this code and I am willing to release it under Unlicense
I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)
grub4k, pukkandan

What is the purpose of your pull request?

Fix or improvement to an extractor (Make sure to add/update tests)
New extractor (Piracy websites will not be accepted)
Core bug fix/improvement
New feature (It is strongly recommended to open an issue first)

…plugins-revive # Conflicts: # yt_dlp/YoutubeDL.py # yt_dlp/__init__.py # yt_dlp/options.py # yt_dlp/plugins.py # yt_dlp/postprocessor/__init__.py # yt_dlp/utils/_utils.py

…ame plugin

…plugins-revive # Conflicts: # test/test_plugins.py # yt_dlp/plugins.py

Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>

README.md

bashonly

First impressions. I'll look at it again soon and hopefully have a better grasp on it then

Don't miss my lazy extractors comment amid my sea of pedantry

yt_dlp/options.py

yt_dlp/extractor/extractors.py

yt_dlp/plugins.py

yt_dlp/options.py

Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

pyproject.toml

yt_dlp/plugins.py

Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>

Grub4K and others added 19 commits January 7, 2023 08:19

Rough demo

40f772c

Its lazy now at least?

8c297d1

Fix tests and add test for reloading

00709d5

Merge with 'upstream/master'

25b921b

Merge remote-tracking branch 'upstream/master' into misc/globals-and-…

3d939b6

…plugins-revive # Conflicts: # yt_dlp/YoutubeDL.py # yt_dlp/__init__.py # yt_dlp/options.py # yt_dlp/plugins.py # yt_dlp/postprocessor/__init__.py # yt_dlp/utils/_utils.py

fix

9f1f2c5

Get plugin overrides working

4266658

Make globals internal-only

413ae76

Fix reloading

cb9e38a

load plugins in youtubedl if they haven't already

3561d2a

minor refactoring

6547ff4

Add test for override plugins

42771dd

revert back to init_subclass, add guard against multiple imports of s…

a19dd28

…ame plugin

add test for plugin dirs

9269248

Decouple plugins.py from plugin types

21e13bf

Add public functions to add custom external plugin paths

109c019

Add --no-plugins

97684b0

Fix lazy extractors

9811496

minor refactoring

2723802

coletdjnz added the enhancement New feature or request label Oct 21, 2024

coletdjnz added 9 commits November 30, 2024 12:26

Merge remote-tracking branch 'upstream/master' into misc/globals-and-…

072e680

…plugins-revive # Conflicts: # test/test_plugins.py # yt_dlp/plugins.py

Cleanup after merge

2699951

Move away from contextvars

51f3740

may this help?

97088ae

fix

4e4bc2f

Fix no-external

d48d00d

fix no-external not including PYTHONPATH

37aed89

Avoid searching disk for plugins when plugins are disabled

600b2ec

Improve error message when invalid directory provided

cd490ee

coletdjnz changed the title ~~[plugins] Improvements WIP~~ [plugins] Load plugins on demand Nov 30, 2024

coletdjnz and others added 3 commits February 9, 2025 10:45

cleanup from review

e2883ff

Update yt_dlp/options.py

78bc4e7

Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>

Update README.md

eb6cba8

coletdjnz commented Feb 8, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

Rename --no-plugins to --no-plugin-dirs

b6c3d22

bashonly reviewed Feb 9, 2025

View reviewed changes

bashonly mentioned this pull request Feb 9, 2025

Fix ffmpeg when specifying its location #12318

Open

8 tasks

coletdjnz added 4 commits February 9, 2025 21:26

Implement _CLASS_LOOKUP in make_lazy_extractors.py

68644c3

fix for older python versions

8b8d6c5

ruff

d134dc3

use repr

4e7ee9f

pukkandan reviewed Feb 9, 2025

View reviewed changes

yt_dlp/options.py Outdated Show resolved Hide resolved

coletdjnz and others added 13 commits February 11, 2025 19:31

Autopep8 ignore lazy_extractors.py

a78576e

fix

782b70e

does this work now?

f31a5c2

or this?

c3211e4

Fix lazy_extractors generation

a209cb8

Add some more excludes for autopep8

d25341f

Update yt_dlp/options.py

7b31c53

Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>

Update yt_dlp/plugins.py

35c779d

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Update yt_dlp/plugins.py

92f0776

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Update yt_dlp/options.py

aff93ef

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Update yt_dlp/plugins.py

cf5bad8

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Update yt_dlp/plugins.py

914d554

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Update yt_dlp/plugins.py

f9ef24c

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

coletdjnz commented Feb 11, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

coletdjnz commented Feb 11, 2025

View reviewed changes

yt_dlp/plugins.py Show resolved Hide resolved

coletdjnz and others added 3 commits February 11, 2025 20:19

Apply suggestions from code review

635d550

Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>

formatting

86138e6

Fix --no-config-locations

2589ba4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[plugins] Load plugins on demand #11305

[plugins] Load plugins on demand #11305

coletdjnz commented Oct 21, 2024 •

edited

Loading

bashonly left a comment •

edited

Loading

[plugins] Load plugins on demand #11305

Are you sure you want to change the base?

[plugins] Load plugins on demand #11305

Conversation

coletdjnz commented Oct 21, 2024 • edited Loading

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

What is the purpose of your pull request?

bashonly left a comment • edited Loading

Choose a reason for hiding this comment

coletdjnz commented Oct 21, 2024 •

edited

Loading

bashonly left a comment •

edited

Loading