Performance and maintainability: improve load time and add support for external plugins

[stefannibrasil]

This work is part of the larger ongoing work of improving faker’s performance and usability. We’re currently testing autoloading, with the ultimate goal of having users introduce their own Faker::Base subclasses in gems and have Faker users use them without modifying Faker.

Performance and discoverability been long running issues: #2689, #2939 and #2719

Why

faker ruby has lots of generators, which is good! People love using it and we want to make it even better (more performant, easier to use and maintain).

Some generators are really niche. Each generator may load a set of locale files, which can be slow. The burden of maintaining all these generators is also a problem.

Proposed Solutions

Our main goals are:

• Only load what one needs.
• Add support for external plugins, where the community can create their own plugins instead of merging into faker. This will ease the maintainability work.
• Make as fewer breaking changes as possible to make the changes easier to adopt.

Steps

To ensure we are implementing changes carefully, we want to break the work down into the following steps:

1 - Normalize YML files

Currently underway in #3061. Having all the YML files reformatted will help spotting any errors, and keep the files standardized.

2 - Add a benchmark CI workflow

Run a benchmark as part of our CI to ensure new code does not degrade performance.

3 - Benchmark autoload

Loading time is the main performance gain we want to improve. But before doing anything, we need to have baseline stats.

We have a benchmark task but we want to focus on the loading stats.

4 - Experiment with different loading mechanisms

To make users only pay for what they use, to avoid loading all generators by default, we need to experiment with different ways of loading the generators:

• Lazy load
• Zeitwerk

We want to run experiments to compare benchmarks and the pros and cons of each mechanism taking maintainability, fewer breaking changes, easier adoption factors as guiding points.

5 - Add support for external plugins

Once we have decided the solution from step 4, we can test creating external classes (categories as well).

6 - Launch a pre/beta release

To avoid too many disruptions, we want to have a way for the community to test the changes and report any bugs before releasing an official version.

7 - Release a major version

Everything going according to plan, we release a major version.

How you can help

We will share the results of each experiment before making a decision. A beta/pre release will be available for you to test and report any bugs.

We want to thank @jeremyevans for his mentorship on this project, and for everyone's participation on the related issues.