Significant performance drop when creating batch of pydantic models since pydantic 2.5.0

[g0di]

Hi Folks,

After updating some of my project dependencies I noticed a signifiant increase of my tests running time (went from 30s to more than a minute). After digging I discovered that this was due to the update of pydantic from 2.4.2 to 2.8.2, after refining it looks like the the performance dropped since version 2.5.0 of pydantic.

Starting from version 2.5.0 of pydantic, the performances of batch function of ModelFactory drastically dropped (~x5).

How to reproduce

Considering my system spec

{
  "implementation_name": "cpython",
  "implementation_version": "3.11.4",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.19045",
  "python_full_version": "3.10.11",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "win32"
}

Using the following benchmark script

# snippet.py
from timeit import timeit

from polyfactory.factories.pydantic_factory import ModelFactory
from pydantic import BaseModel


class Foo(BaseModel):
    id: int
    name: str


class FooFactory(ModelFactory[Foo]):
    pass


def create_pydantic_models() -> None:
    FooFactory.batch(1000)


print(timeit(create_pydantic_models, number=1))

I've picked a high number for batch size to make the time difference significant enough for readability

Run the following commands to setup two virtual env and run the benchmark for pydantic==2.5.0 and pydantic<2.5.0

python -m venv .venv-slow
.venv-slow/Scripts/python -m pip install "pydantic==2.5.0" polyfactory
python -m venv .venv-fast
.venv-slow/Scripts/python -m pip install "pydantic<2.5.0" polyfactory
.venv-slow/Scripts/python snippet.py
.venv-fast/Scripts/python snippet.py

On my side this is the output i've got

$ .venv-fast/Scripts/python snippet.py 
0.4096285999985412

$ .venv-slow/Scripts/python snippet.py 
2.147997599997325

So the difference is pretty huge.

Some tracks

After digging a bit more I discovered that the the ModelFactory.process_kwargs(...) function was probably responsible of those numbers. It looks like this function takes way more time with pydantic>=2.5.0. However, I did not dug more as this function was pretty complex to diagnosis for me for the moment.

Perhpas Pydantic change some stuff in his public API causing this performance issue? Maybe polyfactory should use some other api of pydantic? I can't tell.

[g0di]

By the way, if you provide values for model attributes, the performances are a lot better. In the previous benchmark script, if you change the foo objects creation with this

FooFactory.batch(1000, id=1, name="bar")

The performances are drastically better (below second)

[guacs]

Hey @g0di, thanks for reporting this! I was able to reproduce this. I'll look into this a bit more and let you know :)