Idea: Use array type for embedding speed-up

[pamelafox]

Confirm this is a feature request for the Python library and not the underlying OpenAI API.

• This is a feature request for the Python library

Describe the feature or improvement you're requesting

The SDK currently uses numpy to speed up embedding:

            embedding.embedding = np.frombuffer(  # type: ignore[no-untyped-call]
                base64.b64decode(data), dtype="float32"
            ).tolist()

It does seem to improve performance, based on our tests, but we were wondering if similar gains could be made without numpy, using the built-in array type? Have you tried that already?

https://docs.python.org/3/library/array.html

We're having some pains with the numpy dependency for our Azure samples and are looking for ways to move off it without affecting performance.

Additional context

No response

[RobertCraigie]

I have not tested with the array type but I'm also curious if it could provide similar improvements. Would you be able to share some examples of what the changes we'd need to make would look like?

[tonybaloney]

Looking at what numpy is used for in the embeddings type, that is taking a base64 bytes object as a buffer (non-copy reference), then converting it into a compact float32 single-dimension array in numpy, then back out to a list of native floats, you can do that with the builtin array type:

embedding.embedding = array.array("f", base64.b64decode(data)).tolist()

Have submitted this in a draft PR

[tonybaloney]

Benchmark:

import array
import base64
import numpy as np
import json

# Sample data
data = 'yIBZu6KsajycyTO86gLxujWCdbzGqY08VDmHvBE6X7wFrhe8I3v9vCBexTyVoJA87dS1PPEax7pjqGA7zkb9u85G/TzGxqC72mo+PM5G/bt+O3u8E6mkPBg1orsTjBG7NVSVOynZmjxv6bQ7hPDRu+iOpLvYB78696ZEPBAAuTqRlCW9C5FOOsRGjrxzErO8ktoRvE146ruWIKM8MAI+vd6wTzzm5bg8p+GuvHW7HrwhpDG7fQFVPP8y5zueVQy8wpHcvOiOJDxBzw+7OX0TPHiBHbyTa/G762Xwu8RGDrwIrry6irZ1u8ypVzzon/G7I3v9O5X3STz/Mmc6sJaquxDXX7vQ0lW8SL4MPHWeC7xa82S81pj5uoaZvTx7gcK73xNPOxKd3jvw8e08ZAvgu5eU77xNPkQ886afPKQP6jvjE/Q8WZwrvTErl7vocZE8S76xPCb2CDwjarC8rZYFPbdolLwPnbk79myeO9OHBz0e3jI7ljHwuzaagbzgAgK5MB/RuiMwijwKOhW8K835vJ+sxTvnPPK7/T4IvT43cbse+0U82STSvNB7HLzrVKM872COPLCKZLzkdnM837yVPPDDjTzgPCi9VvO/untHHLsuSIU8lJRKvDKOFrwGERe8uT9gPAzXujwamKE8Tmedu8odf7xdqBa8jzEmum1d3DrU0vq7eeScOyofBz16HsM86gLxPH9TB7xNSgq9sIrkPJc9NjwsvKy7MB/RvC0Cmby13Ls65EiTPNB7nLvD18g7dFifPOQ8zTwl3nw8iV+8uj3DJDvMUh68wBHKPJXOcDlqhhA9kaXyuKg4aDyaIEg81KSaO0SVjjz4Q+q8DQCUvIrCuzyhVbE8K3ZAuD4JEbyKwru6CejiPOnUED3D9Fu6Vy3mOykwVLwhpDE7DK7hPFtiqrxsBiM9tegBu30NGzxHW4088RpHvLuREr03/YA6ctgMurSWTzwBhRk96GXLPAbovTvek7w8MoLQO3+e+rzKDDK7NpqBudZeUzu56KY72ROFvM9eibxRSi+/CcvPvMhGszsyjha8AZbmPJ5VDD1xhtq7FO8QvKJyxDsycQM9Jd78vFh/mDsiB7E7nlUMuwL55btpXbe7C4ABPdpqvryA8Ky761SjPG/ptDvODNc7w+MOvFy54zp+ZNS5Y3oAPFf/BbvgAoI7BvSDOnnHiTyx0FC7ytKLPFKtLrysUBk8p/5BPZosDjtpIxE6mKC1PFRWGj3TmFQ9MnEDvQBcwLuH8HY8MoLQOyXefLzCgI+7GW/IPAo6lTybg0e518FSvK75hDu8BV88fjt7PCuCBjzdPIO8+wnpPP6hhzyuJ+U7TXjqOtkTBbwKLk+8IHtYOwYFUbxULUE8B2jQvIxCTjtOocO7WJBlPK7tvjwv9ne8W2KqPEd4oDyu7T68WvNkPFQtQTyhVTE8NuX0O4vfzrtmUUy8/qEHPLwFX7ypm+e7sdwWvX47ezhjegA90m97PIqIFb3dTdC7TXhqu9jqq7xT1oc8B4XjPCkH+7xWyuY7//hAvC0w+Tv/BAe8F9IiPe+aND34CcS8sIpkPNXqBjrOGB28S76xu7kRADwlB9Y8SNufPPgVijt55Jw8OBoUvP1bG7zUpJq8ioiVPGI0FDwqWS27sjPQvKwzhjwz5c+7se1jPPlPsLz5Mh08yEazu1tiKjzRwYg7miBIPC5IhTmKiBW9IHvYvEImSbwLgAG83U3QuzFIqjypbQc9HbXZuTFlPbtVnIa7AwUsPB8HDDyk/pw8DRHhu5Nr8bwYGA88JepCvNaYebyKwrs7AYWZvB8Y2bzPb1a7PaaRO7wF3zyZ9268L5++vFr/Krxe7oK8KdmaPDxgpbv72wi9OuCSu+MT9LtCMg+9CdcVPbk/YDytxGW6gPCsu2X6Ejw2mgG9WsWEO+bluDwpMNS8/XiuvNJBG7yNa6e8qZvnuyHBxLom9oi8qDjoOwpL4rwfGNm8oUlruuoC8bvORn27kghyPHyNCDzrN5C8sb+DPIlfPLvnSLg8HEGNPNeHrLz3sgq8KM3UPCSk1jz/+EA8DmMTvBaMtrpHssa8oUlrOxmM2zxWymY8DnRgPG1dXDy8BV+7WFY/O/1s6LyE8NE8FsbcvM3SsDta4he74XbOO/8Ehzq0hYK8VxyZO8lvjDwm9gi9745uPLNcqTyNaye8aV23PA+AJjxgtIG8OatzOv8Eh7wGImQ8GpghvE5nnTwatbQ8QGwQPKaniDwWqUm7TBVru5ogyLwamKE6OBqUPHh1VzxVkEA8Uq0uu4qIFTxP2+m74x86PKg4aDt6Kok8fQHVu8KAjzuyM1C8UwRoPM9vVjw2q848uL9Nu/JDoLxff+I7sbO9OVbK5rt52Na8RMPuPKJyxDu7riW8L8iXPO5USDp+ZNQ8E+PKPD79SjtiRWE5mKC1Ox21WTz1JrI8p+EuuyufGbwjMAo8CehiPOR287zSNdW8e3V8u71LSzi0vyi8Ye6nPHnHiTzYzRi9C4CBPBv7oDwfJJ881N7AvGTRuby8Bd+6MB/RO3ye1bmc5sY7+OywvEd4oDsmE5y7mQO1PDFlPbvUpJq7wVe2u3YS2LuoJxu8+BWKvJ5ynzzD4w68iFN2uxrv2rzgAoI80m97u9kkUrswDgS8c0zZPKQParnnDhK87Lciup+4CzuH36k8aTRevAdoULx3WES83/Y7PDf9gDqDmRi8QcPJvBbGXLxa8+Q8Ht6yOyjNVLyx3Ja8DK7hu9HemzzJqbI9ToSwPKrQhrzesE88gRmGPDQOqbz6sq+8UXMIvA30zTzdPIO8gnC/uyYwL7wvvNE7o9XDu3nY1jwosEG8HZjGO+QrgLzen4I7JbAcunQ7jLsSgMs6uK6AuFhiBTzgAoK8MA6EO3tHHLzTh4c8mlruPDbl9LxWyuY6AD8tPDWCdTxo+rc7eJ6wPD43cTxNPkS7zIB+PIlrAj1AmvC8745uPL/Xozutij87YdGUO2ZRzDzK41g75EiTvJdaSTyCcL87YkXhvHnHCT19DRu8oPIxvIfwdjsiJMQ7hPDRPAyuYbwdpAy72naEvDsaubs+/Uq8Mrx2vKVy6bta/yq8ybr/Onye1by5EYC8QcNJvEUm7rvsmo+8TmedvASFvrzdWZa83VkWu3o7VjwkpNY8c0zZOw0R4bsVRko7woAPPKg4aLxo3SS8OWCAO5X3ybyaSaE7YLQBPNZNhjyDfAW8IGoLPNleeDz2fWs8pXJpOx9BsruMTpQ7W0WXPN6wz7tmbl88S6EePSz2UjwqPBq7kz0RPb1Ly7xXHBm7VzksOyt2QDz0Guy7j0LzPE/KHLk7N8y8nSDtvHWv2DzTe8G7byNbPHW7HjwKOhU8N/2APLXcuzwcQQ08HbVZu7UWYry0s+K7JvaIvLkizTrdTdA84WUBvSC1fjwMrmG7dDuMvN/2u7vLNYs7o7gwPC5ZUju9ESW6anrKu6VhHL3inye8FqnJuo9CczzBOqO8iogVuoAB+rzTXq66zG8xPF3FKb0uk3g8v8vdvIgZULsfUv+704cHvCIHMTzOGB28DnTguxxS2rzTe0G8MUiquzxDEr2nxBs8zIxEvAcuKj0/JqQ8ytILPUJg77sWb6M74p+nPIfw9rxy2Ay8H0EyO1bKZrzTh4e8l2YPvKKs6jx0hv87ZAvgPAdLvTwQ46W8qAoIPPHgoLqx7WM7TmcdvEdbjbyWA5A8jrETvEHPjzxBw8m74E31u1FziLvEOkg8N/0APccMDT1z9R88WFa/PLHtYzvah9G796ZEO/OJDLyhSeu8ebtDOTACPjzc2QM8y0bYPDxxcrqOlIA8RMNuvMnGxTx29cS7Fm+jOydZiLvrNxC8vRElO13/z7w4Nye8IuodvVyLg7z2bJ68FUbKOy+fvjwJ9Ci8E+NKPGqGELzSQRu9xymgu5kDNTqvUD49l5RvvAiuPDysUJk8S7JruyzlBTuYg6I8BGgrPJGlcrr6pmk6KLyHPLHtY7t8ntW7ebvDvAn0qLvCkdy8opsdvJu97TwEhb67a8A2upGUJbwUAF68OBoUuNUkLTwvyJe8QibJuy5Z0ryrJ8A6HF4gvdC1Qjzc2YO8WznRO1bK5rzQqfy7+AnEO8iAWTxYVr+8nfKMu5ysoLwdtVk8r1C+vMcd2rpr+lw8/WzoOiYTnLwWb6O8UATDux8Y2TziyAC8oTievD79yrts6Y888MONPDoOc7wycYM6kc7LvJX3ybsFv2Q8cC+hPOFlgbv7+Ju7ETpfOwdoULxGMrS6rFAZu0wVa7xUOQc8se3jvDErl7stPD+8UwRouxQMpDv9W5s8RezHOpSUSrtKPp88X1ECvafEmzwT48q8EOMlPLqFzLz72wg863E2PJ+sxTsxWXc8noNsvO3UNblpNN68pXLpu7jLk7tT85o8MWW9vEuy6ztXLWY7UWfCO1y547xI+LK6yW8MPeQrgLiUd7c81k0GPVQtwbzrZfA6cq+zO01KCjxfUYI8nSyzvPfPnbvCnSK8fNj7PMmpsjvV6oa82Pt4vJpJoboIdJa8SOxsPKO4sLy/y127LOWFPE1KijtP56+8eHVXPDVxKLxElY68Nw5OvA5XTbscQQ29xymgvEBskLs91HE7BZGEPJHOS7yeSca8KjwavBTvELyIJZa7fkfBPAyuYTmO6zk84B+VvG/MITyGX5c8t6I6vKbV6DzWTYa8FUZKvBbGXLxEsqG7GYxbu83jfbse+0U66chKvBvejTu+kbc8Jd78O2JRJzwxWXe81ph5O30B1bsp2Zq8Is0KPMyp17s91HE8DK7hvA+6TLxoF0s86dSQvAA/rbxlF6a7wB2QPBQA3rp8jQg8n4+yvFME6LovvFG8/s/nvGOXEzwC+WW8bEDJPI7OpjwfQTK8jrETvLk/YDt3OzE8UWdCvEd4oLxv6bQ7VZyGvPJgszsrzfm8MSuXuwBcwLzSb3s76I6kvCdqVbwUDKS8BIW+PDWOO7xKPp+87ivvuk6EsLxYkOU6zvuJN2CoO7zHAEc8pP6cO0/nr7xjbjq8MWW9uucrpbyoYUG8/T6IPLjc4LvODNc8k2txPn3kwbtEsiE8yzULPROMEbu1FuI8wwCiPE+tibuJawK8ER3MPHceHruu0Cs8FO+QvHkS/Tsv9nc82mo+PNvq0LsC6Bg6YIsovCi8B72x7WO8HFLau6KbnbwZe4671ftTPCSTiTzDACK8RM+0O9ZNBj0LgIE8pDhDvGJF4brMUh68RjK0PCkwVLzenwI8ioiVPKhErjw/JqQ7PHFyu2qjozyhSes6ptXouw5jk7z+vho8uoXMPOzIbzzi2U07I0FXvM3j/TyWMfC8HEENO/OJDDyZ9+48UWfCuwyuYTx4ddc8OzdMO28jWzx7Rxw8L8iXO0Um7jy7ot+8NY67O9JBG7zJun88bwZIvDKOljsyjha8yylFPCuTU7x8jYi6sIpkuy08vzt0WB+74E31vD9DNzznPHI8f576O4glljwOOrq8/T6IPOcrJbyE/Je8YOJhvKFVsbz72wg8g5mYugGW5rxr+lw7LQIZuV9RAr1yr7O6oA/Fu6vtGTuMfPQ65YI5POif8TwKHQI77JoPvIwxAb2D0z69t4UnPGqXXTtWyuY7DNc6vBOppDs3/YC76itKOsRXW7zFnUe8Q08iOS0fLDvcJPc7LR8sPL50JLyE/Be7XiipvFc5LLxy2Aw6QmBvOtqH0bxyr7O8uL9NPCZB/Dld/8+82M0YPMrjWDzERg682TAYveQrgDxmXZK8rYo/PBrv2jpLoR684p+nO76RN7zXwVK8hPDRO9U1ejxmUUw8K5PTOwA/LbxP22k8pBswPH9TB71JT2w85zzyu6rEwLzu/Y68tlzOvFmcqzuWMfC7yGPGuxxBDT1fUQK9rxaYvAdo0Lxa4pc7t3lhPCBqC70xKxe81N5AOki+DLytxGW80HucvG/MIb7YBz+728F3PDDlKjxYkGW8KRNBvETPND3/+MC8fNj7vAWRBLxBw0k8sIrkucQdNb1o+rc7IGqLvEwEnrqtxOW8hl8XPEHPjzwuSAU7EmM4PLjLk7s8YKU8tlzOuzx9ODu8y7g8tflOvEBsEDxbOdG7CHSWvGk03ryRpXK8I0HXu8AuXbwxZT27DkYAPF4c47s8cfI6ktqRvJGl8jzbzb08SPgyPfsJ6TyRpXI8p8Qbu3ye1TsuWdK6Mo6WPASFvjuWIKO7UBCJvAn0KLw5t7k8xqkNvMhGszsPnbk7H0Gyu82Yijxd4jy8mkkhPKEbi7z5T7C8PuA3O4Z8KjvCgA+9zKlXvIDwrLwOV808S4QLvR8YWbwBlma7JQfWu1A+aTzPb1Y8+rIvvMP0WzxeC5a8aV03O3YSWDsWxtw7+9sIvAnoYjwVRsq7xEYOvPUJHzxKPh+761Sjub/LXTstHyw7sdwWOxVGyruYvUi9oNWeu2qjo7zojqQ7F9KiPG1dXDxIvow7OavzO8FXtjvm5bi8xwBHvJogyLzhZQE7q+0ZPGJFYTyA8Kw8BvSDPIU2vjwosMG5ZO7Mu2CouzyFGas7s3m8PEkhDLukG7C6+wlpO9ZeU7sa79o639kovAo6lT0fQbI5K3bAvBKd3rvEHTW8YMXOPCNB170fUv+8uNzgO0dbDTyS2pG7sb+DPDU3ArxaxQQ8eeQcPP2VwTx82Ps7CboCu48xJjmkD+q872AOPYN8BTzThwc9DkYAvGCou7x4r308RRWhuy6CKzzHDA25DfTNvF5FvLy5EYC797IKvRQpNzxzEjM70jXVuz9gSjxHska8YKi7OxVjXbp/Uwc8K5+ZPH47+7p98Ie7DldNPDfxOrwfGFk6DQAUucMAIjxGMrS8HvvFPOcrJTuAAXo8M+VPPOoCcbttXVy56ivKvANcZToHS728gSpTvPkyHT3gTfU7NuV0PJzJszzMUh48AaKsPNjNmDwWxly8+8/Cu0i+DD2k4Qk9oTieu6APxbxHssY64wKnO7IWPbtYkOW8K58ZPY8xJrx3O7E8vMs4vJosDrt9AVW8mkkhvfqVnDxh0RS886afvBDGkrzsmo+7HvvFvO5UyDzm5Tg8uT9gvAMivzxJT+y7sb8DPMrj2Ds0SE887dQ1PBPGN73Gutq8L7xRPPqmaTug5mu7ESkSu3iesDyPFJO8rFCZvAydFL0DLoU8sdyWvF3/T7y89JE8pP6cPHYBi7xKPp+8qsRAvKmb57s1jju8OZomPKgnm7yA8Kw7IaQxPPfPnbzkKwA66as3vBg1Iros2b863xNPPIelg7v9lcE8AuiYu4C2BrxnwBE6xB21uwm6gjwN4wC94rw6vDv9Jbxgiyi9XeI8Ol9izzzBdEk5syKDvPjsMLol3vy7jHx0O4KNUjwX7zW8+wnpvCNNnTv1CR+9rDMGvRxBDTsRKZI7m4PHudjqqzxCJsk64+WTPDErlzzU3kC8Qgk2vH47ezjWTQa9sz8WPC5lGLyvM6s6on4Kvf7PZzyZ5qG7Z5c4PBw1xzw91HE8W0WXu5GxuDrm5bi8QIkjPMD0trzYsAW8xsagPMbGIDvbk5c7Why+OwGFmTrvjm67732hO3oBsLxmUcw7ppvCPA5GALxMFWu79QmfPHdMfjzah9E7P0O3vFAtnLzY6qu7bAajPOblOLxVnAa8HsEfvLGzPbqtij+8eK99O89Sw7xqhhA8mIMiuykwVDw25XQ7f3AaOySkVry2XM687lRIu17uAjxAbBC8UsrBvCLqnTpYkGU7988duuGClLyoJxu87lRIPAt0O7wtMPk85DzNt5zJs7w5YAC972AOPSNBVzzEV9u66I6kPDFlPbz34Go85silO+oCcTzSb/u86J/xvPOmnzuWAxC89mweur+6EDtKPh88L7zRu1r/qrxT8xo8iWsCPUTPtLu5Ik09SOzsPI8Uk7zYzRi7xFfbO0wEnjvIgFk8LoKrPCWwHL1DiUi8GDWiOwSFvrtElY48LlnSvNHem7zY+/i8Zm7fvC6TeLxoF8u7rZaFvCuT0zxB7KK6FVIQPN2HdjsqH4e8JhOcvKanCD3EV9s5edhWu6J+irwwH1G6fjv7OkUm7ryA8Cy8BiLkPAm6grt3Hp48W1ZkPD438Tz9bGi75CuAvBw1RzwlB1a8B0u9uyb2iDxmeiU9DeMAvdjNmDwESxi9'

as_json = json.dumps(array.array("f", base64.b64decode(data)).tolist())

def bench_standard():
    for _ in range(1000):
        json.loads(as_json)

def bench_array():
    for _ in range(1000):
        array.array("f", base64.b64decode(data)).tolist()

def bench_numpy():
    for _ in range(1000):
        np.frombuffer(  # type: ignore[no-untyped-call]
                        base64.b64decode(data), dtype="float32"
                    ).tolist()

__benchmarks__ = [
    (bench_standard, bench_array, "Standard vs array"),
    (bench_standard, bench_numpy, "Standard vs numpy"),
    (bench_numpy, bench_array, "Array vs numpy"),
]

Replace json with a more efficient encoder (orjson)

Results show this array approach is equivalent to the numpy one (10-20% faster) and is significantly faster than the standard approach (10x):

Benchmark	Min	Max	Mean	Min (+)	Max (+)	Mean (+)
Standard vs array	3.115	3.388	3.223	0.225 (13.8x)	0.256 (13.2x)	0.242 (13.3x)
Standard vs numpy	2.941	3.330	3.135	0.263 (11.2x)	0.306 (10.9x)	0.280 (11.2x)
Array vs numpy	0.256	0.273	0.264	0.218 (1.2x)	0.227 (1.2x)	0.222 (1.2x)

[tonybaloney]

Sorry, I'm reading my own benchmark data wrong. It's faster than numpy

[tonybaloney]

Benchmark comparing the pydantic parser which openai uses to the array and numpy approaches:

import array
import base64
import numpy as np
import json
import pydantic

# Sample data
data = 'yIBZu6KsajycyTO86gLxujWCdbzGqY08VDmHvBE6X7wFrhe8I3v9vCBexTyVoJA87dS1PPEax7pjqGA7zkb9u85G/TzGxqC72mo+PM5G/bt+O3u8E6mkPBg1orsTjBG7NVSVOynZmjxv6bQ7hPDRu+iOpLvYB78696ZEPBAAuTqRlCW9C5FOOsRGjrxzErO8ktoRvE146ruWIKM8MAI+vd6wTzzm5bg8p+GuvHW7HrwhpDG7fQFVPP8y5zueVQy8wpHcvOiOJDxBzw+7OX0TPHiBHbyTa/G762Xwu8RGDrwIrry6irZ1u8ypVzzon/G7I3v9O5X3STz/Mmc6sJaquxDXX7vQ0lW8SL4MPHWeC7xa82S81pj5uoaZvTx7gcK73xNPOxKd3jvw8e08ZAvgu5eU77xNPkQ886afPKQP6jvjE/Q8WZwrvTErl7vocZE8S76xPCb2CDwjarC8rZYFPbdolLwPnbk79myeO9OHBz0e3jI7ljHwuzaagbzgAgK5MB/RuiMwijwKOhW8K835vJ+sxTvnPPK7/T4IvT43cbse+0U82STSvNB7HLzrVKM872COPLCKZLzkdnM837yVPPDDjTzgPCi9VvO/untHHLsuSIU8lJRKvDKOFrwGERe8uT9gPAzXujwamKE8Tmedu8odf7xdqBa8jzEmum1d3DrU0vq7eeScOyofBz16HsM86gLxPH9TB7xNSgq9sIrkPJc9NjwsvKy7MB/RvC0Cmby13Ls65EiTPNB7nLvD18g7dFifPOQ8zTwl3nw8iV+8uj3DJDvMUh68wBHKPJXOcDlqhhA9kaXyuKg4aDyaIEg81KSaO0SVjjz4Q+q8DQCUvIrCuzyhVbE8K3ZAuD4JEbyKwru6CejiPOnUED3D9Fu6Vy3mOykwVLwhpDE7DK7hPFtiqrxsBiM9tegBu30NGzxHW4088RpHvLuREr03/YA6ctgMurSWTzwBhRk96GXLPAbovTvek7w8MoLQO3+e+rzKDDK7NpqBudZeUzu56KY72ROFvM9eibxRSi+/CcvPvMhGszsyjha8AZbmPJ5VDD1xhtq7FO8QvKJyxDsycQM9Jd78vFh/mDsiB7E7nlUMuwL55btpXbe7C4ABPdpqvryA8Ky761SjPG/ptDvODNc7w+MOvFy54zp+ZNS5Y3oAPFf/BbvgAoI7BvSDOnnHiTyx0FC7ytKLPFKtLrysUBk8p/5BPZosDjtpIxE6mKC1PFRWGj3TmFQ9MnEDvQBcwLuH8HY8MoLQOyXefLzCgI+7GW/IPAo6lTybg0e518FSvK75hDu8BV88fjt7PCuCBjzdPIO8+wnpPP6hhzyuJ+U7TXjqOtkTBbwKLk+8IHtYOwYFUbxULUE8B2jQvIxCTjtOocO7WJBlPK7tvjwv9ne8W2KqPEd4oDyu7T68WvNkPFQtQTyhVTE8NuX0O4vfzrtmUUy8/qEHPLwFX7ypm+e7sdwWvX47ezhjegA90m97PIqIFb3dTdC7TXhqu9jqq7xT1oc8B4XjPCkH+7xWyuY7//hAvC0w+Tv/BAe8F9IiPe+aND34CcS8sIpkPNXqBjrOGB28S76xu7kRADwlB9Y8SNufPPgVijt55Jw8OBoUvP1bG7zUpJq8ioiVPGI0FDwqWS27sjPQvKwzhjwz5c+7se1jPPlPsLz5Mh08yEazu1tiKjzRwYg7miBIPC5IhTmKiBW9IHvYvEImSbwLgAG83U3QuzFIqjypbQc9HbXZuTFlPbtVnIa7AwUsPB8HDDyk/pw8DRHhu5Nr8bwYGA88JepCvNaYebyKwrs7AYWZvB8Y2bzPb1a7PaaRO7wF3zyZ9268L5++vFr/Krxe7oK8KdmaPDxgpbv72wi9OuCSu+MT9LtCMg+9CdcVPbk/YDytxGW6gPCsu2X6Ejw2mgG9WsWEO+bluDwpMNS8/XiuvNJBG7yNa6e8qZvnuyHBxLom9oi8qDjoOwpL4rwfGNm8oUlruuoC8bvORn27kghyPHyNCDzrN5C8sb+DPIlfPLvnSLg8HEGNPNeHrLz3sgq8KM3UPCSk1jz/+EA8DmMTvBaMtrpHssa8oUlrOxmM2zxWymY8DnRgPG1dXDy8BV+7WFY/O/1s6LyE8NE8FsbcvM3SsDta4he74XbOO/8Ehzq0hYK8VxyZO8lvjDwm9gi9745uPLNcqTyNaye8aV23PA+AJjxgtIG8OatzOv8Eh7wGImQ8GpghvE5nnTwatbQ8QGwQPKaniDwWqUm7TBVru5ogyLwamKE6OBqUPHh1VzxVkEA8Uq0uu4qIFTxP2+m74x86PKg4aDt6Kok8fQHVu8KAjzuyM1C8UwRoPM9vVjw2q848uL9Nu/JDoLxff+I7sbO9OVbK5rt52Na8RMPuPKJyxDu7riW8L8iXPO5USDp+ZNQ8E+PKPD79SjtiRWE5mKC1Ox21WTz1JrI8p+EuuyufGbwjMAo8CehiPOR287zSNdW8e3V8u71LSzi0vyi8Ye6nPHnHiTzYzRi9C4CBPBv7oDwfJJ881N7AvGTRuby8Bd+6MB/RO3ye1bmc5sY7+OywvEd4oDsmE5y7mQO1PDFlPbvUpJq7wVe2u3YS2LuoJxu8+BWKvJ5ynzzD4w68iFN2uxrv2rzgAoI80m97u9kkUrswDgS8c0zZPKQParnnDhK87Lciup+4CzuH36k8aTRevAdoULx3WES83/Y7PDf9gDqDmRi8QcPJvBbGXLxa8+Q8Ht6yOyjNVLyx3Ja8DK7hu9HemzzJqbI9ToSwPKrQhrzesE88gRmGPDQOqbz6sq+8UXMIvA30zTzdPIO8gnC/uyYwL7wvvNE7o9XDu3nY1jwosEG8HZjGO+QrgLzen4I7JbAcunQ7jLsSgMs6uK6AuFhiBTzgAoK8MA6EO3tHHLzTh4c8mlruPDbl9LxWyuY6AD8tPDWCdTxo+rc7eJ6wPD43cTxNPkS7zIB+PIlrAj1AmvC8745uPL/Xozutij87YdGUO2ZRzDzK41g75EiTvJdaSTyCcL87YkXhvHnHCT19DRu8oPIxvIfwdjsiJMQ7hPDRPAyuYbwdpAy72naEvDsaubs+/Uq8Mrx2vKVy6bta/yq8ybr/Onye1by5EYC8QcNJvEUm7rvsmo+8TmedvASFvrzdWZa83VkWu3o7VjwkpNY8c0zZOw0R4bsVRko7woAPPKg4aLxo3SS8OWCAO5X3ybyaSaE7YLQBPNZNhjyDfAW8IGoLPNleeDz2fWs8pXJpOx9BsruMTpQ7W0WXPN6wz7tmbl88S6EePSz2UjwqPBq7kz0RPb1Ly7xXHBm7VzksOyt2QDz0Guy7j0LzPE/KHLk7N8y8nSDtvHWv2DzTe8G7byNbPHW7HjwKOhU8N/2APLXcuzwcQQ08HbVZu7UWYry0s+K7JvaIvLkizTrdTdA84WUBvSC1fjwMrmG7dDuMvN/2u7vLNYs7o7gwPC5ZUju9ESW6anrKu6VhHL3inye8FqnJuo9CczzBOqO8iogVuoAB+rzTXq66zG8xPF3FKb0uk3g8v8vdvIgZULsfUv+704cHvCIHMTzOGB28DnTguxxS2rzTe0G8MUiquzxDEr2nxBs8zIxEvAcuKj0/JqQ8ytILPUJg77sWb6M74p+nPIfw9rxy2Ay8H0EyO1bKZrzTh4e8l2YPvKKs6jx0hv87ZAvgPAdLvTwQ46W8qAoIPPHgoLqx7WM7TmcdvEdbjbyWA5A8jrETvEHPjzxBw8m74E31u1FziLvEOkg8N/0APccMDT1z9R88WFa/PLHtYzvah9G796ZEO/OJDLyhSeu8ebtDOTACPjzc2QM8y0bYPDxxcrqOlIA8RMNuvMnGxTx29cS7Fm+jOydZiLvrNxC8vRElO13/z7w4Nye8IuodvVyLg7z2bJ68FUbKOy+fvjwJ9Ci8E+NKPGqGELzSQRu9xymgu5kDNTqvUD49l5RvvAiuPDysUJk8S7JruyzlBTuYg6I8BGgrPJGlcrr6pmk6KLyHPLHtY7t8ntW7ebvDvAn0qLvCkdy8opsdvJu97TwEhb67a8A2upGUJbwUAF68OBoUuNUkLTwvyJe8QibJuy5Z0ryrJ8A6HF4gvdC1Qjzc2YO8WznRO1bK5rzQqfy7+AnEO8iAWTxYVr+8nfKMu5ysoLwdtVk8r1C+vMcd2rpr+lw8/WzoOiYTnLwWb6O8UATDux8Y2TziyAC8oTievD79yrts6Y888MONPDoOc7wycYM6kc7LvJX3ybsFv2Q8cC+hPOFlgbv7+Ju7ETpfOwdoULxGMrS6rFAZu0wVa7xUOQc8se3jvDErl7stPD+8UwRouxQMpDv9W5s8RezHOpSUSrtKPp88X1ECvafEmzwT48q8EOMlPLqFzLz72wg863E2PJ+sxTsxWXc8noNsvO3UNblpNN68pXLpu7jLk7tT85o8MWW9vEuy6ztXLWY7UWfCO1y547xI+LK6yW8MPeQrgLiUd7c81k0GPVQtwbzrZfA6cq+zO01KCjxfUYI8nSyzvPfPnbvCnSK8fNj7PMmpsjvV6oa82Pt4vJpJoboIdJa8SOxsPKO4sLy/y127LOWFPE1KijtP56+8eHVXPDVxKLxElY68Nw5OvA5XTbscQQ29xymgvEBskLs91HE7BZGEPJHOS7yeSca8KjwavBTvELyIJZa7fkfBPAyuYTmO6zk84B+VvG/MITyGX5c8t6I6vKbV6DzWTYa8FUZKvBbGXLxEsqG7GYxbu83jfbse+0U66chKvBvejTu+kbc8Jd78O2JRJzwxWXe81ph5O30B1bsp2Zq8Is0KPMyp17s91HE8DK7hvA+6TLxoF0s86dSQvAA/rbxlF6a7wB2QPBQA3rp8jQg8n4+yvFME6LovvFG8/s/nvGOXEzwC+WW8bEDJPI7OpjwfQTK8jrETvLk/YDt3OzE8UWdCvEd4oLxv6bQ7VZyGvPJgszsrzfm8MSuXuwBcwLzSb3s76I6kvCdqVbwUDKS8BIW+PDWOO7xKPp+87ivvuk6EsLxYkOU6zvuJN2CoO7zHAEc8pP6cO0/nr7xjbjq8MWW9uucrpbyoYUG8/T6IPLjc4LvODNc8k2txPn3kwbtEsiE8yzULPROMEbu1FuI8wwCiPE+tibuJawK8ER3MPHceHruu0Cs8FO+QvHkS/Tsv9nc82mo+PNvq0LsC6Bg6YIsovCi8B72x7WO8HFLau6KbnbwZe4671ftTPCSTiTzDACK8RM+0O9ZNBj0LgIE8pDhDvGJF4brMUh68RjK0PCkwVLzenwI8ioiVPKhErjw/JqQ7PHFyu2qjozyhSes6ptXouw5jk7z+vho8uoXMPOzIbzzi2U07I0FXvM3j/TyWMfC8HEENO/OJDDyZ9+48UWfCuwyuYTx4ddc8OzdMO28jWzx7Rxw8L8iXO0Um7jy7ot+8NY67O9JBG7zJun88bwZIvDKOljsyjha8yylFPCuTU7x8jYi6sIpkuy08vzt0WB+74E31vD9DNzznPHI8f576O4glljwOOrq8/T6IPOcrJbyE/Je8YOJhvKFVsbz72wg8g5mYugGW5rxr+lw7LQIZuV9RAr1yr7O6oA/Fu6vtGTuMfPQ65YI5POif8TwKHQI77JoPvIwxAb2D0z69t4UnPGqXXTtWyuY7DNc6vBOppDs3/YC76itKOsRXW7zFnUe8Q08iOS0fLDvcJPc7LR8sPL50JLyE/Be7XiipvFc5LLxy2Aw6QmBvOtqH0bxyr7O8uL9NPCZB/Dld/8+82M0YPMrjWDzERg682TAYveQrgDxmXZK8rYo/PBrv2jpLoR684p+nO76RN7zXwVK8hPDRO9U1ejxmUUw8K5PTOwA/LbxP22k8pBswPH9TB71JT2w85zzyu6rEwLzu/Y68tlzOvFmcqzuWMfC7yGPGuxxBDT1fUQK9rxaYvAdo0Lxa4pc7t3lhPCBqC70xKxe81N5AOki+DLytxGW80HucvG/MIb7YBz+728F3PDDlKjxYkGW8KRNBvETPND3/+MC8fNj7vAWRBLxBw0k8sIrkucQdNb1o+rc7IGqLvEwEnrqtxOW8hl8XPEHPjzwuSAU7EmM4PLjLk7s8YKU8tlzOuzx9ODu8y7g8tflOvEBsEDxbOdG7CHSWvGk03ryRpXK8I0HXu8AuXbwxZT27DkYAPF4c47s8cfI6ktqRvJGl8jzbzb08SPgyPfsJ6TyRpXI8p8Qbu3ye1TsuWdK6Mo6WPASFvjuWIKO7UBCJvAn0KLw5t7k8xqkNvMhGszsPnbk7H0Gyu82Yijxd4jy8mkkhPKEbi7z5T7C8PuA3O4Z8KjvCgA+9zKlXvIDwrLwOV808S4QLvR8YWbwBlma7JQfWu1A+aTzPb1Y8+rIvvMP0WzxeC5a8aV03O3YSWDsWxtw7+9sIvAnoYjwVRsq7xEYOvPUJHzxKPh+761Sjub/LXTstHyw7sdwWOxVGyruYvUi9oNWeu2qjo7zojqQ7F9KiPG1dXDxIvow7OavzO8FXtjvm5bi8xwBHvJogyLzhZQE7q+0ZPGJFYTyA8Kw8BvSDPIU2vjwosMG5ZO7Mu2CouzyFGas7s3m8PEkhDLukG7C6+wlpO9ZeU7sa79o639kovAo6lT0fQbI5K3bAvBKd3rvEHTW8YMXOPCNB170fUv+8uNzgO0dbDTyS2pG7sb+DPDU3ArxaxQQ8eeQcPP2VwTx82Ps7CboCu48xJjmkD+q872AOPYN8BTzThwc9DkYAvGCou7x4r308RRWhuy6CKzzHDA25DfTNvF5FvLy5EYC797IKvRQpNzxzEjM70jXVuz9gSjxHska8YKi7OxVjXbp/Uwc8K5+ZPH47+7p98Ie7DldNPDfxOrwfGFk6DQAUucMAIjxGMrS8HvvFPOcrJTuAAXo8M+VPPOoCcbttXVy56ivKvANcZToHS728gSpTvPkyHT3gTfU7NuV0PJzJszzMUh48AaKsPNjNmDwWxly8+8/Cu0i+DD2k4Qk9oTieu6APxbxHssY64wKnO7IWPbtYkOW8K58ZPY8xJrx3O7E8vMs4vJosDrt9AVW8mkkhvfqVnDxh0RS886afvBDGkrzsmo+7HvvFvO5UyDzm5Tg8uT9gvAMivzxJT+y7sb8DPMrj2Ds0SE887dQ1PBPGN73Gutq8L7xRPPqmaTug5mu7ESkSu3iesDyPFJO8rFCZvAydFL0DLoU8sdyWvF3/T7y89JE8pP6cPHYBi7xKPp+8qsRAvKmb57s1jju8OZomPKgnm7yA8Kw7IaQxPPfPnbzkKwA66as3vBg1Iros2b863xNPPIelg7v9lcE8AuiYu4C2BrxnwBE6xB21uwm6gjwN4wC94rw6vDv9Jbxgiyi9XeI8Ol9izzzBdEk5syKDvPjsMLol3vy7jHx0O4KNUjwX7zW8+wnpvCNNnTv1CR+9rDMGvRxBDTsRKZI7m4PHudjqqzxCJsk64+WTPDErlzzU3kC8Qgk2vH47ezjWTQa9sz8WPC5lGLyvM6s6on4Kvf7PZzyZ5qG7Z5c4PBw1xzw91HE8W0WXu5GxuDrm5bi8QIkjPMD0trzYsAW8xsagPMbGIDvbk5c7Why+OwGFmTrvjm67732hO3oBsLxmUcw7ppvCPA5GALxMFWu79QmfPHdMfjzah9E7P0O3vFAtnLzY6qu7bAajPOblOLxVnAa8HsEfvLGzPbqtij+8eK99O89Sw7xqhhA8mIMiuykwVDw25XQ7f3AaOySkVry2XM687lRIu17uAjxAbBC8UsrBvCLqnTpYkGU7988duuGClLyoJxu87lRIPAt0O7wtMPk85DzNt5zJs7w5YAC972AOPSNBVzzEV9u66I6kPDFlPbz34Go85silO+oCcTzSb/u86J/xvPOmnzuWAxC89mweur+6EDtKPh88L7zRu1r/qrxT8xo8iWsCPUTPtLu5Ik09SOzsPI8Uk7zYzRi7xFfbO0wEnjvIgFk8LoKrPCWwHL1DiUi8GDWiOwSFvrtElY48LlnSvNHem7zY+/i8Zm7fvC6TeLxoF8u7rZaFvCuT0zxB7KK6FVIQPN2HdjsqH4e8JhOcvKanCD3EV9s5edhWu6J+irwwH1G6fjv7OkUm7ryA8Cy8BiLkPAm6grt3Hp48W1ZkPD438Tz9bGi75CuAvBw1RzwlB1a8B0u9uyb2iDxmeiU9DeMAvdjNmDwESxi9'

class Float32Array(pydantic.BaseModel):
    data: list[float]

as_json = json.dumps({"data": array.array("f", base64.b64decode(data)).tolist()})

def bench_standard():
    for _ in range(1000):
        Float32Array.model_validate_json(as_json)

def bench_array():
    for _ in range(1000):
        array.array("f", base64.b64decode(data)).tolist()

def bench_numpy():
    for _ in range(1000):
        np.frombuffer(  # type: ignore[no-untyped-call]
                        base64.b64decode(data), dtype="float32"
                    ).tolist()

__benchmarks__ = [
    (bench_standard, bench_array, "Standard vs array"),
    (bench_standard, bench_numpy, "Standard vs numpy"),
    (bench_numpy, bench_array, "numpy vs array"),
]

Benchmark	Min	Max	Mean	Min (+)	Max (+)	Mean (+)
Standard vs array	0.988	1.523	1.233	0.292 (3.4x)	0.313 (4.9x)	0.297 (4.1x)
Standard vs numpy	0.928	1.046	0.991	0.341 (2.7x)	0.370 (2.8x)	0.361 (2.7x)
numpy vs array	0.333	0.383	0.347	0.284 (1.2x)	0.316 (1.2x)	0.298 (1.2x)

If this approach was the new default, it is 4x than the current pydantic parser and 20% faster than the numpy decoder

[RobertCraigie]

Closing this as the PR was merged, thanks!