Description
when one executes node.store() it takes postgres about 1 sec (X sec) to write that node to the database and the connection is blocked untill this is done and all other connections are on wait/idle.
I am not sure if the bottleneck here is postgres I/O to the disks, on SSD this is faster, or the things done around it.
Often one as a case where one loops over nodes, and changes somethings or stores new nodes.
if one calls store for each node, this will take therefore X sec times the number of nodes.
Which is very slow. (1 Mio nodes several months)
The performance might be improved if several quantities can be stored at once.
(if store can be invoked on a list of nodes and if extras can be set also for non stored nodes yet)
Prepare the 1 million nodes in memory and store/flush them to the db at once.
I assume the import export features of aiida also have such methods, that might be exposed.