Clean Architecture, database, data and performance

[ml-deb]

I was wondering, in Clean Architecture how you handle a modification to a lot of data.

What I mean is sometime we have to update hundred or thousand of records in the database on certain user action.

The way I understand it (in the banking kata) is that we should load all the domain entities that need to be updated, then apply the transformation and then save them back one by one. What really happens is:
1 - calls the repository, that call jpa,
2 - returns records from database
3 - convert them to domain object
4 - apply the transformation to the domain
5 - call save
6 - it will convert back the model to records
7 - save the records to the database.

I find it really inefficient compare to calling an update like this in the database update record set value=newvalue where X=Y.

Is there a way to do this nicely with Clean Architecture ?

Another question:
How do you deal with some kind of lazy loading. I understand if JPA is the domain entity, but to keep it clean with split it in different objects so I would have to load it all.

Example: An invoice with a lot of detail lines . I don't want to load all the details every time I want to display or perform an operation on an invoice. Example, in a list that shows all the received invoices, I will only display a summary and then the user clicks on it, he will see all of it's detail lines.

When I create an invoice, it's interesting to load it and then do something like invoice.addDetailLine(qty, price, description...) but when I consult it within a list, it's a lot of to load all this entity (aggregate ?)

In this example there's only a detail lines, but the Invoice can contain a lot more linked information like : addresses, discounts ...

Thanks a lot

[marciovmartins]

In this situation maybe a domain approach is not what you need. Does it have a complex domain? Do you need all the data available to do whatever you need to do in your domain? For example, if you get your process, make is analog. Would the person responsible for the task get all the information from the archive, process it somehow to produce the output?

Sounds to me more a batch process and this can be solved in the infrastructure layer. Maybe using streams and persisting the data in batches.

[marciovmartins]

About your second question. In your example you have two domain objects. Invoice and InvoiceSummary. You would have two different methods in your repository to return those domain objects.

Saying that you would not need to load the Invoice to generate the InvoiceSummary.

The InvoiceSummary would contain several Invoices, but not the full invoice, just what you need to present the InvoiceSummary. And when the user click on one of the items, this could load the full Invoice with all the data.

[ml-deb]

Yes, the InvoiceSummary is juste a part of the Invoice. It's the same but with less information so it won't load all the details.

I think the answer would be to use something like CQRS (without event sourcing) and use only the domain object in the commands and create something more specific where dealing with queries.

[marciovmartins]

CQRS is not mandatory here. For that you should see the benefits and drawbacks to introduce some CQRS architecture to apply it.

If the InvoiceSummary is just a small subset of data from Invoice. Okay, you can have some CQRS and ignore completely the InvoiceSummary domain part and get the data from the databas directly to some DTO to be presented.