-
Notifications
You must be signed in to change notification settings - Fork 12
Target file performance Improvement. #870
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
try { | ||
timeSlots = subscriptionTimeSlotService.findTimeSlotsForSubscriptionsById(subscriptionIds); | ||
try { | ||
timeSlots = subscriptionTimeSlotService.findAllTimeSlots(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why looking for all Time slots here?
imi/src/main/java/org/motechproject/nms/imi/service/impl/TargetFileServiceImpl.java
Outdated
Show resolved
Hide resolved
"WHERE ts.subscriptionId IN (" + idsString + ")"; | ||
"FROM nms_subscriptions_time_slot ts " + | ||
"LEFT JOIN nms_subscriptions s ON ts.subscription_id = s.id;"; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This join isn't efficient. Right now, time_slot has little data because it's in the pilot phase, but it will grow to millions soon. We might also get unnecessary old data. Add a WHERE condition to filter only recent data (e.g., last 72 weeks, exclude deactivated or completed ).
We should run this new method alongside the old way of creating the target file in production, not replace it immediately. We can't risk missing a single day's call. Use the API to generate the target file, but will not send it to Hungama. we have analyze the results first, and if everything works well, then switch to the new method completely. @atishbeehyv123 please suggest |
b0adee4
to
25a6fef
Compare
Below changes has been done to inprove the performance
Approach for Performance Improvement:
Fetching data using a JDBC connection instead of Data Nucleus.
Optimized Query Execution:
Using prepared statements for executing database queries.
Ensuring all operations are properly closed after a successful database interaction.
Query Optimization:
Updated the existing query to fetch all the required columns from 9 tables used for target file generation.
DTO Usage:
Storing query results in a DTO format.
Applied the Builder design pattern in the DTO class for efficient data handling.
Integration with Method:
Updating all columns in the generateFreshCall method.