cloudfront_invalidation - Do not fetch every cloudfront invalidation when caller_reference is None

[kakawait]

SUMMARY

When requesting cloudfront_invalidation the module is firstly trying to fetch every existing/passed invalidation to determine if the invalidation has not already be done based on caller_reference as key

https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/cloudfront_invalidation.py#L161

However, when you've cloudfront that containing large number of executed invalidation, cloudfront_invalidation will take really long time to compute. In our case with time it takes more than 10 minutes...

Since we're not passing caller_reference param, and regarding code:

https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/cloudfront_invalidation.py#L235

It will generate a pseudo-unique one using current timestamp. I think is not really useful to check the existing/passed invalidation

ISSUE TYPE

• Feature Idea

COMPONENT NAME

cloudfront_invalidation

ADDITIONAL INFORMATION

I'm ok to PR but I'll need some feedbacks!

Example of what patched I've applied locally (is a bit dirty but give the idea)

diff --git a/plugins/modules/cloudfront_invalidation.py b/plugins/modules/cloudfront_invalidation.py
index 52e3aea..2849b73 100644
--- a/plugins/modules/cloudfront_invalidation.py
+++ b/plugins/modules/cloudfront_invalidation.py
@@ -157,8 +157,10 @@ class CloudFrontInvalidationServiceManager(object):
         self.module = module
         self.client = module.client('cloudfront')

-    def create_invalidation(self, distribution_id, invalidation_batch):
-        current_invalidation_response = self.get_invalidation(distribution_id, invalidation_batch['CallerReference'])
+    def create_invalidation(self, distribution_id, invalidation_batch, caller_reference):
+        current_invalidation_response = False
+        if caller_reference is not None:
+          current_invalidation_response = self.get_invalidation(distribution_id, invalidation_batch['CallerReference'])
         try:
             response = self.client.create_invalidation(DistributionId=distribution_id, InvalidationBatch=invalidation_batch)
             response.pop('ResponseMetadata', None)
@@ -265,7 +267,7 @@ def main():
     distribution_id = validation_mgr.validate_distribution_id(distribution_id, alias)
     valid_target_paths = validation_mgr.validate_invalidation_batch(target_paths, caller_reference)
     valid_pascal_target_paths = snake_dict_to_camel_dict(valid_target_paths, True)
-    result, changed = service_mgr.create_invalidation(distribution_id, valid_pascal_target_paths)
+    result, changed = service_mgr.create_invalidation(distribution_id, valid_pascal_target_paths, caller_reference)

     module.exit_json(changed=changed, **camel_dict_to_snake_dict(result))

I've moved from ~10mins to 3secs with patch

[ansibullbot]

Files identified in the description:

• plugins/modules/cloudfront_invalidation.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @jillr @s-hertel @tremble @wilvk @wimnat
click here for bot help

[goneri]

Hi @kakawait,

Yes! Please, open up a PR with your change. Can you also add a changelog fragment https://docs.ansible.com/ansible/latest/community/development_process.html#changelogs.
Don't hesitate to ping us on IRC #ansible-aws or #ansible-fr if you speak french :-)