Refactoring/aws/elasticache

[Aboisier]

This PR migrates ElastiCache to the new architecture, see #183 . I had to work quite a bit to do this one. Here are a few of the difficulties I encountered:

• If wasn't really working before. For example, the parameter groups and subnet groups were not showing.
• The clusters are scoped by VPC, but you can't specify a VPC Id when you call the API, and it doesn't tell you which VPC the cluster belongs to, so you have to make another call to the API.
• Because of that, the number of API calls would have exploded, so I had to implement some caching.
• I had to use locks to make sure the caching was thread safe, but the locks weren't working because our stuff is async. You have to use asyncio.Lock instead of threading.Lock, and do async with lock: #your critical code instead of just with lock: #your critical code.

[codecov-io]

Codecov Report

Merging #245 into refactoring/resource-configs will increase coverage by 0.37%.
The diff coverage is 46.37%.

@@                       Coverage Diff                        @@
##           refactoring/resource-configs     #245      +/-   ##
================================================================
+ Coverage                         29.64%   30.02%   +0.37%     
================================================================
  Files                               104      111       +7     
  Lines                              5431     5536     +105     
================================================================
+ Hits                               1610     1662      +52     
- Misses                             3821     3874      +53

Impacted Files	Coverage Δ
ScoutSuite/providers/aws/resources/regions.py	34.61% <0%> (-2.89%)	⬇️
ScoutSuite/providers/aws/resources/ec2/vpcs.py	100% <100%> (+50%)	⬆️
ScoutSuite/providers/aws/resources/vpcs.py	35.29% <35.29%> (ø)
ScoutSuite/providers/aws/facade/elasticache.py	39.53% <39.53%> (ø)
...iders/aws/resources/elasticache/parametergroups.py	40% <40%> (ø)
...viders/aws/resources/elasticache/securitygroups.py	40% <40%> (ø)
...roviders/aws/resources/elasticache/subnetgroups.py	45.45% <45.45%> (ø)
...ite/providers/aws/resources/elasticache/cluster.py	45.45% <45.45%> (ø)
ScoutSuite/providers/aws/configs/services.py	49.18% <50%> (ø)	⬆️
...ite/providers/aws/resources/elasticache/service.py	56.25% <56.25%> (ø)
... and 11 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07dc10b...1e91971. Read the comment docs.

[x4v13r64]

I had to work quite a bit to do this one

How well did the new architecture handle these difficulties? Did you have to break any conventions?

I had to implement some caching.

Would it be worth documenting this case in the wiki, in case other services might be the same?

[Aboisier]

I had to work quite a bit to do this one

How well did the new architecture handle these difficulties? Did you have to break any conventions?

It was a bit hard to make all the pieces fit in the new architecture, but I'd say it's because the previous architecture allowed to make some hacks. If I was to implement this service from scratch, it would have been easier. I did not have to break any conventions.

I had to implement some caching.

Would it be worth documenting this case in the wiki, in case other services might be the same?

Good idea, I'll add a sentence or two regarding this.

[vifor2]

I have a feeling like you've ingested a lot of caffeine in the past weeks. Nice work, good job on the comments for the more obscure parts of the code too.

[Aboisier]

I have a feeling like you've ingested a lot of caffeine in the past weeks.

I most certainly did!

Nice work, good job on the comments for the more obscure parts of the code too.

Thanks!

[misg]

Great job @Aboisier! 🥇 I can understand why you had such a hard time migrating this one! Highlight on your cache system which is very neat 💯