Skip to content

Multi line parameters #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 8, 2020
Merged

Multi line parameters #2

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1f8666f
Handle more than ten secrets
ryan-j-mci Jan 7, 2020
b680e20
Different python base
ryan-j-mci Jan 7, 2020
e917950
Get only one parameter at a time
ryan-j-mci Jan 8, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM jfloff/alpine-python:2.7-slim
FROM python:3.8-slim

MAINTAINER Signiant DevOps <devops@signiant.com>

Expand Down
27 changes: 15 additions & 12 deletions parameter_sync.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
import logging, logging.handlers
import argparse
import os
import boto3
import hashlib
import tempfile
import logging
import logging.handlers
import os
import shutil
import tempfile
import boto3

logging.getLogger("botocore").setLevel(logging.CRITICAL)


def get_sha256_hash(file_path):
logging.debug('Hashing "%s" using SHA256' % file_path)
BUF_SIZE = 65536 # lets read stuff in 64kb chunks!
Expand All @@ -29,13 +31,14 @@ def process_parameters_with_prefix(param_prefix, cred_path, aws_region, aws_acce
def get_parameters(parameter_names_list):
parameter_list = []
if parameter_names_list:
result = ssm.get_parameters(Names=parameter_names_list, WithDecryption=True)
if result:
if 'ResponseMetadata' in result:
if 'HTTPStatusCode' in result['ResponseMetadata']:
if result['ResponseMetadata']['HTTPStatusCode'] == 200:
if 'Parameters' in result:
parameter_list = result['Parameters']
for parameter_name in parameter_names_list:
result = ssm.get_parameters(Names=[parameter_name], WithDecryption=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're only going to get one at a time, why not use get_parameter instead of get_parameters?

For the record, this seems to be WAY less efficient than sending a list of parameter names. If there is a limit to how many parameters you can put in the list (not documented, but clearly a problem) then why not break up the list in chunks of size X and call this method multiple times?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed it to get_parameter.
This is much less efficient, but the get_paramter call is very quick.

if result:
if 'ResponseMetadata' in result:
if 'HTTPStatusCode' in result['ResponseMetadata']:
if result['ResponseMetadata']['HTTPStatusCode'] == 200:
if 'Parameters' in result:
parameter_list = parameter_list + result['Parameters']
return parameter_list

def process_parameter(param_name, param_value):
Expand All @@ -47,7 +50,7 @@ def process_parameter(param_name, param_value):
new_file_full_path = temp_dir + os.sep + filename + '.new'
logging.debug('Storing retrieved value for parameter "%s" in "%s"' % (param_name, new_file_full_path))
with open(new_file_full_path, 'w') as f:
f.write(param_value.replace('\\n', '\n'))
f.write(param_value)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm this should still work with files that did have unescaped new lines so that's good.

new_file_sha256_hash = get_sha256_hash(new_file_full_path)
logging.debug('Comparing file hashes')
if existing_file_sha256_hash != new_file_sha256_hash:
Expand Down