tag:github.com,2008:https://github.com/awslabs/athena-glue-service-logs/releasesRelease notes from athena-glue-service-logs2021-05-05T18:19:22Ztag:github.com,2008:Repository/134778020/v6.0.02021-05-05T18:25:47Zv6.0.0 - Python3 Support<p>This release adds support for Python 3 (thanks <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ryandeivert/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ryandeivert">@ryandeivert</a>).</p>
<p>Please note that this is likely the last release of this library in its current format. As part of researching many of the new Glue features, it's likely we'll make use of the new Blueprints and Workflows functionality to replace a good chunk of this code.</p>
<p>Follow along in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="833028463" data-permission-text="Title is private" data-url="https://github.com/awslabs/athena-glue-service-logs/issues/23" data-hovercard-type="issue" data-hovercard-url="/awslabs/athena-glue-service-logs/issues/23/hovercard" href="https://github.com/awslabs/athena-glue-service-logs/issues/23">#23</a> for more updates! 😄</p>dacorttag:github.com,2008:Repository/134778020/v5.3.52019-09-12T17:58:58Zv5.3.5 - Fix partition pagination<p>This release fixes an issue (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="492945652" data-permission-text="Title is private" data-url="https://github.com/awslabs/athena-glue-service-logs/issues/17" data-hovercard-type="pull_request" data-hovercard-url="/awslabs/athena-glue-service-logs/pull/17/hovercard" href="https://github.com/awslabs/athena-glue-service-logs/pull/17">#17</a>) where we weren't fetching the entire set of partitions from the Glue Data Catalog and could result in that data not showing up in the raw or optimized tables. This would have surfaced for data sources grouped by region with a large (>1000) number of partitions.</p>
<p>You can update your Glue jobs by just pointing the "Python lib path" to this new build. You may notice that on the next run, data from the past 30 days in those missing regions are now added.</p>dacorttag:github.com,2008:Repository/134778020/v5.3.42019-05-07T10:13:24Zv5.3.4 - New ALB and S3 fields<p>This update adds a few new fields for both ALB (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="425138510" data-permission-text="Title is private" data-url="https://github.com/awslabs/athena-glue-service-logs/issues/9" data-hovercard-type="pull_request" data-hovercard-url="/awslabs/athena-glue-service-logs/pull/9/hovercard" href="https://github.com/awslabs/athena-glue-service-logs/pull/9">#9</a>) and S3 Access Logs (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="441144348" data-permission-text="Title is private" data-url="https://github.com/awslabs/athena-glue-service-logs/issues/11" data-hovercard-type="pull_request" data-hovercard-url="/awslabs/athena-glue-service-logs/pull/11/hovercard" href="https://github.com/awslabs/athena-glue-service-logs/pull/11">#11</a>) as well as updates the Makefile to provide a location for Glue temp files (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="428048145" data-permission-text="Title is private" data-url="https://github.com/awslabs/athena-glue-service-logs/issues/10" data-hovercard-type="issue" data-hovercard-url="/awslabs/athena-glue-service-logs/issues/10/hovercard" href="https://github.com/awslabs/athena-glue-service-logs/issues/10">#10</a>).</p>dacorttag:github.com,2008:Repository/134778020/v5.3.02018-09-12T20:29:06Zv5.3.0 - VPC Flow Support<p>Now that VPC Flow Logs can be published <a href="https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-s3.html" rel="nofollow">directly to S3</a>, it makes it easy to add support for them to this framework. The logs are automatically partitioned by region and date.</p>dacort