Skip to content

chore: Sync main with develop for release 1.14 #648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 45 commits into from
Sep 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
6f50d34
Initial commit. New Serialization context. .NET 8 serialization speci…
hjgraca Aug 2, 2024
34070d6
move to static PowertoolsLoggerHelpers
hjgraca Aug 20, 2024
98c6733
Merge branch 'develop' into aot(logging)-support-logging
hjgraca Aug 21, 2024
2bd8915
Merge branch 'develop' of https://github.com/hjgraca/powertools-lambd…
hjgraca Aug 27, 2024
cd9babf
Refactor case logic. new lambdacontext using LambdaCore. Tests, docs …
hjgraca Aug 29, 2024
70c996d
Merge branch 'develop' of https://github.com/hjgraca/powertools-lambd…
hjgraca Aug 29, 2024
19e82b8
Fix AOT warnings, new way to extract properties from Exceptions. New …
hjgraca Aug 29, 2024
5216450
remove non-deterministic string from tests
hjgraca Aug 29, 2024
fca9fcd
Merge branch 'develop' of https://github.com/hjgraca/powertools-lambd…
hjgraca Aug 30, 2024
35e613b
remove comments
hjgraca Sep 2, 2024
4f12325
merge
hjgraca Sep 2, 2024
d5e981b
Update idempotency 1.2.2
hjgraca Sep 2, 2024
f5f7b7b
Update tracing
hjgraca Sep 2, 2024
1f2d649
Update batch
hjgraca Sep 2, 2024
f6c2ae0
Update LambdaPowertoolsAPI.csproj
hjgraca Sep 2, 2024
e3cb52f
Update HelloWorld.csproj
hjgraca Sep 2, 2024
5a483b7
Merge pull request #645 from aws-powertools/chore(examples)Update-exa…
hjgraca Sep 9, 2024
42138f7
refactor, change the way we create Logger in tests to use LoggerProvider
hjgraca Sep 11, 2024
c7e95b4
refactor configuration and sampling rate code
hjgraca Sep 11, 2024
27e7433
address sonar issues
hjgraca Sep 11, 2024
1a97e37
add Serialize method to the correct place
hjgraca Sep 11, 2024
6c23b03
add tests for serializer
hjgraca Sep 11, 2024
f65d3cb
refactoring and serializer tests
hjgraca Sep 11, 2024
86e624d
refactor and add tests for logging handler.
hjgraca Sep 13, 2024
80815ea
Refactor Aspect Injection to Factory. Add more tests for full path us…
hjgraca Sep 16, 2024
68395c3
fix sonar and cleanup
hjgraca Sep 16, 2024
681de15
fix sonar issues
hjgraca Sep 17, 2024
a18f29e
more tests
hjgraca Sep 17, 2024
e9e71b0
fix sonar issues
hjgraca Sep 17, 2024
e92559b
remove ApiGateway and LoadBalancer Events from serializer
hjgraca Sep 17, 2024
67dc1cf
fix LogLevel and default to None when not set. more tests
hjgraca Sep 18, 2024
0595e8e
fix serialization on first request. the context was not set properly,…
hjgraca Sep 20, 2024
0b4e96b
replace ApplicationException with JsonSerializerException
hjgraca Sep 20, 2024
86b8f85
addressed issue when anonymous type on AddKeys. Refactored serializat…
hjgraca Sep 20, 2024
af66bcd
Fix custom serializer on AOT. Add tests
hjgraca Sep 23, 2024
6446e95
for aot, ILogFormatter instance needs to be passed to PowertoolsSourc…
hjgraca Sep 23, 2024
52f6066
comments
hjgraca Sep 23, 2024
4d52cc1
add documentation.
hjgraca Sep 23, 2024
0ffbbbc
update doc
hjgraca Sep 23, 2024
8e81bfa
Merge branch 'develop' of https://github.com/hjgraca/powertools-lambd…
hjgraca Sep 23, 2024
b6f7988
refactor AOT examples to each folder. Add Logging AOT examples and up…
hjgraca Sep 23, 2024
cbb78fa
update examples
hjgraca Sep 23, 2024
eabc183
update doc and examples
hjgraca Sep 23, 2024
e1d1181
fix sonar issues
hjgraca Sep 23, 2024
335eef7
Merge pull request #628 from hjgraca/aot(logging)-support-logging
hjgraca Sep 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 139 additions & 0 deletions docs/core/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ The logging utility provides a Lambda optimized logger with output structured as
* Log Lambda event when instructed (disabled by default)
* Log sampling enables DEBUG log level for a percentage of requests (disabled by default)
* Append additional keys to structured log at any point in time
* Ahead-of-Time compilation to native code support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) from version 1.6.0

## Installation

Expand All @@ -22,6 +23,12 @@ Powertools for AWS Lambda (.NET) are available as NuGet packages. You can instal

## Getting started

!!! info

AOT Support
If loooking for AOT specific configurations navigate to the [AOT section](#aot-support)


Logging requires two settings:

Setting | Description | Environment variable | Attribute parameter
Expand Down Expand Up @@ -166,6 +173,10 @@ When debugging in non-production environments, you can instruct Logger to log th

You can set a Correlation ID using `CorrelationIdPath` parameter by passing a [JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"}.

!!! Attention
The JSON Pointer expression is `case sensitive`. In the bellow example `/headers/my_request_id_header` would work but `/Headers/my_request_id_header` would not find the element.


=== "Function.cs"

```c# hl_lines="6"
Expand Down Expand Up @@ -656,3 +667,131 @@ You can customize the structure (keys and values) of your log entries by impleme
}
}
```

## AOT Support

Logging utility supports native AOT serialization by default without any changes needed.

!!! info

In case you want to use the `LogEvent`, `Custom Log Formatter` features or serialize your own types when Logging events it is required
that you do some changes in your Lambda `Main` method.

!!! info

Starting from version 1.6.0 it is required to update `Amazon.Lambda.Serialization.SystemTextJson` to `version 2.4.3` in your `csproj`.

### Configure

The change needed is to replace `SourceGeneratorLambdaJsonSerializer` with `PowertoolsSourceGeneratorSerializer`.

This change enables Powertools to construct an instance of JsonSerializerOptions that is used to customize the serialization and deserialization of the Lambda JSON events and your own types.

=== "Before"

```csharp
Func<APIGatewayHttpApiV2ProxyRequest, ILambdaContext, Task<APIGatewayHttpApiV2ProxyResponse>> handler = FunctionHandler;
await LambdaBootstrapBuilder.Create(handler, new SourceGeneratorLambdaJsonSerializer<MyCustomJsonSerializerContext>())
.Build()
.RunAsync();
```

=== "After"

```csharp hl_lines="2"
Func<APIGatewayHttpApiV2ProxyRequest, ILambdaContext, Task<APIGatewayHttpApiV2ProxyResponse>> handler = FunctionHandler;
await LambdaBootstrapBuilder.Create(handler, new PowertoolsSourceGeneratorSerializer<MyCustomJsonSerializerContext>())
.Build()
.RunAsync();
```

For example when you have your own Demo type

```csharp
public class Demo
{
public string Name { get; set; }
public Headers Headers { get; set; }
}
```

To be able to serialize it in AOT you have to have your own `JsonSerializerContext`

```csharp
[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))]
[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))]
[JsonSerializable(typeof(Demo))]
public partial class MyCustomJsonSerializerContext : JsonSerializerContext
{
}
```

When you change to `PowertoolsSourceGeneratorSerializer<MyCustomJsonSerializerContext>` we are
combining your `JsonSerializerContext` types with Powertools `JsonSerializerContext`. This allows Powertools to serialize your types and Lambda events.

### Custom Log Formatter

To be able to use a custom log formatter with AOT we need to pass an instance of ` ILogFormatter` to `PowertoolsSourceGeneratorSerializer`
instead of using the static `Logger.UseFormatter` in the Function contructor.

=== "Function Main method"

```csharp hl_lines="5"

Func<APIGatewayHttpApiV2ProxyRequest, ILambdaContext, Task<APIGatewayHttpApiV2ProxyResponse>> handler = FunctionHandler;
await LambdaBootstrapBuilder.Create(handler,
new PowertoolsSourceGeneratorSerializer<LambdaFunctionJsonSerializerContext>
(
new CustomLogFormatter()
)
)
.Build()
.RunAsync();

```

=== "CustomLogFormatter.cs"

```csharp
public class CustomLogFormatter : ILogFormatter
{
public object FormatLogEntry(LogEntry logEntry)
{
return new
{
Message = logEntry.Message,
Service = logEntry.Service,
CorrelationIds = new
{
AwsRequestId = logEntry.LambdaContext?.AwsRequestId,
XRayTraceId = logEntry.XRayTraceId,
CorrelationId = logEntry.CorrelationId
},
LambdaFunction = new
{
Name = logEntry.LambdaContext?.FunctionName,
Arn = logEntry.LambdaContext?.InvokedFunctionArn,
MemoryLimitInMB = logEntry.LambdaContext?.MemoryLimitInMB,
Version = logEntry.LambdaContext?.FunctionVersion,
ColdStart = logEntry.ColdStart,
},
Level = logEntry.Level.ToString(),
Timestamp = logEntry.Timestamp.ToString("o"),
Logger = new
{
Name = logEntry.Name,
SampleRate = logEntry.SamplingRate
},
};
}
}
```

### Anonymous types

!!! note

Although we support anonymous type serialization by converting to a `Dictionary<string, object>`,
this is not a best practice and is not recommendede when using native AOT.

Recommendation is to use concrete classes and add them to your `JsonSerializerContext`.
25 changes: 25 additions & 0 deletions examples/AOT/AOT_Logging/src/AOT_Logging/AOT_Logging.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<AWSProjectType>Lambda</AWSProjectType>
<!-- This property makes the build directory similar to a publish directory and helps the AWS .NET Lambda Mock Test Tool find project dependencies. -->
<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
<!-- Generate Native AOT image during publishing to improve cold start time. -->
<PublishAot>true</PublishAot>
<!-- StripSymbols tells the compiler to strip debugging symbols from the final executable if we're on Linux and put them into their own file.
This will greatly reduce the final executable's size.-->
<StripSymbols>true</StripSymbols>
<!-- TrimMode partial will only trim assemblies marked as trimmable. To reduce package size make all assemblies trimmable and set TrimMode to full.
If there are trim warnings during build, you can hit errors at runtime.-->
<TrimMode>partial</TrimMode>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Amazon.Lambda.RuntimeSupport" Version="1.10.0"/>
<PackageReference Include="Amazon.Lambda.Core" Version="2.2.0"/>
<PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.4.3"/>
<PackageReference Include="AWS.Lambda.Powertools.Logging" Version="1.5.1" />
</ItemGroup>
</Project>
76 changes: 76 additions & 0 deletions examples/AOT/AOT_Logging/src/AOT_Logging/Function.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
using Amazon.Lambda.Core;
using Amazon.Lambda.RuntimeSupport;
using Amazon.Lambda.Serialization.SystemTextJson;
using System.Text.Json.Serialization;
using AWS.Lambda.Powertools.Logging;
using LogLevel = Microsoft.Extensions.Logging.LogLevel;

namespace AOT_Logging;

public static class Function
{
/// <summary>
/// The main entry point for the Lambda function. The main function is called once during the Lambda init phase. It
/// initializes the .NET Lambda runtime client passing in the function handler to invoke for each Lambda event and
/// the JSON serializer to use for converting Lambda JSON format to the .NET types.
/// </summary>
private static async Task Main()
{
Func<string, ILambdaContext, string> handler = FunctionHandler;
await LambdaBootstrapBuilder.Create(handler,
new SourceGeneratorLambdaJsonSerializer<LambdaFunctionJsonSerializerContext>())
.Build()
.RunAsync();
}

/// <summary>
/// A simple function that takes a string and does a ToUpper.
///
/// To use this handler to respond to an AWS event, reference the appropriate package from
/// https://github.com/aws/aws-lambda-dotnet#events
/// and change the string input parameter to the desired event type. When the event type
/// is changed, the handler type registered in the main method needs to be updated and the LambdaFunctionJsonSerializerContext
/// defined below will need the JsonSerializable updated. If the return type and event type are different then the
/// LambdaFunctionJsonSerializerContext must have two JsonSerializable attributes, one for each type.
///
// When using Native AOT extra testing with the deployed Lambda functions is required to ensure
// the libraries used in the Lambda function work correctly with Native AOT. If a runtime
// error occurs about missing types or methods the most likely solution will be to remove references to trim-unsafe
// code or configure trimming options. This sample defaults to partial TrimMode because currently the AWS
// SDK for .NET does not support trimming. This will result in a larger executable size, and still does not
// guarantee runtime trimming errors won't be hit.
/// </summary>
/// <param name="input">The event for the Lambda function handler to process.</param>
/// <param name="context">The ILambdaContext that provides methods for logging and describing the Lambda environment.</param>
/// <returns></returns>
[Logging(LogEvent = true, Service = "pt_service", LogLevel = LogLevel.Debug)]
public static string FunctionHandler(string input, ILambdaContext context)
{
Logger.LogInformation("FunctionHandler invocation");
return ToUpper(input);
}

private static string ToUpper(string input)
{
Logger.LogInformation("ToUpper invocation");

var upper = input.ToUpper();

Logger.LogInformation("ToUpper result: {Result}", upper);

return upper;
}
}

/// <summary>
/// This class is used to register the input event and return type for the FunctionHandler method with the System.Text.Json source generator.
/// There must be a JsonSerializable attribute for each type used as the input and return type or a runtime error will occur
/// from the JSON serializer unable to find the serialization information for unknown types.
/// </summary>
[JsonSerializable(typeof(string))]
public partial class LambdaFunctionJsonSerializerContext : JsonSerializerContext
{
// By using this partial class derived from JsonSerializerContext, we can generate reflection free JSON Serializer code at compile time
// which can deserialize our class and properties. However, we must attribute this class to tell it what types to generate serialization code for.
// See https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-source-generation
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# AWS Lambda Native AOT Project with Powertools for AWS Lambda (.NET)
# AWS Lambda Native AOT Project

This starter project consists of:
* Function.cs - contains a class with a `Main` method that starts the bootstrap and a single function handler method.
Expand Down Expand Up @@ -71,12 +71,12 @@ If already installed check if new version is available.

Execute unit tests
```
cd "AOT/test/AOT.Tests"
cd "AOT_Logging/test/AOT_Logging.Tests"
dotnet test
```

Deploy function to AWS Lambda
```
cd "AOT/src/AOT"
cd "AOT_Logging/src/AOT_Logging"
dotnet lambda deploy-function
```
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,6 @@
"function-runtime": "dotnet8",
"function-memory-size": 512,
"function-timeout": 30,
"function-handler": "AOT",
"function-handler": "AOT_Logging",
"msbuild-parameters": "--self-contained true"
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\AOT\AOT.csproj" />
<ProjectReference Include="..\..\src\AOT_Logging\AOT_Logging.csproj" />
</ItemGroup>
</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
using Amazon.Lambda.Core;
using Amazon.Lambda.TestUtilities;

namespace AOT.Tests;
namespace AOT_Logging.Tests;

public class FunctionTest
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@
<ItemGroup>
<PackageReference Include="Amazon.Lambda.RuntimeSupport" Version="1.10.0"/>
<PackageReference Include="Amazon.Lambda.Core" Version="2.2.0"/>
<PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.4.1"/>
<PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.4.3"/>
<PackageReference Include="AWS.Lambda.Powertools.Metrics" Version="1.7.1" />
<PackageReference Include="AWS.Lambda.Powertools.Tracing" Version="1.5.0" />
</ItemGroup>
</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,10 @@
using Amazon.Lambda.Serialization.SystemTextJson;
using System.Text.Json.Serialization;
using AWS.Lambda.Powertools.Metrics;
using AWS.Lambda.Powertools.Tracing;

namespace AOT;
namespace AOT_Metrics;

public class Function
public static class Function
{
/// <summary>
/// The main entry point for the Lambda function. The main function is called once during the Lambda init phase. It
Expand Down Expand Up @@ -43,27 +42,19 @@ await LambdaBootstrapBuilder.Create(handler,
/// <param name="input">The event for the Lambda function handler to process.</param>
/// <param name="context">The ILambdaContext that provides methods for logging and describing the Lambda environment.</param>
/// <returns></returns>

// You can optionally capture cold start metrics by setting CaptureColdStart parameter to true.
[Metrics(Namespace = "ns", Service = "svc", CaptureColdStart = true)]
[Tracing(CaptureMode = TracingCaptureMode.ResponseAndError)]
public static string FunctionHandler(string input, ILambdaContext context)
{
// You can create metrics using AddMetric
// MetricUnit enum facilitates finding a supported metric unit by CloudWatch.
Metrics.AddMetric("Handler invocation", 1, MetricUnit.Count);
return ToUpper(input);
}

[Tracing(SegmentName = "ToUpper Call")]

private static string ToUpper(string input)
{
Metrics.AddMetric("ToUpper invocation", 1, MetricUnit.Count);

var upper = input.ToUpper();

Tracing.AddAnnotation("Upper text", upper);


// You can add high-cardinality data as part of your Metrics log with AddMetadata method.
// This is useful when you want to search highly contextual information along with your metrics in your logs.
Metrics.AddMetadata("Input Uppercase", upper);
Expand Down
Loading
Loading