Skip to content

chore: Logging AOT Support #628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 38 commits into from
Sep 23, 2024
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
6f50d34
Initial commit. New Serialization context. .NET 8 serialization speci…
hjgraca Aug 2, 2024
34070d6
move to static PowertoolsLoggerHelpers
hjgraca Aug 20, 2024
98c6733
Merge branch 'develop' into aot(logging)-support-logging
hjgraca Aug 21, 2024
2bd8915
Merge branch 'develop' of https://github.com/hjgraca/powertools-lambd…
hjgraca Aug 27, 2024
cd9babf
Refactor case logic. new lambdacontext using LambdaCore. Tests, docs …
hjgraca Aug 29, 2024
70c996d
Merge branch 'develop' of https://github.com/hjgraca/powertools-lambd…
hjgraca Aug 29, 2024
19e82b8
Fix AOT warnings, new way to extract properties from Exceptions. New …
hjgraca Aug 29, 2024
5216450
remove non-deterministic string from tests
hjgraca Aug 29, 2024
fca9fcd
Merge branch 'develop' of https://github.com/hjgraca/powertools-lambd…
hjgraca Aug 30, 2024
35e613b
remove comments
hjgraca Sep 2, 2024
4f12325
merge
hjgraca Sep 2, 2024
42138f7
refactor, change the way we create Logger in tests to use LoggerProvider
hjgraca Sep 11, 2024
c7e95b4
refactor configuration and sampling rate code
hjgraca Sep 11, 2024
27e7433
address sonar issues
hjgraca Sep 11, 2024
1a97e37
add Serialize method to the correct place
hjgraca Sep 11, 2024
6c23b03
add tests for serializer
hjgraca Sep 11, 2024
f65d3cb
refactoring and serializer tests
hjgraca Sep 11, 2024
86e624d
refactor and add tests for logging handler.
hjgraca Sep 13, 2024
80815ea
Refactor Aspect Injection to Factory. Add more tests for full path us…
hjgraca Sep 16, 2024
68395c3
fix sonar and cleanup
hjgraca Sep 16, 2024
681de15
fix sonar issues
hjgraca Sep 17, 2024
a18f29e
more tests
hjgraca Sep 17, 2024
e9e71b0
fix sonar issues
hjgraca Sep 17, 2024
e92559b
remove ApiGateway and LoadBalancer Events from serializer
hjgraca Sep 17, 2024
67dc1cf
fix LogLevel and default to None when not set. more tests
hjgraca Sep 18, 2024
0595e8e
fix serialization on first request. the context was not set properly,…
hjgraca Sep 20, 2024
0b4e96b
replace ApplicationException with JsonSerializerException
hjgraca Sep 20, 2024
86b8f85
addressed issue when anonymous type on AddKeys. Refactored serializat…
hjgraca Sep 20, 2024
af66bcd
Fix custom serializer on AOT. Add tests
hjgraca Sep 23, 2024
6446e95
for aot, ILogFormatter instance needs to be passed to PowertoolsSourc…
hjgraca Sep 23, 2024
52f6066
comments
hjgraca Sep 23, 2024
4d52cc1
add documentation.
hjgraca Sep 23, 2024
0ffbbbc
update doc
hjgraca Sep 23, 2024
8e81bfa
Merge branch 'develop' of https://github.com/hjgraca/powertools-lambd…
hjgraca Sep 23, 2024
b6f7988
refactor AOT examples to each folder. Add Logging AOT examples and up…
hjgraca Sep 23, 2024
cbb78fa
update examples
hjgraca Sep 23, 2024
eabc183
update doc and examples
hjgraca Sep 23, 2024
e1d1181
fix sonar issues
hjgraca Sep 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 135 additions & 0 deletions docs/core/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ The logging utility provides a Lambda optimized logger with output structured as
* Log Lambda event when instructed (disabled by default)
* Log sampling enables DEBUG log level for a percentage of requests (disabled by default)
* Append additional keys to structured log at any point in time
* Ahead-of-Time compilation to native code support [AOT](https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html) from version 1.6.0

## Installation

Expand All @@ -22,6 +23,12 @@ Powertools for AWS Lambda (.NET) are available as NuGet packages. You can instal

## Getting started

!!! info

AOT Support
If loooking for AOT specific configurations navigate to the [AOT section](#aot-support)


Logging requires two settings:

Setting | Description | Environment variable | Attribute parameter
Expand Down Expand Up @@ -166,6 +173,10 @@ When debugging in non-production environments, you can instruct Logger to log th

You can set a Correlation ID using `CorrelationIdPath` parameter by passing a [JSON Pointer expression](https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-json-pointer-03){target="_blank"}.

!!! Attention
The JSON Pointer expression is `case sensitive`. In the bellow example `/headers/my_request_id_header` would work but `/Headers/my_request_id_header` would not find the element.


=== "Function.cs"

```c# hl_lines="6"
Expand Down Expand Up @@ -656,3 +667,127 @@ You can customize the structure (keys and values) of your log entries by impleme
}
}
```

## AOT Support

Logging utility supports native AOT serialization by default without any changes needed.

!!! info

In case you want to use the `LogEvent`, `Custom Log Formatter` features or serialize your own types when Logging events it is required
that you do some changes in your Lambda `Main` method.

### Configure

The change needed is to replace `SourceGeneratorLambdaJsonSerializer` with `PowertoolsSourceGeneratorSerializer`.

This change enables Powertools to construct an instance of JsonSerializerOptions that is used to customize the serialization and deserialization of the Lambda JSON events and your own types.

=== "Before"

```csharp
Func<APIGatewayHttpApiV2ProxyRequest, ILambdaContext, Task<APIGatewayHttpApiV2ProxyResponse>> handler = FunctionHandler;
await LambdaBootstrapBuilder.Create(handler, new SourceGeneratorLambdaJsonSerializer<MyCustomJsonSerializerContext>())
.Build()
.RunAsync();
```

=== "After"

```csharp hl_lines="2"
Func<APIGatewayHttpApiV2ProxyRequest, ILambdaContext, Task<APIGatewayHttpApiV2ProxyResponse>> handler = FunctionHandler;
await LambdaBootstrapBuilder.Create(handler, new PowertoolsSourceGeneratorSerializer<MyCustomJsonSerializerContext>())
.Build()
.RunAsync();
```

For example when you have your own Demo type

```csharp
public class Demo
{
public string Name { get; set; }
public Headers Headers { get; set; }
}
```

To be able to serialize it in AOT you have to have your own `JsonSerializerContext`

```csharp
[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))]
[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))]
[JsonSerializable(typeof(Demo))]
public partial class MyCustomJsonSerializerContext : JsonSerializerContext
{
}
```

When you change to `PowertoolsSourceGeneratorSerializer<MyCustomJsonSerializerContext>` we are
combining your `JsonSerializerContext` types with Powertools `JsonSerializerContext`. This allows Powertools to serialize your types and Lambda events.

### Custom Log Formatter

To be able to use a custom log formatter with AOT we need to pass an instance of ` ILogFormatter` to `PowertoolsSourceGeneratorSerializer`
instead of using the static `Logger.UseFormatter` in the Function contructor.

=== "Function Main method"

```csharp hl_lines="5"

Func<APIGatewayHttpApiV2ProxyRequest, ILambdaContext, Task<APIGatewayHttpApiV2ProxyResponse>> handler = FunctionHandler;
await LambdaBootstrapBuilder.Create(handler,
new PowertoolsSourceGeneratorSerializer<LambdaFunctionJsonSerializerContext>
(
new CustomLogFormatter()
)
)
.Build()
.RunAsync();

```

=== "CustomLogFormatter.cs"

```csharp
public class CustomLogFormatter : ILogFormatter
{
public object FormatLogEntry(LogEntry logEntry)
{
return new
{
Message = logEntry.Message,
Service = logEntry.Service,
CorrelationIds = new
{
AwsRequestId = logEntry.LambdaContext?.AwsRequestId,
XRayTraceId = logEntry.XRayTraceId,
CorrelationId = logEntry.CorrelationId
},
LambdaFunction = new
{
Name = logEntry.LambdaContext?.FunctionName,
Arn = logEntry.LambdaContext?.InvokedFunctionArn,
MemoryLimitInMB = logEntry.LambdaContext?.MemoryLimitInMB,
Version = logEntry.LambdaContext?.FunctionVersion,
ColdStart = logEntry.ColdStart,
},
Level = logEntry.Level.ToString(),
Timestamp = logEntry.Timestamp.ToString("o"),
Logger = new
{
Name = logEntry.Name,
SampleRate = logEntry.SamplingRate
},
};
}
}
```

### Anonymous types

!!! note

Although we support anonymous type serialization by converting to a `Dictionary<string, object>`,
this is not a best practice and is not recommendede when using native AOT.

Recommendation is to use concrete classes and add them to your `JsonSerializerContext`.
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ public interface IPowertoolsConfigurations
/// Gets the logger sample rate.
/// </summary>
/// <value>The logger sample rate.</value>
double? LoggerSampleRate { get; }
double LoggerSampleRate { get; }

/// <summary>
/// Gets a value indicating whether [logger log event].
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
* permissions and limitations under the License.
*/

using System.Globalization;

namespace AWS.Lambda.Powertools.Common;

/// <summary>
Expand Down Expand Up @@ -157,10 +159,10 @@ public bool GetEnvironmentVariableOrDefault(string variable, bool defaultValue)
/// Gets the logger sample rate.
/// </summary>
/// <value>The logger sample rate.</value>
public double? LoggerSampleRate =>
double.TryParse(_systemWrapper.GetEnvironmentVariable(Constants.LoggerSampleRateNameEnv), out var result)
public double LoggerSampleRate =>
double.TryParse(_systemWrapper.GetEnvironmentVariable(Constants.LoggerSampleRateNameEnv), NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var result)
? result
: null;
: 0;

/// <summary>
/// Gets a value indicating whether [logger log event].
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@
<ItemGroup>
<!-- Package versions are Centrally managed in Directory.Packages.props file -->
<!-- More info https://learn.microsoft.com/en-us/nuget/consume-packages/central-package-management -->
<PackageReference Include="Microsoft.Extensions.Logging"/>
<PackageReference Include="Amazon.Lambda.Core" />
<PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" />
<PackageReference Include="Microsoft.Extensions.Logging" />
<ProjectReference Include="..\AWS.Lambda.Powertools.Common\AWS.Lambda.Powertools.Common.csproj" Condition="'$(Configuration)'=='Debug'"/>
</ItemGroup>

</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
using System.Linq;
using System.Text.Json;
using System.Text.Json.Serialization;
using AWS.Lambda.Powertools.Logging.Serializers;

namespace AWS.Lambda.Powertools.Logging.Internal.Converters;

Expand Down Expand Up @@ -57,15 +58,13 @@
public override void Write(Utf8JsonWriter writer, Exception value, JsonSerializerOptions options)
{
var exceptionType = value.GetType();
var properties = exceptionType.GetProperties()
.Where(prop => prop.Name != nameof(Exception.TargetSite))
.Select(prop => new { prop.Name, Value = prop.GetValue(value) });
var properties = ExceptionPropertyExtractor.ExtractProperties(value);

if (options.DefaultIgnoreCondition == JsonIgnoreCondition.WhenWritingNull)
properties = properties.Where(prop => prop.Value != null);

var props = properties.ToArray();
if (!props.Any())
if (props.Length == 0)
return;

writer.WriteStartObject();
Expand All @@ -76,17 +75,16 @@
switch (prop.Value)
{
case IntPtr intPtr:
writer.WriteNumber(ApplyPropertyNamingPolicy(prop.Name, options), intPtr.ToInt64());
writer.WriteNumber(ApplyPropertyNamingPolicy(prop.Key, options), intPtr.ToInt64());

Check warning on line 78 in libraries/src/AWS.Lambda.Powertools.Logging/Internal/Converters/ExceptionConverter.cs

View check run for this annotation

Codecov / codecov/patch

libraries/src/AWS.Lambda.Powertools.Logging/Internal/Converters/ExceptionConverter.cs#L78

Added line #L78 was not covered by tests
break;
case UIntPtr uIntPtr:
writer.WriteNumber(ApplyPropertyNamingPolicy(prop.Name, options), uIntPtr.ToUInt64());
writer.WriteNumber(ApplyPropertyNamingPolicy(prop.Key, options), uIntPtr.ToUInt64());

Check warning on line 81 in libraries/src/AWS.Lambda.Powertools.Logging/Internal/Converters/ExceptionConverter.cs

View check run for this annotation

Codecov / codecov/patch

libraries/src/AWS.Lambda.Powertools.Logging/Internal/Converters/ExceptionConverter.cs#L81

Added line #L81 was not covered by tests
break;
case Type propType:
writer.WriteString(ApplyPropertyNamingPolicy(prop.Name, options), propType.FullName);
writer.WriteString(ApplyPropertyNamingPolicy(prop.Key, options), propType.FullName);

Check warning on line 84 in libraries/src/AWS.Lambda.Powertools.Logging/Internal/Converters/ExceptionConverter.cs

View check run for this annotation

Codecov / codecov/patch

libraries/src/AWS.Lambda.Powertools.Logging/Internal/Converters/ExceptionConverter.cs#L84

Added line #L84 was not covered by tests
break;
default:
writer.WritePropertyName(ApplyPropertyNamingPolicy(prop.Name, options));
JsonSerializer.Serialize(writer, prop.Value, options);
case string propString:
writer.WriteString(ApplyPropertyNamingPolicy(prop.Key, options), propString);
break;
}
}
Expand Down
Loading
Loading