Feature request: Create an environment variable to allow customer disable custom metrics

[leandrodamascena]

Use case

Original discussion: #2986

Similar to the Tracer utility (https://docs.powertools.aws.dev/lambda/python/latest/core/tracer/#environment-variables), our customers want to disable custom metrics in environments like dev and others.

More information here: https://discord.com/channels/1006478942305263677/1006478942787604491/1143575948994826260

Solution/User Experience

Create an environment variable to explicitly disable custom metrics.

Alternative solutions

No response

Acknowledgment

• This feature request meets Powertools for AWS Lambda (Python) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

[dreamorosi]

The discussion on Discord seems to suggest that this is going to be a feature for non-production environments and/or testing, if this is the case, should this be part of the existing POWERTOOLS_DEV variable described in aws-powertools/powertools-lambda#86?

[leandrodamascena]

The discussion on Discord seems to suggest that this is going to be a feature for non-production environments and/or testing, if this is the case, should this be part of the existing POWERTOOLS_DEV variable described in aws-powertools/powertools-lambda#86?

That's a great point @dreamorosi! I think we need to create an explicit variable like POWERTOOLS_METRICS_DISABLED to allow customers to only disable custom metrics and no matter the environment, it's a customer choice.

But what you're suggesting is: if the POWERTOOLS_DEV env is set, we automatically disable custom metrics the same way we do with Tracer, right? It makes lot of sense to me and I think we should go in this way.

[dreamorosi]

But what you're suggesting is: if the POWERTOOLS_DEV env is set, we automatically disable custom metrics the same way we do with Tracer, right? It makes lot of sense to me and I think we should go in this way.

Yes

I think we need to create an explicit variable like POWERTOOLS_METRICS_DISABLED to allow customers to only disable custom metrics and no matter the environment, it's a customer choice.

I think the question here is: what type of use case does this environment variable enable, and when do we want customers to use it?

The reasoning we have been using with Tracer and Event Handler is that the default behavior is for tracing to be enabled and for Event Handler to have (for example) CORS enabled & scoped down. By providing an environment we allow customers to override this behavior in particular occasions like testing or development environments.

The key proposition of these env variables is not to provide yet another configuration option but rather allow customers to temporarily tweak the behavior in some cases without having to change their code: i.e. I can test my production code without changes but I want to disable tracing because the X-Ray Daemon is not available in my testing environment.

While it's indeed a customer choice, I think it should be explicit that we want these behaviors (i.e. tracing being disabled, metrics not being emitted, event handler allowing any origins, etc.) only in development environments and not in production.

Having these settings behind an env variable that is explicitly named after the use case (POWERTOOLS_DEV) conveys this message and avoids foot guns like customer inadvertently leaves/sets POWERTOOLS_METRICS_DISABLED or POWERTOOLS_TRACER_DISABLED enabled in production & loses data.

If they really don't want to emit metrics / traces in any of their environments, including production, then they should remove the dependency altogether to avoid having dead code / code that is a no-op but still gets imported and has potential impact on their Init phase.

To sum up: I'm not against giving customers choice, but I think that we also have the responsibility to educate customers and be opinionated on topics like this, otherwise we'll end up becoming yet another k8s with dozens of knobs & handles.

This by the way, was also the aim of the POWERTOOLS_DEV RFC which also mentions a potential future deprecation of one-off variables like POWERTOOLS_EVENT_HANDLER_DEBUG:

This change future-proof additional convenience for customers and lower cognitive load on needing multiple env vars to the same end. We will create a new section in the documentation to keep track of behaviours this env var will enable.

In my opinion adding a POWERTOOLS_METRICS_DISABLED would go against that and encourage the sprawl.

With that said, I'm happy to discuss this further and commit in case everyone else wants to go in this direction.

[leandrodamascena]

I think the question here is: what type of use case does this environment variable enable, and when do we want customers to use it?

I see many cases where customers can use this: development environment, stage environment, stress testing where custom metrics don't matter, even in production if the customer just wants to disable metrics globally without changing code. There are many use cases where I don't think we can cover them all, and forcing clients to use POWERTOOLS_DEV in all those cases is not the best choice.

The key proposition of these env variables is not to provide yet another configuration option but rather allow customers to temporarily tweak the behavior in some cases without having to change their code: i.e. I can test my production code without changes but I want to disable tracing because the X-Ray Daemon is not available in my testing environment.

I think this is a configuration that makes the adoption of a library like Powertools more flexible and allows customers to choose their desired behavior. As I said in the first paragraph, I can't think of all the use cases customers might have, but I would like to have the option to only disable custom business metrics if I'm running a stress test in the pre-production environment.

If they really don't want to emit metrics / traces in any of their environments, including production, then they should remove the dependency altogether to avoid having dead code / code that is a no-op but still gets imported and has potential impact on their Init phase.

This is true for Tracer, where the customer can remove the X-Ray dependency, but for Metrics there is no such option as our default provider uses print/console.log to generate EMF to the standard output CloudWatch.

To sum up: I'm not against giving customers choice, but I think that we also have the responsibility to educate customers and be opinionated on topics like this, otherwise we'll end up becoming yet another k8s with dozens of knobs & handles.

Yes, I agree that we should avoid creating a complex ecosystem where customers need dozens of resources to deploy a Hello World and we're still far from that. We're just giving the customer more flexibility in a controlled environment.

With that said, I'm happy to discuss this further and commit in case everyone else wants to go in this direction.

I fully agree that we need to listen to others before making a decision.

Thank you very much for raising all these points, as we are always doing an amazing job when we try to cover as many situations as possible to deliver the best customer experience. 🚀

cc @rubenfonseca @heitorlessa

[rubenfonseca]

Great points here! I also believe that running a staging / QA environment with POWERTOOLS_DEV would not be smart. Besides, you might have different reasons for enabling metrics BUT not logs, or vice-verse. So I believe integrating with POWERTOOLS_DEV and having a dedicated environment variable makes sense here.