Add installation & subscription options for Otel log ingestion lambda

README.md

@@ -16,6 +16,9 @@ A CLI to install the New Relic AWS Lambda integration and layers.
     * [AWS Lambda Layers](#aws-lambda-layers)
     * [AWS Lambda Functions](#aws-lambda-functions)
     * [NewRelic Log Subscription](#newRelic-log-subscription)
+    * [NewRelic Otel Log Ingestions](#newRelic-otel-ingestions-install)
+    * [NewRelic Otel Log Subscription](#newRelic-otel-log-subscription)
+
 * **[Docker](#docker)**
 * **[Contributing](#contributing)**
 * **[Code Style](#code-style)**
@@ -255,6 +258,73 @@ newrelic-lambda subscriptions uninstall --function <name or arn>
 | `--aws-profile` or `-p` | No | The AWS profile to use for this command. Can also use `AWS_PROFILE`. Will also check `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables if not using AWS CLI. |
 | `--aws-region` or `-r` | No | The AWS region this function is located. Can use `AWS_DEFAULT_REGION` environment variable. Defaults to AWS session region. |
 
+### NewRelic Otel Ingestions Install
+
+#### Install Otel Log Ingestion
+
+```bash
+newrelic-lambda otel-ingestions install \
+    --nr-account-id <account id> \
+    --nr-api-key <api key>
+```
+
+| Option | Required? | Description |
+|--------|-----------|-------------|
+| `--nr-account-id` or `-a` | Yes | The [New Relic Account ID](https://docs.newrelic.com/docs/accounts/install-new-relic/account-setup/account-id) for this integration. Can also use the `NEW_RELIC_ACCOUNT_ID` environment variable. |
+| `--nr-api-key` or `-k` | Yes | Your [New Relic User API Key](https://docs.newrelic.com/docs/apis/get-started/intro-apis/types-new-relic-api-keys#user-api-key). Can also use the `NEW_RELIC_API_KEY` environment variable. |
+| `--linked-account-name` or `-n` | No | A label for the New Relic Linked Account. This is how this integration will appear in New Relic. Defaults to "New Relic Lambda Integration - <AWS Account ID>". |
+| `--memory-size` or `-m` | No | Memory size (in MiB) for the New Relic log ingestion function. Default to 128MB. |
+| `--nr-region` | No | The New Relic region to use for the integration. Can use the `NEW_RELIC_REGION` environment variable. Can be either `eu` or `us`. Defaults to `us`. |
+| `--timeout` or `-t` | No | Timeout (in seconds) for the New Relic log ingestion function. Defaults to 30 seconds. |
+| `--role-name` | No | Role name for the ingestion function. If you prefer to create and manage an IAM role for the function to assume out of band, do so and specify that role's name here. This avoids needing CAPABILITY_IAM. |
+| `--aws-profile` or `-p` | No | The AWS profile to use for this command. Can also use `AWS_PROFILE`. Will also check `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables if not using AWS CLI. |
+| `--aws-region` or `-r` | No | The AWS region for the integration. Can use `AWS_DEFAULT_REGION` environment variable. Defaults to AWS session region. |
+| `--aws-role-policy` | No | Specify an alternative IAM role policy ARN for this integration. |
+| `--tag <key> <value>` | No | Sets tags on the CloudFormation Stacks this CLI creates. Can be used multiple times, example: `--tag key1 value1 --tag key2 value2`. |
+| `--stackname` | No | The AWS Cloudformation stack name which contains the newrelic-log-ingestion lambda function. If no value is provided, the command searches for the NewRelicLogIngestion stack |
+
+
+#### Uninstall Otel Log Ingestion
+
+```bash
+newrelic-lambda otel-ingestions uninstall
+```
+
+| Option | Required? | Description |
+|--------|-----------|-------------|
+| `--aws-profile` or `-p` | No | The AWS profile to use for this command. Can also use `AWS_PROFILE`. Will also check `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables if not using AWS CLI. |
+| `--aws-region` or `-r` | No | The AWS region for the integration. Can use `AWS_DEFAULT_REGION` environment variable. Defaults to AWS session region. |
+| `--force` or `-f` | No | Forces uninstall non-interactively |
+| `--nr-account-id` or `-a` | No | The [New Relic Account ID](https://docs.newrelic.com/docs/accounts/install-new-relic/account-setup/account-id) for the integration. Only required if also uninstalling the New Relic AWS Lambda integration. Can also use the `NEW_RELIC_ACCOUNT_ID` environment variable. |
+| `--stackname` | No | The AWS Cloudformation stack name which contains the newrelic-log-ingestion lambda function. If no value is provided, the command searches for the NewRelicLogIngestion stack |
+
+### NewRelic Otel Log Subscription
+
+#### Install Otel Log Subscription
+
+```bash
+newrelic-lambda subscriptions install --function <name or arn> --otel
+```
+
+| Option | Required? | Description |
+|--------|-----------|-------------|
+| `--function` or `-f` | Yes | The AWS Lambda function name or ARN in which to remove a log subscription. Can provide multiple `--function` arguments. Will also accept `all`, `installed` and `not-installed` similar to `newrelic-lambda functions list`. |
+| `--otel` or `-o` | Yes | Use this flag to install subscription filters for Lambdas that are instrumented with OpenTelemetry (Otel) |
+| `--stackname` | No | The AWS Cloudformation stack name which contains the newrelic-otel-log-ingestion lambda function. If no value is provided, the command searches for the NewRelicOtelLogIngestion stack |
+
+
+#### Uninstall Otel Log Subscription
+
+```bash
+newrelic-lambda subscriptions uninstall --function <name or arn> --otel
+```
+
+| Option | Required? | Description |
+|--------|-----------|-------------|
+| `--function` or `-f` | Yes | The AWS Lambda function name or ARN in which to remove a log subscription. Can provide multiple `--function` arguments. Will also accept `all`, `installed` and `not-installed` similar to `newrelic-lambda functions list`. |
+| `--otel` or `-o` | Yes | Use this flag to install subscription filters for Lambdas that are instrumented with OpenTelemetry (Otel) |
+| `--stackname` | No | The AWS Cloudformation stack name which contains the newrelic-log-ingestion lambda function. If no value is provided, the command searches for the NewRelicLogIngestion stack |
+
 ## Docker
 
 Now, you can run newrelic-lambda-cli as a container.

newrelic_lambda_cli/api.py

@@ -21,6 +21,9 @@
     IntegrationInstall,
     IntegrationUpdate,
     LayerInstall,
+    OtelIngestionInstall,
+    OtelIngestionUninstall,
+    OtelIngestionUpdate,
 )
 from newrelic_lambda_cli.utils import parse_arn
 
@@ -332,8 +335,11 @@
         return res
 
 
-def validate_gql_credentials(input):
-    assert isinstance(input, (IntegrationInstall, IntegrationUpdate, LayerInstall))
+def validate_gql_credentials(input, otel: bool = False):
+    if otel:
+        assert isinstance(input, (OtelIngestionInstall, OtelIngestionUpdate))
+    else:
+        assert isinstance(input, (IntegrationInstall, IntegrationUpdate, LayerInstall))
 
     try:
         return NewRelicGQL(input.nr_account_id, input.nr_api_key, input.nr_region)

newrelic_lambda_cli/cli/__init__.py

@@ -2,7 +2,13 @@
 
 import click
 
-from newrelic_lambda_cli.cli import functions, integrations, layers, subscriptions
+from newrelic_lambda_cli.cli import (
+    functions,
+    integrations,
+    layers,
+    otel_ingestions,
+    subscriptions,
+)
 
 
 @click.group()
@@ -17,6 +23,7 @@ def cli(ctx, verbose):
 def register_groups(group):
     functions.register(group)
     integrations.register(group)
+    otel_ingestions.register(group)
     layers.register(group)
     subscriptions.register(group)
 

newrelic_lambda_cli/cli/otel_ingestions.py

@@ -0,0 +1,242 @@
+# -*- coding: utf-8 -*-
+
+import boto3
+import click
+
+from newrelic_lambda_cli import api, otel_ingestions, permissions
+from newrelic_lambda_cli.types import (
+    OtelIngestionInstall,
+    OtelIngestionUninstall,
+    OtelIngestionUpdate,
+)
+from newrelic_lambda_cli.cli.decorators import add_options, AWS_OPTIONS, NR_OPTIONS
+from newrelic_lambda_cli.cliutils import done, failure
+
+
+@click.group(name="otel-ingestions")
+def ingestion_group():
+    """Manage New Relic AWS Lambda Otel Log Ingestion lambda"""
+    pass
+
+
+def register(group):
+    group.add_command(ingestion_group)
+    ingestion_group.add_command(install)
+    ingestion_group.add_command(uninstall)
+    ingestion_group.add_command(update)
+
+
+@click.command(name="install")
+@add_options(AWS_OPTIONS)
+@click.option(
+    "--aws-role-policy",
+    help="Alternative AWS role policy to use for integration",
+    metavar="<arn>",
+)
+@click.option(
+    "--stackname",
+    default="NewRelicOtelLogIngestion",
+    help="The AWS Cloudformation stack name which contains the newrelic-log-ingestion lambda function",
+    metavar="<arn>",
+    show_default=False,
+    required=False,
+)
+@click.option(
+    "--memory-size",
+    "-m",
+    default=128,
+    help="Memory size (in MiB) for the log ingestion function",
+    metavar="<size>",
+    show_default=True,
+    type=click.INT,
+)
+@click.option(
+    "--linked-account-name",
+    "-n",
+    help="New Relic Linked Account Label",
+    metavar="<name>",
+    required=False,
+)
+@add_options(NR_OPTIONS)
+@click.option(
+    "--timeout",
+    "-t",
+    default=30,
+    help="Timeout (in seconds) for the New Relic log ingestion function",
+    metavar="<secs>",
+    show_default=True,
+    type=click.INT,
+)
+@click.option(
+    "--role-name",
+    default=None,
+    help="The name of a pre-created execution role for the log ingest function",
+    metavar="<role_name>",
+    show_default=False,
+)
+@click.option(
+    "--integration-arn",
+    default=None,
+    help="The ARN of a pre-existing AWS IAM role for the New Relic Lambda integration",
+    metavar="<role_arn>",
+    show_default=False,
+)
+@click.option(
+    "--tag",
+    "tags",
+    default=[],
+    help="A tag to be added to the CloudFormation Stack (can be used multiple times)",
+    metavar="<key> <value>",
+    multiple=True,
+    nargs=2,
+)
+@click.pass_context
+def install(ctx, **kwargs):
+    """Install New Relic AWS Lambda Integration"""
+    input = OtelIngestionInstall(session=None, verbose=ctx.obj["VERBOSE"], **kwargs)
+
+    input = input._replace(
+        session=boto3.Session(
+            profile_name=input.aws_profile, region_name=input.aws_region
+        )
+    )
+
+    if input.aws_permissions_check:
+        permissions.ensure_integration_install_permissions(input)
+
+    click.echo("Validating New Relic credentials")
+    gql_client = api.validate_gql_credentials(input, otel=True)
+
+    click.echo("Retrieving integration license key")
+    nr_license_key = api.retrieve_license_key(gql_client)
+
+    install_success = True
+
+    click.echo("Creating newrelic-otel-log-ingestion Lambda function in AWS account")
+    res = otel_ingestions.install_otel_log_ingestion(input, nr_license_key)
+    install_success = res and install_success
+
+
+@click.command(name="uninstall")
+@add_options(AWS_OPTIONS)
+@click.option(
+    "--nr-account-id",
+    "-a",
+    envvar="NEW_RELIC_ACCOUNT_ID",
+    help="New Relic Account ID",
+    metavar="<id>",
+    required=False,
+    type=click.INT,
+)
+@click.option(
+    "--stackname",
+    default="NewRelicOtelLogIngestion",
+    help="The AWS Cloudformation stack name which contains the newrelic-log-ingestion lambda function",
+    metavar="<arn>",
+    show_default=False,
+    required=False,
+)
+@click.option("--force", "-f", help="Force uninstall non-interactively", is_flag=True)
+def uninstall(**kwargs):
+    """Uninstall New Relic AWS Lambda Integration"""
+    input = OtelIngestionUninstall(session=None, **kwargs)
+
+    input = input._replace(
+        session=boto3.Session(
+            profile_name=input.aws_profile, region_name=input.aws_region
+        )
+    )
+
+    if input.aws_permissions_check:
+        permissions.ensure_integration_uninstall_permissions(input)
+
+    otel_ingestions.remove_otel_log_ingestion_function(input)
+
+    done("Uninstall Complete")
+
+
+@click.command(name="update")
+@add_options(AWS_OPTIONS)
+@click.option(
+    "--enable-logs/--disable-logs",
+    default=None,
+    help="Determines if logs are forwarded to New Relic Logging",
+)
+@click.option(
+    "--stackname",
+    default="NewRelicLogIngestion",
+    help="The AWS Cloudformation stack name which contains the newrelic-log-ingestion lambda function",
+    metavar="<arn>",
+    show_default=False,
+    required=False,
+)
+@click.option(
+    "--memory-size",
+    "-m",
+    help="Memory size (in MiB) for the log ingestion function",
+    metavar="<size>",
+    type=click.INT,
+)
+@add_options(NR_OPTIONS)
+@click.option(
+    "--timeout",
+    "-t",
+    help="Timeout (in seconds) for the New Relic log ingestion function",
+    metavar="<secs>",
+    type=click.INT,
+)
+@click.option(
+    "--role-name",
+    default=None,
+    help="The name of a new pre-created execution role for the log ingest function",
+    metavar="<role_name>",
+    show_default=False,
+)
+@click.option(
+    "--stackname",
+    default="NewRelicOtelLogIngestion",
+    help="The AWS Cloudformation stack name which contains the newrelic-log-ingestion lambda function",
+    metavar="<arn>",
+    show_default=False,
+    required=False,
+)
+@click.option(
+    "--enable-license-key-secret/--disable-license-key-secret",
+    default=True,
+    show_default=True,
+    help="Enable/disable the license key managed secret",
+)
+@click.option(
+    "--tag",
+    "tags",
+    default=[],
+    help="A tag to be added to the CloudFormation Stack (can be used multiple times)",
+    metavar="<key> <value>",
+    multiple=True,
+    nargs=2,
+)
+def update(**kwargs):
+    """UpdateNew Relic AWS Lambda Integration"""
+    input = OtelIngestionUpdate(session=None, **kwargs)
+
+    input = input._replace(
+        session=boto3.Session(
+            profile_name=input.aws_profile, region_name=input.aws_region
+        )
+    )
+
+    if input.aws_permissions_check:
+        permissions.ensure_integration_install_permissions(input)
+
+    update_success = True
+
+    click.echo(
+        "Updating newrelic-aws-otel-log-ingestion Lambda function in AWS account"
+    )
+    res = otel_ingestions.update_otel_log_ingestion(input)
+    update_success = res and update_success
+
+    if update_success:
+        done("Update Complete")
+    else:
+        failure("Update Incomplete. See messages above for details.", exit=True)