Note that this project is experimental, and may change considerably before it is released.

newrelic-azure-log-ingestion

An Azure Resource Manager template to export Azure Monitoring logs, metrics and traces to New Relic.

How Does It Work?

This integration creates and configures the Azure resources necessary to efficiently export telemetry data from an Azure account into New Relic. It relies on Azure Diagnostic Settings to stream telemetry data to an Azure Event Hub, this Event Hub subsequently batches and triggers an Azure Function to handle the transport of telemetry data on to New Relic.

Currently this integration only targets App Insights. If you have other telemetry data you would like to see exported using Diagnostic Settings, tell us about your use case.

Supported Data Types

Data Type	Supported?	In New Relic One
AppAvailabilityResults	Yes	Distributed Tracing and Custom Events (as AzureAppAvailabilityResult
AppBrowserTimings	Yes	Distributed Tracing and Custom Events (as AzureAppBrowserTiming
AppDependencies	Yes	Distributed Tracing
AppEvents	Yes	Custom Events (as AzureAppEvent
AppExceptions	Yes	Distributed Tracing and Custom Events (as AzureAppException
AppMetrics	Planned	Metrics
AppPageViews	Yes	Distributed Tracing and Custom Events (as AzureAppPageView
AppPerformanceCounters	Yes	Metrics
AppRequests	Yes	Distributed Tracing and Custom Events (as AzureAppRequest
AppTraces	Yes	Logs and Logs In Context

We also plan to support Azure Resource Logs. Let us know which resources we should target first.

Installation

This integration requires both a New Relic and Azure account.

• If you don't have a NewRelic account yet, you can create one for free.
• If you don't have an Azure account yet, you can create one for free.

Install Using Azure Portal

Retrieve your New Relic Insights Insert Key.

Then click the button below to start the installation process via the Azure Portal.

Install Using Azure CLI

1. Install the Azure CLI.

2. Retrieve your New Relic Insights Insert Key.

3. Run az login to login to Azure.

4. Run az group create --name NewRelicLogs --location westus to create an Azure Resource Group. Replace NewRelicLogs with your preferred Resource Group name and westus with your prefered Azure location. This step is optional but recommended. If you have an existing Resource Group you want to use, you can skip this step.

5. Deploy the Resource Manager template:

      az deployment group create \
          --name NewRelicLogs \
          --resource-group NewRelicLogs \
          --template-uri https://raw.githubusercontent.com/newrelic/newrelic-azure-log-ingestion/main/templates/azure-log-ingestion.json \
          --parameters NewRelicInsertKey=<new relic insert key here>
   

   Make sure the --resource-group argument is the same as the one you created in the previous step. Also make sure you replace <new relic insert key here> with your New Relic Insert Key.

Getting Started

Now that you have your New Relic export stack setup in your Azure account, you need to configure your App Insights resources to start sending telemetry data to it.

Automatic Reporting with Azure Policy

You can automatically configure your App Insights resources to report telemetry to New Relic using an Azure Policy.

This is the easiest option as it only needs to be setup once and all existing and future App Insights resources will be automatically configured to report to New Relic. Keep in mind there are costs associated with exporting telemetry data from Azure. You will want to monitor integration utilization to ensure they meet your expectations.

Using Azure Portal

Click the button below to start the installation process via the Azure Portal.

Using Azure CLI

To set up this Azure Policy, run the following commands using the Azure CLI:

az policy definition create \
    --name New-Relic-Diagnostic \
    --mode All \
    --rules "https://raw.githubusercontent.com/newrelic/newrelic-azure-log-ingestion/main/policies/rules.json" 
    --params "https://raw.githubusercontent.com/newrelic/newrelic-azure-log-ingestion/main/policies/params.json"

az policy assignment create \
    --name New-Relic-Diagnostic-assignment \
    --policy New-Relic-Diagnostic \
    --assign-identity \
    --role Contributor \
    --identity-scope /subscriptions/${subscriptionId} \
    --location westus2 
    -p '{ "eventHubAuthorizationRuleId": { "value": "/subscriptions/${subscriptionId}/resourceGroups/${resource-group}/providers/Microsoft.EventHub/namespaces/${namespace-name)/authorizationRules/NewRelicLogsSharedAccessKey" } }'

az policy remediation create \
    --name New-Relic-Diagnostic-remediation \
    --policy-assignment New-Relic-Diagnostic-assignment

Replacing, as necessary, the ${subscriptionId}, ${resource-group}, and ${namespace-name} placeholders with the values appropriate to your installation.

Manually Configure Diagnostic Settings

You can manually configure Diagnostic Settings for only the App Insights resources you want to report telemetry data to New Relic. This option is more involved, but offers more fine grained control over the telemetry data being exported.

1. Find the AppInsights resource you want to export telemetry data from in the Azure Portal.
2. In the resource's menu, click Diagnostic Settings under Monitor.
3. Click Add diagnostic setting.
4. Enter a Name for your setting.
5. Under Category details, Check the box for each category of data you want to export to New Relic.
6. Under Destination details, check the Event Hub box.
7. Under the Event Hub destination configuration, you'll need to specify the following:
   • The subscription which the event hub is part of. This should be the same subscription used during the installation steps above.
   • The Event hub namespace. This should be NewRelicLogs.
   • An Event hub name. This should be newrelic-log-ingestion.
   • An Event Hub policy. The policy selected must grant the permissions necessary to stream data to the Event Hub.
8. Click Save.

After a few moments, the new setting appears in your list of settings for this resource, and logs are streamed to the specified destination as new event data is generated.

See the Azure documentation for more details.

Usage

From here, App Insights will begin streamingplatform logs and metrics out-of-the-box. However, to get the most out of your telemetry, you'll want to start instrumenting.

See below for details on how to instrument your code for your specific runtime:

• C#
• Java
• Node.js
• Python

Building

1. Run npm install to install the dependencies.
2. Run npm run build to run the build.

Testing

Run npm test to run tests.

Support

New Relic hosts and moderates an online forum where customers can interact with New Relic employees as well as other customers to get help and share best practices. Like all official New Relic open source projects, there's a related Community topic in the New Relic Explorers Hub. You can find this project's topic/threads here:

Add the url for the support thread here: discuss.newrelic.com

Contribute

We encourage your contributions to improve newrelic-azure-log-ingestion! Keep in mind that when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project.

If you have any questions, or to execute our corporate CLA (which is required if your contribution is on behalf of a company), drop us an email at opensource@newrelic.com.

A note about vulnerabilities

As noted in our security policy, New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.

If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through HackerOne.

If you would like to contribute to this project, review these guidelines.

To all contributors, we thank you! Without your contribution, this project would not be what it is today. We also host a community project page dedicated to Project Name.

License

newrelic-azure-log-ingestion is licensed under the Apache 2.0 License.