Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What to do when the CI IP address changes? #65128

Closed
ghost opened this issue Oct 28, 2020 · 7 comments
Closed

What to do when the CI IP address changes? #65128

ghost opened this issue Oct 28, 2020 · 7 comments

Comments

@ghost
Copy link

ghost commented Oct 28, 2020

So what are we supposed to do when the IP address of the underlying private container instance group changes? This happens from time to time upon instance restarts or redeployments.


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

@psmanhas
Copy link
Contributor

@mattp-rab Thank you for sharing your query! We are currently investigating and will get back to you shortly on this.

@psmanhas
Copy link
Contributor

psmanhas commented Oct 28, 2020

@mattp-rab I had discussion internally and got below response :

It seems to be a common ask. Currently we are suggesting to customers automating client configuration updates every time ACI IP changes. Here is an example for updating AppGw backend configuration using an Automation Account usin below script

#Variables declarations
$RGAPPGW="cases" # Resource Group of Application Gateway
$APPGWNAME="test-1" # Application Gateway name
$APPGWBKNAME="aci" # Application Gateway backend pool name
$RGACI="cases" # Resource Group of Azure Container Instance
$ACINAME="test" # Azure Container Instance name

#login to Azure
$connectionName = "AzureRunAsConnection"
try
{
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection = Get-AutomationConnection -Name $connectionName
"Logging in to Azure..."
$connectionResult = Connect-AzAccount -Tenant $servicePrincipalConnection.TenantID -ApplicationId $servicePrincipalConnection.ApplicationID
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint `
-ServicePrincipal
"Logged in."
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $.Exception
throw $
.Exception
}
}

$ipaddraci=(Get-AzContainerGroup -ResourceGroupName $RGACI -Name $ACINAME).IpAddress

$AppGw1 = Get-AzApplicationGateway -Name $APPGWNAME -ResourceGroupName $RGAPPGW

$ipaddraappgwbk1=(Get-AzApplicationGatewayBackendAddressPool -Name $APPGWBKNAME -ApplicationGateway $AppGw1)

$ipaddraappgwbk=$ipaddraappgwbk1.BackendAddresses.ToArray().IpAddress

If ("$ipaddraci" -eq "$ipaddraappgwbk") {
'IPs are equal.'

} Else {

'IPs are not equal.'
$AppGw = Get-AzApplicationGateway -Name $APPGWNAME -ResourceGroupName $RGAPPGW

$AppGw =Set-AzApplicationGatewayBackendAddressPool -ApplicationGateway $AppGw -Name $APPGWBKNAME -BackendIPAddresses "$ipaddraci"
$UpdatedAppGw = Set-AzApplicationGateway -ApplicationGateway $AppGw
}

Hope it helps!!!

@ghost
Copy link
Author

ghost commented Oct 28, 2020

Thank you very much for your response, @psmanhas. I actually wasn't expecting a response :-)

I had arrived at that process as a workaround too. In our case we are looking at deploying a private DNS zone and having each ACI instance update their record with the private DNS service upon boot. Then we reference container instances by DNS name instead of IP address.

To us to have to code in a workaround for this is a deal-breaker for using ACI in earnest. Is there a plan with an ETA for a built-in solution to this problem?

The alternative is to use Azure functions in a private VNET, but the costs are outrageous.

Any suggestions would be welcomed!

By the way, I couldn't access your referenced URL. I get redirected to https://spsprodcus2.vssps.visualstudio.com/_signedin which responds with a 401: 401 - Uh-oh, you do not have access.

@psmanhas
Copy link
Contributor

psmanhas commented Oct 28, 2020

@mattp-rab Thank you for sharing out your inputs. I will again reach out to our Product Group and will keep you updated on thread and apologies for adding up link since it wont be accessible you can indeed take a look on script updated in comment :)

@psmanhas
Copy link
Contributor

psmanhas commented Nov 3, 2020

@mattp-rab Apologies for all the delay in response and the inconvenience caused because of the issue.
I had discussion internally and below is their response:

IP changes upon restart/stop/start is expected behavior and published in our documentation.
The sample for using Runbook to automate updates to AG given IP changes is the best solution we have for the customer at this time.

Hope it helps!!!

Do let me know if you have any further queries on this.

@psmanhas
Copy link
Contributor

@mattp-rab
Hope the provided information is helpful.

We will now close this issue. If there are further questions regarding this, please tag me in a comment. I will reopen it and we will continue the discussion.

@gpltaylor
Copy link

gpltaylor commented May 3, 2022

Saying "is expected behaviour" is a little poor.
An IP address randomly changing is pointless for companies that take security seriously.

We would never expose a service directly. Everything is private, on VNET subnets with NSG and Azure Firewalls. Access is controlled by Private and Public DNS pointing using Azure App Gateway as a proxy with Network Monitoring.

This ensures only the right services can communicate with each other and is a core security principle.

Using ACI and Application Gateway is documented without any mention that an IP will suddenly swap out.

If I am making a simple Cat site demo then this is fine, but for a ISO/GDPR company, we simply can't use ACI for web services.

https://docs.microsoft.com/en-us/azure/container-instances/container-instances-application-gateway

Based on these comments, I don't think Azure POs even understands this is poor?

Solution:

  • Allow a Private IP to be assigned to the ACI
  • Implement IP affinity where a service will get the same IP after a restart
  • DNS update (Like with VMs)
  • Don't randomly restart our services, hydrate containers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants