Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only the first function in my serverless.yml is placed in a nested stack with the VPC defined as a parameter #175

Open
william-matz opened this issue Dec 23, 2021 · 1 comment

Comments

@william-matz
Copy link

Serverless version

% sls version
Framework Core: 1.83.3 (local)
Plugin: 3.8.4
SDK: 2.3.2
Components: 2.34.9

serverless-plugin-split-stacks version

"serverless-plugin-split-stacks": "^1.11.2",

stacks-map.js
All the lambda function resources get resolved into stacks based on their domain

module.exports = (resource, logicalId) => {
  if (logicalId.startsWith("ServerlessDeploymentBucket")) {
    return;
  }
  if (logicalId.startsWith("Lambda")) {
    return;
  }
  if (logicalId.startsWith("ApiGatewayMethod")) {
    const firstKey = logicalId
      .replace("ApiGatewayMethod", "")
      .split(/(?=[A-Z])/)[0];
    return {
      destination: firstKey,
    };
  }
  if (logicalId.startsWith("ApiGatewayResource")) {
    const firstKey = logicalId
      .replace("ApiGatewayResource", "")
      .split(/(?=[A-Z])/)[0];
    return {
      destination: firstKey,
    };
  }
  if (
    logicalId.startsWith("ApiGatewayDeployment") ||
    logicalId.startsWith("Migration") ||
    resource.Type === "AWS::ApiGateway::GatewayResponse" ||
    resource.Type === "AWS::IAM::Role" ||
    logicalId.startsWith("Serverless") ||
    logicalId.startsWith("Website") ||
    logicalId.startsWith("Migration") ||
    logicalId.startsWith("Response") ||
    logicalId.startsWith("ApiGateway")
  ) {
    return;
  }
  if (logicalId.startsWith("Auth")) {
    return {
      destination: "Auth",
    };
  }
  if (logicalId.startsWith("WarmUp")) {
    return {
      destination: "WarmUp",
    };
  }
  const firstKey = logicalId.split("Dash")[0];
  return { destination: firstKey };
};

Here's a snippet from our serverless.yml file, defining configs for the split-stacks config and our VPC config, which reference VPC resources defined in external files

provider:
  vpc:
    securityGroupIds:
      - !GetAtt ServerlessSecurityGroup.GroupId
    subnetIds:
      - Ref: ServerlessSubnetA
      - Ref: ServerlessSubnetB
      - Ref: ServerlessSubnetC
 ...
 custom:
  splitStacks:
    perFunction: true
    perType: false
    perGroupFunction: false

When I run sls package with only one function in my serverless file, it correctly adds the referenced parameters in the parameters of the stack

{
  "Parameters": {
    "ServerlessSubnetCParameter": {
      "Type": "String"
    },
    "ServerlessSubnetBParameter": {
      "Type": "String"
    },
    "ServerlessSubnetAParameter": {
      "Type": "String"
    },
    "ServerlessSecurityGroupGroupIdParameter": {
      "Type": "String"
    },
  },
  "Resources": {
    "Function1": {
      "Properties": {
        "VpcConfig": {
          "SecurityGroupIds": [
            {
              "Ref": "ServerlessSecurityGroupGroupIdParameter"
            }
          ],
          "SubnetIds": [
            {
              "Ref": "ServerlessSubnetAParameter"
            },
            {
              "Ref": "ServerlessSubnetBParameter"
            },
            {
              "Ref": "ServerlessSubnetCParameter"
            }
          ]
        }
      },
    }
  },
}

But as soon as I have two or more resources, the subsequent stacks do not have the parameters referenced, and the stack with the first function still does reference those parameters

{
  "Parameters": {
    ... A few parameters, but not ServerlessSubnetAParameter, etc
  },
  "Resources": {
    "Function2": {
      "Properties": {
        "VpcConfig": {
          "SecurityGroupIds": [
            {
              "Ref": "ServerlessSecurityGroupGroupIdParameter"
            }
          ],
          "SubnetIds": [
            {
              "Ref": "ServerlessSubnetAParameter"
            },
            {
              "Ref": "ServerlessSubnetBParameter"
            },
            {
              "Ref": "ServerlessSubnetCParameter"
            }
          ]
        }
      },
    }
  },
}

I have 128 lambda functions I'm working on deploying
Here's what I've tried to deploy:

  1. The current setup
    I get the following error message:
Template format error: Unresolved resource dependencies [ServerlessSecurityGroupGroupIdParameter, ServerlessSubnetAParameter, ServerlessSubnetBParameter, ServerlessSubnetCParameter] in the Resources block of the template
  1. Sending all API gateway and Lambda Function resources to the first stack
  if (resource.Type === "AWS::Lambda::Function") {
    return { destination: "TestStack" };
  }

Doesn't work, circular dependency error

Error: The CloudFormation template is invalid: Circular dependency between resources: [TestStackNestedStack, TestFunctionNestedStack]
  1. Hard-code the deployed VPC resource ids - everything works fine, but it defeats the purpose of CF :(

Here's what I'm wondering:

  1. Is this a bug or expected behavior?
  2. Is there any way I can forcefully inject those parameters into the generated stacks?
  3. Is there any other setup (splitting technique) I can use here to solve this issue?
@william-matz
Copy link
Author

william-matz commented Dec 23, 2021

Update: I believe the root cause is described here: serverless/serverless#7206
Specifically, this comment: serverless/serverless#7206 (comment)

Switching my serverless version down to 1.59 solved this
Leaving this issue open because it seems like something this plugin should account for, though I don't understand fully understand what's going on

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant