tests: Standardize acceptance test naming #17847
Description
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Fixed by #21306
Description
Primary Goal
Acceptance test names should be standardized to facilitate one primary goal:
- Run all tests for a given AWS service (e.g.
ec2) with one wildcard e.g.-run='TestAccAWSEC2'
Second Goals
Secondary goals might include:
- Reduce the number of changes needed in test names
- Run all data sources for a given service with one wildcard e.g.
-run='TestAccAWSEC2DataSource' - Run all data source for all services with a single wildcard (likely incompatible with test #1 and resource/ebs_snapshot: Add support for tags #3)
- Run all resource for a given service with one wildcard e.g.
-run='TestAccAWSEC2Resource'
Problems
- Older tests (and resources, data sources) often do not include the service name (e.g.,
TestAccAWSAMI_,TestAccAWSVPC_,TestAccAWSALBTargetGroup_) - Some tests do not follow convention with a
TestAccAWSprefix and useTestAccAwsinstead - Some tests include
DataSourcebeforeAWS/Aws(e.g.,TestAccDataSourceAwsEbsVolumes_)
Proposal
Use CI-testing (semgrep or tfproviderlint) to require the following (after fixing existing problems):
- All tests are prefixed with
TestAcc("AWS" and "Aws" are dropped) - After the prefix, all tests include the "service test name".
- For non-service tests (e.g.
TestAccAWSCallerIdentity_), after the prefix, include "Meta." - For data sources, immediately before the first underscore, include "DataSource."
Service Test Names
| service package | "service test name" |
|---|---|
| accessanalyzer | AccessAnalyzer |
| acm | ACM |
| acmpca | ACMPCA |
| alexaforbusiness | AlexaForBusiness |
| amplify | Amplify |
| amplifybackend | AmplifyBackend |
| apigateway | APIGateway |
| apigatewaymanagementapi | APIGatewayManagementAPI |
| apigatewayv2 | APIGatewayV2 |
| appconfig | AppConfig |
| appflow | AppFlow |
| appintegrationsservice | AppIntegrations (*) |
| applicationautoscaling | ApplicationAutoScaling |
| applicationdiscoveryservice | ApplicationDiscovery (*) |
| applicationinsights | ApplicationInsights |
| appmesh | AppMesh |
| appregistry | AppRegistry |
| appstream | AppStream |
| appsync | AppSync |
| athena | Athena |
| auditmanager | AuditManager |
| augmentedairuntime | AugmentedAiruntime |
| autoscaling | AutoScaling |
| autoscalingplans | AutoScalingPlans |
| backup | Backup |
| batch | Batch |
| braket | Braket |
| budgets | Budgets |
| chime | Chime |
| cloud9 | Cloud9 |
| clouddirectory | CloudDirectory |
| cloudformation | CloudFormation |
| cloudfront | CloudFront |
| cloudhsm | CloudHSM |
| cloudhsmv2 | CloudHSMV2 |
| cloudsearch | CloudSearch |
| cloudsearchdomain | CloudSearchDomain |
| cloudtrail | CloudTrail |
| cloudwatch | CloudWatch |
| cloudwatchevents | CloudWatchEvents |
| cloudwatchlogs | CloudWatchLogs |
| codeartifact | CodeArtifact |
| codebuild | CodeBuild |
| codecommit | CodeCommit |
| codedeploy | CodeDeploy |
| codeguruprofiler | CodeGuruProfiler |
| codegurureviewer | CodeGuruReviewer |
| codepipeline | CodePipeline |
| codestar | CodeStar |
| codestarconnections | CodeStarConnections |
| codestarnotifications | CodeStarNotifications |
| cognitoidentity | CognitoIdentity |
| cognitoidentityprovider | CognitoIdentityProvider |
| cognitosync | CognitoSync |
| comprehend | Comprehend |
| comprehendmedical | ComprehendMedical |
| computeoptimizer | ComputeOptimizer |
| configservice | Config (*) |
| connect | Connect |
| connectcontactlens | ConnectContactLens |
| connectparticipant | ConnectParticipant |
| costandusagereportservice | CUR (**) |
| costexplorer | CostExplorer |
| customerprofiles | CustomerProfiles |
| databasemigrationservice | DMS (**) |
| dataexchange | DataExchange |
| datapipeline | DataPipeline |
| datasync | DataSync |
| dax | DAX |
| detective | Detective |
| devicefarm | DeviceFarm |
| devopsguru | DevOpsGuru |
| directconnect | DirectConnect |
| directoryservice | Directory (*) |
| dlm | DLM |
| docdb | DocDB |
| dynamodb | DynamoDB |
| dynamodbattribute | DynamoDBAttribute |
| dynamodbstreams | DynamoDBStreams |
| ebs | EBS |
| ec2 | EC2 |
| ec2instanceconnect | EC2InstanceConnect |
| ecr | ECR |
| ecrpublic | ECRPublic |
| ecs | ECS |
| efs | EFS |
| eks | EKS |
| elasticache | ElastiCache |
| elasticbeanstalk | ElasticBeanstalk |
| elasticinference | ElasticInference |
| elasticsearchservice | ElasticSearch (*) |
| elastictranscoder | ElasticTranscoder |
| elb | ELB |
| elbv2 | ELBV2 |
| emr | EMR |
| emrcontainers | EMRContainers |
| eventbridge | EventBridge |
| expression | Expression |
| firehose | Firehose |
| fms | FMS |
| forecastqueryservice | ForecastQuery (*) |
| forecastservice | Forecast (*) |
| frauddetector | FraudDetector |
| fsx | FSx |
| gamelift | GameLift |
| glacier | Glacier |
| globalaccelerator | GlobalAccelerator |
| glue | Glue |
| gluedatabrew | GlueDataBrew |
| greengrass | Greengrass |
| greengrassv2 | GreengrassV2 |
| groundstation | GroundStation |
| guardduty | GuardDuty |
| health | Health |
| healthlake | HealthLake |
| honeycode | HoneyCode |
| iam | IAM |
| identitystore | IdentityStore |
| imagebuilder | ImageBuilder |
| inspector | Inspector |
| iot | IoT |
| iot1clickdevicesservice | IoT1ClickDevices (*) |
| iot1clickprojects | IoT1ClickProjects |
| iotanalytics | IoTAnalytics |
| iotdataplane | IoTDataPlane |
| iotdeviceadvisor | IoTDeviceAdvisor |
| iotevents | IoTEvents |
| ioteventsdata | IoTEventsData |
| iotfleethub | IoTFleetHub |
| iotjobsdataplane | IoTJobsDataPlane |
| iotsecuretunneling | IoTSecureTunneling |
| iotsitewise | IoTSiteWise |
| iotthingsgraph | IoTThingsGraph |
| iotwireless | IoTWireless |
| ivs | IVS |
| kafka | Kafka |
| kendra | Kendra |
| kinesis | Kinesis |
| kinesisanalytics | KinesisAnalytics |
| kinesisanalyticsv2 | KinesisAnalyticsV2 |
| kinesisvideo | KinesisVideo |
| kinesisvideoarchivedmedia | KinesisVideoArchivedMedia |
| kinesisvideomedia | KinesisVideoMedia |
| kinesisvideosignalingchannels | KinesisVideoSignalingChannels |
| kms | KMS |
| lakeformation | LakeFormation |
| lambda | Lambda |
| lexmodelbuildingservice | Lex (**) |
| lexmodelsv2 | LexModelsV2 |
| lexruntimeservice | LexRuntime (*) |
| lexruntimev2 | LexRuntimeV2 |
| licensemanager | LicenseManager |
| lightsail | Lightsail |
| locationservice | Location (*) |
| lookoutforvision | LookoutForVision |
| machinelearning | MachineLearning |
| macie | Macie |
| macie2 | Macie2 |
| managedblockchain | ManagedBlockchain |
| marketplacecatalog | MarketplaceCatalog |
| marketplacecommerceanalytics | MarketplaceCommerceAnalytics |
| marketplaceentitlementservice | MarketplaceEntitlement (*) |
| marketplacemetering | MarketplaceMetering |
| mediaconnect | MediaConnect |
| mediaconvert | MediaConvert |
| medialive | MediaLive |
| mediapackage | MediaPackage |
| mediapackagevod | MediaPackageVOD |
| mediastore | MediaStore |
| mediastoredata | MediaStoreData |
| mediatailor | MediaTailor |
| meta | Meta (not an AWS service) |
| migrationhub | MigrationHub |
| migrationhubconfig | MigrationHubConfig |
| mobile | Mobile |
| mobileanalytics | MobileAnalytics |
| mq | MQ |
| mturk | MTurk |
| mwaa | MWAA |
| neptune | Neptune |
| networkfirewall | NetworkFirewall |
| networkmanager | NetworkManager |
| opsworks | OpsWorks |
| opsworkscm | OpsWorksCM |
| organizations | Organizations |
| outposts | Outposts |
| personalize | Personalize |
| personalizeevents | PersonalizeEvents |
| personalizeruntime | PersonalizeRuntime |
| pi | PI |
| pinpoint | Pinpoint |
| pinpointemail | PinpointEmail |
| pinpointsmsvoice | PinpointSMSVoice |
| polly | Polly |
| pricing | Pricing |
| prometheusservice | Prometheus (*) |
| qldb | QLDB |
| qldbsession | QLDBSession |
| quicksight | QuickSight |
| ram | RAM |
| rds | RDS |
| rdsdataservice | RDSData (*) |
| rdsutils | RDSUtils |
| redshift | Redshift |
| redshiftdataapiservice | RedshiftDataAPI (*) |
| rekognition | Rekognition |
| resourcegroups | ResourceGroups |
| resourcegroupstaggingapi | ResourceGroupStaggingAPI |
| robomaker | RoboMaker |
| route53 | Route53 |
| route53domains | Route53Domains |
| route53resolver | Route53Resolver |
| s3 | S3 |
| s3control | S3Control |
| s3crypto | S3Crypto |
| s3manager | S3Manager |
| s3outposts | S3Outposts |
| sagemaker | SageMaker |
| sagemakeredgemanager | SageMakerEdgeManager |
| sagemakerfeaturestoreruntime | SageMakerFeatureStoreRuntime |
| sagemakerruntime | SageMakerRuntime |
| savingsplans | SavingsPlans |
| schemas | Schemas |
| secretsmanager | SecretsManager |
| securityhub | SecurityHub |
| serverlessapplicationrepository | ServerlessApplicationRepository |
| servicecatalog | ServiceCatalog |
| servicediscovery | ServiceDiscovery |
| servicequotas | ServiceQuotas |
| ses | SES |
| sesv2 | SESV2 |
| sfn | SFN |
| shield | Shield |
| sign | Sign |
| signer | Signer |
| simpledb | SimpleDB |
| sms | SMS |
| snowball | Snowball |
| sns | SNS |
| sqs | SQS |
| ssm | SSM |
| sso | SSO |
| ssoadmin | SSOAdmin |
| ssooidc | SSOOIDC |
| storagegateway | StorageGateway |
| sts | STS |
| support | Support |
| swf | SWF |
| synthetics | Synthetics |
| textract | Textract |
| timestreamquery | TimeStreamQuery |
| timestreamwrite | TimeStreamWrite |
| transcribeservice | Transcribe (*) |
| transcribestreamingservice | TranscribeStreaming (*) |
| transfer | Transfer |
| translate | Translate |
| waf | WAF |
| wafregional | WAFRegional |
| wafv2 | WAFV2 |
| wellarchitected | WellArchitected |
| workdocs | WorkDocs |
| worklink | WorkLink |
| workmail | WorkMail |
| workmailmessageflow | WorkMailMessageFlow |
| workspaces | WorkSpaces |
| xray | XRay |
(*) Indicates "service" dropped
(**) Indicates changes beyond just dropping "service"
New or Affected Resource(s)
- aws_* (most)