roachtest: cluster_creation failed

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-26.1 @ e15b1ecc444676ba53266770e0b758de9ae72d22:

test cdc/frontier-persistence-benchmark/interval=10m/tables=10/ranges=1000 failed: (test_runner.go:990).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.4xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-QwczjNpONpmi3wvdfBJYQ-C6ON4= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-26-1},{Key=created,Value=2025-12-11T07:44:24Z},{Key=Name,Value=teamcity-20847153-1765437052-31-n4cpu16-0003},{Key=Cluster,Value=teamcity-20847153-1765437052-31-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-20847153-1765437052-31-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-26-1},{Key=created,Value=2025-12-11T07:44:24Z},{Key=Name,Value=teamcity-20847153-1765437052-31-n4cpu16-0003},{Key=Cluster,Value=teamcity-20847153-1765437052-31-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-20847153-1765437052-31-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64}] --user-data file:///tmp/aws-startup-script2599156857 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping246025332 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m6id.4xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m6id.4xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

• cloud=aws
• coverageBuild=false
• cpu=16
• runtimeAssertionsBuild=false
• ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

• roachtest: cluster_creation failed #159040 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159039 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159038 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159032 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159031 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2]
• roachtest: cluster_creation failed #158857 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158856 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158855 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158854 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158853 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-57743

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-26.1 @ e15b1ecc444676ba53266770e0b758de9ae72d22:

test cdc/frontier-persistence-benchmark/interval=10m/tables=100/ranges=100 failed: (test_runner.go:990).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.4xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-QwczjNpONpmi3wvdfBJYQ-C6ON4= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-26-1},{Key=Cluster,Value=teamcity-20847153-1765437052-34-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-20847153-1765437052-34-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-12-11T07:46:19Z},{Key=Name,Value=teamcity-20847153-1765437052-34-n4cpu16-0002}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-26-1},{Key=Cluster,Value=teamcity-20847153-1765437052-34-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-20847153-1765437052-34-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2025-12-11T07:46:19Z},{Key=Name,Value=teamcity-20847153-1765437052-34-n4cpu16-0002}] --user-data file:///tmp/aws-startup-script1817972267 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3387553320 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m6id.4xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m6id.4xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]

Parameters:

• cloud=aws
• coverageBuild=false
• cpu=16
• runtimeAssertionsBuild=false
• ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

• roachtest: cluster_creation failed #159040 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159039 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159038 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159032 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159031 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2]
• roachtest: cluster_creation failed #158857 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158856 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158855 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158854 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158853 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-26.1 @ e15b1ecc444676ba53266770e0b758de9ae72d22:

test cdc/frontier-persistence-benchmark/interval=5s/tables=10000/ranges=1 failed: (test_runner.go:990).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7g.xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-QwczjNpONpmi3wvdfBJYQ-C6ON4= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-26-1},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-20847153-1765437052-48-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-12-11T08:01:02Z},{Key=Name,Value=teamcity-20847153-1765437052-48-n4cpu16-0004},{Key=Cluster,Value=teamcity-20847153-1765437052-48-n4cpu16},{Key=spot,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-26-1},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-20847153-1765437052-48-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-12-11T08:01:02Z},{Key=Name,Value=teamcity-20847153-1765437052-48-n4cpu16-0004},{Key=Cluster,Value=teamcity-20847153-1765437052-48-n4cpu16},{Key=spot,Value=true}] --user-data file:///tmp/aws-startup-script2365903484 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping745549908 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

• cloud=aws
• coverageBuild=false
• cpu=16
• runtimeAssertionsBuild=false
• ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

• roachtest: cluster_creation failed #159040 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159039 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159038 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159032 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159031 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2]
• roachtest: cluster_creation failed #158857 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158856 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158855 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158854 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158853 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

[cockroach-teamcity]

roachtest.cluster_creation failed with artifacts on release-26.1 @ e15b1ecc444676ba53266770e0b758de9ae72d22:

test cdc/multi-table-pts-benchmark/per-table-pts=false/num-tables=500/num-ranges=10 failed: (test_runner.go:990).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m7gd.4xlarge --image-id ami-05983a09f7dc1c18f --key-name teamcity-runner-QwczjNpONpmi3wvdfBJYQ-C6ON4= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-26-1},{Key=cluster,Value=teamcity-20847153-1765437052-52-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-12-11T08:04:07Z},{Key=Name,Value=teamcity-20847153-1765437052-52-n4cpu16-0002},{Key=Cluster,Value=teamcity-20847153-1765437052-52-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-26-1},{Key=cluster,Value=teamcity-20847153-1765437052-52-n4cpu16},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=arm64},{Key=created,Value=2025-12-11T08:04:07Z},{Key=Name,Value=teamcity-20847153-1765437052-52-n4cpu16-0002},{Key=Cluster,Value=teamcity-20847153-1765437052-52-n4cpu16},{Key=spot,Value=true},{Key=roachprod,Value=true}] --user-data file:///tmp/aws-startup-script551053464 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping3277846918 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr: 
An error occurred (MaxSpotInstanceCountExceeded) when calling the RunInstances operation: Max spot instance count exceeded: exit status 254 [owner=test-eng]

Parameters:

• cloud=aws
• coverageBuild=false
• cpu=16
• runtimeAssertionsBuild=false
• ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

• roachtest: cluster_creation failed #159040 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159039 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159038 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159032 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
• roachtest: cluster_creation failed #159031 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2]
• roachtest: cluster_creation failed #158857 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158856 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158855 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158854 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
• roachtest: cluster_creation failed #158853 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!