Add MySQL workload detection by Klarsen04 · Pull Request #2177 · aws/amazon-cloudwatch-agent

Klarsen04 · 2026-06-25T14:01:23Z

Description of the issue

Adds workload detection support for MySQL database servers as part of the workload-discovery feature.

Description of changes

Implements a MySQL process detector that:

Identifies MySQL server instances by checking for the mysqld executable
Extracts the listening port from process arguments (--port/-P flag) or the MYSQL_TCP_PORT environment variable, defaulting to 3306 if neither is set. This is consistent with the upstream OTel MySQL receiver default
Always returns READY status with the resolved port

New files:

mysql.go - Main detector implementation
port.go - Port extraction logic
Unit tests for both components

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Added unit tests for MySQL detector (mysql_test.go)
Added unit tests for port extractor (port_test.go)
make fmt and make lint pass

Test on AL2:


$ sudo /tmp/workload-discovery -debug
time=2026-06-25T13:48:04.298Z level=DEBUG source=internal/detector/nvidia/nvidia.go:36 msg="Starting NVIDIA GPU detection"
time=2026-06-25T13:48:04.299Z level=DEBUG source=internal/detector/nvidia/nvidia.go:39 msg="No NVIDIA GPU devices found"
time=2026-06-25T13:48:04.347Z level=DEBUG source=cmd/workload-discovery/discovery.go:105 msg="Starting discovery" num_process=271 num_worker=8
time=2026-06-25T13:48:04.348Z level=DEBUG source=internal/detector/mysql/mysql.go:46 msg="MySQL process detected" pid=19573
time=2026-06-25T13:48:04.348Z level=DEBUG source=cmd/workload-discovery/discovery.go:201 msg="Detected supported workload(s) for process" pid=19573 categories=[MYSQL]
time=2026-06-25T13:48:04.348Z level=DEBUG source=cmd/workload-discovery/discovery.go:93 msg="Discovered metadata" elapsed=49.58251ms
[
  {
    "categories": [
      "MYSQL"
    ],
    "name": "mysql",
    "telemetry_port": 3306,
    "status": "READY"
  }
]

Requirements

Run make fmt and make fmt-sh
Run make lint

Create the core MySQL receiver translator with functional options pattern: - translator.go with endpoint, username, passfile, TLS, localhost detection - translator_test.go with unit tests for all option functions - Register mysqlreceiver.NewFactory() in defaultcomponents - Add MySQLKey and MySQL DBI component name constants to common.go - Add go.mod dependency on mysqlreceiver

Add YAML configs and translator wiring for MySQL DBI pipelines: - dbi_dbload_mysql.yaml: count connector for active sessions by wait/user/db/sql/host - dbi_topsql_mysql.yaml: signaltometrics connector for top query metrics - transform_dbi_fix_start_time_mysql.yaml: transform processor for MySQL timestamps - Update count, signaltometrics, and transform translators to handle MySQL component names - Add unit tests for MySQL connector/processor creation

Add mysql[] array schema to database_insights in schema.json: - endpoint, instance_name, username, password_file - tls.ca_file for TLS configuration - logs.file_path for server log ingestion

Extend the DBI pipeline orchestrator to support MySQL instances: - Add translator_mysql.go with dbiMysqlTranslator implementing 4 pipeline types: - metrics/dbi_mysql_N: counter metrics + DBLoad/TopSQL from connectors - logs/dbi_mysql_N: log-to-metrics via count + signaltometrics connectors - logs/dbi_mysql_rawevents_N: raw query sample/top query events - logs/dbi_mysql_serverlogs_N: MySQL server log file ingestion - Add parseDbiMysqlInstances() to parse mysql[] config array - MySQL-specific resource attributes (db.system.name = "mysql") - MySQL-specific log group paths and monitor user exclusion filter - Unit tests for pipeline wiring with single/multiple instances

Add end-to-end config translation test for MySQL DBI: - dbi_mysql_config_linux.conf: input agent config fixture - dbi_mysql_config_linux.json: parsed JSON intermediate fixture - dbi_mysql_config_linux.yaml: golden YAML output (full OTel pipeline) - .mysql_credentials: test passfile for credential loading - Register TestMySQLDbiConfig in tocwconfig_unix_test.go

- Unify the DBI pipeline translator: merge translator_mysql.go into a single dbiTranslator that branches by engine (carried on the instance config), instead of maintaining a separate MySQL translator. - Make component naming consistent across engines using an engine suffix: count/dbi_dbload_<engine>, signaltometrics/dbi_topsql_<engine>, transform/dbi_fix_start_time_<engine>, and per-instance dbi_resource_<engine>_N, dbi_scope_<engine>_N, dbi_logs_<engine>_<dest>_N, dbi_exclude_monitor_<engine>_N. PostgreSQL components now carry the _postgresql suffix for consistency. - Rename engine-specific connector/processor YAMLs for symmetry (dbi_dbload_postgresql.yaml, dbi_topsql_postgresql.yaml, transform_dbi_fix_start_time_postgresql.yaml); the connector and transform translators select the config by engine. - Add the dbi_scope transform processor (cloudwatch.source/solution) to all MySQL pipelines, matching the PostgreSQL implementation. - Use the shared named resourcedetection/opentelemetry processor. - Add the validateOttlSafe check to the unified Translate() so username and instance_name are validated for both engines. - Fix the host metrics process scraper to match the mysqld process (mysqld.*) when MySQL DBI is configured. - Remove TLS/ca_file from the MySQL schema and receiver; MySQL DBI is localhost-only and always connects with TLS disabled. - Fix the MySQL passfile test fixture to the pgpass-style format (host:port:database:username:password) and assert all seven DB Load metrics. - Regenerate golden YAMLs (mysql, postgres, combined configs).

…upport The CloudWatch Console (Cinch) creates separate SSM parameters for each workload configuration (e.g. one for database-insights-postgres and one for database-insights-mysql). When the agent loads multiple config files from the .d/ directory, the config merger needs to deep-merge the database_insights section so that both postgresql and mysql arrays coexist. Without this rule, the agent treats database_insights from two separate config sources as conflicting values and drops one, causing missing log groups in CloudWatch. Register database_insights as a SectionMergeRule under the collect rule, matching the existing pattern for opentelemetry and collect.

- Move the complete 14-metric Top SQL connector config (dbi_topsql_mysql.yaml) into this PR. The full metric set was previously committed in the MySQL workload-detection PR by mistake, which left this PR's embedded config with only 2 metrics while the golden YAML had 14 (mismatch that failed the translation unit tests). All 14 metrics match what the mysqlreceiver publishes on db.server.top_query events. - Remove the user.name attribute from all Top SQL metrics: top queries come from events_statements_summary_by_digest, aggregated by digest and schema, so the receiver does not publish user.name on top_query events (validated against the forked mysqlreceiver RecordDbServerTopQueryEvent). - Remove the duplicate dbiMysqlKey variable in favor of the shared common.DatabaseInsightsMysqlKey. - Regenerate the MySQL golden YAML.

Implement detection of mysqld processes for the workload discovery framework: - Add mysql.NewDetector that matches mysqld process by executable name - Add port extraction from --port/-P flags, MYSQL_TCP_PORT env var, or default 3306 - Add CategoryMySQL constant to metadata categories - Register MySQL detector in workload-discovery main - Unit tests for detection logic and port extraction (all flag/env variants)

Update the test credential file to use the MySQL-native INI format (Option C) with required host, port, user, and password fields.

Points to amazon-contributing/opentelemetry-collector-contrib PR #577 which replaces pgpass format with .my.cnf INI format for MySQL DBI credential files.

The DB Load connector condition only matched processlist_command == "Query", missing prepared statement executions ("Execute") used by sysbench, ORMs, JDBC, and most real-world applications. This caused DB Load metrics to undercount active sessions. Replace the command-based filter with session.status != "idle" to match all active sessions regardless of command type.

github-actions · 2026-07-20T02:49:07Z

This PR was marked stale due to lack of activity.

Adjust MySQL query sample configuration to optimize resource usage: - Increase MaxRowsPerQuery from 500 to 5000 (aligns with PostgreSQL) - Decrease MaxQuerySampleCount from 5000 to 1000 (conservative limit) This balances query plan collection with CloudWatch Logs limits.

…orkload-detection

…e-translator

…386b) Update go.mod to point to the latest commit on the upstream dbi-mysql branch which includes PR #617 (InnoDB deadlocks and transaction metrics).

…erySampleCollection.MaxRowsPerQuery from 5000 to 500 to match PostgreSQL - Change TopQueryCollection.MaxQuerySampleCount from 1000 to 5000 to match PostgreSQL - Make TopQueryCollection.CollectionInterval configurable via WithTopQueryInterval option - Set 60s interval for 'events' receiver - Add test for custom interval configuration

- Log decode errors to stderr for visibility when MySQL config is malformed - Return empty slice instead of nil to allow PostgreSQL instances to still be processed - Addresses PR review feedback about error handling

This merge brings in updates from main and resolves conflicts: - Updated MySQL DBI pipeline translator to support both MySQL and PostgreSQL engines - Fixed translateMetrics() to use engine-specific receiver logic - Fixed translateLogToMetrics() and translateRawEvents() for MySQL support - Resolved duplicate receivers in combined_v1_v2_eks_config.yaml - Fixed syntax error in translators_test.go (missing closing brace) - Updated go.mod and go.sum with latest dependencies All tests pass and lint is clean.

…e-translator

…mysql-pipeline-translator

…etection

Run make fmt fmt-sh to fix formatting issues detected by CI.

github-actions · 2026-08-11T16:55:21Z

Binary Size Report

linux/amd64

Binary	PR	vs `main` (`9884add`)	vs `v1.300072.0`
amazon-cloudwatch-agent	194.5 MB	${\color{red}▲}$ +1.2 MB (+0.6%)	${\color{red}▲}$ +1.2 MB (+0.6%)
amazon-cloudwatch-agent-config-wizard	2.0 MB	+0 B	+0 B
config-downloader	2.0 MB	+0 B	+0 B
config-translator	2.0 MB	+0 B	+0 B
start-amazon-cloudwatch-agent	2.5 MB	+0 B	+0 B
workload-discovery	3.0 MB	${\color{red}▲}$ +8.2 KB (+0.3%)	${\color{red}▲}$ +8.2 KB (+0.3%)
Total	206.1 MB	${\color{red}▲}$ +1.2 MB (+0.6%)	${\color{red}▲}$ +1.2 MB (+0.6%)

linux/amd64 amazon-cloudwatch-agent (last 9 main commits + this PR)

 195 ┤                                        
     ┤                                    ███ 
     ┤                                    ███ 
     ┤                                    ███ 
     ┤                                    ███ 
     ┤                                    ███ 
     ┤                                    ███ 
     ┤                                    ███ 
 193 ┤▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ███ 
 MB  └────────────────────────────────────────
      ab55b8d                     v1.30007PR0

Notable changes:

+1.2 MB at PR

linux/arm64

Binary	PR	vs `main` (`9884add`)	vs `v1.300072.0`
amazon-cloudwatch-agent	178.3 MB	${\color{red}▲}$ +1.2 MB (+0.7%)	${\color{red}▲}$ +1.2 MB (+0.7%)
amazon-cloudwatch-agent-config-wizard	2.0 MB	+0 B	+0 B
config-downloader	2.0 MB	+0 B	+0 B
config-translator	2.0 MB	+0 B	+0 B
start-amazon-cloudwatch-agent	2.4 MB	+0 B	+0 B
workload-discovery	2.9 MB	+0 B	+0 B
Total	189.5 MB	${\color{red}▲}$ +1.2 MB (+0.6%)	${\color{red}▲}$ +1.2 MB (+0.6%)

windows/amd64

Binary	PR	vs `main` (`9884add`)	vs `v1.300072.0`
amazon-cloudwatch-agent.exe	197.3 MB	${\color{red}▲}$ +1.2 MB (+0.6%)	${\color{red}▲}$ +1.2 MB (+0.6%)
amazon-cloudwatch-agent-config-wizard.exe	2.1 MB	+0 B	+0 B
config-downloader.exe	2.1 MB	+0 B	+0 B
config-translator.exe	2.1 MB	+0 B	+0 B
start-amazon-cloudwatch-agent.exe	2.3 MB	+0 B	+0 B
workload-discovery.exe	2.9 MB	${\color{red}▲}$ +6.1 KB (+0.2%)	${\color{red}▲}$ +6.1 KB (+0.2%)
Total	209.0 MB	${\color{red}▲}$ +1.3 MB (+0.6%)	${\color{red}▲}$ +1.3 MB (+0.6%)

Investigating size changes

Use go-size-analyzer to compare binaries:

GOEXPERIMENT=jsonv2 go install github.com/Zxilly/go-size-analyzer/cmd/gsa@latest
gsa diff --old <baseline-binary> --new <new-binary>

- Add error logging to PostgreSQL config decoder (similar to MySQL) - Fix receiver() function to properly handle PostgreSQL with conditional check - Remove duplicate combined_v1_v2_ec2_config_main.yaml file - Change instance name from 'pentest-mysql' to 'mysql-instance' in test configs - Keep per-instance scope processors for consistency and future flexibility

- Fix PostgreSQL receiver conditional to return nil for unknown engines - Add error logging to PostgreSQL and MySQL config decoders - Change instance name from 'pentest-mysql' to 'mysql-instance' in test configs - Remove duplicate combined_v1_v2_ec2_config_main.yaml file

- Fix README typo: change telemetryPort to telemetry_port (snake_case) in both MySQL and PostgreSQL READMEs - Document port detection behavior and fallback logic in MySQL README - Improve port validation: explicitly reject port 0 and out-of-range values - Distinguish between ErrExtractPort (not found) and ErrInvalidPort (malformed/invalid) - Fix test precedence: add competing env variable to CmdlineTakesPrecedence test - Add test coverage for invalid port scenarios (port 0, malformed values, out-of-range) - Rename test: Error/CmdlineSliceWithContext -> Success/DefaultPortOnCmdlineError to match actual behavior All tests pass. Lint clean.

Comment 1 - README wording: - Fix incorrect fallback description that contradicted priority chain - Now correctly states invalid values fall back to next source (env → default) - Example: mysqld --port=abc with MYSQL_TCP_PORT=3308 uses 3308, not 3306 Comment 2 - Multiple --port flags: - Fix bug where invalid value masked later valid values - Change from immediate return to continue on invalid values - Now matches MySQL server behavior: last valid occurrence wins - Example: mysqld --port=abc --port=3307 now correctly uses 3307 Comment 3 - Test coverage: - Add 15 new test cases covering all flag formats and validation paths - Equals format: --port=0, --port=abc, --port=99999 - Short flag attached: -P0, -Pabc, -P99999 - Environment variable: MYSQL_TCP_PORT with invalid values - Multiple flag scenarios to verify Comment 2 fix All tests pass. Lint clean.

…fan-out The DBI count and signaltometrics connectors were created with engine-scoped IDs (count/dbi_dbload_<engine>), so all instances of an engine shared a single connector whose output was consumed by every per-instance metrics pipeline. Each pipeline unconditionally set db.instance.name, cross-tagging active_sessions/top-query datapoints with every configured instance name (a cartesian product across instances) for mysql and postgresql. Make the connector IDs per-instance (append the instance index) in both translateMetrics and translateLogToMetrics so each instance's stream is isolated end to end.

- Add MySQL-specific case in excludeMonitorFilter for consistency with PostgreSQL pattern - MySQL only checks user.name attribute (no separate role attribute like PostgreSQL) - Add test case TestDbiExcludeMonitorMySQL to verify MySQL filter condition - Filter condition: attributes["user.name"] == configured_username - This ensures MySQL monitoring sessions are filtered from DB Load metrics

Klarsen04 added 3 commits June 24, 2026 15:58

Add MySQL config schema definition

137120b

Add mysql[] array schema to database_insights in schema.json: - endpoint, instance_name, username, password_file - tls.ca_file for TLS configuration - logs.file_path for server log ingestion

Klarsen04 requested a review from a team as a code owner June 25, 2026 14:01

Klarsen04 added the ready for testing Indicates this PR is ready for integration tests to run label Jun 25, 2026

Klarsen04 added 2 commits June 29, 2026 19:19

Klarsen04 force-pushed the larkirs/mysql-workload-detection branch 2 times, most recently from 615b5d5 to 7352bdb Compare June 30, 2026 14:11

Klarsen04 force-pushed the larkirs/mysql-workload-detection branch from 7352bdb to 3a7fea5 Compare June 30, 2026 17:16

Klarsen04 force-pushed the larkirs/mysql-workload-detection branch from 3a7fea5 to 97bfc15 Compare June 30, 2026 17:24

Klarsen04 force-pushed the larkirs/mysql-workload-detection branch from 97bfc15 to 6814222 Compare July 1, 2026 18:59

Klarsen04 force-pushed the larkirs/mysql-workload-detection branch from 6814222 to c266a98 Compare July 1, 2026 19:09

Klarsen04 added 3 commits July 2, 2026 19:43

test: Update .mysql_credentials test fixture to .my.cnf INI format

4c84ea5

Update the test credential file to use the MySQL-native INI format (Option C) with required host, port, user, and password fields.

test: Update .mysql_credentials test fixture to .my.cnf INI format

b97916d

Update the test credential file to use the MySQL-native INI format (Option C) with required host, port, user, and password fields.

chore: Update mysqlreceiver replace to .my.cnf INI format commit

2789a79

Points to amazon-contributing/opentelemetry-collector-contrib PR #577 which replaces pgpass format with .my.cnf INI format for MySQL DBI credential files.

github-actions Bot added the Stale label Jul 9, 2026

Klarsen04 force-pushed the feature/dbi-mysql branch 2 times, most recently from 042dfff to 66280f8 Compare July 10, 2026 15:38

aws deleted a comment from github-actions Bot Jul 10, 2026

github-actions Bot removed the Stale label Jul 11, 2026

Klarsen04 added 3 commits July 15, 2026 20:01

chore: Update mysqlreceiver replace to .my.cnf INI format commit

15156a6

Points to amazon-contributing/opentelemetry-collector-contrib PR #577 which replaces pgpass format with .my.cnf INI format for MySQL DBI credential files.

github-actions Bot added the Stale label Jul 20, 2026

Klarsen04 added 18 commits July 30, 2026 13:30

Merge branch 'larkirs/mysql-pipeline-translator' into larkirs/mysql

e629c72

Merge branch 'larkirs/mysql-pipeline-translator' into larkirs/mysql

8e35863

Merge branch 'larkirs/mysql-pipeline-translator' into larkirs/mysql-w…

7b6c45c

…orkload-detection

Merge remote-tracking branch 'origin/main' into larkirs/mysql-pipelin…

3660a9e

…e-translator

Merge branch 'main' into larkirs/mysql

9e310e2

Merge branch 'main' into larkirs/mysql-workload-detection

c502010

chore(dbi): update mysqlreceiver to latest dbi-mysql branch (83b5027c…

b7e259f

…386b) Update go.mod to point to the latest commit on the upstream dbi-mysql branch which includes PR #617 (InnoDB deadlocks and transaction metrics).

chore(dbi): update mysqlreceiver to latest dbi-mysql branch (83b5027c…

9a48cd1

…386b) Update go.mod to point to the latest commit on the upstream dbi-mysql branch which includes PR #617 (InnoDB deadlocks and transaction metrics).

Merge branch 'larkirs/mysql' into larkirs/mysql-workload-detection

4de19fc

fix(dbi): Add error logging for MySQL config decode failures

1aa9733

- Log decode errors to stderr for visibility when MySQL config is malformed - Return empty slice instead of nil to allow PostgreSQL instances to still be processed - Addresses PR review feedback about error handling

Merge remote-tracking branch 'origin/main' into larkirs/mysql-pipelin…

d348170

…e-translator

Merge remote-tracking branch 'origin/feature/dbi-mysql' into larkirs/…

7826e2f

…mysql-pipeline-translator

Merge larkirs/mysql-pipeline-translator into larkirs/mysql-workload-d…

674a4d2

…etection

fix: Remove extra blank line in mysql translator_test.go

879e7cb

Run make fmt fmt-sh to fix formatting issues detected by CI.

fix: Remove extra blank line in mysql translator_test.go

0f3cd76

Run make fmt fmt-sh to fix formatting issues detected by CI.

musa-asad reviewed Aug 13, 2026

View reviewed changes

Klarsen04 added 3 commits August 13, 2026 13:10

musa-asad reviewed Aug 13, 2026

View reviewed changes

Comment thread internal/detector/mysql/README.md Outdated

Comment thread internal/detector/mysql/extract/port.go Outdated

Comment thread internal/detector/mysql/extract/port_test.go

musa-asad approved these changes Aug 14, 2026

View reviewed changes

Paamicky approved these changes Aug 14, 2026

View reviewed changes

Klarsen04 added 2 commits August 17, 2026 21:54

Klarsen04 force-pushed the larkirs/mysql-workload-detection branch from 3c6b0dd to 2f30359 Compare August 18, 2026 13:38

Navigation Menu

Uh oh!

Add MySQL workload detection - #2177

Add MySQL workload detection#2177
Klarsen04 wants to merge 59 commits into
feature/dbi-mysqlfrom
larkirs/mysql-workload-detection

Klarsen04 commented Jun 25, 2026

Uh oh!

github-actions Bot commented Jul 20, 2026

Uh oh!

github-actions Bot commented Aug 11, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Klarsen04 commented Jun 25, 2026

Description of the issue

Description of changes

License

Tests

Test on AL2:

Requirements

Uh oh!

github-actions Bot commented Jul 20, 2026

Uh oh!

github-actions Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Binary Size Report

linux/amd64

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Aug 11, 2026 •

edited

Loading