Add Health Check Endpoint

[Yash Shrivastava (alephys26)]

Background

We did not have health checks for the clusters in Heimdall. Introducing health check endpoint for greater observability and to allow features such as AWS's blue-green deployment with configurable test endpoint to shift production traffic.

Changes

This pull request introduces a comprehensive health check framework for clusters and their associated plugins. It adds a new /health API endpoint that performs health checks on active clusters using plugin-specific logic, and reports their status. To support this, a HealthCheck method is implemented for each relevant plugin, and a health_check flag is added to cluster configuration and model.

The most important changes are:

Health Check Framework and API:

• Introduced a new health check system in heimdall, including a /health HTTP endpoint that concurrently checks the health of all active clusters with health_check: true, using plugin-specific logic. The endpoint returns a JSON response with per-cluster/plugin status and overall health. (internal/pkg/heimdall/health.go, internal/pkg/heimdall/heimdall.go) [1] [2]
• Added a health_check boolean field to the Cluster struct and the cluster configuration file, allowing clusters to opt in to health checks. (pkg/object/cluster/cluster.go, configs/local.yaml) [1] [2]

Plugin Interface and Implementations:

• Defined a new HealthChecker interface in the plugin package, and implemented the HealthCheck method for all major plugins (ClickHouse, DynamoDB, ECS, Glue, Ping, Shell, Snowflake, Spark, SparkEKS, Trino). Each implementation performs a minimal operation to verify connectivity and readiness. (pkg/plugin/plugin.go, plus various plugin files) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

General Codebase Improvements:

• Added necessary imports and minor cleanups in plugin files to support health check logic. [1] [2] [3]

These changes collectively enable robust, extensible cluster health monitoring across all supported backends.

Tests

Tested with go test files. All tests passed.

[wiz-55ccc8b716]

Wiz Scan Summary

Scanner	Findings
Vulnerabilities	-
Sensitive Data	-
Secrets	-
IaC Misconfigurations	-
SAST Findings	2  1
Software Management Findings	-
Total	2  1

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

[wiz-55ccc8b716]

 SAST Finding

Hardcoded AWS Credentials in Go Application

More Details

Embedding static AWS credentials directly into a Go application using the credentials.NewStaticCredentialsProvider function poses a significant security risk. This practice exposes the AWS access keys and secret keys in plaintext within the application code, making them vulnerable to theft or misuse. If an attacker gains access to the application code or the compiled binary, they can extract the hardcoded credentials and potentially gain unauthorized access to AWS resources, leading to data breaches, financial losses, or other malicious activities.

Hardcoded credentials should never be used in production environments. Instead, applications should retrieve credentials securely from trusted sources, such as environment variables, secure key management services, or temporary credentials obtained through AWS Identity and Access Management (IAM) roles. Failing to properly manage and protect AWS credentials can lead to severe consequences, including data exfiltration, resource hijacking, and compliance violations.

Attribute	Value
Impact
Likelihood

Remediation

Hardcoding AWS credentials into an application poses a significant security risk. If the application's code is compromised or accidentally exposed, the hardcoded credentials can be easily extracted and misused by attackers to gain unauthorized access to AWS resources, potentially leading to data breaches, financial losses, and other severe consequences.

To fix this issue securely, applications should retrieve AWS credentials from secure sources at runtime, such as environment variables, AWS credential files, or AWS credential providers. This approach ensures that credentials are not embedded in the application's code and can be easily rotated or revoked if needed.

Code examples:

// VULNERABLE CODE - Hardcoded AWS credentials
import (
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/credentials"
)

creds := credentials.NewStaticCredentialsProvider(
"AKIAIOSFODNN7EXAMPLE",
"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"",
)

// SECURE CODE - Using AWS credential provider
import (
"github.com/aws/aws-sdk-go-v2/config"
)

cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
// Handle error
}

// AWS credentials are retrieved securely from the environment or other sources

Additional recommendations:

• Follow the AWS best practices for managing AWS access keys and secret access keys.
• Implement least privilege access principles by granting only the necessary permissions to AWS resources.
• Regularly rotate AWS credentials and revoke unused or compromised credentials.
• Consider using temporary security credentials (AWS STS) for enhanced security and auditing capabilities.
• Adhere to relevant security standards and guidelines, such as the AWS Security Best Practices and the OWASP Application Security Verification Standard (ASVS).

Rule ID: WS-GO-00051

[wiz-55ccc8b716]

 SAST Finding

Hardcoded AWS Credentials in Go Application

More Details

Embedding static AWS credentials directly into a Go application using the credentials.NewStaticCredentialsProvider function poses a significant security risk. This practice exposes the AWS access keys and secret keys in plaintext within the application code, making them vulnerable to theft or misuse. If an attacker gains access to the application code or the compiled binary, they can extract the hardcoded credentials and potentially gain unauthorized access to AWS resources, leading to data breaches, financial losses, or other malicious activities.

Hardcoded credentials should never be used in production environments. Instead, applications should retrieve credentials securely from trusted sources, such as environment variables, secure key management services, or temporary credentials obtained through AWS Identity and Access Management (IAM) roles. Failing to properly manage and protect AWS credentials can lead to severe consequences, including data exfiltration, resource hijacking, and compliance violations.

Attribute	Value
Impact
Likelihood

Remediation

Hardcoding AWS credentials into an application poses a significant security risk. If the application's code is compromised or accidentally exposed, the hardcoded credentials can be easily extracted and misused by attackers to gain unauthorized access to AWS resources, potentially leading to data breaches, financial losses, and other severe consequences.

To fix this issue securely, applications should retrieve AWS credentials from secure sources at runtime, such as environment variables, AWS credential files, or AWS credential providers. This approach ensures that credentials are not embedded in the application's code and can be easily rotated or revoked if needed.

Code examples:

// VULNERABLE CODE - Hardcoded AWS credentials
import (
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/credentials"
)

creds := credentials.NewStaticCredentialsProvider(
"AKIAIOSFODNN7EXAMPLE",
"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"",
)

// SECURE CODE - Using AWS credential provider
import (
"github.com/aws/aws-sdk-go-v2/config"
)

cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
// Handle error
}

// AWS credentials are retrieved securely from the environment or other sources

Additional recommendations:

• Follow the AWS best practices for managing AWS access keys and secret access keys.
• Implement least privilege access principles by granting only the necessary permissions to AWS resources.
• Regularly rotate AWS credentials and revoke unused or compromised credentials.
• Consider using temporary security credentials (AWS STS) for enhanced security and auditing capabilities.
• Adhere to relevant security standards and guidelines, such as the AWS Security Best Practices and the OWASP Application Security Verification Standard (ASVS).

Rule ID: WS-GO-00051