[AWSX-1559] feat: update Python version to 3.13 (#924)

sanassi-cisse · web-flow · commit ba1039c5c6d8 · 2025-07-23T13:24:43.000+02:00
* [AWSX-1559] feat: update Python version to 3.13
* [AWSX-1559] feat: update version
* [AWSX-1559] feat: bump cfn-lint to 1.19.0
* [AWSX-1559] feat: use ddtrace v3.0.0
* [AWSX-1559] chore: update python version in docs
* [AWSX-1559] refactor: update requirements for ddd-lambda, dd, ddtrace
* [AWSX-1559] chore: update version in docs for next forwarder version
* [AWSX-1559] refactor: apply doc suggestion
diff --git a/.github/workflows/aws_integration_test.yml b/.github/workflows/aws_integration_test.yml
@@ -28,4 +28,4 @@ jobs:
 
       - name: AWS Python integration tests
         run: |
-          ./aws/logs_monitoring/tools/integration_tests/integration_tests.sh --python-version=3.12
+          ./aws/logs_monitoring/tools/integration_tests/integration_tests.sh --python-version=3.13
diff --git a/.github/workflows/aws_lint.yml b/.github/workflows/aws_lint.yml
@@ -11,7 +11,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v3
         with:
-          python-version: "3.12"
+          python-version: "3.13"
 
       - name: Install pip
         run: |
@@ -33,7 +33,7 @@ jobs:
 
       - name: Install cfn-lint print version and check formatting
         run: |
-          pip install cfn-lint==1.11.1
+          pip install cfn-lint==1.19.0
           cfn-lint --version
           cfn-lint -t aws/logs_monitoring/template.yaml
           cfn-lint -t aws/rds_enhanced_monitoring/rds-enhanced-sam-template.yaml
diff --git a/.github/workflows/aws_unit_test.yml b/.github/workflows/aws_unit_test.yml
@@ -8,10 +8,10 @@ jobs:
     steps:
       - name: Checkout source
         uses: actions/checkout@v3
-      - name: Setup Python 3.12
+      - name: Setup Python 3.13
         uses: actions/setup-python@v3
         with:
-          python-version: 3.12
+          python-version: 3.13
 
       - name: AWS Python unit tests
         env:
diff --git a/aws/logs_monitoring/README.md b/aws/logs_monitoring/README.md
@@ -115,7 +115,7 @@ resource "aws_cloudformation_stack" "datadog_forwarder" {
 
 If you can't install the Forwarder using the provided CloudFormation template, you can install the Forwarder manually following the steps below. Feel free to open an issue or pull request to let us know if there is anything we can improve to make the template work for you.
 
-1. Create a Python 3.12 Lambda function using `aws-dd-forwarder-<VERSION>.zip` from the latest [releases][101].
+1. Create a Python 3.13 Lambda function using `aws-dd-forwarder-<VERSION>.zip` from the latest [releases][101].
 2. Save your [Datadog API key][102] in AWS Secrets Manager, set environment variable `DD_API_KEY_SECRET_ARN` with the secret ARN on the Lambda function, and add the `secretsmanager:GetSecretValue` permission to the Lambda execution role.
 3. If you need to forward logs from S3 buckets, add the `s3:GetObject` permission to the Lambda execution role.
 4. Set the environment variable `DD_ENHANCED_METRICS` to `false` on the Forwarder. This stops the Forwarder from generating enhanced metrics itself, but it will still forward custom metrics from other lambdas.DdFetchLambdaTags
@@ -148,8 +148,11 @@ The <a href="#cloudformation-parameters">environment variables provided on this
 
 If you encounter issues upgrading to the latest version, check the Troubleshooting section.
 
+### Upgrade an older verison to 4.12.0+
+Starting version 4.12.0+ Lambda function has been updated to require **Python 3.13**. If upgrading an older forwarder installation to 4.12.0+, ensure AWS Lambda function is configured to use Python 3.13
+
 ### Upgrade an older verison to 4.3.0+
-Starting verison 4.3.0 Lambda forwarder will support a single python version only. The supported Python version of this release is 3.12. 
+Starting verison 4.3.0 Lambda forwarder will support a single python version only. The supported Python version of this release is 3.12.
 
 ### Upgrade an older version to +4.0.0
 Starting version 4.0.0 `source`, `service` and `host` identification logic will be pulled out from the Lambda forwarder's code and set in directly in Datadog's backend. The first migrated log source is `RDS`.
@@ -748,7 +751,7 @@ The value of the `service` tag is determined based on multiple inputs. These inp
 1. Log message custom tags: If the log message has a `ddtags` key which contains a `service` tag value, it will be used to override the `service` tag in the log event.
 2. Lambda tags cache (applicable for Lambda logs only): Activating `DdFetchLambdaTags` will fetch and store all Lambda functions tags and will override the `service` tag if it wasn't set previously or was set to a default value i.e. `source` value.
 3. Cloudwatch log group tags cache (applicable for Cloudwatch logs only): Activating `DdFetchLogGroupTags` will fetch and store all Cloudwatch log groups tags which are added to the `ddtags` entry in the log event. If `service` tag value was set in the tags cache it will be used to set the `service` tag for the log event.
-4. Directly setting a `service` tag value in the forwarder's `ddtags` ENV var. 
+4. Directly setting a `service` tag value in the forwarder's `ddtags` ENV var.
 5. Default value equal to the `source` tag.
 
 
diff --git a/aws/logs_monitoring/requirements.txt b/aws/logs_monitoring/requirements.txt
@@ -3,10 +3,10 @@ bytecode
 cattrs
 certifi
 charset-normalizer
-datadog-lambda==6.104.0
-datadog==0.50.2
+datadog-lambda==7.112.0
+datadog==0.52.0
 ddsketch==3.0.1
-ddtrace==2.17.3
+ddtrace==3.10.2
 deprecated
 envier
 exceptiongroup
diff --git a/aws/logs_monitoring/setup.py b/aws/logs_monitoring/setup.py
@@ -12,10 +12,10 @@
     author="Datadog, Inc.",
     author_email="dev@datadoghq.com",
     classifiers=[
-        "Programming Language :: Python :: 3.12",
+        "Programming Language :: Python :: 3.13",
     ],
     keywords="datadog aws lambda layer",
-    python_requires=">=3.12, <3.13",
+    python_requires=">=3.13, <3.14",
     extras_require={
         "dev": ["nose2==0.9.1", "flake8==3.7.9", "requests==2.22.0", "boto3==1.10.33"]
     },
diff --git a/aws/logs_monitoring/template.yaml b/aws/logs_monitoring/template.yaml
@@ -432,7 +432,7 @@ Resources:
             - {DdForwarderVersion: !FindInMap [Constants, DdForwarder, Version]}
         - ZipFile: " "
       MemorySize: !Ref MemorySize
-      Runtime: python3.12
+      Runtime: python3.13
       Architectures:
         - arm64
       Timeout: !Ref Timeout
@@ -843,7 +843,7 @@ Resources:
     Properties:
       Description: Copies Datadog Forwarder zip to the destination S3 bucket
       Handler: index.handler
-      Runtime: python3.12
+      Runtime: python3.13
       Timeout: 600
       Code:
         ZipFile: |
diff --git a/aws/logs_monitoring/tools/build_bundle.sh b/aws/logs_monitoring/tools/build_bundle.sh
@@ -42,7 +42,7 @@ else
     VERSION=$1
 fi
 
-PYTHON_VERSION="${PYTHON_VERSION:-3.12}"
+PYTHON_VERSION="${PYTHON_VERSION:-3.13}"
 FORWARDER_PREFIX="aws-dd-forwarder"
 FORWARDER_DIR="../.forwarder"
 
diff --git a/aws/logs_monitoring/tools/integration_tests/cache_test_lambda/serverless.yml b/aws/logs_monitoring/tools/integration_tests/cache_test_lambda/serverless.yml
@@ -1,7 +1,7 @@
 service: integration-tests
 provider:
   name: aws
-  runtime: python3.12
+  runtime: python3.13
 
 functions:
   cache_test_lambda:
diff --git a/aws/logs_monitoring/tools/integration_tests/integration_tests.sh b/aws/logs_monitoring/tools/integration_tests/integration_tests.sh
@@ -7,9 +7,9 @@
 
 set -e
 
-PYTHON_VERSION="python3.12"
-PYTHON_VERSION_TAG="3.12"
-PYTHON_IMAGE="python:3.12"
+PYTHON_VERSION="python3.13"
+PYTHON_VERSION_TAG="3.13"
+PYTHON_IMAGE="python:3.13"
 SKIP_FORWARDER_BUILD=false
 UPDATE_SNAPSHOTS=false
 LOG_LEVEL=info
@@ -36,7 +36,7 @@ for arg in "$@"; do
                 SKIP_FORWARDER_BUILD=true
                 shift
                 ;;
-                
+
 
         # -u or --update
         # Update the snapshots to reflect this test run
diff --git a/aws/logs_monitoring/tools/publish_layers.sh b/aws/logs_monitoring/tools/publish_layers.sh
@@ -92,7 +92,7 @@ user_confirm() {
 # Makes sure any subprocesses will be terminated with this process
 trap "pkill -P $$; exit 1;" INT
 
-PYTHON_VERSIONS_FOR_AWS_CLI=("python3.12")
+PYTHON_VERSIONS_FOR_AWS_CLI=("python3.13")
 LAYER_PATHS=(".forwarder/aws-dd-forwarder-${FORWARDER_VERSION}-layer.zip")
 AVAILABLE_LAYERS=("Datadog-Forwarder")
 AVAILABLE_REGIONS=$(aws ec2 describe-regions | jq -r '.[] | .[] | .RegionName')
diff --git a/aws/logs_monitoring/trace_forwarder/scripts/build_linux_go_bin.sh b/aws/logs_monitoring/trace_forwarder/scripts/build_linux_go_bin.sh
@@ -17,7 +17,7 @@ rm -rf ./bin
 # between different python runtimes.
 
 if [[ $(docker image ls | grep -c golang) -lt 1 ]]; then
-    docker buildx build --platform linux/arm64 -t golang . --no-cache --build-arg "runtime=python:3.12"
+    docker buildx build --platform linux/arm64 -t golang . --no-cache --build-arg "runtime=python:3.13"
 fi
 
 id=$(docker create --platform linux/arm64 golang)
diff --git a/aws/logs_monitoring/trace_forwarder/scripts/run_tests.sh b/aws/logs_monitoring/trace_forwarder/scripts/run_tests.sh
@@ -11,5 +11,5 @@ set -e
 # Change to the parent of the directory this script is in
 cd $(dirname "$0")/..
 
-docker buildx build --platform linux/arm64 -t datadog-go-layer . --build-arg runtime=python:3.12
+docker buildx build --platform linux/arm64 -t datadog-go-layer . --build-arg runtime=python:3.13
 docker run --rm datadog-go-layer go test -v ./...
diff --git a/aws/rds_enhanced_monitoring/rds-enhanced-sam-template.yaml b/aws/rds_enhanced_monitoring/rds-enhanced-sam-template.yaml
@@ -20,7 +20,7 @@ Resources:
           KeyId: !Ref KMSKeyId
       Architectures:
         - arm64
-      Runtime: python3.12
+      Runtime: python3.13
       Timeout: 10
       KmsKeyArn:
         !Sub
diff --git a/aws/vpc_flow_log_monitoring/README.md b/aws/vpc_flow_log_monitoring/README.md
@@ -49,7 +49,7 @@ version, account, eni, source, destination, srcport, destport="22", protocol="6"
 
    - Create a `lambda_execution` role and attach this policy
 
-   - Create a lambda function: Skip the blueprint, name it `functionname`, set the Runtime to `Python 3.12`, the Architecture to `arm64`,, the handle to `lambda_function.lambda_handler`, and the role to `lambda_execution`.
+   - Create a lambda function: Skip the blueprint, name it `functionname`, set the Runtime to `Python 3.13`, the Architecture to `arm64`,, the handle to `lambda_function.lambda_handler`, and the role to `lambda_execution`.
 
    - Copy the content of `functionname/lambda_function.py` in the code section, make sure to update the `KMS_ENCRYPTED_KEYS` environment variable with the encrypted key generated in step 1
 
diff --git a/aws/vpc_flow_log_monitoring/vpc-flow-log-sam-template.yaml b/aws/vpc_flow_log_monitoring/vpc-flow-log-sam-template.yaml
@@ -18,7 +18,7 @@ Resources:
       Policies:
         KMSDecryptPolicy:
           KeyId: !Ref KMSKeyId
-      Runtime: python3.12
+      Runtime: python3.13
       Architectures:
         - arm64
       Timeout: 10
