Log format unification for kyma module controllers

[kwiatekus]

Description
Adjust logging in istio, so that deployments labelled with kyma-project.io/module=serverless produce logs that are compliant with recommendations documented here.

Configuration should be stored in a config map.
Logger configuration should be reloaded after changes (hot deploy).

Reason
Our goal is unify log format in order to reduce the time needed for users and support engineers to create error and warning log dumps.
kyma-project/cli#2651

Expected logs

• make all log entries JSON parsable (aslo for operator)
• include level : debug, info, warn, error
• include timestamp (not ts) in RFC3339 format
• include message (not msg)

Actual logs

{
  "level": "INFO",
  "timestamp": "2025-10-24T14:55:39Z",
  "caller": "state/deployment_status.go:53",
  "message": "deployment unhealthy: \"test-me-sdx6w\"",
  "context": {
    "request": "default/test-me"
  }
}
{
  "level": "INFO",
  "timestamp": "2025-10-24T14:55:39Z",
  "caller": "fsm/fsm.go:95",
  "message": "reconciliation done",
  "context": {
    "request": "default/test-me",
    "error": null,
    "result": {
      "Requeue": false,
      "RequeueAfter": 1000000000
    }
  }
}

[pPrecel]

QA

I've tested changes using the latest changes from main (commit 61edcfa3fd6031d4e01b7c3e61ea3d481790c12d)

I've checked all workloads with the kyma-project.io/module=serverless label:

kubectl get deploy -A -l kyma-project.io/module=serverless
NAMESPACE     NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
kyma-system   serverless-ctrl-mngr   1/1     1            1           59m
kyma-system   serverless-operator    1/1     1            1           59m

🆗 Both workloads (operator and serverless-controller) met logs expectations (format, timestamp, fields, levels)

🆗 It's possible to reconfigure the serverless-controller logger

❗ I changed log level and log format manually for the operator, but format was not configured properly:

kubectl logs -n kyma-system serverless-operator-648ccdb47b-2t5ft -f:

...
{"level":"info","timestamp":"2026-01-07T11:38:49Z","logger":"notifier","caller":"config/watcher.go:46","message":"config file change detected","context":{}}
{"level":"info","timestamp":"2026-01-07T11:38:49Z","logger":"notifier","caller":"config/dynamic.go:21","message":"loggers reconfigured with level 'debug' and format 'text'","context":{}}
{"level":"info","timestamp":"2026-01-07T11:41:09Z","caller":"controllers/serverless_controller.go:80","message":"reconciliation started","context":{"request":"kyma-system/default"}}
{"level":"info","timestamp":"2026-01-07T11:41:09Z","caller":"state/fsm.go:126","message":"switching state: sFnServedFilter","context":{"request":"kyma-system/default"}}
{"level":"info","timestamp":"2026-01-07T11:41:09Z","caller":"state/fsm.go:126","message":"switching state: sFnAddFinalizer","context":{"request":"kyma-system/default"}}
{"level":"info","timestamp":"2026-01-07T11:41:09Z","caller":"state/fsm.go:126","message":"switching state: sFnInitialize","context":{"request":"kyma-system/default"}}
{"level":"info","timestamp":"2026-01-07T11:41:09Z","caller":"state/fsm.go:126","message":"switching state: sFnRegistryConfiguration","context":{"request":"kyma-system/default"}}
{"level":"debug","timestamp":"2026-01-07T11:41:09Z","caller":"state/update_status.go:20","message":"updating serverless status to '{EventingEndpoint:http://eventing-publisher-proxy.kyma-system.svc.cluster.local/publish TracingEndpoint: CPUUtilizationPercentage: RequeueDuration: BuildExecutorArgs: BuildMaxSimultaneousJobs: HealthzLivenessTimeout: DefaultBuildJobPreset: DefaultRuntimePodPreset:XS LogLevel:info LogFormat:json DockerRegistry: State:Processing Served:True NetworkPoliciesEnabled:True Conditions:[{Type:Configured Status:True ObservedGeneration:0 LastTransitionTime:2026-01-07 11:15:23 +0000 UTC Reason:Configured Message:Configuration ready} {Type:Installed Status:True ObservedGeneration:0 LastTransitionTime:2026-01-07 11:15:39 +0000 UTC Reason:Installed Message:Serverless installed}]}'","context":{"request":"kyma-system/default"}}
{"level":"info","timestamp":"2026-01-07T11:41:09Z","caller":"state/fsm.go:126","message":"switching state: sFnOptionalDependencies","context":{"request":"kyma-system/default"}}
...

❗ After adding the log configuration to the Serverless CR, the whole serverless-controller pod is recreated, so the dynamic watcher has no chance of logger reconfiguration because the old pod is terminated.

old pod logs:

...
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"manager/internal.go:555","message":"Stopping and waiting for non leader election runnables","context":{}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"manager/internal.go:559","message":"Stopping and waiting for leader election runnables","context":{}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"manager/internal.go:550","message":"Stopping and waiting for warmup runnables","context":{}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"controller/controller.go:309","message":"Shutdown signal received, waiting for all workers to finish","context":{"controller":"function","controllerGroup":"serverless.kyma-project.io","controllerKind":"Function"}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"controller/controller.go:311","message":"All workers finished","context":{"controller":"function","controllerGroup":"serverless.kyma-project.io","controllerKind":"Function"}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"manager/internal.go:567","message":"Stopping and waiting for caches","context":{}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"manager/internal.go:571","message":"Stopping and waiting for webhooks","context":{}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"manager/internal.go:574","message":"Stopping and waiting for HTTP servers","context":{}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"manager/server.go:68","message":"shutting down server","context":{"name":"health probe","addr":"[::]:8090"}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","logger":"controller-runtime.metrics","caller":"server/server.go:254","message":"Shutting down metrics server with timeout of 1 minute","context":{}}
{"level":"info","timestamp":"2026-01-07T11:41:22Z","caller":"manager/internal.go:578","message":"Wait completed, proceeding to shutdown the manager","context":{}}
{"level":"error","timestamp":"2026-01-07T11:41:22Z","caller":"manager/internal.go:517","message":"error received after stop sequence was engaged","context":{"error":"leader election lost"}}

new pod logs:

2026-01-07T11:41:11Z    info    cmd/main.go:117 Generating Kubernetes client config     {"context": {}}
2026-01-07T11:41:11Z    info    cmd/main.go:120 Initializing controller manager {"context": {}}
2026-01-07T11:41:11Z    info    notifier        config/watcher.go:24    config notifier started {"context": {}}
{"level":"info","ts":"2026-01-07T11:41:11Z","logger":"setup","msg":"starting manager"}
2026-01-07T11:41:11Z    info    orphaned-resources/delete_orphaned_resources.go:20      cleaning orphan deprecated resources      {"context": {}}
2026-01-07T11:41:11Z    info    controller-runtime.metrics      server/server.go:208    Starting metrics server {"context": {}}
2026-01-07T11:41:11Z    info    controller-runtime.metrics      server/server.go:247    Serving metrics server  {"context": {"bindAddress": ":8080", "secure": false}}
2026-01-07T11:41:11Z    info    manager/server.go:83    starting server {"context": {"name": "health probe", "addr": "[::]:8090"}}
2026-01-07T11:41:11Z    info    leaderelection/leaderelection.go:257    attempting to acquire leader lease kyma-system/serverless-controller-leader-election-helper...    {"context": {}}
2026-01-07T11:41:30Z    debug   healthz controller/health.go:83 Liveness handler triggered      {"context": {}}
...