From 3731a391c09a4dea689f79f35658d7c6eb1aae44 Mon Sep 17 00:00:00 2001 From: "A.Arnold" Date: Thu, 2 Jul 2026 16:06:44 +0100 Subject: [PATCH 01/12] MTA-7134-LLM-Proxy-Transaction-Logging-Schema-Audit-Trail-Integration Signed-off-by: A.Arnold --- docs/developer-lightspeed-guide/master.adoc | 2 + .../proc_auditing-llm-proxy-transactions.adoc | 105 ++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc diff --git a/docs/developer-lightspeed-guide/master.adoc b/docs/developer-lightspeed-guide/master.adoc index 042d59c50d4..c7c97e09da5 100644 --- a/docs/developer-lightspeed-guide/master.adoc +++ b/docs/developer-lightspeed-guide/master.adoc @@ -23,4 +23,6 @@ include::topics/developer-lightspeed/assembly_run-dev-lightspeed-analysis.adoc[l include::topics/developer-lightspeed/con_developer-lightspeed-logs.adoc[leveloffset=+1] +include::topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc[leveloffset=+1] + :!mta-developer-lightspeed: diff --git a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc new file mode 100644 index 00000000000..00908f358cf --- /dev/null +++ b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc @@ -0,0 +1,105 @@ +// Module included in the following assemblies: +// +// * docs/developer-lightspeed-guide/master.adoc + +:_template-generated: 2026-07-05 + +:_mod-docs-content-type: CONCEPT + +[id="monitoring-llm-proxy-interactions_{context}"] += LLM proxy interaction monitoring + +[role="_abstract"] +The {mta-dl-plugin} LLM proxy acts as a single, administrator-controlled access point between developers and external large language model (LLM) providers. Understanding its data handling and logging behavior is important for organizations with artificial intelligence (AI) system governance requirements. + +[NOTE] +==== +Red{nbsp}Hat plans to provide structured per-request audit logging and security incident and event management (SIEM) forwarding. The current release generates only standard server logs. For more information, see xref:llm-proxy-future-logging_{context}[Planned audit logging capabilities]. +==== + +[id="llm-proxy-data-handling_{context}"] +== LLM proxy request and data handling + +The LLM proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. It provides the following controls: + +Credential isolation:: LLM provider application programming interface (API) keys are stored in a Kubernetes secret and never distributed to client applications. Only the proxy pod reads these credentials. + +JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub's Keycloak instance by using its `JWKS` endpoint: ++ +[subs="+quotes"] +---- +https://____/realms/____/protocol/openid-connect/certs +---- ++ +Requests that fail validation are rejected with a `401` or `403` response before they reach the LLM provider. + +Header redaction:: Authorization headers and other sensitive fields are redacted from proxy logs. + +Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL database in the following tables: ++ +* `llm_proxy_inference_store`: Individual inference records +* `llm_proxy_conversations`: Conversation context records + +[NOTE] +==== +As of MTA 8.2, OIDC user identity is terminated at the Hub and is not passed to or stored by the proxy. +==== + +[id="llm-proxy-log-access_{context}"] +== Accessing LLM proxy pod logs + +The proxy writes standard server output to stdout. You can access these logs by using the `oc logs` command. The logs are useful for diagnosing connectivity issues and errors between the proxy and the LLM provider. + +[NOTE] +==== +Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. +==== + +.Prerequisites +* You have installed the {ocp-short} {CLIName} (`oc`). +* You are logged in to the cluster with the `cluster-admin` role or an equivalent role that grants read access to pods in the {ProductShortName} namespace. + +.Procedure + +. Identify the proxy pod name: ++ +[source,terminal,subs="+quotes"] +---- +$ *oc get pods -n ____ | grep proxy* +---- ++ +.Example output +[source,terminal] +---- +mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d +---- + +. Stream the current log output from the proxy pod: ++ +[source,terminal,subs="+quotes"] +---- +$ *oc logs -f ____ -n ____* +---- ++ +.Example output +[source,terminal] +---- +INFO: 10.128.2.14:52310 - "POST /v1/chat/completions HTTP/1.1" 200 OK +INFO: 10.128.2.14:52311 - "POST /v1/chat/completions HTTP/1.1" 503 Service Unavailable +---- + +. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag: ++ +[source,terminal,subs="+quotes"] +---- +$ *oc logs --previous ____ -n ____* +---- + +[id="llm-proxy-future-logging_{context}"] +== Planned audit logging capabilities + +Red{nbsp}Hat plans to provide the following audit logging capabilities: + +* Structured JSON log events per completed LLM transaction, including request routing, response latency, and token count metadata. +* User-to-request correlation by using anonymized user identifiers. +* Forwarding structured log streams to a SIEM system (such as Splunk or Elasticsearch) by using the Red{nbsp}Hat OpenShift Logging Operator. From 701a92e254497ba4b6905cd9b19dc5516829bbcc Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Mon, 20 Jul 2026 15:38:17 +0100 Subject: [PATCH 02/12] Update docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc Co-authored-by: Prabha Kylasamiyer Sundara Rajan --- .../proc_auditing-llm-proxy-transactions.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc index 00908f358cf..58a317256f0 100644 --- a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc +++ b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc @@ -42,7 +42,7 @@ Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL dat [NOTE] ==== -As of MTA 8.2, OIDC user identity is terminated at the Hub and is not passed to or stored by the proxy. +As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated at the Hub and is not passed to or stored by the proxy. ==== [id="llm-proxy-log-access_{context}"] From 5e030ef118d5ea86df76088012291067a94e31e4 Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Mon, 20 Jul 2026 15:38:35 +0100 Subject: [PATCH 03/12] Update docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc Co-authored-by: Mariya Pershina <53339200+mpershina@users.noreply.github.com> --- .../proc_auditing-llm-proxy-transactions.adoc | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc index 58a317256f0..53980627790 100644 --- a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc +++ b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc @@ -50,10 +50,7 @@ As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated The proxy writes standard server output to stdout. You can access these logs by using the `oc logs` command. The logs are useful for diagnosing connectivity issues and errors between the proxy and the LLM provider. -[NOTE] -==== Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. -==== .Prerequisites * You have installed the {ocp-short} {CLIName} (`oc`). From c683babaf82b5d12f6121837b078a9f119966d1b Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Mon, 20 Jul 2026 15:38:51 +0100 Subject: [PATCH 04/12] Update docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc Co-authored-by: Mariya Pershina <53339200+mpershina@users.noreply.github.com> --- .../proc_auditing-llm-proxy-transactions.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc index 53980627790..a1a1f876046 100644 --- a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc +++ b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc @@ -48,7 +48,7 @@ As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated [id="llm-proxy-log-access_{context}"] == Accessing LLM proxy pod logs -The proxy writes standard server output to stdout. You can access these logs by using the `oc logs` command. The logs are useful for diagnosing connectivity issues and errors between the proxy and the LLM provider. +To diagnose connectivity errors between the proxy and the LLM provider, use the `oc logs` command. The proxy writes standard server output to stdout. Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. From 684a8d67e57c4fb01b94c7220c21572027b411dd Mon Sep 17 00:00:00 2001 From: "A.Arnold" Date: Mon, 20 Jul 2026 15:48:15 +0100 Subject: [PATCH 05/12] Modularize Signed-off-by: A.Arnold --- 0001-modularize-llm-proxy-monitoring.patch | 263 ++++++++++++++++++ docs/developer-lightspeed-guide/master.adoc | 2 +- ...mbly_llm-proxy-interaction-monitoring.adoc | 14 + .../con_llm-proxy-data-handling.adoc | 34 +++ .../proc_access-llm-proxy-logs.adoc | 45 +++ .../proc_auditing-llm-proxy-transactions.adoc | 102 ------- 6 files changed, 357 insertions(+), 103 deletions(-) create mode 100644 0001-modularize-llm-proxy-monitoring.patch create mode 100644 docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc create mode 100644 docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc create mode 100644 docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc delete mode 100644 docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc diff --git a/0001-modularize-llm-proxy-monitoring.patch b/0001-modularize-llm-proxy-monitoring.patch new file mode 100644 index 00000000000..56ef3593265 --- /dev/null +++ b/0001-modularize-llm-proxy-monitoring.patch @@ -0,0 +1,263 @@ +From d70ad5ecee5206452332b943dd6ce327b09a6bff Mon Sep 17 00:00:00 2001 +From: Patch Generator +Date: Mon, 20 Jul 2026 14:46:38 +0000 +Subject: [PATCH] fix(MTA-7134): modularize and resolve review comments + +- Replace proc_auditing-llm-proxy-transactions.adoc with + assembly + concept + procedure modules +- Remove 'Planned audit logging capabilities' section (IBM Style Guide: + no future-plans claims per mpershina) +- Remove NOTE referencing removed section (same reason) +- Split log-access content into PROCEDURE module per mpershina + (migration risk if left as == inside a CONCEPT) +- Drop .Example output label and fabricated log lines from oc logs -f + step per mpershina suggestion +- Change 'Keycloak instance' to 'OIDC provider' per rromannissen + (Keycloak removed in 8.2; generic term is forward-safe) +- Update master.adoc include to point at new assembly +--- + docs/developer-lightspeed-guide/master.adoc | 2 +- + ...mbly_llm-proxy-interaction-monitoring.adoc | 14 +++ + .../con_llm-proxy-data-handling.adoc | 34 ++++++ + .../proc_access-llm-proxy-logs.adoc | 45 ++++++++ + .../proc_auditing-llm-proxy-transactions.adoc | 102 ------------------ + 5 files changed, 94 insertions(+), 103 deletions(-) + create mode 100644 docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc + create mode 100644 docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc + create mode 100644 docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc + delete mode 100644 docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc + +diff --git a/docs/developer-lightspeed-guide/master.adoc b/docs/developer-lightspeed-guide/master.adoc +index 8e258eb..ffffb85 100644 +--- a/docs/developer-lightspeed-guide/master.adoc ++++ b/docs/developer-lightspeed-guide/master.adoc +@@ -6,6 +6,6 @@ include::topics/developer-lightspeed/assembly_run-dev-lightspeed-analysis.adoc[l + + include::topics/developer-lightspeed/con_developer-lightspeed-logs.adoc[leveloffset=+1] + +-include::topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc[leveloffset=+1] ++include::topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc[leveloffset=+1] + + :!mta-developer-lightspeed: +diff --git a/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc b/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc +new file mode 100644 +index 0000000..10b3f82 +--- /dev/null ++++ b/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc +@@ -0,0 +1,14 @@ ++// Module included in the following assemblies: ++// ++// * docs/developer-lightspeed-guide/master.adoc ++ ++:_mod-docs-content-type: ASSEMBLY ++ ++[id="assembly-llm-proxy-interaction-monitoring_{context}"] ++= LLM proxy interaction monitoring ++ ++The {mta-dl-plugin} LLM proxy acts as a single, administrator-controlled access point between developers and external large language model (LLM) providers. Understanding its data handling and logging behavior is important for organizations with artificial intelligence (AI) system governance requirements. ++ ++include::con_llm-proxy-data-handling.adoc[leveloffset=+1] ++ ++include::proc_access-llm-proxy-logs.adoc[leveloffset=+1] +diff --git a/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc b/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc +new file mode 100644 +index 0000000..1e66ce6 +--- /dev/null ++++ b/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc +@@ -0,0 +1,34 @@ ++// Module included in the following assemblies: ++// ++// * docs/developer-lightspeed-guide/master.adoc ++ ++:_mod-docs-content-type: CONCEPT ++ ++[id="llm-proxy-data-handling_{context}"] ++= LLM proxy request and data handling ++ ++[role="_abstract"] ++The LLM proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. It provides the following controls: ++ ++Credential isolation:: LLM provider application programming interface (API) keys are stored in a Kubernetes secret and never distributed to client applications. Only the proxy pod reads these credentials. ++ ++JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub's OIDC provider by using its JWKS endpoint: +++ ++[subs="+quotes"] ++---- ++https://____/realms/____/protocol/openid-connect/certs ++---- +++ ++Requests that fail validation are rejected with a `401` or `403` response before they reach the LLM provider. ++ ++Header redaction:: Authorization headers and other sensitive fields are redacted from proxy logs. ++ ++Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL database in the following tables: +++ ++* `llm_proxy_inference_store`: Individual inference records ++* `llm_proxy_conversations`: Conversation context records ++ ++[NOTE] ++==== ++As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated at the Hub and is not passed to or stored by the proxy. ++==== +diff --git a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc +new file mode 100644 +index 0000000..605beb9 +--- /dev/null ++++ b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc +@@ -0,0 +1,45 @@ ++// Module included in the following assemblies: ++// ++// * docs/developer-lightspeed-guide/master.adoc ++ ++:_mod-docs-content-type: PROCEDURE ++ ++[id="llm-proxy-log-access_{context}"] ++= Accessing LLM proxy pod logs ++ ++To diagnose connectivity errors between the proxy and the LLM provider, use the `oc logs` command. The proxy writes standard server output to stdout. ++ ++Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. ++ ++.Prerequisites ++* You have installed the {ocp-short} {CLIName} (`oc`). ++* You are logged in to the cluster with the `cluster-admin` role or an equivalent role that grants read access to pods in the {ProductShortName} namespace. ++ ++.Procedure ++ ++. Identify the proxy pod name: +++ ++[source,terminal,subs="+quotes"] ++---- ++$ *oc get pods -n ____ | grep proxy* ++---- +++ ++.Example output ++[source,terminal] ++---- ++mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d ++---- ++ ++. Stream the current log output from the proxy pod: +++ ++[source,terminal,subs="+quotes"] ++---- ++$ *oc logs -f ____ -n ____* ++---- ++ ++. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag: +++ ++[source,terminal,subs="+quotes"] ++---- ++$ *oc logs --previous ____ -n ____* ++---- +diff --git a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc +deleted file mode 100644 +index a1a1f87..0000000 +--- a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc ++++ /dev/null +@@ -1,102 +0,0 @@ +-// Module included in the following assemblies: +-// +-// * docs/developer-lightspeed-guide/master.adoc +- +-:_template-generated: 2026-07-05 +- +-:_mod-docs-content-type: CONCEPT +- +-[id="monitoring-llm-proxy-interactions_{context}"] +-= LLM proxy interaction monitoring +- +-[role="_abstract"] +-The {mta-dl-plugin} LLM proxy acts as a single, administrator-controlled access point between developers and external large language model (LLM) providers. Understanding its data handling and logging behavior is important for organizations with artificial intelligence (AI) system governance requirements. +- +-[NOTE] +-==== +-Red{nbsp}Hat plans to provide structured per-request audit logging and security incident and event management (SIEM) forwarding. The current release generates only standard server logs. For more information, see xref:llm-proxy-future-logging_{context}[Planned audit logging capabilities]. +-==== +- +-[id="llm-proxy-data-handling_{context}"] +-== LLM proxy request and data handling +- +-The LLM proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. It provides the following controls: +- +-Credential isolation:: LLM provider application programming interface (API) keys are stored in a Kubernetes secret and never distributed to client applications. Only the proxy pod reads these credentials. +- +-JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub's Keycloak instance by using its `JWKS` endpoint: +-+ +-[subs="+quotes"] +----- +-https://____/realms/____/protocol/openid-connect/certs +----- +-+ +-Requests that fail validation are rejected with a `401` or `403` response before they reach the LLM provider. +- +-Header redaction:: Authorization headers and other sensitive fields are redacted from proxy logs. +- +-Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL database in the following tables: +-+ +-* `llm_proxy_inference_store`: Individual inference records +-* `llm_proxy_conversations`: Conversation context records +- +-[NOTE] +-==== +-As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated at the Hub and is not passed to or stored by the proxy. +-==== +- +-[id="llm-proxy-log-access_{context}"] +-== Accessing LLM proxy pod logs +- +-To diagnose connectivity errors between the proxy and the LLM provider, use the `oc logs` command. The proxy writes standard server output to stdout. +- +-Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. +- +-.Prerequisites +-* You have installed the {ocp-short} {CLIName} (`oc`). +-* You are logged in to the cluster with the `cluster-admin` role or an equivalent role that grants read access to pods in the {ProductShortName} namespace. +- +-.Procedure +- +-. Identify the proxy pod name: +-+ +-[source,terminal,subs="+quotes"] +----- +-$ *oc get pods -n ____ | grep proxy* +----- +-+ +-.Example output +-[source,terminal] +----- +-mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d +----- +- +-. Stream the current log output from the proxy pod: +-+ +-[source,terminal,subs="+quotes"] +----- +-$ *oc logs -f ____ -n ____* +----- +-+ +-.Example output +-[source,terminal] +----- +-INFO: 10.128.2.14:52310 - "POST /v1/chat/completions HTTP/1.1" 200 OK +-INFO: 10.128.2.14:52311 - "POST /v1/chat/completions HTTP/1.1" 503 Service Unavailable +----- +- +-. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag: +-+ +-[source,terminal,subs="+quotes"] +----- +-$ *oc logs --previous ____ -n ____* +----- +- +-[id="llm-proxy-future-logging_{context}"] +-== Planned audit logging capabilities +- +-Red{nbsp}Hat plans to provide the following audit logging capabilities: +- +-* Structured JSON log events per completed LLM transaction, including request routing, response latency, and token count metadata. +-* User-to-request correlation by using anonymized user identifiers. +-* Forwarding structured log streams to a SIEM system (such as Splunk or Elasticsearch) by using the Red{nbsp}Hat OpenShift Logging Operator. +-- +2.52.0 + diff --git a/docs/developer-lightspeed-guide/master.adoc b/docs/developer-lightspeed-guide/master.adoc index c7c97e09da5..32b734263f9 100644 --- a/docs/developer-lightspeed-guide/master.adoc +++ b/docs/developer-lightspeed-guide/master.adoc @@ -23,6 +23,6 @@ include::topics/developer-lightspeed/assembly_run-dev-lightspeed-analysis.adoc[l include::topics/developer-lightspeed/con_developer-lightspeed-logs.adoc[leveloffset=+1] -include::topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc[leveloffset=+1] +include::topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc[leveloffset=+1] :!mta-developer-lightspeed: diff --git a/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc b/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc new file mode 100644 index 00000000000..10b3f823abb --- /dev/null +++ b/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc @@ -0,0 +1,14 @@ +// Module included in the following assemblies: +// +// * docs/developer-lightspeed-guide/master.adoc + +:_mod-docs-content-type: ASSEMBLY + +[id="assembly-llm-proxy-interaction-monitoring_{context}"] += LLM proxy interaction monitoring + +The {mta-dl-plugin} LLM proxy acts as a single, administrator-controlled access point between developers and external large language model (LLM) providers. Understanding its data handling and logging behavior is important for organizations with artificial intelligence (AI) system governance requirements. + +include::con_llm-proxy-data-handling.adoc[leveloffset=+1] + +include::proc_access-llm-proxy-logs.adoc[leveloffset=+1] diff --git a/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc b/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc new file mode 100644 index 00000000000..1e66ce6865c --- /dev/null +++ b/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc @@ -0,0 +1,34 @@ +// Module included in the following assemblies: +// +// * docs/developer-lightspeed-guide/master.adoc + +:_mod-docs-content-type: CONCEPT + +[id="llm-proxy-data-handling_{context}"] += LLM proxy request and data handling + +[role="_abstract"] +The LLM proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. It provides the following controls: + +Credential isolation:: LLM provider application programming interface (API) keys are stored in a Kubernetes secret and never distributed to client applications. Only the proxy pod reads these credentials. + +JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub's OIDC provider by using its JWKS endpoint: ++ +[subs="+quotes"] +---- +https://____/realms/____/protocol/openid-connect/certs +---- ++ +Requests that fail validation are rejected with a `401` or `403` response before they reach the LLM provider. + +Header redaction:: Authorization headers and other sensitive fields are redacted from proxy logs. + +Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL database in the following tables: ++ +* `llm_proxy_inference_store`: Individual inference records +* `llm_proxy_conversations`: Conversation context records + +[NOTE] +==== +As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated at the Hub and is not passed to or stored by the proxy. +==== diff --git a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc new file mode 100644 index 00000000000..605beb969e3 --- /dev/null +++ b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc @@ -0,0 +1,45 @@ +// Module included in the following assemblies: +// +// * docs/developer-lightspeed-guide/master.adoc + +:_mod-docs-content-type: PROCEDURE + +[id="llm-proxy-log-access_{context}"] += Accessing LLM proxy pod logs + +To diagnose connectivity errors between the proxy and the LLM provider, use the `oc logs` command. The proxy writes standard server output to stdout. + +Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. + +.Prerequisites +* You have installed the {ocp-short} {CLIName} (`oc`). +* You are logged in to the cluster with the `cluster-admin` role or an equivalent role that grants read access to pods in the {ProductShortName} namespace. + +.Procedure + +. Identify the proxy pod name: ++ +[source,terminal,subs="+quotes"] +---- +$ *oc get pods -n ____ | grep proxy* +---- ++ +.Example output +[source,terminal] +---- +mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d +---- + +. Stream the current log output from the proxy pod: ++ +[source,terminal,subs="+quotes"] +---- +$ *oc logs -f ____ -n ____* +---- + +. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag: ++ +[source,terminal,subs="+quotes"] +---- +$ *oc logs --previous ____ -n ____* +---- diff --git a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc deleted file mode 100644 index a1a1f876046..00000000000 --- a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc +++ /dev/null @@ -1,102 +0,0 @@ -// Module included in the following assemblies: -// -// * docs/developer-lightspeed-guide/master.adoc - -:_template-generated: 2026-07-05 - -:_mod-docs-content-type: CONCEPT - -[id="monitoring-llm-proxy-interactions_{context}"] -= LLM proxy interaction monitoring - -[role="_abstract"] -The {mta-dl-plugin} LLM proxy acts as a single, administrator-controlled access point between developers and external large language model (LLM) providers. Understanding its data handling and logging behavior is important for organizations with artificial intelligence (AI) system governance requirements. - -[NOTE] -==== -Red{nbsp}Hat plans to provide structured per-request audit logging and security incident and event management (SIEM) forwarding. The current release generates only standard server logs. For more information, see xref:llm-proxy-future-logging_{context}[Planned audit logging capabilities]. -==== - -[id="llm-proxy-data-handling_{context}"] -== LLM proxy request and data handling - -The LLM proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. It provides the following controls: - -Credential isolation:: LLM provider application programming interface (API) keys are stored in a Kubernetes secret and never distributed to client applications. Only the proxy pod reads these credentials. - -JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub's Keycloak instance by using its `JWKS` endpoint: -+ -[subs="+quotes"] ----- -https://____/realms/____/protocol/openid-connect/certs ----- -+ -Requests that fail validation are rejected with a `401` or `403` response before they reach the LLM provider. - -Header redaction:: Authorization headers and other sensitive fields are redacted from proxy logs. - -Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL database in the following tables: -+ -* `llm_proxy_inference_store`: Individual inference records -* `llm_proxy_conversations`: Conversation context records - -[NOTE] -==== -As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated at the Hub and is not passed to or stored by the proxy. -==== - -[id="llm-proxy-log-access_{context}"] -== Accessing LLM proxy pod logs - -To diagnose connectivity errors between the proxy and the LLM provider, use the `oc logs` command. The proxy writes standard server output to stdout. - -Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. - -.Prerequisites -* You have installed the {ocp-short} {CLIName} (`oc`). -* You are logged in to the cluster with the `cluster-admin` role or an equivalent role that grants read access to pods in the {ProductShortName} namespace. - -.Procedure - -. Identify the proxy pod name: -+ -[source,terminal,subs="+quotes"] ----- -$ *oc get pods -n ____ | grep proxy* ----- -+ -.Example output -[source,terminal] ----- -mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d ----- - -. Stream the current log output from the proxy pod: -+ -[source,terminal,subs="+quotes"] ----- -$ *oc logs -f ____ -n ____* ----- -+ -.Example output -[source,terminal] ----- -INFO: 10.128.2.14:52310 - "POST /v1/chat/completions HTTP/1.1" 200 OK -INFO: 10.128.2.14:52311 - "POST /v1/chat/completions HTTP/1.1" 503 Service Unavailable ----- - -. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag: -+ -[source,terminal,subs="+quotes"] ----- -$ *oc logs --previous ____ -n ____* ----- - -[id="llm-proxy-future-logging_{context}"] -== Planned audit logging capabilities - -Red{nbsp}Hat plans to provide the following audit logging capabilities: - -* Structured JSON log events per completed LLM transaction, including request routing, response latency, and token count metadata. -* User-to-request correlation by using anonymized user identifiers. -* Forwarding structured log streams to a SIEM system (such as Splunk or Elasticsearch) by using the Red{nbsp}Hat OpenShift Logging Operator. From 5c51b9ddf29374dfe2e3dc5661384b9ae045bd82 Mon Sep 17 00:00:00 2001 From: "A.Arnold" Date: Mon, 20 Jul 2026 15:49:20 +0100 Subject: [PATCH 06/12] removing patch Signed-off-by: A.Arnold --- 0001-modularize-llm-proxy-monitoring.patch | 263 --------------------- 1 file changed, 263 deletions(-) delete mode 100644 0001-modularize-llm-proxy-monitoring.patch diff --git a/0001-modularize-llm-proxy-monitoring.patch b/0001-modularize-llm-proxy-monitoring.patch deleted file mode 100644 index 56ef3593265..00000000000 --- a/0001-modularize-llm-proxy-monitoring.patch +++ /dev/null @@ -1,263 +0,0 @@ -From d70ad5ecee5206452332b943dd6ce327b09a6bff Mon Sep 17 00:00:00 2001 -From: Patch Generator -Date: Mon, 20 Jul 2026 14:46:38 +0000 -Subject: [PATCH] fix(MTA-7134): modularize and resolve review comments - -- Replace proc_auditing-llm-proxy-transactions.adoc with - assembly + concept + procedure modules -- Remove 'Planned audit logging capabilities' section (IBM Style Guide: - no future-plans claims per mpershina) -- Remove NOTE referencing removed section (same reason) -- Split log-access content into PROCEDURE module per mpershina - (migration risk if left as == inside a CONCEPT) -- Drop .Example output label and fabricated log lines from oc logs -f - step per mpershina suggestion -- Change 'Keycloak instance' to 'OIDC provider' per rromannissen - (Keycloak removed in 8.2; generic term is forward-safe) -- Update master.adoc include to point at new assembly ---- - docs/developer-lightspeed-guide/master.adoc | 2 +- - ...mbly_llm-proxy-interaction-monitoring.adoc | 14 +++ - .../con_llm-proxy-data-handling.adoc | 34 ++++++ - .../proc_access-llm-proxy-logs.adoc | 45 ++++++++ - .../proc_auditing-llm-proxy-transactions.adoc | 102 ------------------ - 5 files changed, 94 insertions(+), 103 deletions(-) - create mode 100644 docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc - create mode 100644 docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc - create mode 100644 docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc - delete mode 100644 docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc - -diff --git a/docs/developer-lightspeed-guide/master.adoc b/docs/developer-lightspeed-guide/master.adoc -index 8e258eb..ffffb85 100644 ---- a/docs/developer-lightspeed-guide/master.adoc -+++ b/docs/developer-lightspeed-guide/master.adoc -@@ -6,6 +6,6 @@ include::topics/developer-lightspeed/assembly_run-dev-lightspeed-analysis.adoc[l - - include::topics/developer-lightspeed/con_developer-lightspeed-logs.adoc[leveloffset=+1] - --include::topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc[leveloffset=+1] -+include::topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc[leveloffset=+1] - - :!mta-developer-lightspeed: -diff --git a/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc b/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc -new file mode 100644 -index 0000000..10b3f82 ---- /dev/null -+++ b/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc -@@ -0,0 +1,14 @@ -+// Module included in the following assemblies: -+// -+// * docs/developer-lightspeed-guide/master.adoc -+ -+:_mod-docs-content-type: ASSEMBLY -+ -+[id="assembly-llm-proxy-interaction-monitoring_{context}"] -+= LLM proxy interaction monitoring -+ -+The {mta-dl-plugin} LLM proxy acts as a single, administrator-controlled access point between developers and external large language model (LLM) providers. Understanding its data handling and logging behavior is important for organizations with artificial intelligence (AI) system governance requirements. -+ -+include::con_llm-proxy-data-handling.adoc[leveloffset=+1] -+ -+include::proc_access-llm-proxy-logs.adoc[leveloffset=+1] -diff --git a/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc b/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc -new file mode 100644 -index 0000000..1e66ce6 ---- /dev/null -+++ b/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc -@@ -0,0 +1,34 @@ -+// Module included in the following assemblies: -+// -+// * docs/developer-lightspeed-guide/master.adoc -+ -+:_mod-docs-content-type: CONCEPT -+ -+[id="llm-proxy-data-handling_{context}"] -+= LLM proxy request and data handling -+ -+[role="_abstract"] -+The LLM proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. It provides the following controls: -+ -+Credential isolation:: LLM provider application programming interface (API) keys are stored in a Kubernetes secret and never distributed to client applications. Only the proxy pod reads these credentials. -+ -+JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub's OIDC provider by using its JWKS endpoint: -++ -+[subs="+quotes"] -+---- -+https://____/realms/____/protocol/openid-connect/certs -+---- -++ -+Requests that fail validation are rejected with a `401` or `403` response before they reach the LLM provider. -+ -+Header redaction:: Authorization headers and other sensitive fields are redacted from proxy logs. -+ -+Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL database in the following tables: -++ -+* `llm_proxy_inference_store`: Individual inference records -+* `llm_proxy_conversations`: Conversation context records -+ -+[NOTE] -+==== -+As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated at the Hub and is not passed to or stored by the proxy. -+==== -diff --git a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc -new file mode 100644 -index 0000000..605beb9 ---- /dev/null -+++ b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc -@@ -0,0 +1,45 @@ -+// Module included in the following assemblies: -+// -+// * docs/developer-lightspeed-guide/master.adoc -+ -+:_mod-docs-content-type: PROCEDURE -+ -+[id="llm-proxy-log-access_{context}"] -+= Accessing LLM proxy pod logs -+ -+To diagnose connectivity errors between the proxy and the LLM provider, use the `oc logs` command. The proxy writes standard server output to stdout. -+ -+Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. -+ -+.Prerequisites -+* You have installed the {ocp-short} {CLIName} (`oc`). -+* You are logged in to the cluster with the `cluster-admin` role or an equivalent role that grants read access to pods in the {ProductShortName} namespace. -+ -+.Procedure -+ -+. Identify the proxy pod name: -++ -+[source,terminal,subs="+quotes"] -+---- -+$ *oc get pods -n ____ | grep proxy* -+---- -++ -+.Example output -+[source,terminal] -+---- -+mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d -+---- -+ -+. Stream the current log output from the proxy pod: -++ -+[source,terminal,subs="+quotes"] -+---- -+$ *oc logs -f ____ -n ____* -+---- -+ -+. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag: -++ -+[source,terminal,subs="+quotes"] -+---- -+$ *oc logs --previous ____ -n ____* -+---- -diff --git a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc b/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc -deleted file mode 100644 -index a1a1f87..0000000 ---- a/docs/topics/developer-lightspeed/proc_auditing-llm-proxy-transactions.adoc -+++ /dev/null -@@ -1,102 +0,0 @@ --// Module included in the following assemblies: --// --// * docs/developer-lightspeed-guide/master.adoc -- --:_template-generated: 2026-07-05 -- --:_mod-docs-content-type: CONCEPT -- --[id="monitoring-llm-proxy-interactions_{context}"] --= LLM proxy interaction monitoring -- --[role="_abstract"] --The {mta-dl-plugin} LLM proxy acts as a single, administrator-controlled access point between developers and external large language model (LLM) providers. Understanding its data handling and logging behavior is important for organizations with artificial intelligence (AI) system governance requirements. -- --[NOTE] --==== --Red{nbsp}Hat plans to provide structured per-request audit logging and security incident and event management (SIEM) forwarding. The current release generates only standard server logs. For more information, see xref:llm-proxy-future-logging_{context}[Planned audit logging capabilities]. --==== -- --[id="llm-proxy-data-handling_{context}"] --== LLM proxy request and data handling -- --The LLM proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. It provides the following controls: -- --Credential isolation:: LLM provider application programming interface (API) keys are stored in a Kubernetes secret and never distributed to client applications. Only the proxy pod reads these credentials. -- --JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub's Keycloak instance by using its `JWKS` endpoint: --+ --[subs="+quotes"] ------ --https://____/realms/____/protocol/openid-connect/certs ------ --+ --Requests that fail validation are rejected with a `401` or `403` response before they reach the LLM provider. -- --Header redaction:: Authorization headers and other sensitive fields are redacted from proxy logs. -- --Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL database in the following tables: --+ --* `llm_proxy_inference_store`: Individual inference records --* `llm_proxy_conversations`: Conversation context records -- --[NOTE] --==== --As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated at the Hub and is not passed to or stored by the proxy. --==== -- --[id="llm-proxy-log-access_{context}"] --== Accessing LLM proxy pod logs -- --To diagnose connectivity errors between the proxy and the LLM provider, use the `oc logs` command. The proxy writes standard server output to stdout. -- --Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. -- --.Prerequisites --* You have installed the {ocp-short} {CLIName} (`oc`). --* You are logged in to the cluster with the `cluster-admin` role or an equivalent role that grants read access to pods in the {ProductShortName} namespace. -- --.Procedure -- --. Identify the proxy pod name: --+ --[source,terminal,subs="+quotes"] ------ --$ *oc get pods -n ____ | grep proxy* ------ --+ --.Example output --[source,terminal] ------ --mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d ------ -- --. Stream the current log output from the proxy pod: --+ --[source,terminal,subs="+quotes"] ------ --$ *oc logs -f ____ -n ____* ------ --+ --.Example output --[source,terminal] ------ --INFO: 10.128.2.14:52310 - "POST /v1/chat/completions HTTP/1.1" 200 OK --INFO: 10.128.2.14:52311 - "POST /v1/chat/completions HTTP/1.1" 503 Service Unavailable ------ -- --. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag: --+ --[source,terminal,subs="+quotes"] ------ --$ *oc logs --previous ____ -n ____* ------ -- --[id="llm-proxy-future-logging_{context}"] --== Planned audit logging capabilities -- --Red{nbsp}Hat plans to provide the following audit logging capabilities: -- --* Structured JSON log events per completed LLM transaction, including request routing, response latency, and token count metadata. --* User-to-request correlation by using anonymized user identifiers. --* Forwarding structured log streams to a SIEM system (such as Splunk or Elasticsearch) by using the Red{nbsp}Hat OpenShift Logging Operator. --- -2.52.0 - From c699e7b040e2afdffbf275352e234d55549069d2 Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Mon, 20 Jul 2026 15:51:08 +0100 Subject: [PATCH 07/12] Update assembly_llm-proxy-interaction-monitoring.adoc --- .../assembly_llm-proxy-interaction-monitoring.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc b/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc index 10b3f823abb..93258d36bee 100644 --- a/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc +++ b/docs/topics/developer-lightspeed/assembly_llm-proxy-interaction-monitoring.adoc @@ -7,7 +7,8 @@ [id="assembly-llm-proxy-interaction-monitoring_{context}"] = LLM proxy interaction monitoring -The {mta-dl-plugin} LLM proxy acts as a single, administrator-controlled access point between developers and external large language model (LLM) providers. Understanding its data handling and logging behavior is important for organizations with artificial intelligence (AI) system governance requirements. +[role="_abstract"] +The {mta-dl-plugin} large language model (LLM) proxy acts as a single, administrator-controlled access point to external LLM providers. You can monitor its data handling and logging behaviors to meet artificial intelligence (AI) governance requirements. include::con_llm-proxy-data-handling.adoc[leveloffset=+1] From b78c04023274c476eae2cc7569f977473e312522 Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Mon, 20 Jul 2026 15:53:25 +0100 Subject: [PATCH 08/12] Update con_llm-proxy-data-handling.adoc --- .../con_llm-proxy-data-handling.adoc | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc b/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc index 1e66ce6865c..d722e34af99 100644 --- a/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc +++ b/docs/topics/developer-lightspeed/con_llm-proxy-data-handling.adoc @@ -8,27 +8,34 @@ = LLM proxy request and data handling [role="_abstract"] -The LLM proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. It provides the following controls: +The large language model (LLM) proxy is built on llama-stack and runs as a dedicated pod within the {ProductShortName} namespace. + +The proxy provides the following controls: Credential isolation:: LLM provider application programming interface (API) keys are stored in a Kubernetes secret and never distributed to client applications. Only the proxy pod reads these credentials. -JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub's OIDC provider by using its JWKS endpoint: +JWT-based authentication:: When authentication is enabled, the proxy validates every incoming JSON web token (JWT) request against the Hub OpenID Connect (OIDC) provider by using its JWKS endpoint: + [subs="+quotes"] ---- https://____/realms/____/protocol/openid-connect/certs ---- + +where: ++ +``:: Specifies the name of the OIDC provider. +``:: Specifies the name of the realm. ++ Requests that fail validation are rejected with a `401` or `403` response before they reach the LLM provider. Header redaction:: Authorization headers and other sensitive fields are redacted from proxy logs. Data persistence:: LLM interactions are stored in the llama-stack PostgreSQL database in the following tables: + -* `llm_proxy_inference_store`: Individual inference records -* `llm_proxy_conversations`: Conversation context records +* `llm_proxy_inference_store`: Individual inference records. +* `llm_proxy_conversations`: Conversation context records. [NOTE] ==== -As of {ProductShortName} 8.2, OpenID Connect (OIDC) user identity is terminated at the Hub and is not passed to or stored by the proxy. +As of {ProductShortName} 8.2, OIDC user identity is terminated at the Hub and is not passed to or stored by the proxy. ==== From a911fa99a8d724fc27d771587493167d06b426d7 Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Mon, 20 Jul 2026 15:55:07 +0100 Subject: [PATCH 09/12] Update proc_access-llm-proxy-logs.adoc --- .../proc_access-llm-proxy-logs.adoc | 28 +++++++++++++++---- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc index 605beb969e3..5c5ad5018ba 100644 --- a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc +++ b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc @@ -5,9 +5,10 @@ :_mod-docs-content-type: PROCEDURE [id="llm-proxy-log-access_{context}"] -= Accessing LLM proxy pod logs += Access LLM proxy pod logs -To diagnose connectivity errors between the proxy and the LLM provider, use the `oc logs` command. The proxy writes standard server output to stdout. +[role="_abstract"] +To diagnose connectivity errors between the proxy and the large language model (LLM) provider, use the `oc logs` command. The proxy writes standard server output to stdout. Current proxy logs are unstructured uvicorn server output. Each entry contains the HTTP method, request path, and status code, or a Python exception traceback for error conditions. They do not contain structured per-transaction records or user identity information. @@ -21,9 +22,13 @@ Current proxy logs are unstructured uvicorn server output. Each entry contains t + [source,terminal,subs="+quotes"] ---- -$ *oc get pods -n ____ | grep proxy* +$ oc get pods -n ____ | grep proxy ---- + +where: ++ +``:: Specifies the name of the {ProductShortName} namespace. ++ .Example output [source,terminal] ---- @@ -34,12 +39,23 @@ mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d + [source,terminal,subs="+quotes"] ---- -$ *oc logs -f ____ -n ____* +$ oc logs -f ____ -n ____ ---- ++ +where: ++ +``:: Specifies the name of the proxy pod. +``:: Specifies the name of the {ProductShortName} namespace. -. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag: +. To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag to the command: + [source,terminal,subs="+quotes"] ---- -$ *oc logs --previous ____ -n ____* +$ oc logs --previous ____ -n ____ ---- ++ +where: ++ +``:: Specifies the name of the proxy pod. +``:: Specifies the name of the {ProductShortName} namespace. + From 3495b4e67a177f3637e0a979c59ecf6dc85e38a7 Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Thu, 23 Jul 2026 14:51:49 +0100 Subject: [PATCH 10/12] Update docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc Co-authored-by: Mariya Pershina <53339200+mpershina@users.noreply.github.com> --- docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc index 5c5ad5018ba..2a219be0bf4 100644 --- a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc +++ b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc @@ -45,7 +45,6 @@ $ oc logs -f ____ -n ____ where: + ``:: Specifies the name of the proxy pod. -``:: Specifies the name of the {ProductShortName} namespace. . To retrieve logs from a previous (terminated) proxy pod, add the `--previous` flag to the command: + From 072ac2b61526b41fdf69e0bce8cbaa21f072b7ee Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Thu, 23 Jul 2026 14:52:36 +0100 Subject: [PATCH 11/12] Update docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc Co-authored-by: Mariya Pershina <53339200+mpershina@users.noreply.github.com> --- .../developer-lightspeed/proc_access-llm-proxy-logs.adoc | 5 ----- 1 file changed, 5 deletions(-) diff --git a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc index 2a219be0bf4..1bf82268d33 100644 --- a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc +++ b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc @@ -52,9 +52,4 @@ where: ---- $ oc logs --previous ____ -n ____ ---- -+ -where: -+ -``:: Specifies the name of the proxy pod. -``:: Specifies the name of the {ProductShortName} namespace. From a5681b346781c511523e8435bccd4352b28a3c2a Mon Sep 17 00:00:00 2001 From: Andy Arnold Date: Thu, 23 Jul 2026 14:53:01 +0100 Subject: [PATCH 12/12] Update docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc Co-authored-by: Mariya Pershina <53339200+mpershina@users.noreply.github.com> --- .../developer-lightspeed/proc_access-llm-proxy-logs.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc index 1bf82268d33..647ba2bd232 100644 --- a/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc +++ b/docs/topics/developer-lightspeed/proc_access-llm-proxy-logs.adoc @@ -29,7 +29,8 @@ where: + ``:: Specifies the name of the {ProductShortName} namespace. + -.Example output +The command returns output similar to the following example: ++ [source,terminal] ---- mta-ui-mta-proxy-7d6f9b8c4-xk2pq 1/1 Running 0 2d