ACM-35599 Fix MCE pod log errors

[fxiang1]

📝 Summary

Ticket Summary (Title):
ACM Console pods are continuously logging ERR_STREAM_UNABLE_TO_PIPE and ERR_HTTP_HEADERS_SENT errors

Ticket Link:
https://redhat.atlassian.net/browse/ACM-35599

Type of Change:

• 🐞 Bug Fix
• ✨ Feature
• 🔧 Refactor
• 💸 Tech Debt
• 🧪 Test-related
• 📄 Docs

---

✅ Checklist

General

• PR title follows the convention (e.g. ACM-12340 Fix bug with...)
• Code builds and runs locally without errors
• No console logs, commented-out code, or unnecessary files
• All commits are meaningful and well-labeled
• All new display strings are externalized for localization (English only)
• (Nice to have) JSDoc comments added for new functions and interfaces

If Feature

• UI/UX reviewed (if applicable)
• All acceptance criteria met
• Unit test coverage added or updated
• Relevant documentation or comments included

If Bugfix

• Root cause and fix summary are documented in the ticket (for future reference / errata)
• Fix tested thoroughly and resolves the issue
• Test(s) added to prevent regression

---

🗒️ Notes for Reviewers

Root Cause

// Line 82-84 in serve.ts
if (req.headers['if-modified-since'] === modificationTime) {
  res.writeHead(constants.HTTP_STATUS_NOT_MODIFIED).end()
  // ← BUG: no `return` here!
}

When a browser sends a conditional request with If-Modified-Since matching the file's modification time, the code correctly sends a 304 Not Modified and ends the response — but then falls through into the compression/streaming code below (lines 86–152).
The Error Chain
Here's exactly what happens:

1. 304 response sent and stream ended (line 83)
2. Code falls through to the brotli/gzip/uncompressed sections
3. A readStream is created and pipeline(readStream, res, ...) is called on the already-ended response stream → ERR_STREAM_UNABLE_TO_PIPE
4. The readStream.on('open') callback fires and calls res.writeHead(200, ...) on a response that already has headers sent → ERR_HTTP_HEADERS_SENT
5. The pipeline error propagates to serveHandler → catchInternalServerError → respondInternalServerError which tries res.writeHead(500) on the same already-finished response → ERR_HTTP_HEADERS_SENT (unhandled rejection)
   All three errors in your log trace back to this single missing return.

Analysis for if this was caused by the Nodejs 24 upgrade

Why it was silent before: In Node.js 20/22, when pipeline() was called with an already-ended response stream, the behavior was more lenient — it would either silently fail or just call the error callback without throwing. The response data was effectively dropped, but no loud errors were produced. The 304 was correctly sent to the client, so functionally everything appeared to work.

What changed in Node.js 24: The streams implementation tightened up:

1. pipelineImpl now validates upfront — the check at node:internal/streams/pipeline:264 (visible in your stack trace) verifies the destination stream isn't closed/destroyed before attempting to pipe. This throws ERR_STREAM_UNABLE_TO_PIPE immediately instead of failing silently.
2. HTTP response streams are finalized more aggressively after .end() — the stream is marked as destroyed sooner, so by the time pipeline() runs, it sees a closed stream.
3. Stricter writeHead enforcement — calling writeHead() after headers are sent now consistently throws rather than being silently ignored.
   So the bug was always there — the 304 path was always falling through and attempting to pipe/write headers on a finished response — but older Node.js versions silently swallowed it. Node.js 24's stricter stream semantics turned it into the three visible errors you're seeing. The fix is correct regardless of Node.js version.

Summary by CodeRabbit

• Bug Fixes
  • Fixed error response handling to prevent errors when headers have already been sent to the client.
  • Improved cache revalidation to skip unnecessary content processing when serving cached responses.

[coderabbitai]

📝 Walkthrough

Walkthrough

Two defensive fixes in the backend: catchInternalServerError now checks res.headersSent before writing an error response, and the serve route adds an explicit return after sending 304 Not Modified to prevent the compression/streaming path from executing on cache hits.

Changes

HTTP Response Double-Write Guards

Layer / File(s)	Summary
headersSent guard in error middleware backend/src/lib/respond.ts	catchInternalServerError's returned handler checks res.headersSent before calling respondInternalServerError, skipping the write if headers are already committed.
Early return after 304 in static file server backend/src/routes/serve.ts	serve returns immediately after sending 304 Not Modified for if-modified-since matches, preventing downstream .br/.gz resolution and file streaming from running.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'ACM-35599 Fix MCE pod log errors' directly references the ticket number and accurately summarizes the main change—fixing pod logging errors.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	PR description includes ticket link, change type (Bug Fix), and detailed technical root cause analysis addressing the specific errors and fix rationale.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
66.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[KevinFCormier]

@fxiang1 Looks good. Can you patch this onto weekly and check there? I don't usually see this error when running locally, and often I only see it in one of the pods on the cluster (probably session affinity).

[fxiang1]

@KevinFCormier Yep, I patched console-mce earlier on the weekly and it looks good. I just realized I need to patch the console pods as well. Patching those now. You can take a look at the console-mce pods.

[KevinFCormier]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fxiang1, KevinFCormier

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [KevinFCormier,fxiang1]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[fxiang1]

@KevinFCormier should we backport this fix all the way to ACM 2.13?

[KevinFCormier]

@KevinFCormier should we backport this fix all the way to ACM 2.13?

Let's start with 2.17 and 2.16 for now.

[fxiang1]

/cherrypick release-2.17

[fxiang1]

/cherrypick release-2.16

[openshift-cherrypick-robot]

@fxiang1: new pull request created: #6355

Details

In response to this:

/cherrypick release-2.17

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-cherrypick-robot]

@fxiang1: new pull request created: #6356

Details

In response to this:

/cherrypick release-2.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.