Chore/logging fixes

[peshence]

Description

• no more double entries on request.processed
• added properly formatted request.Waiting logs
• added frontend http error codes in logs
• removed a spam logs like broker saying a user has reached their quota of requests, frontend reporting it's alive, etc
• logs now contain a summary for a long array in the request. The full array is logged once at the time of coercion. That's also the only time the user_request field is logged anymore. This allows splunk to parse our logs.

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

[Copilot]

Pull request overview

This PR focuses on improving log quality and structure across the worker, frontend, broker, and request stores to reduce spam, prevent oversized log events, and make logs more reliably parseable (e.g., in Splunk).

Changes:

• Introduce PolytopeRequest.serialize_logging() with recursive truncation/summarization for log-safe request payloads.
• Update request-store / worker / frontend logging to include structured fields (e.g., request payload, HTTP status/error code) and remove noisy log lines.
• Add/extend unit tests to validate shutdown behavior and new logging/serialization behavior.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/unit/test_worker_shutdown.py	New tests ensuring worker clears in-flight state before termination and reschedules processing requests on shutdown.
tests/unit/test_request.py	Adds tests for string truncation and recursive oversized-list summarization in serialize_logging().
tests/unit/test_mongodb_request_store.py	Adds assertions that MongoDB request-store logs include serialize_logging() payloads.
tests/unit/test_flask_handler.py	Adds tests for structured HTTP error logging and ensuring healthcheck success isn’t logged.
tests/unit/test_dynamodb.py	Adds assertions that DynamoDB request-store logs include serialize_logging() payloads.
polytope_server/worker/worker.py	Ensures worker clears request state before termination; switches processing log payload to serialize_logging().
polytope_server/frontend/flask_handler.py	Adds helper for structured exception logging (http.status, error.code) and changes healthcheck response to avoid success logging.
polytope_server/frontend/common/data_transfer.py	Adds “Received user request” logging and removes some “request added/succeeded/revoked” info logs.
polytope_server/common/request_store/mongodb_request_store.py	Logs structured serialize_logging() payload on add/update; updates add-request log message.
polytope_server/common/request_store/dynamodb_request_store.py	Logs structured serialize_logging() payload on add/update; updates add-request log message.
polytope_server/common/request.py	Implements bounded logging serialization (truncate strings, summarize long lists) and removes user_request from logging serialization.
polytope_server/common/collection.py	Logs both coerced request and raw user request at coercion time.
polytope_server/common/authotron.py	Switches auth failure logging to structured fields (status + WWW-Authenticate).
polytope_server/broker/broker.py	Downgrades some broker informational logs to debug; sets a user-facing message on queue failure.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.