Use ephemeral server-side sessions to temporarily persist messages-list across requests

[yangm2]

There may be a variable scoping issue in the backend. The current code has 2 scopes for backend variables ...

• View-class variables: these are global-scope, which can be dangerous if any response or session data is stored there.
• request arguments: these are scoped to the lifetime of a single request

For multi-turn conversations, each turn (human prompt + ai response) is effectively 1 (or more) requests. In order to properly maintain the context of multi-turn conversation, we probably need to use server-side session storage so that the message history (including reasoning w/ reasoning signatures) is retained for the duration of the session. Server-side storage will have a larger limit on context size than client-side (aka cookie) storage.

Here we see the class definition with a class variable.

tenantfirstaid/backend/tenantfirstaid/chat.py

Lines 18 to 35 in 17a235b

	class ChatView(View):
	def __init__(self) -> None:
	self.chat_manager = LangChainChatManager()
	def dispatch_request(self, *args, **kwargs) -> Response:
	"""
	Handle client POST request
	Expects JSON body with:
	- messages: List of AnyMessage dicts
	- city: Optional city name
	- state: State abbreviation
	"""
	data: Dict[str, Any] = request.json
	messages: List[AnyMessage] = data["messages"]
	city: Optional[OregonCity] = OregonCity.from_maybe_str(data["city"])
	state: UsaState = UsaState.from_maybe_str(data["state"])

The LangChainChatManager will be instantiated once for the backend, but called many times by one or more clients per request/response (aka turn). Thus the LangChainChatManager.agent is also a global singleton which services all request/responses. So there may be a problem in how the agent is created with a system prompt for "portland, oregon" but a subsequent session is supposed to prompted for "eugene, oregon".

Data like city/state could be stored in the client-side session (aka cookie) since they are small and sent on a per-request basis.

Note: message history should not be stored in any global-scoped variable because that will pollute concurrent conversations of different users/sessions!

[yangm2]

@claude read the issue description in the context of the current code base and evaluate whether we have a real problem. How can we test whether this is happening? Or do we just have safe, but forgetful context?

[claude]

Claude finished @yangm2's task in 3m 19s —— View job

---

Analysis Complete

• Read chat.py to understand view class structure
• Read langchain_chat_manager.py to understand agent lifecycle
• Analyze message history management
• Evaluate session pollution risk
• Propose testing strategies
• Provide final analysis

---

Findings: Safe but Forgetful (with performance concerns) ✅

After analyzing the codebase and Flask's View lifecycle, here's what I found:

Good News: No Session Pollution Risk 🎉

Flask's View Behavior: By default, Flask creates a NEW instance of ChatView for every HTTP request. This means:

class ChatView(View):
    def __init__(self) -> None:
        self.chat_manager = LangChainChatManager()  # NEW instance per request

Each request gets:

• A new ChatView instance
• A new LangChainChatManager instance
• A new agent (created on first use within that request)

Result: No risk of location prompt pollution or message history leakage between different users/sessions.

---

Current Architecture Analysis

1. Message History: Correctly Forgetful ✅

Location: chat.py:40

tid: Optional[str] = None  # Thread ID hardcoded to None

Without a thread_id:

• No checkpointer is created (langchain_chat_manager.py:94-95)
• Agent doesn't persist conversation history
• Each request is stateless

Verdict: Safe by design, but users can't have multi-turn conversations.

2. Location Context: Per-Request Isolation ✅

Location: langchain_chat_manager.py:161-162

if self.agent is None:
    self.agent = self.__create_agent_for_session(city, state, thread_id)

Since each request gets a fresh LangChainChatManager:

• self.agent starts as None
• Agent created with correct city/state for that specific request
• Next request gets a completely new manager/agent

Verdict: Location context is correctly isolated per request.

3. Potential Performance Issue ⚠️

Current behavior:

• Every request creates a new ChatGoogleGenerativeAI instance (langchain_chat_manager.py:59-71)
• Every request creates a new agent graph (langchain_chat_manager.py:100-106)

Impact:

• Unnecessary overhead for model initialization
• Agent compilation happens repeatedly
• Memory churn from creating/destroying heavy objects

Note: This is a performance concern, not a safety concern.

---

Testing Strategy

Even though the current code is safe, here's how to verify and prevent future regressions:

Test 1: Verify View Instance Isolation

# backend/tests/test_session_isolation.py
import pytest
from tenantfirstaid.chat import ChatView
from flask import Flask

def test_view_instances_are_isolated():
    """Verify that each request gets a new ChatView instance."""
    app = Flask(__name__)
    app.add_url_rule("/api/query", view_func=ChatView.as_view("chat"), methods=["POST"])
    
    instances = []
    
    # Patch __init__ to track instances
    original_init = ChatView.__init__
    def tracking_init(self):
        instances.append(id(self))
        original_init(self)
    
    ChatView.__init__ = tracking_init
    
    with app.test_client() as client:
        # Make two requests
        client.post("/api/query", json={
            "messages": [{"role": "human", "content": "Test 1"}],
            "city": "Portland", "state": "or"
        })
        client.post("/api/query", json={
            "messages": [{"role": "human", "content": "Test 2"}],
            "city": "Eugene", "state": "or"
        })
    
    ChatView.__init__ = original_init
    
    # Different instances were created
    assert len(instances) == 2
    assert instances[0] != instances[1]

Test 2: Verify Agent Recreation Per Request

def test_agent_created_per_request():
    """Verify that each request creates its own agent."""
    app = Flask(__name__)
    app.add_url_rule("/api/query", view_func=ChatView.as_view("chat"), methods=["POST"])
    
    agent_ids = []
    
    # Spy on agent creation
    from tenantfirstaid import langchain_chat_manager
    original_create = langchain_chat_manager.LangChainChatManager._LangChainChatManager__create_agent_for_session
    
    def tracking_create(self, city, state, thread_id):
        agent = original_create(self, city, state, thread_id)
        agent_ids.append(id(agent))
        return agent
    
    langchain_chat_manager.LangChainChatManager._LangChainChatManager__create_agent_for_session = tracking_create
    
    with app.test_client() as client:
        client.post("/api/query", json={
            "messages": [{"role": "human", "content": "Test 1"}],
            "city": "Portland", "state": "or"
        })
        client.post("/api/query", json={
            "messages": [{"role": "human", "content": "Test 2"}],
            "city": "Eugene", "state": "or"
        })
    
    langchain_chat_manager.LangChainChatManager._LangChainChatManager__create_agent_for_session = original_create
    
    assert len(agent_ids) == 2
    assert agent_ids[0] != agent_ids[1]

Test 3: Verify No Message History Persistence

def test_no_message_history_persistence():
    """Verify that messages don't persist between requests."""
    app = Flask(__name__)
    app.add_url_rule("/api/query", view_func=ChatView.as_view("chat"), methods=["POST"])
    
    with app.test_client() as client:
        # First request mentions a name
        resp1 = client.post("/api/query", json={
            "messages": [{"role": "human", "content": "My name is Alice"}],
            "city": "Portland", "state": "or"
        })
        
        # Second request asks about the name (should not remember)
        resp2 = client.post("/api/query", json={
            "messages": [{"role": "human", "content": "What's my name?"}],
            "city": "Portland", "state": "or"
        })
        
        response2_text = b"".join(resp2.response).decode()
        
        # Agent should not know the name from previous request
        assert "Alice" not in response2_text.lower()

---

When Session Management IS Implemented

The issue's concerns will become relevant when you implement server-side sessions. At that point, you'll need to:

1. Implement Thread ID Generation

Replace chat.py:40:

# Current
tid: Optional[str] = None

# Future (example using Flask sessions)
from flask import session
tid = session.get('thread_id') or str(uuid.uuid4())
session['thread_id'] = tid

2. Use Shared Checkpointer (CRITICAL!)

Currently, InMemorySaver() is created per-request (langchain_chat_manager.py:97). This needs to become a shared, persistent store:

# In app.py or singleton
from langgraph.checkpoint.postgres import PostgresSaver
# or
from langgraph.checkpoint.redis import RedisSaver

checkpointer = PostgresSaver(connection_string=...)

# In LangChainChatManager.__init__
def __init__(self, checkpointer):
    self.checkpointer = checkpointer

# In __create_agent_for_session
return create_agent(
    self.llm,
    self.tools,
    system_prompt=system_prompt,
    state_schema=TFAAgentStateSchema,
    checkpointer=self.checkpointer,  # Shared across requests
)

3. Keep City/State in Session Storage

Store location preferences in Flask session (client-side cookie) since they're small:

session['city'] = city
session['state'] = state

4. Performance Optimization (Optional)

Consider caching the LLM and tools at application level:

# In app.py
from tenantfirstaid.langchain_chat_manager import create_shared_llm, create_shared_tools

shared_llm = create_shared_llm()
shared_tools = create_shared_tools()
shared_checkpointer = PostgresSaver(...)

# Pass to ChatView
class ChatView(View):
    def __init__(self, llm, tools, checkpointer):
        self.llm = llm
        self.tools = tools
        self.checkpointer = checkpointer

---

Summary

Is this a real problem? NO - not currently

• Location context: Safe ✅ (new agent per request)
• Session history: Safe ✅ (no persistence, thread_id=None)
• Concurrent requests: Safe ✅ (isolated instances)

Current state: Safe but forgetful - exactly as designed

Future risk: When implementing server-side sessions:

• ⚠️ Must use shared persistent checkpointer (PostgreSQL/Redis)
• ⚠️ Must generate stable thread IDs per session
• ⚠️ Must avoid storing session data in view/manager instance variables

Performance note: Consider optimizing LLM/agent initialization when you implement sessions, but this is not a correctness issue.

---