feat: add thinking/reasoning display UI

[gadenbuie]

Summary

• Adds a collapsible ThinkingDisplay component that shows LLM reasoning tokens in real-time, with topic labels, animated transitions, and auto-collapse on completion
• Supports thinking via content_type: "thinking" transport (R/ellmer, Python/chatlas) and client-side <thinking> tag detection (local models, Python bookmark restore)
• Includes accessibility (aria-expanded, inert, focus-visible, prefers-reduced-motion), stick-to-bottom for overflow, and comprehensive reducer tests (54 cases)
• Handles edge cases: disconnect mid-stream (finalizes in-flight blocks), <thinking> tags inside fenced code blocks (preserved as content), empty thinking blocks (hidden), history replay (suppresses flash/duration artifacts), and multiple thinking-response cycles within a single turn

Examples

Local thinking models

Models that emit <thinking>...</thinking> tags (DeepSeek, QwQ, Gemma, etc.) get thinking UI automatically with no server-side changes:

Python:

from chatlas import ChatOpenAI

chat_model = ChatOpenAI(
    base_url="http://localhost:1234/v1",
    model="google/gemma-4-26b-a4b",
    api_key="lm-studio",
)

app = chat_model.app()

R:

library(shinychat)
library(ellmer)

chat <- chat_lmstudio(
    base_url = "http://localhost:1234",
    model = "google/gemma-4-26b-a4b"
)

live_browser(chat)

Structured thinking APIs

Models with dedicated reasoning APIs (Claude with extended thinking, OpenAI with reasoning) emit ContentThinking objects that flow through the content_type: "thinking" transport:

from chatlas import ChatAnthropic, ChatOpenAI

# Claude with extended thinking
chat = ChatAnthropic(reasoning=2000, model="claude-sonnet-4-6")
chat.app()

# OpenAI with reasoning
chat = ChatOpenAI(reasoning="medium", model="gpt-5.5")
chat.app()

Full demo app (Python) — mock thinking stream for testing

import asyncio
import random

from chatlas._content import ContentThinking
from shiny import App, reactive, ui



async def split_words_stream(texts: list[str]):
    """Yield characters with random delays to simulate streaming."""
    full = "".join(texts)
    for char in full:
        yield char
        await asyncio.sleep(random.uniform(0.005, 0.05))


thinking_chunks = [
    "The user is asking about the relationship between violent video games ",
    "and violent behavior in children. This is a well-studied topic in ",
    "psychology and media studies. I need to consider the research carefully.\n\n",
    "There have been several major meta-analyses on this topic. ",
    "Anderson et al. (2010) found small but significant effects, ",
    "while Ferguson (2015) argued these effects disappear when ",
    "publication bias is accounted for. The APA task force (2015) ",
    "concluded there's a link to aggression but insufficient evidence ",
    "for a link to criminal violence.\n\n",
    "Important confounding variables include family environment, ",
    "pre-existing behavioral issues, socioeconomic status, and ",
    "peer influences. Longitudinal studies have had mixed results ",
    "when controlling for these factors.\n\n",
    "I should present the nuanced scientific consensus: ",
    "short-term increases in aggressive thoughts/feelings are documented, ",
    "but the leap to real-world violent behavior is not well-supported. ",
    "I'll note the difference between aggression and violence.",
]

response_chunks = [
    "## The Short Answer\n\n",
    "The relationship between violent video games and real-world violence ",
    "is **far more nuanced** than popular media often suggests.\n\n",
    "## What the Research Shows\n\n",
    "### Short-term effects\n",
    "Studies do find that playing violent video games can temporarily ",
    "increase aggressive *thoughts* and *feelings*. ",
    "However, these effects are small and short-lived.\n\n",
    "### Long-term behavioral effects\n",
    "The evidence for a causal link to actual violent *behavior* is ",
    "much weaker. Major meta-analyses disagree:\n\n",
    "- **Anderson et al. (2010)**: Found small correlations with aggressive behavior\n",
    "- **Ferguson (2015)**: Argued effects vanish when correcting for publication bias\n",
    "- **APA (2015, 2020)**: Insufficient evidence to link games to criminal violence\n\n",
    "### The bigger picture\n",
    "As video game sales have skyrocketed globally since the 1990s, ",
    "youth violence rates have actually *declined* significantly in most countries. ",
    "This ecological data doesn't prove games reduce violence, but it strongly ",
    "undermines claims of a major causal effect.\n\n",
    "## Key Takeaway\n\n",
    "There's a meaningful difference between *aggression* (competitive feelings, ",
    "frustration) and *violence* (intent to harm). Video games may briefly ",
    "increase the former, but the scientific consensus does not support a ",
    "causal link to the latter.",
]

local_model_thinking_chunks = [
    "The user is asking about the relationship between violent video games ",
    "and violent behavior in children. This is a well-studied topic in ",
    "psychology and media studies. I need to consider the research carefully.\n\n",
    "<topic>Reviewing the research</topic>\n",
    "There have been several major meta-analyses on this topic. ",
    "Anderson et al. (2010) found small but significant effects, ",
    "while Ferguson (2015) argued these effects disappear when ",
    "publication bias is accounted for. The APA task force (2015) ",
    "concluded there's a link to aggression but insufficient evidence ",
    "for a link to criminal violence.\n\n",
    "I should present the nuanced scientific consensus.",
]

local_model_response_chunks = [
    "## The Short Answer\n\n",
    "The relationship between violent video games and real-world violence ",
    "is **far more nuanced** than popular media often suggests.\n\n",
    "## What the Research Shows\n\n",
    "Studies do find that playing violent video games can temporarily ",
    "increase aggressive *thoughts* and *feelings*. ",
    "However, these effects are small and short-lived.\n\n",
    "The evidence for a causal link to actual violent *behavior* is ",
    "much weaker.",
]


async def mock_thinking_stream(n: int = 1):
    """Yield ContentThinking and str chunks simulating structured thinking API."""
    indexes: list[dict[str, list[list[int]]]] = [
        {
            "thinking": [list(range(len(thinking_chunks)))],
            "response": [list(range(len(response_chunks)))],
        },
        {
            "thinking": [list(range(11)), list(range(11, len(thinking_chunks)))],
            "response": [list(range(15)), list(range(15, len(response_chunks)))],
        },
        {
            "thinking": [
                list(range(5)),
                list(range(5, 10)),
                list(range(10, len(thinking_chunks))),
            ],
            "response": [
                list(range(10)),
                list(range(10, 20)),
                list(range(20, len(response_chunks))),
            ],
        },
    ]

    idx = indexes[n - 1]

    for i in range(n):
        selected_thinking = [thinking_chunks[j] for j in idx["thinking"][i]]
        async for char in split_words_stream(selected_thinking):
            yield ContentThinking(thinking=char)

        selected_response = [response_chunks[j] for j in idx["response"][i]]
        async for char in split_words_stream(selected_response):
            yield char


async def mock_local_model_stream():
    """Yield plain strings with <thinking> tags (simulates local model output)."""
    async for char in split_words_stream(["<thinking>\n"]):
        yield char
    async for char in split_words_stream(local_model_thinking_chunks):
        yield char
    async for char in split_words_stream(["\n</thinking>\n\n"]):
        yield char
    async for char in split_words_stream(local_model_response_chunks):
        yield char


app_ui = ui.page_sidebar(
    ui.sidebar(
        ui.p(
            "This app demonstrates the thinking/reasoning display using mock data."
        ),
        ui.p(
            "Type anything (or just hit Enter) to trigger a simulated thinking response."
        ),
        ui.input_numeric(
            "num_responses",
            "Number of responses to simulate:",
            value=1,
            min=1,
            max=3,
        ),
        ui.input_checkbox(
            "local_model",
            "Simulate local model (raw <thinking> tags)",
            value=False,
        ),
        ui.input_action_button(
            "reset", "Clear Chat", class_="btn-outline-secondary btn-sm"
        ),
        width=300,
    ),
    ui.chat_ui("chat", placeholder="Ask me anything..."),
    title="Thinking UI Demo",
)


def server(input, output, session):
    chat = ui.Chat("chat")

    @chat.on_user_submit
    async def _():
        if input.local_model():
            stream = mock_local_model_stream()
        else:
            stream = mock_thinking_stream(input.num_responses())
        await chat.append_message_stream(stream)

    @reactive.effect
    @reactive.event(input.reset)
    async def _():
        await chat.clear_messages()


app = App(app_ui, server)

Full demo app (R) — mock thinking stream for testing

library(shiny)
library(bslib)
library(shinychat)
library(coro)
library(ellmer)

split_words <- function(texts) {
  texts <- unlist(strsplit(texts, ""))

  new_word <- TRUE

  purrr::reduce(texts, .init = list(), .f = function(acc, char) {
    if (new_word) {
      new_word <<- FALSE
      acc[[length(acc) + 1]] <- list(word = char, delay = runif(1, 0.01, 0.1))
      return(acc)
    }
    last_word <- acc[[length(acc)]]
    last_word$word <- paste0(last_word$word, char)
    acc[[length(acc)]] <- last_word
    if (char == " ") {
      new_word <<- TRUE
    }
    acc
  })
}

mock_local_model_stream <- async_generator(function(n = 1) {
  thinking_chunks <- c(
    "The user is asking about the relationship between violent video games ",
    "and violent behavior in children. This is a well-studied topic in ",
    "psychology and media studies. I need to consider the research carefully.\n\n",
    "<topic>Reviewing the research</topic>\n",
    "There have been several major meta-analyses on this topic. ",
    "Anderson et al. (2010) found small but significant effects, ",
    "while Ferguson (2015) argued these effects disappear when ",
    "publication bias is accounted for. The APA task force (2015) ",
    "concluded there's a link to aggression but insufficient evidence ",
    "for a link to criminal violence.\n\n",
    "I should present the nuanced scientific consensus."
  )

  response_chunks <- c(
    "## The Short Answer\n\n",
    "The relationship between violent video games and real-world violence ",
    "is **far more nuanced** than popular media often suggests.\n\n",
    "## What the Research Shows\n\n",
    "Studies do find that playing violent video games can temporarily ",
    "increase aggressive *thoughts* and *feelings*. ",
    "However, these effects are small and short-lived.\n\n",
    "The evidence for a causal link to actual violent *behavior* is ",
    "much weaker."
  )

  for (chunk in split_words(c("<thinking>\n"))) {
    yield(ContentText(chunk$word))
    await(async_sleep(chunk$delay))
  }
  for (chunk in split_words(thinking_chunks)) {
    yield(ContentText(chunk$word))
    await(async_sleep(chunk$delay))
  }
  for (chunk in split_words(c("\n</thinking>\n\n"))) {
    yield(ContentText(chunk$word))
    await(async_sleep(chunk$delay))
  }
  for (chunk in split_words(response_chunks)) {
    yield(ContentText(chunk$word))
    await(async_sleep(chunk$delay))
  }
})

mock_thinking_stream <- async_generator(function(n = 1) {
  thinking_chunks <- c(
    "The user is asking about the relationship between violent video games ",
    "and violent behavior in children. This is a well-studied topic in ",
    "psychology and media studies. I need to consider the research carefully.\n\n",
    "There have been several major meta-analyses on this topic. ",
    "Anderson et al. (2010) found small but significant effects, ",
    "while Ferguson (2015) argued these effects disappear when ",
    "publication bias is accounted for. The APA task force (2015) ",
    "concluded there's a link to aggression but insufficient evidence ",
    "for a link to criminal violence.\n\n",
    "Important confounding variables include family environment, ",
    "pre-existing behavioral issues, socioeconomic status, and ",
    "peer influences. Longitudinal studies have had mixed results ",
    "when controlling for these factors.\n\n",
    "I should present the nuanced scientific consensus: ",
    "short-term increases in aggressive thoughts/feelings are documented, ",
    "but the leap to real-world violent behavior is not well-supported. ",
    "I'll note the difference between aggression and violence.",
    paste(lorem::ipsum(4), collapse = "\n\n")
  )

  response_chunks <- c(
    "## The Short Answer\n\n",
    "The relationship between violent video games and real-world violence ",
    "is **far more nuanced** than popular media often suggests.\n\n",
    "## What the Research Shows\n\n",
    "### Short-term effects\n",
    "Studies do find that playing violent video games can temporarily ",
    "increase aggressive *thoughts* and *feelings*. ",
    "However, these effects are small and short-lived.\n\n",
    "### Long-term behavioral effects\n",
    "The evidence for a causal link to actual violent *behavior* is ",
    "much weaker. Major meta-analyses disagree:\n\n",
    "- **Anderson et al. (2010)**: Found small correlations with aggressive behavior\n",
    "- **Ferguson (2015)**: Argued effects vanish when correcting for publication bias\n",
    "- **APA (2015, 2020)**: Insufficient evidence to link games to criminal violence\n\n",
    "### The bigger picture\n",
    "As video game sales have skyrocketed globally since the 1990s, ",
    "youth violence rates have actually *declined* significantly in most countries. ",
    "This ecological data doesn't prove games reduce violence, but it strongly ",
    "undermines claims of a major causal effect.\n\n",
    "## Key Takeaway\n\n",
    "There's a meaningful difference between *aggression* (competitive feelings, ",
    "frustration) and *violence* (intent to harm). Video games may briefly ",
    "increase the former, but the scientific consensus does not support a ",
    "causal link to the latter."
  )

  indexes <- list(
    list(
      thinking = list(seq_along(thinking_chunks)),
      response = list(seq_along(response_chunks))
    ),
    list(
      thinking = list(1:11, 12:length(thinking_chunks)),
      response = list(1:15, 16:length(response_chunks))
    ),
    list(
      thinking = list(1:5, 6:10, 11:length(thinking_chunks)),
      response = list(1:10, 11:20, 21:length(response_chunks))
    )
  )

  for (i in seq_len(n)) {
    for (chunk in split_words(thinking_chunks[indexes[[n]]$thinking[[i]]])) {
      yield(ContentThinking(chunk$word))
      await(async_sleep(chunk$delay))
    }

    for (chunk in split_words(response_chunks[indexes[[n]]$response[[i]]])) {
      yield(ContentText(chunk$word))
      await(async_sleep(chunk$delay))
    }
  }
})

ui <- page_sidebar(
  title = "Thinking UI Demo",
  sidebar = sidebar(
    width = 300,
    p("This app demonstrates the thinking/reasoning display using mock data."),
    p(
      "Type anything (or just hit Enter) to trigger a simulated thinking response."
    ),
    numericInput(
      "num_responses",
      "Number of responses to simulate:",
      value = 1,
      min = 1,
      max = 3
    ),
    checkboxInput(
      "local_model",
      "Simulate local model (raw <thinking> tags)",
      value = FALSE
    ),
    actionButton("reset", "Clear Chat", class = "btn-outline-secondary btn-sm")
  ),
  chat_ui("chat", placeholder = "Ask me anything...")
)

server <- function(input, output, session) {
  observeEvent(input$chat_user_input, {
    stream <- if (input$local_model) {
      mock_local_model_stream()
    } else {
      mock_thinking_stream(input$num_responses)
    }
    chat_append("chat", stream)
  })

  observeEvent(input$reset, {
    chat_clear("chat")
  })
}

shinyApp(ui, server)

Test plan

• cd js && npm run lint — TypeScript + ESLint pass
• npx vitest run — 54 state reducer tests pass
• Manual test with thinking-enabled model (Claude extended thinking via ellmer)
• Verify collapsed/expanded states, topic transitions, auto-collapse behavior
• Verify <thinking> tags inside fenced code blocks are NOT promoted to thinking UI
• Verify bookmark/restore preserves thinking blocks (R via stream replay, Python via <thinking> tag wrapper)

[cpsievert]

I would like to avoid adding this piece of state -- #211 proposes a way of doing that.

Admittedly, the state management stuff in Python has always been problematic from the standpoint of complexity with somewhat questionable upside. By adding more state, we're compounding that problem and also contributing to drift between R and Python.

It's also worth calling out that for nested streams we track a separate piece of "checkpoint" state. I'm pretty sure if we were to keep this approach, we would also need a piece of checkpoint state for thinking as well.

[cpsievert]

Another problem I just realized with the current implementation: thinking state never gets reset when the stream is finished.

[gadenbuie]

Yeah most of this is about me not being sure about the machinery and usage behind _current_stream_message. My more direct take is that I'd rather consider dropping _current_stream_thinking entirely unless it's absolutely required. So my first question in response is really: do we even need it in the first place?

[gadenbuie]

Okay, I took another look and understand how _current_stream_message operates now. As an aside, I think we should revisit this at some point to move away from accumulating strings into this field, but that's for another time.

In 1b2a514, I updated the internal logic to buffer thinking tokens until thinking is complete and then append them to _current_stream_message when other content is received (or in a finally block if interrupted). This way, we stream tokens to the UI as they arrive, but also keep the current semantics of _current_stream_message.

[cpsievert]

So my first question in response is really: do we even need it in the first place?

If we want to properly restore streams that are getting transformed as they're sent out, then yes. I know we'll probably disagree on whether that is an important use case, but I can tell you right now there are real apps doing useful things by transforming the stream (shiny assistant, etc).

Can I follow up by asking what your objection to heading in the direction of #211 would be?

[cpsievert]

In 1b2a514, I updated the internal logic to buffer thinking tokens

This is a step in the right direction, but I still feel like it's more complex than it needs to be. If part of your resistance to #211 is that we currently don't store and restore the content type properly, then I think we should fix that separately (we already have this problem on main). Once we have that, I think all we would really need is to make sure we set the right content type (similar to the R changes).

[gadenbuie]

If we want to properly restore streams that are getting transformed as they're sent out, then yes.

Do you mean _current_stream_thinking? That's what I was asking if we need (or more specifically if we need to accumulate thinking tokens somewhere). But it seems the answer is yes, we need to accumulate the tokens somewhere but not in a separate field. Does the new approach that appends them to _current_stream_message work?

Can I follow up by asking what your objection to heading in the direction of #211 would be?

I would say that it's less about having an objection and more about needing context about the problems you're seeing and wanting to solve in #211. I'm finding this conversation here to be a lot more productive (because I'm talking with you and not Claude) than it is to try to read between the lines of Claude's PR description and code changes.

As I see it now, I generally like the idea of richer data types for delta objects, but at this stage it seems like #211 is trying to solve shinychat problems by pushing behavior changes into chatlas and ellmer. I think that at this point in time, we have enough available to us that we can solve problems locally in shinychat before we need to make changes in chatlas/ellmer.

[cpsievert]

Do you mean _current_stream_thinking?

Not specifically. I mean more generally any state that is tracking outgoing messages with the intention of restoring them on bookmark.

Does the new approach that appends them to _current_stream_message work?

Mostly yes, but not if the content type is changing mid-stream. Doing that actually wasn't supported before the React migration. Now that we more officially support it, I think we should update the Python state management to mirror how the JS side works.

solve problems locally in shinychat before we need to make changes in chatlas/ellmer.

I think I agree, but can we also agree that the chatlas/ellmer changes are an improvement regardless of shinychat? If so, I think we'll want to wait for those changes to land so that we can detect the right delta class.

[gadenbuie]

Mostly yes, but not if the content type is changing mid-stream.

That part of this PR does work, though, because we're storing the thinking as <thinking>...</thinking> in _current_stream_message, and those are handled client side. I think this an acceptable trade-off for now. I think in the long run we should stop storing or passing state via strings, but that's not something we should try to thread through this PR.

I think I agree, but can we also agree that the chatlas/ellmer changes are an improvement regardless of shinychat? If so, I think we'll want to wait for those changes to land so that we can detect the right delta class.

I think ContentDelta types would be useful, but they're a big enough feature that they should be developed carefully and independently. I think they'll be an improvement, but from ellmer and chatlas' perspectives the thinking deltas are just one type of delta that we'd want to support. I don't think it's a good idea to commit to a design now that without taking into consideration the other ways that delta types would be used.

Also, because ellmer just released, it will likely be more than a month before delta content classes will land in a usable way in ellmer. So I don't think we should wait for a ContentDeltaThinking class to land upstream.

[cpsievert]

I think in the long run we should stop storing or passing state via strings, but that's not something we should try to thread through this PR.

Yeah, agreed. If we do that, it would open a path where the python thinking logic becomes very minimal, while also more generally fixing the "incorrect restoration of mixed content types" problem.

BTW, I appreciate your pragmatic approach, but it would also bother me to keep kicking this pre-existing (& self-inflicted) problem down the road, making it harder to undo. It doesn't necessarily have to block this PR, but I'd at least like to take a look first.

[cpsievert]

If is_content_thinking was a TypeGuard, you could drop the hasattr() part

[cpsievert]

I'm not sure the message_content_chunk() is necessary?

[cpsievert]

I think this path leaves the reducer in an inconsistent state: the outer message is streaming, but the inner thinking block can still have streaming: false. Since ThinkingDisplay keys off the block-level flag, should we set that block to streaming: true here (or at creation time) so the UI state stays aligned?

[cpsievert]

Here's another area where I'm now realizing that I introduced a problem that's getting more exposed. That is, there are two levels of content and content type: top level content as well as block level content. I think it'd be worth simplifying this to have just one (block) level of content and building that in at the protocol level. That way we can restore properly and have less confusion around the source of truth.