Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions content/event/260211.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: "From Empirical Study to Runtime Mitigation: Addressing Integration Failures in LLM-Enabled Software"

event: Weekly Talk
event_url:

location: MR24@COM3-02-64
address:
street:
city:
region:
postcode:
country: Singapore

summary:
abstract: "Large language models (LLMs) and Retrieval-Augmented Generation (RAG) are increasingly integrated into software systems to realize intelligent features. However, this integration poses significant challenges due to undefined interface specifications, diverse software context requirements, and complex system management.In this talk, we first present a comprehensive empirical study on the correctness of LLM integration. By analyzing 100 open-source LLM-enabled applications, we identified 18 distinct defect patterns located across the LLM agent, vector database, software components, and system management. Our study reveals that integration defects are widespread, with 77% of these applications containing more than three types of defects that degrade functionality, efficiency, and security. To facilitate future research, we constructed Hydrangea, a defect library containing 546 identified defects.Guided by the findings from our empirical study, we then introduce Comfrey, a runtime framework designed to prevent integration failures in LLM-enabled software. Serving as a middle layer between AI and software components, Comfrey automatically detects and resolves potential integration failures through a three-stage workflow targeting format, syntax, and repetition errors. Our evaluation demonstrates that Comfrey effectively detects 75.1% and prevents 63.3% of potential integration failures with only 8.4% overhead, significantly outperforming existing baselines."

# Talk start and end times.
# End time can optionally be hidden by prefixing the line with `#`.
date: "2026-02-11T14:00:00Z"
date_end: "2026-02-11T15:00:00Z"
all_day: false

# Schedule page publish date (NOT talk date).
publishDate: "2026-02-02T00:00:00Z"

authors: [Yuchen Shao]
tags: [Weekly Talk]

# Is this a featured talk? (true/false)
featured: false

image:
caption: 'Image credit: [**Unsplash**](https://unsplash.com/photos/bzdhc5b3Bxs)'
focal_point: Right

url_code: ""
url_pdf: ""
url_slides: ""
url_video: ""

# Markdown Slides (optional).
# Associate this talk with Markdown slides.
# Simply enter your slide deck's filename without extension.
# E.g. `slides = "example-slides"` references `content/slides/example-slides.md`.
# Otherwise, set `slides = ""`.
slides:

# Projects (optional).
# Associate this post with one or more of your projects.
# Simply enter your project's folder or file name without extension.
# E.g. `projects = ["internal-project"]` references `content/project/deep-learning/index.md`.
# Otherwise, set `projects = []`.
projects:

# Slides can be added in a few ways:
#
# - **Create** slides using Wowchemy's [*Slides*](https://wowchemy.com/docs/managing-content/#create-slides) feature and link using `slides` parameter in the front matter of the talk file
# - **Upload** an existing slide deck to `static/` and link using `url_slides` parameter in the front matter of the talk file
# - **Embed** your slides (e.g. Google Slides) or presentation video on this page using [shortcodes](https://wowchemy.com/docs/writing-markdown-latex/).
#
# Further event details, including page elements such as image galleries, can be added to the body of this page.

---
Speaker Info:

[Yuchen Shao](https://ycshao12.github.io/) is a third-year Ph.D. student at the Software Engineering Institute, East China Normal University (ECNU) and the Shanghai Innovation Institute, co-advised by Prof. Chengcheng Wan and Prof. Ting Su. Her research interests lie in SE/Sys for AI and software testing. Her recent work centers on the correctness and reliability of Large Language Model (LLM) integration in software systems, including analyzing integration patterns and mitigating runtime failures in LLM-enabled software.
2 changes: 1 addition & 1 deletion content/event/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Timetable for upcoming events in AY25/26 (subject to changes):
| March 4 | <font color=blue>TBD</font> | TEST-lab | Flavien Solt|
| Feb 25 | <font color=gray>Recess Week</font> | --- | ---|
| Feb 18 | <font color=gray>Skip: Chinese New Year</font> | --- | ---|
| Feb 11 | <font color=blue>Comfrey: Mitigating Integration Failures in LLM-enabled Software at Run-Time </font> | Yibo Dong | Yuchen Shao |
| Feb 11 | <font color=blue>From Empirical Study to Runtime Mitigation: Addressing Integration Failures in LLM-Enabled Software</font> | Yibo Dong | Yuchen Shao |
| Feb 10 | <font color=green>Reading Group</font> | Junwen | ---|
| Feb 4 | <font color=brown>Suyang Dry Run</font> | --- | ---|
| Jan 28 | <font color=gray>Skip</font> | --- | ---|
Expand Down