Skip to content

Commit fb8a09a

Browse files
yaschoyascho
committed
commit
0 parents  commit fb8a09a

File tree

10 files changed

+5083
-0
lines changed

10 files changed

+5083
-0
lines changed

‎404.html‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="utf-8" />
5+
<meta http-equiv="x-ua-compatible" content="ie=edge" />
6+
<meta name="viewport" content="width=device-width, initial-scale=1" />
7+
8+
<title>
9+
A gentle introduction to collapse-based machine unlearning
10+
</title>
11+
12+
<!-- Begin Jekyll SEO tag v2.8.0 -->
13+
<meta name="generator" content="Jekyll v4.3.4" />
14+
<meta property="og:title" content="A gentle introduction to collapse-based machine unlearning" />
15+
<meta property="og:locale" content="en_US" />
16+
<link rel="canonical" href="http://localhost:4000/404.html" />
17+
<meta property="og:url" content="http://localhost:4000/404.html" />
18+
<meta property="og:site_name" content="A gentle introduction to collapse-based machine unlearning" />
19+
<meta property="og:type" content="website" />
20+
<meta name="twitter:card" content="summary" />
21+
<meta property="twitter:title" content="A gentle introduction to collapse-based machine unlearning" />
22+
<script type="application/ld+json">
23+
{"@context":"https://schema.org","@type":"WebPage","headline":"A gentle introduction to collapse-based machine unlearning","url":"http://localhost:4000/404.html"}</script>
24+
<!-- End Jekyll SEO tag -->
25+
26+
27+
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.0.0/dist/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
28+
<link rel="stylesheet" href="/css/main.css">
29+
<link
30+
href="https://fonts.googleapis.com/css?family=Open+Sans:400,300,700,800,600"
31+
rel="stylesheet"
32+
type="text/css"
33+
/>
34+
<link
35+
href="https://fonts.googleapis.com/css?family=Muli:400,300"
36+
rel="stylesheet"
37+
type="text/css"
38+
/>
39+
40+
<script>
41+
MathJax = {
42+
tex: {
43+
inlineMath: [['$', '$'], ['\\(', '\\)']],
44+
displayMath: [['$$','$$'], ['\\[','\\]']]
45+
},
46+
options: {
47+
skipHtmlTags: ['script','noscript','style','textarea','pre','code']
48+
}
49+
};
50+
</script>
51+
<script id="MathJax-script" async
52+
src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js">
53+
</script>
54+
<script src="https://cdn.plot.ly/plotly-3.1.0.min.js" charset="utf-8"></script>
55+
</head>
56+
<body class="d-flex flex-column h-100">
57+
<main class="container flex-shrink-0 p-0">
58+
<article class="container">
59+
60+
<div class="title">A gentle introduction to collapse-based <br>machine unlearning</div>
61+
62+
<br>
63+
Blogpost about the preprint:<br>
64+
<a href="https://arxiv.org/pdf/2507.04219">Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs</a>
65+
<br>
66+
<a href="https://yascho.github.io/" target="_blank" >Yan Scholten</a>$^1$,
67+
<a href="https://mila.quebec/en/directory/sophie-xhonneux" target="_blank">Sophie Xhonneux</a>$^2$,
68+
<a href="https://schwinnl.github.io/" target="_blank">Leo Schwinn</a>$^{*,1}$,
69+
and <a href="https://www.cs.cit.tum.de/daml/guennemann/" target="_blank">Stephan Günnemann</a>$^{*,1}$<br/>
70+
<br>
71+
72+
<div style="display: flex; justify-content: space-between; margin-top: 0em;">
73+
<div>$^1$ TUM<br>$^2$ Mila, Université de Montréal</div>
74+
<!--<div><i>Correspondence to YS.</i></div>-->
75+
</div>
76+
<br>
77+
<div id="collapseGroup" class="mt-4">
78+
<a class="btn btn-primary btn-sm collapsed"
79+
style="color:white;"
80+
role="button" href="https://arxiv.org/pdf/2507.04219" target="_blank">
81+
<b>&#9656; PDF</b>
82+
</a>
83+
<a class="btn btn-primary btn-sm collapsed"
84+
style="color:white;"
85+
role="button"
86+
data-bs-toggle="collapse"
87+
data-bs-target="#abs"
88+
aria-controls="abs">
89+
<b>&#9656; Abstract</b>
90+
</a>
91+
<a class="btn btn-primary btn-sm collapsed"
92+
style="color:white;"
93+
role="button" href="https://github.com/yascho/partial-model-collapse-unlearning" target="_blank">
94+
<b>&#9656; Code</b>
95+
</a>
96+
<a class="btn btn-primary btn-sm collapsed"
97+
style="color:white;"
98+
role="button"
99+
data-bs-toggle="collapse"
100+
data-bs-target="#bibtex"
101+
aria-controls="bibtex">
102+
<b>&#9656; Cite</b>
103+
</a>
104+
105+
<div id="abs" class="collapse mt-2" data-bs-parent="#collapseGroup" style="text-align:justify;">
106+
<p>Current unlearning methods for LLMs optimize on the private information they seek to remove by incorporating it into their fine-tuning data. We argue this not only risks reinforcing exposure to sensitive data, it also fundamentally contradicts the principle of minimizing its use. As a remedy, we propose a novel unlearning method - Partial Model Collapse (PMC), which does not require unlearning targets in the unlearning objective. Our approach is inspired by recent observations that training generative models on their own generations leads to distribution collapse, effectively removing information from model outputs. Our central insight is that model collapse can be leveraged for machine unlearning by deliberately triggering it for data we aim to remove. We theoretically analyze that our approach converges to the desired outcome, i.e. the model unlearns the data targeted for removal. We empirically demonstrate that PMC overcomes three key limitations of existing unlearning methods that explicitly optimize on unlearning targets, and more effectively removes private information from model outputs while preserving general model utility. Overall, our contributions represent an important step toward more comprehensive unlearning that aligns with real-world privacy constraints.</p>
107+
</div>
108+
109+
110+
<div id="bibtex" class="collapse mt-2" data-bs-parent="#collapseGroup" style="text-align:justify; position: relative;">
111+
<!-- Copy Button -->
112+
<button class="btn btn-sm btn-secondary"
113+
style="position: absolute; right: 0.5rem;"
114+
onclick="copyBibTeX()">
115+
Copy
116+
</button>
117+
118+
<br>
119+
<figure class="highlight"><pre><code class="language-bibtex" data-lang="bibtex"> <span class="nc">@misc</span><span class="p">{</span><span class="nl">scholten2025modelcollapse</span><span class="p">,</span>
120+
<span class="na">title</span><span class="p">=</span><span class="s">{Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs}</span><span class="p">,</span>
121+
<span class="na">author</span><span class="p">=</span><span class="s">{Yan Scholten and Sophie Xhonneux and Leo Schwinn and Stephan Günnemann}</span><span class="p">,</span>
122+
<span class="na">year</span><span class="p">=</span><span class="s">{2025}</span><span class="p">,</span>
123+
<span class="na">eprint</span><span class="p">=</span><span class="s">{2507.04219}</span><span class="p">,</span>
124+
<span class="na">archivePrefix</span><span class="p">=</span><span class="s">{arXiv}</span><span class="p">,</span>
125+
<span class="na">primaryClass</span><span class="p">=</span><span class="s">{cs.LG}</span><span class="p">,</span>
126+
<span class="na">url</span><span class="p">=</span><span class="s">{https://arxiv.org/abs/2507.04219}</span><span class="p">,</span>
127+
<span class="p">}</span>
128+
</code></pre></figure>
129+
</div>
130+
131+
132+
133+
</div>
134+
<hr>
135+
136+
<div style="text-align: justify;">
137+
<div class="container">
138+
<h1>404</h1>
139+
140+
<p><strong>Page not found :(</strong></p>
141+
<p>The requested page could not be found.</p>
142+
</div>
143+
144+
</div>
145+
146+
</article>
147+
</main>
148+
149+
<script>
150+
function copyBibTeX() {
151+
// Get the code content inside the highlight block
152+
const codeBlock = document.querySelector('#bibtex pre');
153+
const text = codeBlock.innerText;
154+
155+
navigator.clipboard.writeText(text)
156+
}
157+
</script>
158+
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.bundle.min.js" integrity="sha384-ka7Sk0Gln4gmtz2MlQnikT1wXgYsOg+OMhuP+IlRH9sENBO0LRn5q+8nbTov4+1p" crossorigin="anonymous"></script>
159+
160+
</body>
161+
162+
</html>

0 commit comments

Comments
 (0)