Fix DDIMParallelScheduler batch path ignoring final_alpha_cumprod#14216
Fix DDIMParallelScheduler batch path ignoring final_alpha_cumprod#14216Osamaali313 wants to merge 1 commit into
Conversation
`DDIMParallelScheduler`'s parallel path must match its sequential path
(the ParaDiGMS correctness guarantee is parallel == sequential). For the
terminal step (`prev_timestep < 0`), the scalar methods use
`self.final_alpha_cumprod`:
# _get_variance (l.274) and step (l.448)
alpha_prod_t_prev = self.alphas_cumprod[prev_timestep] if prev_timestep >= 0 else self.final_alpha_cumprod
but the batched methods hardcode `1.0`:
# _batch_get_variance and batch_step_no_noise
alpha_prod_t_prev[prev_t < 0] = torch.tensor(1.0)
`self.final_alpha_cumprod` equals `1.0` only when `set_alpha_to_one=True`;
with `set_alpha_to_one=False` (the default in Stable Diffusion 1.x scheduler
configs) it is `alphas_cumprod[0]` (~0.999). So on the final denoising step
the parallel path diverges from the sequential one. Use
`self.final_alpha_cumprod` in both batched methods, matching the scalar
siblings (it is already moved to the correct device before use).
|
Hi @Osamaali313, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. |
|
This is a standalone correctness fix — there's no separate tracked issue. The bug and a bit-exact before/after repro are in the PR description: |
Problem
DDIMParallelSchedulerimplements ParaDiGMS, whose whole premise is that the parallel path produces the same result as the sequential DDIM path. For the terminal step (prev_timestep < 0), the scalar methods useself.final_alpha_cumprod:but the batched methods hardcode
1.0:self.final_alpha_cumprodis set at init to:So it equals
1.0only whenset_alpha_to_one=True. Withset_alpha_to_one=False— the default in Stable Diffusion 1.x scheduler configs — it isalphas_cumprod[0](~0.999). On the final denoising step the batched path therefore diverges from the scalar path, breaking the parallel==sequential guarantee.Fix
Use
self.final_alpha_cumprodin both batched methods, matching the scalar siblings:final_alpha_cumprodis already moved to the correct device beforebatch_step_no_noiseuses it (line 577), and the scalar siblings reference it the same way.Reproduction
Porting the verbatim terminal-step arithmetic of both paths, with
set_alpha_to_one=False(final_alpha_cumprod ≈ 0.999):The current code diverges by ~6.8e-2 on the final step; the fix makes the parallel path bit-exactly equal to the sequential path.