-
Notifications
You must be signed in to change notification settings - Fork 143
Open
Description
Currently, changing num_frames (or any generation-related config) requires re-initializing the runner via create_generator, which reloads the entire model weights:
def create_generator(
self,
attn_mode="flash_attn2",
infer_steps=50,
num_frames=81,
height=480,
width=832,
guidance_scale=5.0,
sample_shift=5.0,
fps=16,
aspect_ratio="16:9",
boundary=0.900,
boundary_step_index=2,
denoising_step_list=[1000, 750, 500, 250],
config_json=None,
rope_type="torch",
resize_mode=None,
):
self.resize_mode = resize_mode
if config_json is not None:
self.set_infer_config_json(config_json)
else:
self.set_infer_config(
attn_mode,
rope_type,
infer_steps,
num_frames,
height,
width,
guidance_scale,
sample_shift,
fps,
aspect_ratio,
boundary,
boundary_step_index,
denoising_step_list,
)
config = set_config(self)
print_config(config)
self.runner = self._init_runner(config) <-- Re-initialize the runner (model) again!!!
This is correct behavior from an implementation standpoint, but it is inefficient in practice since many of these parameters are generation-only and do not require model weight reloading.
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
Todo