Skip to content

feat(sae): SAE steering — delta clamps and full reconstruction#271

Draft
RhizoNymph wants to merge 327 commits into
feat/integrationfrom
feat/sae-steering
Draft

feat(sae): SAE steering — delta clamps and full reconstruction#271
RhizoNymph wants to merge 327 commits into
feat/integrationfrom
feat/sae-steering

Conversation

@RhizoNymph

Copy link
Copy Markdown
Owner

Brings SAE-based steering into the integration line: per-(layer, hook) SAE feature surgery composing with the existing additive steering, capture, and patching stack.

What this adds

SAE delta tier (sae_steering.py, SAEClampManager): encode the live residual with a registered SAE, replace a chosen set of feature activations with caller-supplied clamp values (absolute or additive), and add the resulting decoder-direction delta back into the residual stream. Clamp tables use row 0 as the no-op sentinel, rows 1/2 as global prefill/decode tiers, rows 3+ per-request; global and per-request clamps validate against overlap in both directions and set_global_clamps is atomic.

SAE full-reconstruction tier (sae_full_reconstruction.py, SAEFullReconstructionManager): replace the residual with decode(activate(encode(h))) + b_dec, optionally clamping the activation before decode (Scaling Monosemanticity / Golden Gate semantics). Rows are gated per-site by an active-row table (a row owned by another module's site never reconstructs here) and keyed by FR-only phase hashes so additive state can't route tokens into a reconstruction row in the wrong phase.

Runtime integration: both tiers ride the existing machinery — SamplingParams.sae_clamp_specs / sae_full_reconstruction_specs with cached phase hashes folded into the steering config hashes (prefix-cache aware), three independent scheduler capacity pools (additive / SAE clamp / full-recon) under the config-pool backpressure model, admission/release/streaming-refresh/preemption-resumption lifecycle in the steering runner mixin, TP/PP via the same collective_rpc discipline, and SAE module registration (manifest + encoder/decoder weights) through the steering modules registry and /v1/steering/modules router with kind: sae_delta | sae_full_reconstruction.

Hook semantics: SAE surgery at the block output operates on the true block output residual + hidden_states (the #198/#202 post_block consolidation applied to SAE), writing back through the residual summand as a delta so untouched tokens stay bit-identical; the sum is only formed when SAE buffers are attached at the site.

Notes for review

  • This branch already contains feat/integration (merged; 898 conflicts resolved with integration's framework as canonical), so the diff here is purely the SAE feature.
  • SteeringManager.register_config gains optional content-hash row aliasing (identical vector content shares a physical row); implicit aliasing applies to prefill rows only, keeping decode rows 1:1 for the per-row scale/monitor machinery.
  • GPU-dependent suites (tests/models/language/generation/test_sae_*_real_weights.py, test_steering.py, tests/v1/core/test_steering_config_reservation.py) still need a pass on GPU hardware.

RhizoNymph and others added 30 commits March 30, 2026 23:00
fix: stop writing phase-specific globals to shared steering buffers
…registration

fix: register decode steering configs during lazy manager init
…phase-capacity

fix: scheduler steering capacity counts only active phase
…k-hash

fix: include decode steering hash in generated cache block keys
…sition

fix: graceful steering transition + phase-tracked release
…idation

fix: remove double validation and add phase-specific norms to steering status
…ion-capacity

fix: transition-aware steering capacity and decode-start admission
fix: include decode hash in boundary block cache keys
fix: track req_id in deferred decode registrations to prevent leak
…obals

fix: report pending phase-specific globals in steering status
docs: update steering docs for deferred cleanup and pre-init status
…alidation

fix: validate matching dimensions between base and phase steering vectors
fix: swap steering hashes in InputBatch.swap_states()
…ropping

Replace the silent fallback-to-global approach with a proper deferral
mechanism.  Failed registrations during lazy init and prefill-to-decode
transitions are now appended to a unified
`_pending_steering_registrations` list that is retried with priority
before new admissions on every step.

The retry loop also filters stale entries:
- Entries for finished requests are dropped (prevents row leaks).
- Entries whose request changed phase are dropped (e.g. prefill
  deferred but request already transitioned to decode).

This subsumes the decode-only `_pending_decode_registrations` list
and the row-leak fix from PR #46.
RhizoNymph and others added 30 commits July 11, 2026 13:23
fix(sae-steering): full-recon row routing, hash separation, and scheduler capacity
# Conflicts:
#	.buildkite/ci_config.yaml
#	.buildkite/hardware_tests/amd.yaml
#	.buildkite/hardware_tests/cpu.yaml
#	.buildkite/intel_jobs/lora_intel.yaml
#	.buildkite/intel_jobs/misc_intel.yaml
#	.buildkite/intel_jobs/test-intel.yaml
#	.buildkite/performance-benchmarks/tests/serving-tests.json
#	.buildkite/release-pipeline.yaml
#	.buildkite/scripts/annotate-release.sh
#	.buildkite/scripts/hardware_ci/run-amd-test.sh
#	.buildkite/scripts/hardware_ci/run-cpu-test-arm.sh
#	.buildkite/scripts/hardware_ci/run-intel-test.sh
#	.buildkite/scripts/hardware_ci/run-xpu-test.sh
#	.buildkite/test-amd.yaml
#	.buildkite/test_areas/attention.yaml
#	.buildkite/test_areas/benchmarks.yaml
#	.buildkite/test_areas/disaggregated.yaml
#	.buildkite/test_areas/distributed.yaml
#	.buildkite/test_areas/e2e_integration.yaml
#	.buildkite/test_areas/engine.yaml
#	.buildkite/test_areas/entrypoints.yaml
#	.buildkite/test_areas/kernels.yaml
#	.buildkite/test_areas/lm_eval.yaml
#	.buildkite/test_areas/lora.yaml
#	.buildkite/test_areas/misc.yaml
#	.buildkite/test_areas/model_executor.yaml
#	.buildkite/test_areas/model_runner_v2.yaml
#	.buildkite/test_areas/models_basic.yaml
#	.buildkite/test_areas/models_language.yaml
#	.buildkite/test_areas/models_multimodal.yaml
#	.buildkite/test_areas/pytorch.yaml
#	.buildkite/test_areas/samplers.yaml
#	.buildkite/test_areas/spec_decode.yaml
#	.github/CODEOWNERS
#	.github/mergify.yml
#	.github/workflows/pre-commit.yml
#	.github/workflows/scripts/build.sh
#	.gitignore
#	.pre-commit-config.yaml
#	CMakeLists.txt
#	README.md
#	benchmarks/kernels/benchmark_router_gemm.py
#	benchmarks/multi_turn/benchmark_serving_multi_turn.py
#	cmake/cpu_extension.cmake
#	cmake/external_projects/deepgemm.cmake
#	cmake/external_projects/qutlass.cmake
#	cmake/external_projects/vllm_flash_attn.cmake
#	csrc/attention/dtype_fp8.cuh
#	csrc/attention/merge_attn_states.cu
#	csrc/cache.h
#	csrc/cpu/cpu_attn.cpp
#	csrc/cpu/cpu_attn_impl.hpp
#	csrc/cpu/cpu_attn_vec.hpp
#	csrc/cpu/cpu_fused_moe.cpp
#	csrc/cpu/cpu_types_riscv_defs.hpp
#	csrc/cpu/cpu_types_riscv_impl.hpp
#	csrc/cpu/generate_cpu_attn_dispatch.py
#	csrc/cpu/sgl-kernels/gemm.h
#	csrc/cpu/sgl-kernels/gemm_int4.cpp
#	csrc/cpu/torch_bindings.cpp
#	csrc/cpu/utils.hpp
#	csrc/cutlass_extensions/torch_utils.hpp
#	csrc/fused_qknorm_rope_kernel.cu
#	csrc/libtorch_stable/activation_kernels.cu
#	csrc/libtorch_stable/cache_kernels.cu
#	csrc/libtorch_stable/cutlass_extensions/epilogue/scaled_mm_epilogues_c2x.hpp
#	csrc/libtorch_stable/dispatch_utils.h
#	csrc/libtorch_stable/layernorm_kernels.cu
#	csrc/libtorch_stable/layernorm_quant_kernels.cu
#	csrc/libtorch_stable/moe/marlin_moe_wna16/marlin_template.h
#	csrc/libtorch_stable/moe/marlin_moe_wna16/ops.cu
#	csrc/libtorch_stable/moe/torch_bindings.cpp
#	csrc/libtorch_stable/ops.h
#	csrc/libtorch_stable/quantization/fp4/activation_nvfp4_quant_fusion_kernels.cu
#	csrc/libtorch_stable/quantization/fp4/mxfp4_blockwise_moe_kernel.cu
#	csrc/libtorch_stable/quantization/fp4/mxfp4_experts_quant.cu
#	csrc/libtorch_stable/quantization/fp4/nvfp4_blockwise_moe_kernel.cu
#	csrc/libtorch_stable/quantization/fp4/nvfp4_experts_quant.cu
#	csrc/libtorch_stable/quantization/fp4/nvfp4_quant_entry.cu
#	csrc/libtorch_stable/quantization/fp4/nvfp4_quant_kernels.cu
#	csrc/libtorch_stable/quantization/fp4/nvfp4_scaled_mm_entry.cu
#	csrc/libtorch_stable/quantization/fp4/nvfp4_scaled_mm_kernels.cu
#	csrc/libtorch_stable/quantization/fp4/nvfp4_scaled_mm_sm120_kernels.cu
#	csrc/libtorch_stable/quantization/fused_kernels/fused_silu_mul_block_quant.cu
#	csrc/libtorch_stable/quantization/fused_kernels/quant_conversions.cuh
#	csrc/libtorch_stable/quantization/marlin/marlin.cu
#	csrc/libtorch_stable/quantization/marlin/marlin_template.h
#	csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/scaled_mm_blockwise_sm90_fp8_dispatch.cuh
#	csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/scaled_mm_sm100_fp8.cu
#	csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/scaled_mm_sm100_fp8_dispatch.cuh
#	csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/scaled_mm_sm120_fp8.cu
#	csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/scaled_mm_sm90_fp8.cu
#	csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/scaled_mm_sm90_fp8_dispatch.cuh
#	csrc/libtorch_stable/quantization/w8a8/cutlass/scaled_mm_c2x.cu
#	csrc/libtorch_stable/quantization/w8a8/cutlass/scaled_mm_entry.cu
#	csrc/libtorch_stable/quantization/w8a8/fp8/per_token_group_quant.cu
#	csrc/libtorch_stable/sampler.cu
#	csrc/libtorch_stable/torch_bindings.cpp
#	csrc/minimax_reduce_rms_kernel.h
#	csrc/moe/moe_ops.h
#	csrc/ops.h
#	csrc/pos_encoding_kernels.cu
#	csrc/quantization/w8a8/fp8/amd/quant_utils.cuh
#	csrc/quantization/w8a8/fp8/nvidia/quant_utils.cuh
#	csrc/topk.cu
#	csrc/torch_bindings.cpp
#	docker/Dockerfile
#	docker/Dockerfile.cpu
#	docker/Dockerfile.nightly_torch
#	docker/Dockerfile.rocm
#	docker/Dockerfile.rocm_base
#	docker/Dockerfile.xpu
#	docker/versions.json
#	docs/OVERVIEW.md
#	docs/assets/contributing/dockerfile-stages-dependency.png
#	docs/benchmarking/cli.md
#	docs/contributing/profiling.md
#	docs/design/attention_backends.md
#	docs/design/capture_consumers.md
#	docs/design/cuda_graphs_multimodal.md
#	docs/design/moe_kernel_features.md
#	docs/features/batch_invariance.md
#	docs/features/capture_consumers.md
#	docs/features/disagg_prefill.md
#	docs/features/quantization/online.md
#	docs/features/speculative_decoding/README.md
#	docs/features/steering.md
#	docs/getting_started/installation/cpu.arm.inc.md
#	docs/getting_started/installation/cpu.x86.inc.md
#	docs/getting_started/installation/gpu.cuda.inc.md
#	docs/getting_started/installation/gpu.xpu.inc.md
#	docs/models/pooling_models/README.md
#	docs/models/pooling_models/reward.md
#	docs/models/pooling_models/token_classify.md
#	docs/models/pooling_models/token_embed.md
#	docs/models/supported_models.md
#	docs/serving/openai_compatible_server.md
#	docs/training/layerwise.md
#	docs/usage/security.md
#	examples/disaggregated/disaggregated_serving/disagg_proxy_multiturn.py
#	examples/disaggregated/disaggregated_serving/example_mm_serve.py
#	examples/features/speculative_decoding/extract_hidden_states_offline.py
#	examples/generate/multimodal/vision_language_multi_image_offline.py
#	examples/generate/multimodal/vision_language_offline.py
#	examples/online_serving/disaggregated_serving/moriio_toy_proxy_server.py
#	examples/speech_to_text/lid/openai_lid_client.py
#	examples/speech_to_text/openai/openai_transcription_client.py
#	examples/tool_chat_template_gemma4.jinja
#	pyproject.toml
#	requirements/build/cpu.txt
#	requirements/build/cuda.txt
#	requirements/common.txt
#	requirements/cuda.txt
#	requirements/kv_connectors.txt
#	requirements/rocm.txt
#	requirements/test/cuda.in
#	requirements/test/cuda.txt
#	requirements/test/nightly-torch.txt
#	requirements/test/rocm.in
#	requirements/test/rocm.txt
#	requirements/test/xpu.in
#	requirements/test/xpu.txt
#	requirements/tpu.txt
#	requirements/xpu.txt
#	setup.py
#	tests/basic_correctness/test_mem.py
#	tests/compile/conftest.py
#	tests/compile/fusions_e2e/conftest.py
#	tests/compile/fusions_e2e/test_tp2_ar_rms.py
#	tests/compile/passes/distributed/test_fusion_all_reduce.py
#	tests/compile/passes/ir/test_clone_cleanup.py
#	tests/compile/passes/test_rope_kvcache_fusion.py
#	tests/compile/test_dynamic_shapes_compilation.py
#	tests/conftest.py
#	tests/distributed/test_dcp_a2a.py
#	tests/distributed/test_eplb_execute.py
#	tests/distributed/test_kv_cache_events.py
#	tests/distributed/test_mnnvl_alltoall.py
#	tests/distributed/test_ray_v2_executor.py
#	tests/distributed/test_torchrun_example.py
#	tests/distributed/test_torchrun_example_moe.py
#	tests/engine/test_arg_utils.py
#	tests/entrypoints/openai/chat_completion/test_chat_completion_with_prompt_embeds.py
#	tests/entrypoints/openai/chat_completion/test_chat_error.py
#	tests/entrypoints/openai/chat_completion/test_serving_chat.py
#	tests/entrypoints/openai/chat_completion/test_thinking_token_budget.py
#	tests/entrypoints/openai/completion/test_shutdown.py
#	tests/entrypoints/openai/correctness/test_transcription_api_correctness.py
#	tests/entrypoints/openai/responses/test_function_call.py
#	tests/entrypoints/openai/responses/test_responses_utils.py
#	tests/entrypoints/openai/responses/test_serving_responses.py
#	tests/entrypoints/openai/test_capture_protocol.py
#	tests/entrypoints/openai/test_openai_schema.py
#	tests/entrypoints/openai/test_steering_modules.py
#	tests/entrypoints/openai/test_steering_protocol.py
#	tests/entrypoints/pooling/embed/test_io_processor.py
#	tests/entrypoints/serve/disagg/test_generate_stream.py
#	tests/entrypoints/serve/disagg/test_serving_tokens.py
#	tests/entrypoints/serve/steering/test_api_router.py
#	tests/entrypoints/serve/steering/test_api_router_distributed.py
#	tests/entrypoints/serve/steering/test_protocol.py
#	tests/entrypoints/serve/steering/test_worker_steering.py
#	tests/entrypoints/speech_to_text/transcription/test_transcription_inter_chunk_spacing.py
#	tests/evals/gsm8k/configs/models-mi3xx.txt
#	tests/evals/gsm8k/configs/models-qwen35-mi355.txt
#	tests/ir/test_op.py
#	tests/kernels/attention/test_attention_selector.py
#	tests/kernels/attention/test_cpu_attn.py
#	tests/kernels/attention/test_lightning_attn.py
#	tests/kernels/core/test_fused_q_kv_rmsnorm.py
#	tests/kernels/core/test_layernorm.py
#	tests/kernels/core/test_minimax_reduce_rms.py
#	tests/kernels/helion/test_register.py
#	tests/kernels/ir/test_layernorm.py
#	tests/kernels/mamba/test_causal_conv1d.py
#	tests/kernels/mamba/test_mamba_ssm.py
#	tests/kernels/mamba/test_ssu_dispatch.py
#	tests/kernels/moe/modular_kernel_tools/mk_objects.py
#	tests/kernels/moe/modular_kernel_tools/parallel_utils.py
#	tests/kernels/moe/test_batched_deepgemm.py
#	tests/kernels/moe/test_cpu_fused_moe.py
#	tests/kernels/moe/test_cutlass_moe.py
#	tests/kernels/moe/test_deepgemm.py
#	tests/kernels/moe/test_moe.py
#	tests/kernels/moe/test_moe_layer.py
#	tests/kernels/moe/test_moe_weight_loading_padded.py
#	tests/kernels/moe/test_mxfp4_moe.py
#	tests/kernels/moe/test_topk_softplus_sqrt.py
#	tests/kernels/moe/test_trtllm_nvfp4_moe.py
#	tests/kernels/moe/test_unquantized_backend_selection.py
#	tests/kernels/moe/test_zero_expert_moe.py
#	tests/kernels/moe/utils.py
#	tests/kernels/quantization/test_mxfp4_triton_ep.py
#	tests/kernels/quantization/test_per_token_group_quant.py
#	tests/kernels/test_awq_int4_to_int8.py
#	tests/kernels/test_compressor_kv_cache.py
#	tests/kernels/test_flex_attention.py
#	tests/kernels/test_fused_deepseek_v4_qnorm_rope_kv_insert.py
#	tests/kernels/test_fused_indexer_q_rope_quant.py
#	tests/kernels/test_fused_inv_rope_fp8_quant.py
#	tests/kernels/test_kda.py
#	tests/lora/test_layers.py
#	tests/lora/test_lora_manager.py
#	tests/lora/test_olmoe_tp.py
#	tests/lora/test_worker.py
#	tests/model_executor/layers/test_steering_any_active.py
#	tests/model_executor/layers/test_steering_op.py
#	tests/model_executor/model_loader/test_reload.py
#	tests/model_executor/test_routed_experts_capture.py
#	tests/models/language/pooling/test_colbert.py
#	tests/models/language/pooling/test_max_tokens_per_doc.py
#	tests/models/language/pooling/test_nomic_max_model_len.py
#	tests/models/multimodal/generation/test_nemotron_parse.py
#	tests/models/multimodal/generation/test_qwen2_5_vl.py
#	tests/models/multimodal/generation/test_vit_cudagraph.py
#	tests/models/multimodal/generation/vlm_utils/model_utils.py
#	tests/models/multimodal/processing/test_gemma4.py
#	tests/models/quantization/test_gpt_oss.py
#	tests/models/quantization/test_nvfp4.py
#	tests/models/registry.py
#	tests/models/test_deepseek_v4_mega_moe.py
#	tests/models/utils.py
#	tests/multimodal/test_video.py
#	tests/quantization/test_compressed_tensors.py
#	tests/quantization/test_cpu_wna16.py
#	tests/quantization/test_cutlass_w4a16.py
#	tests/quantization/test_gfx950_moe.py
#	tests/quantization/test_mi3xx_moe.py
#	tests/quantization/test_modelopt.py
#	tests/quantization/test_online.py
#	tests/quantization/test_quark.py
#	tests/quantization/test_turboquant.py
#	tests/reasoning/test_gemma4_reasoning_parser.py
#	tests/renderers/test_gemma4_chat_template.py
#	tests/test_config.py
#	tests/tokenizers_/test_mistral.py
#	tests/tool_parsers/test_deepseekv32_tool_parser.py
#	tests/tool_parsers/test_deepseekv4_tool_parser.py
#	tests/tool_parsers/test_gemma4_tool_parser.py
#	tests/tool_parsers/test_glm4_moe_tool_parser.py
#	tests/tool_parsers/test_minimax_m2_tool_parser.py
#	tests/tool_parsers/test_mistral_tool_parser.py
#	tests/tool_parsers/test_qwen3coder_tool_parser.py
#	tests/tool_parsers/test_qwen3xml_tool_parser.py
#	tests/tool_use/test_gemma4_responses_adjust_request.py
#	tests/transformers_utils/test_utils.py
#	tests/utils.py
#	tests/utils_/test_numa_utils.py
#	tests/v1/attention/test_mla_prefill_selector.py
#	tests/v1/attention/utils.py
#	tests/v1/capture/consumers/filesystem/test_consumer.py
#	tests/v1/capture/consumers/test_logging.py
#	tests/v1/capture/test_config.py
#	tests/v1/capture/test_consumer_base.py
#	tests/v1/capture/test_driver_bridge.py
#	tests/v1/capture/test_manager.py
#	tests/v1/capture/test_multi_consumer_runner.py
#	tests/v1/capture/test_plan.py
#	tests/v1/capture/test_registry.py
#	tests/v1/capture/test_runner_integration.py
#	tests/v1/capture/test_sampling_params.py
#	tests/v1/capture/test_types.py
#	tests/v1/core/test_async_scheduler.py
#	tests/v1/core/test_prefix_caching.py
#	tests/v1/core/test_scheduler.py
#	tests/v1/core/test_single_type_kv_cache_manager.py
#	tests/v1/core/test_steering_hash_determinism.py
#	tests/v1/core/test_steering_scheduler.py
#	tests/v1/cudagraph/test_cudagraph_dispatch.py
#	tests/v1/cudagraph/test_encoder_cudagraph.py
#	tests/v1/determinism/test_batch_invariance.py
#	tests/v1/determinism/test_rms_norm_batch_invariant.py
#	tests/v1/e2e/general/test_mamba_prefix_cache.py
#	tests/v1/e2e/spec_decode/test_async_spec_decode.py
#	tests/v1/e2e/spec_decode/test_spec_decode.py
#	tests/v1/engine/test_input_processor_steering.py
#	tests/v1/kv_connector/nixl_integration/config_sweep_accuracy_test.sh
#	tests/v1/kv_connector/unit/offloading_connector/test_scheduler.py
#	tests/v1/kv_connector/unit/offloading_connector/test_worker_metadata.py
#	tests/v1/kv_connector/unit/offloading_connector/utils.py
#	tests/v1/kv_connector/unit/test_bidirectional_kv_transfer.py
#	tests/v1/kv_connector/unit/test_mooncake_connector.py
#	tests/v1/kv_connector/unit/test_mooncake_connector_hma.py
#	tests/v1/kv_connector/unit/test_moriio_connector.py
#	tests/v1/kv_connector/unit/test_multi_connector.py
#	tests/v1/kv_connector/unit/test_nixl_connector.py
#	tests/v1/kv_connector/unit/test_nixl_connector_hma.py
#	tests/v1/kv_connector/unit/test_offloading_connector.py
#	tests/v1/kv_connector/unit/test_remote_prefill_lifecycle.py
#	tests/v1/kv_connector/unit/test_rixl_gpu_mem_diag.py
#	tests/v1/kv_connector/unit/utils.py
#	tests/v1/kv_offload/cpu/test_gpu_worker.py
#	tests/v1/kv_offload/cpu/test_manager.py
#	tests/v1/kv_offload/cpu/test_shared_offload_region.py
#	tests/v1/logits_processors/test_correctness.py
#	tests/v1/sample/test_rejection_sampler.py
#	tests/v1/sample/test_topk_topp_sampler.py
#	tests/v1/simple_kv_offload/test_integration.py
#	tests/v1/simple_kv_offload/test_scheduler.py
#	tests/v1/spec_decode/test_acceptance_length.py
#	tests/v1/spec_decode/test_eagle.py
#	tests/v1/spec_decode/test_tree_attention.py
#	tests/v1/streaming_input/test_gpu_model_runner_streaming.py
#	tests/v1/test_request_steering.py
#	tests/v1/test_steering_inline_packed.py
#	tests/v1/test_steering_types.py
#	tests/v1/worker/test_gpu_input_batch.py
#	tests/v1/worker/test_gpu_model_runner.py
#	tests/v1/worker/test_steering_manager.py
#	tests/v1/worker/test_steering_manager_ownership.py
#	tests/v1/worker/test_steering_named_resolve_cache.py
#	tools/pre_commit/generate_attention_backend_docs.py
#	vllm/_aiter_ops.py
#	vllm/_custom_ops.py
#	vllm/_xpu_ops.py
#	vllm/benchmarks/datasets/__init__.py
#	vllm/benchmarks/datasets/datasets.py
#	vllm/compilation/backends.py
#	vllm/compilation/caching.py
#	vllm/compilation/codegen.py
#	vllm/compilation/passes/fusion/allreduce_rms_fusion.py
#	vllm/compilation/passes/fusion/collective_fusion.py
#	vllm/compilation/passes/fusion/matcher_utils.py
#	vllm/compilation/passes/fusion/rms_quant_fusion.py
#	vllm/compilation/passes/fusion/rocm_aiter_fusion.py
#	vllm/compilation/passes/fusion/rope_kvcache_fusion.py
#	vllm/compilation/passes/fusion/sequence_parallelism.py
#	vllm/compilation/passes/inductor_pass.py
#	vllm/compilation/passes/ir/clone_elimination.py
#	vllm/compilation/passes/pass_manager.py
#	vllm/compilation/passes/vllm_inductor_pass.py
#	vllm/compilation/wrapper.py
#	vllm/config/__init__.py
#	vllm/config/attention.py
#	vllm/config/cache.py
#	vllm/config/compilation.py
#	vllm/config/kernel.py
#	vllm/config/model.py
#	vllm/config/parallel.py
#	vllm/config/pooler.py
#	vllm/config/quantization.py
#	vllm/config/speculative.py
#	vllm/config/steering.py
#	vllm/config/steering_types.py
#	vllm/config/vllm.py
#	vllm/distributed/device_communicators/all2all.py
#	vllm/distributed/device_communicators/cuda_communicator.py
#	vllm/distributed/device_communicators/xpu_communicator.py
#	vllm/distributed/elastic_ep/elastic_execute.py
#	vllm/distributed/eplb/async_worker.py
#	vllm/distributed/eplb/eplb_communicator.py
#	vllm/distributed/eplb/eplb_state.py
#	vllm/distributed/eplb/eplb_utils.py
#	vllm/distributed/eplb/rebalance_execute.py
#	vllm/distributed/kv_events.py
#	vllm/distributed/kv_transfer/kv_connector/factory.py
#	vllm/distributed/kv_transfer/kv_connector/utils.py
#	vllm/distributed/kv_transfer/kv_connector/v1/lmcache_mp_connector.py
#	vllm/distributed/kv_transfer/kv_connector/v1/mooncake/mooncake_connector.py
#	vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py
#	vllm/distributed/kv_transfer/kv_connector/v1/multi_connector.py
#	vllm/distributed/kv_transfer/kv_connector/v1/nixl/__init__.py
#	vllm/distributed/kv_transfer/kv_connector/v1/nixl/connector.py
#	vllm/distributed/kv_transfer/kv_connector/v1/nixl/metadata.py
#	vllm/distributed/kv_transfer/kv_connector/v1/nixl/scheduler.py
#	vllm/distributed/kv_transfer/kv_connector/v1/nixl/stats.py
#	vllm/distributed/kv_transfer/kv_connector/v1/nixl/utils.py
#	vllm/distributed/kv_transfer/kv_connector/v1/nixl/worker.py
#	vllm/distributed/kv_transfer/kv_connector/v1/offloading/common.py
#	vllm/distributed/kv_transfer/kv_connector/v1/offloading/scheduler.py
#	vllm/distributed/kv_transfer/kv_connector/v1/offloading/worker.py
#	vllm/distributed/kv_transfer/kv_connector/v1/offloading_connector.py
#	vllm/distributed/kv_transfer/kv_connector/v1/simple_cpu_offload_connector.py
#	vllm/distributed/kv_transfer/kv_connector/v1/ssm_conv_transfer_utils.py
#	vllm/distributed/nixl_utils.py
#	vllm/engine/arg_utils.py
#	vllm/engine/protocol.py
#	vllm/entrypoints/anthropic/protocol.py
#	vllm/entrypoints/anthropic/serving.py
#	vllm/entrypoints/chat_utils.py
#	vllm/entrypoints/cli/serve.py
#	vllm/entrypoints/generate/factories.py
#	vllm/entrypoints/llm.py
#	vllm/entrypoints/openai/api_server.py
#	vllm/entrypoints/openai/chat_completion/batch_serving.py
#	vllm/entrypoints/openai/chat_completion/protocol.py
#	vllm/entrypoints/openai/chat_completion/serving.py
#	vllm/entrypoints/openai/cli_args.py
#	vllm/entrypoints/openai/completion/protocol.py
#	vllm/entrypoints/openai/completion/serving.py
#	vllm/entrypoints/openai/engine/protocol.py
#	vllm/entrypoints/openai/engine/serving.py
#	vllm/entrypoints/openai/models/protocol.py
#	vllm/entrypoints/openai/parser/harmony_utils.py
#	vllm/entrypoints/openai/parser/responses_parser.py
#	vllm/entrypoints/openai/responses/context.py
#	vllm/entrypoints/openai/responses/protocol.py
#	vllm/entrypoints/openai/responses/serving.py
#	vllm/entrypoints/openai/responses/streaming_events.py
#	vllm/entrypoints/openai/steering/registry.py
#	vllm/entrypoints/pooling/base/serving.py
#	vllm/entrypoints/pooling/classify/api_router.py
#	vllm/entrypoints/pooling/classify/protocol.py
#	vllm/entrypoints/pooling/embed/api_router.py
#	vllm/entrypoints/pooling/embed/io_processor.py
#	vllm/entrypoints/pooling/embed/protocol.py
#	vllm/entrypoints/pooling/embed/serving.py
#	vllm/entrypoints/pooling/factories.py
#	vllm/entrypoints/pooling/pooling/api_router.py
#	vllm/entrypoints/pooling/pooling/serving.py
#	vllm/entrypoints/pooling/scoring/api_router.py
#	vllm/entrypoints/pooling/scoring/io_processor.py
#	vllm/entrypoints/pooling/scoring/protocol.py
#	vllm/entrypoints/pooling/utils.py
#	vllm/entrypoints/serve/__init__.py
#	vllm/entrypoints/serve/disagg/protocol.py
#	vllm/entrypoints/serve/disagg/serving.py
#	vllm/entrypoints/serve/render/serving.py
#	vllm/entrypoints/serve/sagemaker/api_router.py
#	vllm/entrypoints/serve/steering/_merge.py
#	vllm/entrypoints/serve/steering/api_router.py
#	vllm/entrypoints/serve/steering/modules_protocol.py
#	vllm/entrypoints/serve/steering/modules_router.py
#	vllm/entrypoints/serve/steering/protocol.py
#	vllm/entrypoints/speech_to_text/base/serving.py
#	vllm/entrypoints/speech_to_text/transcription/protocol.py
#	vllm/entrypoints/speech_to_text/transcription/serving.py
#	vllm/env_override.py
#	vllm/envs.py
#	vllm/ir/__init__.py
#	vllm/ir/op.py
#	vllm/ir/util.py
#	vllm/kernels/helion/register.py
#	vllm/kernels/helion/utils.py
#	vllm/kernels/vllm_c.py
#	vllm/kernels/xpu_ops.py
#	vllm/lora/layers/fused_moe.py
#	vllm/lora/punica_wrapper/punica_base.py
#	vllm/lora/worker_manager.py
#	vllm/model_executor/kernels/linear/__init__.py
#	vllm/model_executor/kernels/linear/base.py
#	vllm/model_executor/kernels/linear/mixed_precision/__init__.py
#	vllm/model_executor/kernels/linear/mixed_precision/triton_w4a16.py
#	vllm/model_executor/kernels/linear/mxfp8/emulation.py
#	vllm/model_executor/kernels/linear/mxfp8/flashinfer.py
#	vllm/model_executor/kernels/linear/nvfp4/base.py
#	vllm/model_executor/kernels/linear/nvfp4/cutlass.py
#	vllm/model_executor/kernels/linear/nvfp4/flashinfer.py
#	vllm/model_executor/kernels/linear/scaled_mm/BlockScaledMMLinearKernel.py
#	vllm/model_executor/kernels/linear/scaled_mm/__init__.py
#	vllm/model_executor/kernels/linear/scaled_mm/aiter.py
#	vllm/model_executor/kernels/linear/scaled_mm/cutlass.py
#	vllm/model_executor/kernels/linear/scaled_mm/flashinfer.py
#	vllm/model_executor/kernels/linear/scaled_mm/triton.py
#	vllm/model_executor/kernels/linear/scaled_mm/xpu.py
#	vllm/model_executor/layers/activation.py
#	vllm/model_executor/layers/activation_capture.py
#	vllm/model_executor/layers/attention/attention.py
#	vllm/model_executor/layers/attention/mla_attention.py
#	vllm/model_executor/layers/attention/mm_encoder_attention.py
#	vllm/model_executor/layers/batch_invariant.py
#	vllm/model_executor/layers/fla/ops/chunk.py
#	vllm/model_executor/layers/fla/ops/chunk_delta_h.py
#	vllm/model_executor/layers/fla/ops/chunk_o.py
#	vllm/model_executor/layers/fla/ops/kda.py
#	vllm/model_executor/layers/fused_moe/__init__.py
#	vllm/model_executor/layers/fused_moe/all2all_utils.py
#	vllm/model_executor/layers/fused_moe/config.py
#	vllm/model_executor/layers/fused_moe/cpu_fused_moe.py
#	vllm/model_executor/layers/fused_moe/experts/cutlass_moe.py
#	vllm/model_executor/layers/fused_moe/experts/deep_gemm_moe.py
#	vllm/model_executor/layers/fused_moe/experts/flashinfer_cutedsl_batched_moe.py
#	vllm/model_executor/layers/fused_moe/experts/flashinfer_cutedsl_moe.py
#	vllm/model_executor/layers/fused_moe/experts/fused_batched_moe.py
#	vllm/model_executor/layers/fused_moe/experts/gpt_oss_triton_kernels_moe.py
#	vllm/model_executor/layers/fused_moe/experts/marlin_moe.py
#	vllm/model_executor/layers/fused_moe/experts/nvfp4_emulation_moe.py
#	vllm/model_executor/layers/fused_moe/experts/ocp_mx_emulation_moe.py
#	vllm/model_executor/layers/fused_moe/experts/triton_cutlass_moe.py
#	vllm/model_executor/layers/fused_moe/experts/trtllm_bf16_moe.py
#	vllm/model_executor/layers/fused_moe/experts/trtllm_fp8_moe.py
#	vllm/model_executor/layers/fused_moe/experts/trtllm_mxfp4_moe.py
#	vllm/model_executor/layers/fused_moe/experts/trtllm_nvfp4_moe.py
#	vllm/model_executor/layers/fused_moe/experts/xpu_moe.py
#	vllm/model_executor/layers/fused_moe/fused_moe.py
#	vllm/model_executor/layers/fused_moe/fused_moe_method_base.py
#	vllm/model_executor/layers/fused_moe/fused_moe_modular_method.py
#	vllm/model_executor/layers/fused_moe/layer.py
#	vllm/model_executor/layers/fused_moe/modular_kernel.py
#	vllm/model_executor/layers/fused_moe/oracle/fp8.py
#	vllm/model_executor/layers/fused_moe/oracle/int8.py
#	vllm/model_executor/layers/fused_moe/oracle/int_wna16.py
#	vllm/model_executor/layers/fused_moe/oracle/mxfp4.py
#	vllm/model_executor/layers/fused_moe/oracle/mxfp8.py
#	vllm/model_executor/layers/fused_moe/oracle/nvfp4.py
#	vllm/model_executor/layers/fused_moe/oracle/unquantized.py
#	vllm/model_executor/layers/fused_moe/prepare_finalize/flashinfer_nvlink_one_sided.py
#	vllm/model_executor/layers/fused_moe/prepare_finalize/flashinfer_nvlink_two_sided.py
#	vllm/model_executor/layers/fused_moe/prepare_finalize/naive_dp_ep.py
#	vllm/model_executor/layers/fused_moe/prepare_finalize/no_dp_ep.py
#	vllm/model_executor/layers/fused_moe/routed_experts_capturer.py
#	vllm/model_executor/layers/fused_moe/router/base_router.py
#	vllm/model_executor/layers/fused_moe/router/custom_routing_router.py
#	vllm/model_executor/layers/fused_moe/router/fused_moe_router.py
#	vllm/model_executor/layers/fused_moe/router/fused_topk_bias_router.py
#	vllm/model_executor/layers/fused_moe/router/gate_linear.py
#	vllm/model_executor/layers/fused_moe/router/router_factory.py
#	vllm/model_executor/layers/fused_moe/router/zero_expert_router.py
#	vllm/model_executor/layers/fused_moe/runner/moe_runner.py
#	vllm/model_executor/layers/fused_moe/runner/moe_runner_interface.py
#	vllm/model_executor/layers/fused_moe/runner/shared_experts.py
#	vllm/model_executor/layers/fused_moe/unquantized_fused_moe_method.py
#	vllm/model_executor/layers/fused_moe/utils.py
#	vllm/model_executor/layers/layernorm.py
#	vllm/model_executor/layers/linear.py
#	vllm/model_executor/layers/mamba/gdn/kimi_gdn_linear_attn.py
#	vllm/model_executor/layers/mamba/gdn_linear_attn.py
#	vllm/model_executor/layers/mamba/linear/minimax_linear_attn.py
#	vllm/model_executor/layers/mamba/mamba_utils.py
#	vllm/model_executor/layers/mamba/ops/mamba_ssm.py
#	vllm/model_executor/layers/mamba/ops/ssu_dispatch.py
#	vllm/model_executor/layers/mhc.py
#	vllm/model_executor/layers/pooler/seqwise/methods.py
#	vllm/model_executor/layers/pooler/seqwise/poolers.py
#	vllm/model_executor/layers/pooler/tokwise/methods.py
#	vllm/model_executor/layers/pooler/tokwise/poolers.py
#	vllm/model_executor/layers/quantization/__init__.py
#	vllm/model_executor/layers/quantization/auto_gptq.py
#	vllm/model_executor/layers/quantization/awq_marlin.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w4a4_mxfp4.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w4a4_nvfp4.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w4a8_fp8.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w4a8_int8.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w8a8_fp8.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w8a8_int8.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w8a8_mxfp8.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_wna16.py
#	vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_wna16_marlin.py
#	vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4.py
#	vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8.py
#	vllm/model_executor/layers/quantization/cpu_wna16.py
#	vllm/model_executor/layers/quantization/experts_int8.py
#	vllm/model_executor/layers/quantization/fp8.py
#	vllm/model_executor/layers/quantization/gguf.py
#	vllm/model_executor/layers/quantization/gptq.py
#	vllm/model_executor/layers/quantization/humming.py
#	vllm/model_executor/layers/quantization/inc.py
#	vllm/model_executor/layers/quantization/inc/__init__.py
#	vllm/model_executor/layers/quantization/modelopt.py
#	vllm/model_executor/layers/quantization/mxfp4.py
#	vllm/model_executor/layers/quantization/online/base.py
#	vllm/model_executor/layers/quantization/online/fp8.py
#	vllm/model_executor/layers/quantization/online/int8.py
#	vllm/model_executor/layers/quantization/online/moe_base.py
#	vllm/model_executor/layers/quantization/online/mxfp8.py
#	vllm/model_executor/layers/quantization/quark/quark.py
#	vllm/model_executor/layers/quantization/quark/quark_moe.py
#	vllm/model_executor/layers/quantization/turboquant/__init__.py
#	vllm/model_executor/layers/quantization/turboquant/config.py
#	vllm/model_executor/layers/quantization/utils/flashinfer_fp4_moe.py
#	vllm/model_executor/layers/quantization/utils/flashinfer_utils.py
#	vllm/model_executor/layers/quantization/utils/fp8_utils.py
#	vllm/model_executor/layers/quantization/utils/marlin_utils_fp8.py
#	vllm/model_executor/layers/quantization/utils/mxfp8_utils.py
#	vllm/model_executor/layers/quantization/utils/quant_utils.py
#	vllm/model_executor/layers/rotary_embedding/deepseek_scaling_rope.py
#	vllm/model_executor/layers/sparse_attn_indexer.py
#	vllm/model_executor/layers/steering.py
#	vllm/model_executor/layers/steering_kernel.py
#	vllm/model_executor/layers/utils.py
#	vllm/model_executor/model_loader/default_loader.py
#	vllm/model_executor/model_loader/gguf_loader.py
#	vllm/model_executor/model_loader/reload/layerwise.py
#	vllm/model_executor/model_loader/reload/meta.py
#	vllm/model_executor/model_loader/reload/utils.py
#	vllm/model_executor/model_loader/weight_utils.py
#	vllm/model_executor/models/AXK1.py
#	vllm/model_executor/models/afmoe.py
#	vllm/model_executor/models/apertus.py
#	vllm/model_executor/models/arcee.py
#	vllm/model_executor/models/arctic.py
#	vllm/model_executor/models/aria.py
#	vllm/model_executor/models/baichuan.py
#	vllm/model_executor/models/bailing_moe_linear.py
#	vllm/model_executor/models/commandr.py
#	vllm/model_executor/models/config.py
#	vllm/model_executor/models/dbrx.py
#	vllm/model_executor/models/deepseek_mtp.py
#	vllm/model_executor/models/deepseek_v2.py
#	vllm/model_executor/models/dots1.py
#	vllm/model_executor/models/ernie45_moe.py
#	vllm/model_executor/models/exaone.py
#	vllm/model_executor/models/exaone4.py
#	vllm/model_executor/models/exaone4_5.py
#	vllm/model_executor/models/exaone4_5_mtp.py
#	vllm/model_executor/models/exaone_moe.py
#	vllm/model_executor/models/falcon.py
#	vllm/model_executor/models/flex_olmo.py
#	vllm/model_executor/models/gemma.py
#	vllm/model_executor/models/gemma2.py
#	vllm/model_executor/models/gemma3.py
#	vllm/model_executor/models/gemma3n.py
#	vllm/model_executor/models/gemma4.py
#	vllm/model_executor/models/gemma4_mm.py
#	vllm/model_executor/models/glm4.py
#	vllm/model_executor/models/glm4_moe.py
#	vllm/model_executor/models/glm4_moe_lite.py
#	vllm/model_executor/models/glm4_moe_lite_mtp.py
#	vllm/model_executor/models/glm4_moe_mtp.py
#	vllm/model_executor/models/gpt_neox.py
#	vllm/model_executor/models/granite.py
#	vllm/model_executor/models/granite4_vision.py
#	vllm/model_executor/models/granitemoe.py
#	vllm/model_executor/models/granitemoeshared.py
#	vllm/model_executor/models/grok1.py
#	vllm/model_executor/models/hunyuan_v1.py
#	vllm/model_executor/models/hy_v3.py
#	vllm/model_executor/models/hy_v3_mtp.py
#	vllm/model_executor/models/hyperclovax.py
#	vllm/model_executor/models/interfaces.py
#	vllm/model_executor/models/internlm2.py
#	vllm/model_executor/models/internlm2_ve.py
#	vllm/model_executor/models/interns1_pro.py
#	vllm/model_executor/models/iquest_loopcoder.py
#	vllm/model_executor/models/jais2.py
#	vllm/model_executor/models/kimi_linear.py
#	vllm/model_executor/models/laguna.py
#	vllm/model_executor/models/llama.py
#	vllm/model_executor/models/llama4.py
#	vllm/model_executor/models/mimo_v2.py
#	vllm/model_executor/models/mimo_v2_mtp.py
#	vllm/model_executor/models/minicpm.py
#	vllm/model_executor/models/minimax_m2.py
#	vllm/model_executor/models/minimax_text_01.py
#	vllm/model_executor/models/mistral.py
#	vllm/model_executor/models/mistral_eagle.py
#	vllm/model_executor/models/mixtral.py
#	vllm/model_executor/models/molmo.py
#	vllm/model_executor/models/molmo2.py
#	vllm/model_executor/models/nemotron.py
#	vllm/model_executor/models/nemotron_nas.py
#	vllm/model_executor/models/olmo.py
#	vllm/model_executor/models/olmo2.py
#	vllm/model_executor/models/olmo_hybrid.py
#	vllm/model_executor/models/olmoe.py
#	vllm/model_executor/models/openpangu.py
#	vllm/model_executor/models/opt.py
#	vllm/model_executor/models/orion.py
#	vllm/model_executor/models/ouro.py
#	vllm/model_executor/models/param2moe.py
#	vllm/model_executor/models/persimmon.py
#	vllm/model_executor/models/phi.py
#	vllm/model_executor/models/phimoe.py
#	vllm/model_executor/models/plamo2.py
#	vllm/model_executor/models/plamo3.py
#	vllm/model_executor/models/qwen2.py
#	vllm/model_executor/models/qwen2_5_vl.py
#	vllm/model_executor/models/qwen2_moe.py
#	vllm/model_executor/models/qwen3.py
#	vllm/model_executor/models/qwen3_5.py
#	vllm/model_executor/models/qwen3_5_mtp.py
#	vllm/model_executor/models/qwen3_asr.py
#	vllm/model_executor/models/qwen3_dflash.py
#	vllm/model_executor/models/qwen3_moe.py
#	vllm/model_executor/models/qwen3_next.py
#	vllm/model_executor/models/qwen3_next_mtp.py
#	vllm/model_executor/models/qwen3_omni_moe_thinker.py
#	vllm/model_executor/models/qwen3_vl.py
#	vllm/model_executor/models/qwen3_vl_moe.py
#	vllm/model_executor/models/registry.py
#	vllm/model_executor/models/rnj1.py
#	vllm/model_executor/models/sarvam.py
#	vllm/model_executor/models/seed_oss.py
#	vllm/model_executor/models/solar.py
#	vllm/model_executor/models/stablelm.py
#	vllm/model_executor/models/starcoder2.py
#	vllm/model_executor/models/step1.py
#	vllm/model_executor/models/step3_text.py
#	vllm/model_executor/models/step3p5.py
#	vllm/model_executor/models/transformers/moe.py
#	vllm/model_executor/models/transformers/multimodal.py
#	vllm/model_executor/models/transformers/utils.py
#	vllm/model_executor/warmup/deep_gemm_warmup.py
#	vllm/multimodal/audio.py
#	vllm/multimodal/media/audio.py
#	vllm/multimodal/media/connector.py
#	vllm/multimodal/media/image.py
#	vllm/multimodal/video.py
#	vllm/parser/abstract_parser.py
#	vllm/parser/minimax_m2_parser.py
#	vllm/platforms/cpu.py
#	vllm/platforms/cuda.py
#	vllm/platforms/interface.py
#	vllm/platforms/rocm.py
#	vllm/reasoning/__init__.py
#	vllm/reasoning/abs_reasoning_parsers.py
#	vllm/reasoning/cohere_command_reasoning_parser.py
#	vllm/reasoning/qwen3_reasoning_parser.py
#	vllm/renderers/base.py
#	vllm/renderers/hf.py
#	vllm/renderers/registry.py
#	vllm/sampling_params.py
#	vllm/tokenizers/mistral.py
#	vllm/tokenizers/registry.py
#	vllm/tool_parsers/__init__.py
#	vllm/tool_parsers/abstract_tool_parser.py
#	vllm/tool_parsers/cohere_command_tool_parser.py
#	vllm/tool_parsers/deepseekv32_tool_parser.py
#	vllm/tool_parsers/deepseekv4_tool_parser.py
#	vllm/tool_parsers/gemma4_utils.py
#	vllm/tool_parsers/glm47_moe_tool_parser.py
#	vllm/tool_parsers/glm4_moe_tool_parser.py
#	vllm/tool_parsers/hy_v3_tool_parser.py
#	vllm/tool_parsers/llama_tool_parser.py
#	vllm/tool_parsers/minimax_m2_tool_parser.py
#	vllm/tool_parsers/mistral_tool_parser.py
#	vllm/tool_parsers/poolside_v1_tool_parser.py
#	vllm/tool_parsers/qwen3coder_tool_parser.py
#	vllm/tool_parsers/qwen3xml_tool_parser.py
#	vllm/tool_parsers/streaming.py
#	vllm/tool_parsers/utils.py
#	vllm/transformers_utils/config.py
#	vllm/transformers_utils/configs/__init__.py
#	vllm/transformers_utils/configs/speculators/algos.py
#	vllm/transformers_utils/gguf_utils.py
#	vllm/transformers_utils/model_arch_config_convertor.py
#	vllm/transformers_utils/processors/__init__.py
#	vllm/transformers_utils/processors/pixtral.py
#	vllm/transformers_utils/processors/voxtral.py
#	vllm/utils/__init__.py
#	vllm/utils/cpu_resource_utils.py
#	vllm/utils/cpu_triton_utils.py
#	vllm/utils/deep_gemm.py
#	vllm/utils/flashinfer.py
#	vllm/utils/import_utils.py
#	vllm/utils/mem_utils.py
#	vllm/utils/numa_utils.py
#	vllm/utils/ompmultiprocessing.py
#	vllm/utils/profiling.py
#	vllm/utils/torch_utils.py
#	vllm/v1/attention/backend.py
#	vllm/v1/attention/backends/cpu_attn.py
#	vllm/v1/attention/backends/fa_utils.py
#	vllm/v1/attention/backends/flash_attn.py
#	vllm/v1/attention/backends/flash_attn_diffkv.py
#	vllm/v1/attention/backends/flashinfer.py
#	vllm/v1/attention/backends/flex_attention.py
#	vllm/v1/attention/backends/gdn_attn.py
#	vllm/v1/attention/backends/mla/flashmla_sparse.py
#	vllm/v1/attention/backends/mla/indexer.py
#	vllm/v1/attention/backends/mla/prefill/__init__.py
#	vllm/v1/attention/backends/mla/prefill/base.py
#	vllm/v1/attention/backends/mla/prefill/flash_attn.py
#	vllm/v1/attention/backends/mla/prefill/flashinfer.py
#	vllm/v1/attention/backends/mla/prefill/registry.py
#	vllm/v1/attention/backends/mla/prefill/selector.py
#	vllm/v1/attention/backends/mla/prefill/trtllm_ragged.py
#	vllm/v1/attention/backends/mla/rocm_aiter_mla.py
#	vllm/v1/attention/backends/mla/rocm_aiter_mla_sparse.py
#	vllm/v1/attention/backends/mla/sparse_swa.py
#	vllm/v1/attention/backends/rocm_aiter_fa.py
#	vllm/v1/attention/backends/rocm_aiter_unified_attn.py
#	vllm/v1/attention/backends/rocm_attn.py
#	vllm/v1/attention/backends/tree_attn.py
#	vllm/v1/attention/backends/triton_attn.py
#	vllm/v1/attention/backends/turboquant_attn.py
#	vllm/v1/attention/backends/utils.py
#	vllm/v1/attention/ops/dcp_alltoall.py
#	vllm/v1/attention/ops/rocm_aiter_mla_sparse.py
#	vllm/v1/attention/ops/triton_attention_helpers.py
#	vllm/v1/attention/ops/triton_decode_attention.py
#	vllm/v1/attention/ops/triton_reshape_and_cache_flash.py
#	vllm/v1/attention/ops/triton_unified_attention.py
#	vllm/v1/attention/selector.py
#	vllm/v1/capture/__init__.py
#	vllm/v1/capture/config.py
#	vllm/v1/capture/consumer.py
#	vllm/v1/capture/consumers/filesystem/consumer.py
#	vllm/v1/capture/consumers/filesystem/types.py
#	vllm/v1/capture/consumers/filesystem/validation.py
#	vllm/v1/capture/consumers/filesystem/writer.py
#	vllm/v1/capture/consumers/logging.py
#	vllm/v1/capture/driver_bridge.py
#	vllm/v1/capture/manager.py
#	vllm/v1/capture/plan.py
#	vllm/v1/capture/registry.py
#	vllm/v1/capture/sink.py
#	vllm/v1/capture/types.py
#	vllm/v1/core/kv_cache_coordinator.py
#	vllm/v1/core/kv_cache_manager.py
#	vllm/v1/core/kv_cache_utils.py
#	vllm/v1/core/sched/output.py
#	vllm/v1/core/sched/scheduler.py
#	vllm/v1/core/single_type_kv_cache_manager.py
#	vllm/v1/engine/__init__.py
#	vllm/v1/engine/async_llm.py
#	vllm/v1/engine/core.py
#	vllm/v1/engine/core_client.py
#	vllm/v1/engine/input_processor.py
#	vllm/v1/engine/output_processor.py
#	vllm/v1/engine/utils.py
#	vllm/v1/executor/multiproc_executor.py
#	vllm/v1/executor/ray_executor.py
#	vllm/v1/executor/ray_executor_v2.py
#	vllm/v1/executor/ray_utils.py
#	vllm/v1/kv_cache_interface.py
#	vllm/v1/kv_offload/base.py
#	vllm/v1/kv_offload/cpu/common.py
#	vllm/v1/kv_offload/cpu/gpu_worker.py
#	vllm/v1/kv_offload/cpu/manager.py
#	vllm/v1/kv_offload/cpu/policies/arc.py
#	vllm/v1/kv_offload/cpu/policies/lru.py
#	vllm/v1/kv_offload/cpu/shared_offload_region.py
#	vllm/v1/kv_offload/cpu/spec.py
#	vllm/v1/kv_offload/reuse_manager.py
#	vllm/v1/outputs.py
#	vllm/v1/pool/metadata.py
#	vllm/v1/request.py
#	vllm/v1/sample/ops/topk_topp_sampler.py
#	vllm/v1/sample/rejection_sampler.py
#	vllm/v1/sample/sampler.py
#	vllm/v1/sample/thinking_budget_state.py
#	vllm/v1/simple_kv_offload/cuda_mem_ops.py
#	vllm/v1/simple_kv_offload/manager.py
#	vllm/v1/spec_decode/dflash.py
#	vllm/v1/spec_decode/extract_hidden_states.py
#	vllm/v1/spec_decode/llm_base_proposer.py
#	vllm/v1/worker/cpu_model_runner.py
#	vllm/v1/worker/cpu_worker.py
#	vllm/v1/worker/encoder_cudagraph.py
#	vllm/v1/worker/encoder_cudagraph_defs.py
#	vllm/v1/worker/gpu/attn_utils.py
#	vllm/v1/worker/gpu/block_table.py
#	vllm/v1/worker/gpu/cudagraph_utils.py
#	vllm/v1/worker/gpu/dp_utils.py
#	vllm/v1/worker/gpu/kv_connector.py
#	vllm/v1/worker/gpu/model_runner.py
#	vllm/v1/worker/gpu/model_states/default.py
#	vllm/v1/worker/gpu/sample/gumbel.py
#	vllm/v1/worker/gpu/sample/logprob.py
#	vllm/v1/worker/gpu/sample/prompt_logprob.py
#	vllm/v1/worker/gpu/sample/sampler.py
#	vllm/v1/worker/gpu/spec_decode/eagle/cudagraph.py
#	vllm/v1/worker/gpu/spec_decode/eagle/speculator.py
#	vllm/v1/worker/gpu/spec_decode/rejection_sampler.py
#	vllm/v1/worker/gpu/spec_decode/synthetic_rejection_sampler_utils.py
#	vllm/v1/worker/gpu/warmup.py
#	vllm/v1/worker/gpu_input_batch.py
#	vllm/v1/worker/gpu_model_runner.py
#	vllm/v1/worker/gpu_worker.py
#	vllm/v1/worker/steering_manager.py
#	vllm/v1/worker/steering_model_runner_mixin.py
#	vllm/v1/worker/ubatch_utils.py
#	vllm/v1/worker/worker_base.py
#	vllm/v1/worker/xpu_worker.py
#	vllm/vllm_flash_attn/flash_attn_interface.py
refactor(sae): migrate post_mlp hook to post_block with true block-output semantics
# Conflicts:
#	docs/features/sae_steering.md
# Conflicts:
#	docs/features/sae_steering.md
#	tests/v1/worker/test_sae_mixin_integration.py
feat(sae): close all SAE steering limitations — thresholds, multi-module, Rust, HTTP API
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant