vllm.models.deepseek_v41.nvidia.specialized_decode
¶
DeepSeek-V4.1 specialized decode on NVIDIA GPUs: a decode executor over the
stock backbone, and one over the DSpark draft model's, enabled by
additional_config={"decode_backend": "latency_specialized"}.
The executors run FULL-graph decode steps; every other step runs the stock forward. They issue cross-rank collectives, so the deployment check below is collective over the TP group and every per-step decision reads host state that all TP ranks share.
Modules:
-
ampere–DeepSeek-V4.1 on Ampere datacenter GPUs (A100, SM80).
-
attention– -
boundary– -
embedding– -
engram– -
hopper–DeepSeek-V4.1 decode attention on Hopper datacenter GPUs (H100, H200, SM90).
-
logits– -
moe– -
runtime– -
sampling–
Functions:
-
decode_backend–additional_config["decode_backend"]:auto(the default, the stock -
records_decode_metadata–Whether SM80 or SM90 records the attention metadata builds of FULL
-
register_draft–Have the target model's state build
drafta decode executor. -
specialized_decode_config–The specialized decode settings, or None for the stock decode.
-
take_drafts–The live drafts registered since the last call, in registration
-
unsupported–Every requirement of the specialized decode this deployment misses.
-
uses_checked_flashinfer–Whether the installed FlashInfer is at least the version whose internals
decode_backend(vllm_config)
¶
additional_config["decode_backend"]: auto (the default, the stock
decode) or latency_specialized.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/__init__.py
records_decode_metadata(vllm_config)
¶
Whether SM80 or SM90 records the attention metadata builds of FULL
decode steps (runtime/metadata_graphs.py): with FULL decode CUDA
graphs, without micro-batching or context parallelism.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/__init__.py
register_draft(draft)
¶
Have the target model's state build draft a decode executor.
specialized_decode_config(vllm_config)
¶
The specialized decode settings, or None for the stock decode.
Collective over the TP group when requested: every rank raises if any rank cannot run the specialized decode.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/__init__.py
take_drafts()
¶
The live drafts registered since the last call, in registration order.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/__init__.py
unsupported(vllm_config)
¶
Every requirement of the specialized decode this deployment misses.
Source code in vllm/models/deepseek_v41/nvidia/specialized_decode/__init__.py
uses_checked_flashinfer(feature)
cached
¶
Whether the installed FlashInfer is at least the version whose internals
feature was checked against; logs once per feature when it is not.