vllm.models.minimax_m3.nvidia.specialized_decode.kernels.o_proj.host
¶
MXFP8 x MXFP8 decode GEMV (tcgen05, block-scaled) for already-quantized activations of at most 64 tokens (MAX_TOKENS).
Compiled on the first call per (N, K, split, ring, early trigger, PDL, push, MMA width),
which must happen outside
CUDA-graph capture, or ahead of any call by precompile; once freeze closes
compiles, a call whose kernel is not
compiled declines instead.
Functions:
-
check_cute_dsl–The kernel uses
cutlass.experimental.primitives(tcgen05 block-scaled MMA, -
compile_call–Compile (without launching) exactly the kernel `m3_mx_gemv(xq, xsf, weight,
-
ensure_compiled–Compiles the kernel of this weight's shape outside capture (a one-token dummy
-
freeze–Close (or reopen) compiles: while frozen, a call whose kernel is not compiled yet
-
m3_mx_gemv–xqE4M3 [M <= 64, K] withxsfits 128x4-swizzled UE8M0 scales;weight -
m3_mx_gemv_bf16–The same GEMV for a bf16
x[M <= 8, K] (contiguous), quantized to MXFP8 -
precompile–Compile, without launching, the kernel an M-row call against
weighttakes; -
serves–Whether the GEMV with its default config takes an [M, K] activation (bf16 or
_compile(key, args, m, runtime_ar, stream, extra)
¶
cute.compile the kernel of key (see _resolve) on these argument views;
cached.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
_declines(n, k, m, x_bf16, cfg)
¶
Whether a (split, ring, ...) config declines an [M, K] x [N, K]^T call on shape grounds.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
_extra_args(device)
¶
Placeholder arguments of the kernel's draft-input prologue and RoPE / KV epilogue (compiled off).
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
_kernel()
¶
The TP4 kernel module (imports the CuTe DSL; see check_cute_dsl).
_resolve(m, n, k, x_bf16, trigger_early, cfg, use_pdl, ar, ra_tab=0)
¶
The compile key of an [M, K] x [N, K]^T call and its runtime push words, or None when the call is declined.
ar is the push tuple of _run; only its (world, bucket tokens, mode) enter
the key, the addresses and the
rank are runtime arguments.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 | |
check_cute_dsl()
¶
The kernel uses cutlass.experimental.primitives (tcgen05 block-scaled MMA,
TMEM copies, griddepcontrol)
and cutlass.experimental.cuda tensor maps. vLLM pins nvidia-cutlass-dsl 4.8.0
(requirements/cuda.txt);
raise a clear error if this install does not expose them.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
compile_call(xq, xsf, weight, weight_scale, cfg, ar, ra_tab, out, use_pdl=True)
¶
Compile (without launching) exactly the kernel m3_mx_gemv(xq, xsf, weight,
weight_scale, cfg=cfg,
use_pdl=use_pdl, ar=ar, ra_tab=ra_tab, out=out) will run, so the first live call
never compiles: a live
call that compiles lazily while peers already poll its pushes can run past their
deadlines. Returns the key.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
ensure_compiled(weight, weight_scale, use_pdl)
¶
Compiles the kernel of this weight's shape outside capture (a one-token dummy call).
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
freeze(frozen=True)
¶
Close (or reopen) compiles: while frozen, a call whose kernel is not compiled yet
declines (returns None and
sets last_uncompiled) instead of compiling. precompile and
ensure_compiled still compile.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
m3_mx_gemv(xq, xsf, weight, weight_scale, trigger_early=True, cfg=None, use_pdl=None, ar=None, ra_tab=0, out=None)
¶
xq E4M3 [M <= 64, K] with xsf its 128x4-swizzled UE8M0 scales; weight
E4M3 [N, K] with
weight_scale swizzled. Returns bf16 [M, N], or None when the shape is not
served.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
m3_mx_gemv_bf16(x, weight, weight_scale, trigger_early=True, cfg=None, use_pdl=None, ar=None)
¶
The same GEMV for a bf16 x [M <= 8, K] (contiguous), quantized to MXFP8
inside the kernel exactly as
torch.ops.trtllm.mxfp8_quantize(x, True) does.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
precompile(weight, weight_scale, m, x_bf16, use_pdl=None, ar=None)
¶
Compile, without launching, the kernel an M-row call against weight takes;
returns its key, or None when
such a call is declined. ar = (world, bucket tokens, mode) of a pushing call
(its addresses and rank are
runtime arguments, so none is needed here). Runs outside capture, even while
compiles are frozen.
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/host.py
serves(n, k, m, x_bf16)
¶
Whether the GEMV with its default config takes an [M, K] activation (bf16 or MXFP8) against an [N, K] weight.