vllm.models.minimax_m3.amd.glue.shared
¶
The MXFP8 shared expert of the glue paths, on AITER's preshuffled-weight GEMM.
Weights are derived once, after the checkpoint is loaded (raw [N, K] FP8 rows and per-row [N, K / 32] E8M0 scales, as the Quark OCP-MX loader stores them):
- decode: this rank's TP shard, (16, 16)-preshuffled (gate_up [2 I/TP, H], down [H, I/TP]); the output is a TP partial, like the stock MLP's.
- SP prefill: the full, replicated expert, all-gathered from the TP shards (gate_up = [all ranks' gate rows | all ranks' up rows], down = all ranks' columns), preshuffled; the output of a rank's own rows is complete.
- decode, fused (
fused): the TP shard for AITER's a16w4 MoE kernels' dense shared-expert prologue (routed + shared in the two MoE launches; no separate shared GEMMs). "mxfp8": the checkpoint's FP8 codes, (16, 16)-preshuffled, and their E8M0 scales in AITER'sshuffle_scalelayout; the kernels scale the codes to bf16 in registers. "bf16" (an AITER that takes only bf16 there): the exact bf16 upcast (e4m3 x 2^k fits bf16), (16, 16)-preshuffled, twice the bytes. Both give the same bits.
Functions:
-
aiter_fused_shared_format–The dense shared expert AITER's A16W4 MoE stages take: "mxfp8" (FP8 codes
-
build_shared_weights–Collective when
sp(every TP rank, same layer order).fused("mxfp8" or -
dequant_bf16–FP8 [N, K] x per-32 E8M0 [N, K / 32] -> bf16 [N, K], exact (a 3-bit-mantissa
-
fp8_rows–linear's FP8 weight [N, K] and per-row E8M0 scales [N, K / 32] (uint8). -
fused_shared_supported–The shared-weight format ("mxfp8" or "bf16") in which
mlpruns inside -
shared_decode–TP-partial shared expert of
xq/xsc(the glue's MXFP8 normed rows). -
shared_full–The complete shared expert of a rank's own rows (replicated weights).
_gather(t, dim, group)
¶
All-gather of a byte tensor over the TP group along dim.
aiter_fused_shared_format()
cached
¶
The dense shared expert AITER's A16W4 MoE stages take: "mxfp8" (FP8 codes
+ E8M0 scales: shared_w1 / shared_w1_scale, shared_w2 / shared_w2_scale),
"bf16" (shared_w1_bf16 / shared_w2_bf16, ROCm/aiter#7773 as first
posted), or None.
Source code in vllm/models/minimax_m3/amd/glue/shared.py
build_shared_weights(mlp, decode, sp, group, fused=None)
¶
Collective when sp (every TP rank, same layer order). fused ("mxfp8" or
"bf16", from fused_shared_supported): decode runs the shared expert inside
the MoE kernels, on weights in that format (and not the separate GEMMs').
Source code in vllm/models/minimax_m3/amd/glue/shared.py
dequant_bf16(w, s)
¶
FP8 [N, K] x per-32 E8M0 [N, K / 32] -> bf16 [N, K], exact (a 3-bit-mantissa value times a power of two is a bf16 value).
Source code in vllm/models/minimax_m3/amd/glue/shared.py
fp8_rows(linear)
¶
linear's FP8 weight [N, K] and per-row E8M0 scales [N, K / 32] (uint8).
Source code in vllm/models/minimax_m3/amd/glue/shared.py
fused_shared_supported(mlp)
¶
The shared-weight format ("mxfp8" or "bf16") in which mlp runs inside
the a16w4 MoE kernels: their SwiGLU is mlp's and the installed AITER takes
a dense shared expert there; else None.
Source code in vllm/models/minimax_m3/amd/glue/shared.py
shared_decode(sw, mlp, xq, xsc)
¶
TP-partial shared expert of xq / xsc (the glue's MXFP8 normed rows).
Source code in vllm/models/minimax_m3/amd/glue/shared.py
shared_full(sw, mlp, x_q, x_s)
¶
The complete shared expert of a rank's own rows (replicated weights).