vllm.models.minimax_m3.nvidia.specialized_decode.kernels.o_proj
¶
o_proj stage (TP4): MXFP8 o_proj GEMV + router CTAs, one CuTe DSL launch.
Grid = 16 router CTAs + (6144 / 128) x split GEMV CTAs (split 2: 112 CTAs),
fixed by the layer shapes, not by the SM count. The GEMV CTAs (2-CTA clusters,
tcgen05 block-scaled MMA, A = the o_proj weight, B = the decode's MXFP8 output)
push their bf16 rows into every rank's boundary exchange
(_exchange.push_workspace(transport, 8, "g3"), push mode 4) for the
allreduce_norm stage. The router CTAs compute this rank's share of the MoE
router logits from the folded weights (weights.py) and push them, tagged,
into every rank's RouterExchangeSlot.
Nothing in this grid waits on another CTA outside its own cluster, so it has
no co-residency precondition; dependents launch after each CTA's push. The
allreduce_norm stage (PostRouteStage) must run after every call of this
stage on the same layer (the rx tags pair one o_proj call with one post call).
Explicit buffers and transport; compiled at build (build), never on a live call;
rows are powers of two
(kernel assertion).
Modules:
-
host–MXFP8 x MXFP8 decode GEMV (tcgen05, block-scaled) for already-quantized activations
-
kernel–MXFP8 decode GEMV on tcgen05:
y[M, N] = x[M, K] @ W[N, K]^Tfor M <= 8, block-scaled E4M3 x E4M3. -
weights–Router-hoist weights of the TP4 o_proj router CTAs.
Classes:
-
OProjRouterStage–o_proj + router CTAs for one sparse layer on one rank.
OProjRouterStage
¶
o_proj + router CTAs for one sparse layer on one rank.
Build-time inputs (addresses are baked into the router table, so the
activation buffers are fixed for the stage's lifetime, as in a CUDA graph):
transport: peer transport (world 4).
rx: the layer's RouterExchangeSlot (shared with PostRouteStage).
o_weight: e4m3 [6144, 2048] (this rank's column slice of W_o).
o_weight_scale: uint8 UE8M0, 128x4-swizzled (FlashInfer F8_128x4, as
vLLM's MXFP8 linear stores it).
router_weight: [128, 6144] (fp32 or bf16), post_norm_weight: bf16 [6144].
residual_mid: bf16 [rows, 6144] (the front's residual output).
xq: e4m3 [rows, 2048], xsf: uint8 128x4-swizzled scales (>= 128 * 64 B),
the sparse_decode stage's MXFP8 output.
rows: 1, 2, 4 or 8.
blobs: optional precomputed (blob_a, blob_r) of weights.blobs
for these weights and this rank (shared across row counts).
Methods:
-
build–Compile outside capture (fail early, never on a live call) and check
Source code in vllm/models/minimax_m3/nvidia/specialized_decode/kernels/o_proj/__init__.py
54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 | |
build()
¶
Compile outside capture (fail early, never on a live call) and check the launch fits the device.