vllm.models.mistral.mistral_large4.amd.specialized_decode.mono.common
¶
Mono decode: a model's decode step as one persistent kernel per layer, with the tensor-parallel reductions inside the kernel.
This package holds what every mono model shares -- the mechanisms: runtime
(host: TP consensus, peer memory, kernel argument tables, compile-only builds),
plan (plain Python: the execution model, build keys, the traced hand-off
contract) and device (traced inside kernels: the tagged mailbox, wave / math
primitives). The kernels, their layouts and schedules stay in the model's own
mono package. It imports no model and holds no model constant.
Modules:
-
device–Device-side mechanisms every mono kernel shares, one module each:
ops -
plan–Build-time contracts of a mono kernel, plain Python (no torch, no FlyDSL):
-
runtime–Host-side mechanisms of a mono runner, one module each (imported directly, so
-
weights–Host-side weight preparation every mono kernel needs: the storage