vllm.model_executor.kernels.linear.mxfp8.rocm_native
¶
Native MXFP8 linear GEMM for AMD CDNA4 (gfx950) via Triton tl.dot_scaled.
Consumes the FP8 E4M3 weights + E8M0 block scales directly (no dequant-to-BF16);
activations are MXFP8-quantized per token (inside the GEMM for small M), or
arrive pre-quantized as a QuantizedActivation with row-major (not swizzled)
E8M0 scales. Uses the CDNA4 hardware microscaling matrix cores. dot_scaled
tiles K by 128; a weight whose K is not a multiple of that is dequantized to
BF16 once at load and served by a plain linear instead, since can_implement
does not filter on K.
Weights with 32x32 scale blocks (DeepSeek V4.1) run a block-scaled Triton GEMM, or AITER's tuned FlyDSL GEMMs on gfx950, on a weight preshuffled at load.
Classes:
-
RocmDotScaledMxfp8LinearKernel–Native CDNA4 (gfx950) MXFP8 linear via Triton
tl.dot_scaled.
RocmDotScaledMxfp8LinearKernel
¶
Bases: Mxfp8LinearKernel
Native CDNA4 (gfx950) MXFP8 linear via Triton tl.dot_scaled.
Methods:
-
supports_quantized_input–Whether
apply_weightsreads aQuantizedActivationfor
Source code in vllm/model_executor/kernels/linear/mxfp8/rocm_native.py
464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 | |
_flydsl_fallback(layer)
staticmethod
¶
Why a 32x32 block-scaled layer cannot run AITER's tuned FlyDSL GEMMs (batched wo_a as [G, N / G, K]), or None if it can.
Source code in vllm/model_executor/kernels/linear/mxfp8/rocm_native.py
supports_quantized_input(layer)
staticmethod
¶
Whether apply_weights reads a QuantizedActivation for
layer as is (fp8 weights), rather than dequantizing it back.
Source code in vllm/model_executor/kernels/linear/mxfp8/rocm_native.py
_as_block32_scale(weight_scale)
¶
The [N / 32, K / 32] block scales, if every 32 rows share their scales.
Source code in vllm/model_executor/kernels/linear/mxfp8/rocm_native.py
_launch(x, x_scale, w, w_scale, out, block_m, block_n, block_k, num_warps, num_stages)
¶
out = x @ w.T. x_scale None means x is the unquantized
activation and is MXFP8-quantized inside the kernel.
Source code in vllm/model_executor/kernels/linear/mxfp8/rocm_native.py
_quant_mxfp8_tile(x, FP8_MAX, TINY)
¶
MXFP8-quantize a [BLOCK_M, BLOCK_K] tile: bit-exact with
mxfp8_e4m3_quantize (see _mxfp8_quant_triton_kernel for why the
scale is applied as a reciprocal).
Source code in vllm/model_executor/kernels/linear/mxfp8/rocm_native.py
_select_cfg(M, N, K)
¶
(BLOCK_M, BLOCK_N, BLOCK_K, num_warps, num_stages) — graph-tuned on gfx950.
The M-bucketed, shape-adaptive tile selection here is the speedup over the upstream 2-bucket launcher. Tiles are pipelined (num_stages>=2, larger BLOCK_K) and occupancy- and shape-aware: keyed on the LOCAL (M, N, K), so it adapts to the TP-sharded shapes (e.g. MiniMax-M3 TP=4 vs TP=8, where local N and K differ) — large-K prefill uses BLOCK_K=256; short-K (K=768) widens N. BLOCK_K must divide K (the K-loop is unmasked), so every BLOCK_K below is guarded to be K-divisible (served K: 384/768/1024/2048/6144).
Source code in vllm/model_executor/kernels/linear/mxfp8/rocm_native.py
368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 | |
_select_fused_quant_cfg(M, N, K)
¶
Tile config with the activation quant fused (M <= 16), or None to quantize separately.
Source code in vllm/model_executor/kernels/linear/mxfp8/rocm_native.py
_wide_prefill_tile_pays(M, N)
¶
Whether the 256x256x128 large-prefill tile beats the 128x128 one.