vllm.v1.worker.gpu.spec_decode.fly
¶
Entropy-gated deferred verification for FLy speculative decoding.
FLy defers a high-entropy rejection when the next W draft tokens would be
accepted natively. The rejection kernel applies these decisions.
Functions:
-
compute_fly_entropy–Normalized top-k entropy in
[0, 1].
compute_fly_entropy(values, entropy_top_k, *, from_logits=False)
¶
Normalized top-k entropy in [0, 1].
Takes the largest k target scores, renormalizes them into a
distribution over those k candidates only, then divides the Shannon
entropy by log(k) so that fly_entropy_threshold keeps the same
meaning for every fly_entropy_top_k.