vllm-project vLLM Penalty Handler denial of service can crash the inference engine

A security flaw has been discovered in vllm-project vLLM up to 0.31.0. This impacts the function get_token_bin_counts_and_mask of the file vllm/model_executor/layers/utils.py of the component Penalty Handler. Performing a manipulation results in denial of service. Remote exploitation of the attack is possible. The exploit has been released to the public and may be used for attacks. The project was informed of the problem early through an issue report but has not responded yet.

• vllm-project vLLM: the CNA description identifies the affected scope through version 0.31.0. Updated CNA/GCVE affected data covers versions 0.1 through 0.31.0, including 0.21 through 0.31.0, which were absent from the initial normalized affected_products list.
• Red Hat downstream products: Red Hat VEX marks multiple vLLM components in Red Hat AI Inference Server, Red Hat Enterprise Linux AI 3 and Red Hat OpenShift AI as known affected. It separately marks some CPU, Neuron and torch components as known not affected within Red Hat's product scope. Those downstream statuses do not establish upstream vLLM version boundaries.
• No fixed vLLM release is confirmed in the available evidence. Development commits that address the issue should not be treated as released fixes.

vllm-project vLLM processes token-generation penalties in the Penalty Handler, with the reported failure point in get_token_bin_counts_and_mask in vllm/model_executor/layers/utils.py. The technical issue report states that requests using prompt_embeds do not populate the corresponding prompt positions in the CPU token-ID buffer, even though those positions are marked as not being token IDs. When sampling metadata is built, the penalty path can still use stale contents from that buffer and pass them into the scatter operation without honoring is_token_ids. With repetition, presence or frequency penalties and certain batch shapes, stale or out-of-range values can trigger a CUDA device-side assertion and stop the engine; even when no crash occurs, the penalty may be calculated from incorrect data. A proposed code change converts prompt_embeds positions to an invalid token-ID padding value when sampling metadata is built, but the available evidence does not confirm that the change has been released in an official vLLM version.

The flaw can be reached remotely through an inference endpoint when the server permits prompt_embeds. The public reproducer targets /v1/completions requests that combine prompt embeddings with repetition_penalty, presence_penalty or frequency_penalty; the described trigger includes a batch with different prompt lengths where the embedding request is the longest. The issue report states that one request burst can kill EngineCore, causing in-flight and later requests to fail until the process is restarted. CNA metadata describes remote access with low privileges, while the Red Hat VEX describes unauthenticated remote clients for some downstream deployments; the exact authentication requirement depends on deployment configuration. A public proof of concept is available, but the evidence does not identify a specific campaign, victim or breach.

The direct technical outcome is reduced or lost availability of the model-serving engine. A remote request can trigger a failure in penalty processing, resulting in a CUDA device-side assertion, EngineDeadError, HTTP 500 responses and a required process restart. The affected scope can include all requests using the same engine rather than only the malicious request. The record does not establish data reading, data modification or privilege escalation; the organisational consequence is inference disruption, failures in dependent workloads and recovery effort. Triggerability depends on a deployment that supports prompt embeddings, penalty parameters and a suitable batch shape.

The primary consequence is loss of inference availability:

• A crafted request can stop EngineCore or the serving process, interrupting other users that rely on the same instance.
• In-flight and subsequent requests may fail, and health checks may remain unsuccessful until the service is restarted or recovered.
• Operators may need to restart the workload, investigate GPU errors and handle interrupted inference jobs during recovery.
• The available evidence does not establish an impact on confidentiality or integrity, so this flaw should not be treated as evidence of data disclosure.

1. Inventory all vLLM instances, images or packages and compare them with the affected scope above. Separately identify deployments with --enable-prompt-embeds enabled.
2. Upgrade to an official fixed release published by vLLM or the relevant downstream vendor when one becomes available. No official fixed release is confirmed here, so do not infer that an unverified newer release is safe.
3. Until a confirmed fix is available, restrict network reachability to the inference service to authorized networks or clients. In OpenShift AI environments, use NetworkPolicy to allow ingress only from trusted gateways or namespaces.
4. Where operationally possible, temporarily limit or disable requests that combine prompt_embeds with sampling penalties. This is temporary exposure reduction, not a confirmed patch.
5. After a configuration change or fixed deployment, check the health endpoint, submit a valid inference request and confirm that the engine no longer restarts or fails after penalty-bearing batches.
6. If a crash occurs, isolate the endpoint from untrusted clients, preserve GPU and engine logs for investigation and restart the service through the normal recovery procedure.

1. Inventory all vLLM instances and determine the exact running version, with particular attention to the branches identified as affected by the CNA.
2. Review server configuration for --enable-prompt-embeds and identify inference endpoints that accept a prompt_embeds field.
3. Review request telemetry for combinations of prompt_embeds with repetition_penalty, presence_penalty or frequency_penalty, especially in batches with different prompt lengths. This is a documented exposure condition, not an independent IOC.
4. Search GPU and engine logs for strings such as scatter gather kernel index out of bounds, CUDA error: device-side assert triggered, EngineCore encountered a fatal error or EngineDeadError.
5. Correlate those events with HTTP 500 responses, failed health checks and ordinary requests failing after a penalty-bearing batch. The absence of these signs does not prove that a system is safe.
6. Add general monitoring for unexpected restarts, inference error spikes and health-check interruptions as a precaution. These signals are not confirmed indicators unique to this flaw.