Now it is the turn of vLLM, the other clear contender for a simple library to run LLMs locally. The installation process is easy, everything looks easy after the experience with TensorRT-LLM.

I followed the instructions in the vLLM documentation, creating a virtual environment to avoid messing with existing Pytorch and CUDA libraries:

uv venv --python 3.12 --seed
source .venv/bin/activate

I compiled the library from the source in two different ways, just to see how long they would take. First, using precompiled libraries:

git clone https://github.com/vllm-project/vllm.git
cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install --editable .

What we are doing here is creating the Python installable package, but downloading the precompiled C++ libraries (maybe some C libraries too? Not sure.) It takes 1m33s on my computer.

And then, I compiled everything from the source:

git clone https://github.com/vllm-project/vllm.git
cd vllm
uv pip install -e .

This took 21m40s of intense CPU activity.

The test

Now it is time for some testing. vllm includes a bench tool in the code that I invoked this way:

vllm bench throughput
 --model /home/xavi/.cache/huggingface/hub/models--unsloth--Qwen3-8B-GGUF/snapshots/a6adef130ffb23ddaf1a62fec9dced968c9bc482/Qwen3-8B-Q8_0.gguf
 --dtype float16
 --input-len 512
 --output-len 256
 --max_model_len 16000

Apart from setting the model to a cached quantized version of Qwen4-8B (the same one used in the llama.cpp test), we define the input and output context lengths. I set the max_model_len parameter because it was in the ChatGPT suggestion, but I don’t think it is needed. What is needed is the --dtype float16 part. vLLM defaults to bfloat16, but it does not support it with GGUF quantizations, so we have to force a float type. We can learn here the differences between bfloat16 and float16.

The results for the text:

Throughput: 0.24 requests/s, 180.53 total tokens/s, 60.18 output tokens/s
Total num prompt tokens:  512000
Total num output tokens:  256000

The riddle

We finally test the same riddle as before:

A man is looking at a picture of someone. His friend asks who it is. The man replies, “Brothers and sisters, I have none. But that person’s father is my father’s son.” Who is in the picture?

Generated Outputs:

Prompt messages: [{‘role’: ‘system’, ‘content’: ‘You are a helpful assistant. Provide clear explanations, and avoid repeating yourself.’}, {‘role’: ‘user’, ‘content’: ‘\n A man is looking at a picture of someone. His friend asks who it is. The man replies, “Brothers and sisters, I have none. But that person's father is my father's son.” Who is in the picture?\n ‘}]

Generated text (13826 chars):

Okay, let's try to figure out this riddle. So, there's a man looking at a picture, and his friend asks who it is. The man says, "Brothers and sisters, I have none. But that person's father is my father's son." The question is, who is in the picture? First, let's break down the statements. The man says he has no brothers or sisters. That means he's an only child. So, his siblings are none. Then, he mentions that the person in the picture's father is his father's son. Wait, his father's son... but he himself is his father's son. So, if the person in the picture's father is his father's son, that might mean that the person's father is the man himself. Because the man is his own father's son. But wait, that would imply that the person in the picture is his son, right? Because the person's father would be the man. So, if the man has no brothers or sisters, then the person in the picture's father is the man. Alternatively, could there be another interpretation? Let's think. The man says "my father's son." Since he has no brothers or sisters, his father's son can only be himself. Therefore, the person in the picture's father is the man himself. So the person in the picture must be his son. Therefore, the answer would be the man's son. But wait, let me check again. The man says "that person's father is my father's son." So, the person's father is the man's father's son. But the man's father's son is the man himself, since he has no brothers. Therefore, the person's father is the man. So the person in the picture is the man's son. But is there another possibility? Wait, could it be that the person in the picture is the man himself? Let's see. If the person in the picture is the man's father, then the person's father would be the man's father's son. But the man's father's son is the man. So if the person in the picture is the man's father, then the person's father would be the man's father's son, which is the man. So that would make the person's father the man. So that would mean the person in the picture is the man's father. But the man said he has no brothers or sisters. So that's possible. Wait, but the person's father is the man. So the person is the man's son. So the answer would be the man's son. But maybe it's the man's father? Let me think again. Wait, if the person in the picture's father is the man's father's son. The man's father's son is the man himself because he has no other sons. Therefore, the person's father is the man. Therefore, the person in the picture is the man's son. So the answer is the man's son. But maybe there's another angle. Let me check again. Another way: If the person's father is the man's father's son. The man's father's son can be the man or someone else. But since the man has no brothers or sisters, the only son of his father is the man himself. Therefore, the person's father is the man. So the person in the picture is the man's son. So the answer is the man's son. But maybe the answer is the man's father? That doesn't fit. Wait, if the person in the picture is the man's father, then the person's father would be the man's father's father, which would not be the man. So that doesn't fit. Therefore, the person in the picture must be the man's son. Therefore, the answer is the man's son. So the person in the picture is his son. But wait, maybe the answer is himself? Let me check again. If the person in the picture is the man himself, then the person's father is the man's father. But the man says that the person's father is his father's son, which is the man. So that would mean the man's father is the man, which isn't possible. Therefore, that's not correct. So the correct answer is the man's son. Therefore, the person in the picture is the man's son.

The riddle hinges on interpreting the phrase “my father’s son” in the context of the man’s statement about having no siblings. Here’s the breakdown:

No Brothers or Sisters: The man explicitly states he has no siblings. This means he is the only child of his parents.
“That Person’s Father is My Father’s Son”:
- The man’s father’s son can only be himself (since he has no brothers or sisters).
- Therefore, “that person’s father” = “my father’s son” = the man himself.
- This implies the person in the picture is the man’s son.

Answer: The person in the picture is the man’s son.

Final Answer The person in the picture is the man’s son.

Answer: \boxed{son}

Step-by-Step Explanation:

The man has no brothers or sisters → He is an only child.
“My father’s son” must refer to the man himself (no other sons exist).
Therefore, “that person’s father” = the man → The person in the picture is the man’s son.

Answer: \boxed{son}