Instructions to use tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft")
model = AutoModelForCausalLM.from_pretrained("tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft

SGLang

How to use tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft with Docker Model Runner:
```
docker model run hf.co/tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft
```

Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft (bf16 weights)

Files: bf16 safetensors + tokenizer + chat template. Load with plain AutoModelForCausalLM; saved with transformers 5.x (dtype key, chat_template.jinja). Stage-3c model of a Harvard CS 2881R project (persona: Dr. Sheldon Cooper): a full-parameter RLVR continuation of tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v1 (Qwen2.5-3B-Instruct -> persona SFT -> RLAIF -> LoRA RLVR v1 -> this).

Why this model exists: after the v1 LoRA run plateaued (MATH-500 levels 3-5 at ~55 while about half the prompt groups had become all-correct), two LoRA continuations with a re-labelled, pool-expanded difficulty schedule stayed flat, so the last attempt on this seed trained all parameters instead: AdamW lr 2e-6 (betas 0.9/0.95, eps 1e-15, wd 0.01), fp32 master weights, bf16 autocast, fp32 LM head, WSD (warm-up 5, constant, linear decay over the last 20 of 100 steps to 0.1x), 100 steps x 16 prompts x 16 completions, T 1.0, 2,048-token cap, CISPO loss (eps_high 5), group-std advantages, prompt-level aggregation, truncated and zero-variance groups masked, DAPO soft overlong penalty, no KL. Generation by a vLLM server on one H200, training on two.

Prompts: a 64,686-problem pool (MATH-12k = MATH train + MATH-test minus MATH-500; DeepMath-103K difficulty 5-8; DAPO-Math-17k; DeepScaleR-Preview), deduplicated and filtered against MATH-500, AIME 2024/25/26 and GSM8K test (10-gram or edit-similarity >= 0.9). Each prompt was labelled with the seed model's pass rate over 8 rollouts; training drew prompts with weight p(1-p) inside [1/8, 7/8] plus a 10% slice of never-solved prompts, with per-source caps (MATH-12k 30%, DeepMath 30%, DAPO 20%, DeepScaleR 20%). Prompt format: Qwen system prompt, the problem, then Please reason step by step, and put your final answer within \boxed{}.

Evaluation (vLLM; MATH-500 avg@4 at T 0.6, 4,096 tokens; AIME avg@16 / pass@16; GSM8K test greedy):

model	MATH-500 greedy	avg@4	L3-5 avg@4	L4	L5	GSM8K	AIME 24 / 25 / 26 avg@16	AIME pass@16	mean tokens
Qwen2.5-3B-Instruct (base)	68.0	66.6	59.0	60.5	40.1	86.1	8.1 / 2.3 / 4.4	21.1	633
RLAIF grpo-v2 (RLVR seed)	34.0	29.6	21.0	21.7	10.6	63.6	0.4 / 0.4 / 0.0	3.3	357
RLVR-math-v1 (LoRA, this model's seed)	65.0	63.5	55.2	56.4	37.3	82.0	5.0 / 2.7 / 2.3	17.8	541
this model	65.4	65.1	56.9	60.9	35.3	83.0	5.2 / 1.9 / 3.8	16.7	633

L3-5 avg@4 by checkpoint: 25: 55.2, 50: 55.4, 75: 56.0, 100: 56.9 (monotone; largest step during the LR decay).

Persona (502 held-out prompts, greedy, 400 tokens; GPT-5.6 Luna pairwise judge, both orders, 199 prompts): win rate 0.495 [0.43, 0.56] against grpo-v2, i.e. persona preserved on substance (cast names 38%, Bazinga 6.6% vs 1.2%, canon errors 0.4% vs 2.2%, fewer template openers) at the cost of length: 44% of chat replies hit the 400-token cap (seed 18%), short prompts run 116 words vs 69, and the stage-2 rule penalty rises 0.87 -> 1.20, all from length and truncation terms. Persona leakage into MATH-500 answers 1.2%.

Honest summary: +1.6 (avg@4) / +1.7 (levels 3-5) over the LoRA seed, within ~1.5 points of the untouched base model on MATH-500 and level with it on level 4; still 5 points short on level 5 and 3 on GSM8K. The gain is at the edge of the eval's noise band but monotone over four checkpoints. Verbosity on non-math prompts is the known side effect.

Code, reward, grader and write-up: https://github.com/TBOO-Y/cs2881r-sheldon-sft (rlvr/, results/rlvr/).

Downloads last month: 276

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft

Base model

tbooy/Qwen2.5-3B-Instruct-Sheldon-RLAIF-grpo-v2

Adapter

tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v1

Finetuned

(1)

this model

Finetunes

1 model

tbooy
/

Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft

Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft (bf16 weights)

Model tree for tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft

Datasets used to train tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft