نماذج الاستدلال / Reasoning models

Lesson goals

1. Understand «thinking before answering».
2. Know how they are trained with verifiable rewards.

Reasoning models write out steps before answering, and spend longer on hard problems. They are trained with reinforcement learning on problems whose answers can be checked automatically — maths and code — and are rewarded when right. A common method samples several answers to one question and compares each with the group average.

import numpy as np

def reward(answer: str, correct: str) -> float:
    return 1.0 if answer.strip() == correct else 0.0   # verifiable: no human needed

answers = ["42", "41", "42", "40"]                  # 4 tries at the same question
r = np.array([reward(a, "42") for a in answers])
advantage = (r - r.mean()) / (r.std() + 1e-6)       # better than the group -> pushed up
print(r, advantage.round(2))                        # [1. 0. 1. 0.] [ 1. -1.  1. -1.]

Exercise

Why are reasoning models trained especially on maths and code?