Linear regression from scratch

Build the forward pass, mean squared error, backward pass, and parameter update by hand.

Complete path

import tensorplay as tp

X = tp.randn(100, 1)
y = 3 * X + 2 + tp.randn(100, 1) * 0.1
w = tp.randn(1, 1, requires_grad=True)
b = tp.zeros(1, requires_grad=True)

for step in range(100):
    y_pred = X @ w + b
    loss = ((y_pred - y) ** 2).mean()
    loss.backward()

    with tp.no_grad():
        w -= 0.01 * w.grad
        b -= 0.01 * b.grad
        w.grad.zero_()
        b.grad.zero_()

What this exposes

How requires_grad starts graph tracking.
How loss.backward() triggers reverse-mode differentiation.
How tp.no_grad() isolates parameter updates.
Why gradients must be cleared after each update.