In plain language
One layer transforms the inputs; the next works with the transformed . Nonlinear allow these compositions to represent more complex patterns.
03 / Deep learning
Simple parts. Complex patterns.
Connect into layers, then train their . Watch a network bend its decision boundary around a nonlinear problem.
Watch a network learn
Run, pause and inspect each epoch0epoch
—train loss
—validation accuracy
Inspect a point. Color = predicted probability; shape = observed class.
The initial surface is untrained. Run the experiment to compute the full trajectory.
○ Class 0◇ Class 1 / train lossValidation loss
Ready to trainInspect the layers and the backward pass
Each hidden unit learns a different weighted combination, then applies tanh. Backpropagation multiplies the upstream derivative by 1 − tanh(z)².
Forward: x → h → p → lossBackward: ∂L/∂p → ∂L/∂h → ∂L/∂W
The optimizer subtracts learning rate × gradient from each trainable parameter.
The technical idea
This lab trains a 2 → 6 → 1 with tanh hidden and a output. It is a small neural-network demonstration; modern deep models have many more layers.
From data to proteins
A deep protein model can learn sequence representations instead of relying only on hand-designed descriptors. Architecture and training objectives determine what information those representations retain.
Look deeper: mathematics, methods & limitations
h = tanh(W₁x + b₁); p = σ(W₂h + b₂)
Forward pass: compute hidden and predictions. : compare predictions with the target . : propagate derivatives to all trainable . Update: θ ← θ − η∇θL.
Without nonlinear , a stack of affine layers is still an affine mapping. Depth alone does not make a nonlinear decision boundary. This experiment uses tanh to avoid that collapse.
The curves come from actual optimization in your browser. Training and validation use different synthetic examples. A low training does not establish biological usefulness or performance on unseen protein families.