zongo maqutu.
← Back to the collection

Built and documented by Zongo Maqutu

LEARNING / 2021 / LEARNING STUDY

A small network. A different kind of logic.

Neural Networks, from Scratch

From a hand-written perceptron learning rule to noisy XOR inputs and a handwritten-digit classifier.

MEDIUM
  • Python
BUILT WITH
  • NumPy
  • PyTorch
  • Torchvision
ACCESS

Public source

Source
01

Start with a single neuron

The Perceptron class explicitly implements weighted sums, a threshold, and weight updates. The XOR experiment assembles a network of learned logic gates and introduces noisy inputs around binary values.

02

Then learn from images

A separate PyTorch classifier flattens 28 × 28 grayscale digits into 784 inputs, passes them through hidden layers of 128 and 64 units, and produces ten outputs. ReLU, log-softmax, negative log-likelihood, and SGD make the training pipeline explicit.

SYSTEM SKETCH / CONCEPTUAL OVERVIEW
28 × 28 IMAGE
      │ flatten
      ▼
    [784] → [128] → [64] → [10]
            ReLU    ReLU   log-softmax
A reading of the architecture, not an application screenshot.
SYSTEM MAP / PARALLEL TRAINING PIPELINES

Two experiments, with different learning machinery.

Noisy binary examples → Perceptron: train. Perceptron → Composed logic gates: compose. ImageFolder + DataLoader → 784 → 128 → 64 → 10: flatten 28 × 28. 784 → 128 → 64 → 10 → NLLLoss → backward → SGD: predicted log probabilities. NLLLoss → backward → SGD → 784 → 128 → 64 → 10: parameter update.1234501 / XOR.PYNoisy binary examplesAND / NOT training labelsInput pairs around 0 and 102 / CLASSIFIER.PYImageFolder + DataLoaderGrayscale → tensorBatch of 64 images03 / HAND-WRITTEN MODELPerceptronWeighted sum + thresholdw += η(target − output)x04 / PYTORCH MODEL784 → 128 → 64 → 10ReLU hidden layersLogSoftmax output05 / INFERENCEComposed logic gatesLearned AND / NOT gatesXOR output06 / TRAINING LOOPNLLLoss → backward → SGDUpdate network parametersargmax for digit prediction
  1. 01 / XOR.py

    Noisy binary examples

    AND / NOT training labels

    Input pairs around 0 and 1

    • train → 3. Perceptron
  2. 02 / Classifier.py

    ImageFolder + DataLoader

    Grayscale → tensor

    Batch of 64 images

    • flatten 28 × 28 → 4. 784 → 128 → 64 → 10
  3. 03 / hand-written model

    Perceptron

    Weighted sum + threshold

    w += η(target − output)x

    • compose → 5. Composed logic gates
  4. 04 / PyTorch model

    784 → 128 → 64 → 10

    ReLU hidden layers

    LogSoftmax output

    • predicted log probabilities → 6. NLLLoss → backward → SGD
  5. 05 / inference

    Composed logic gates

    Learned AND / NOT gates

    XOR output

    • 06 / training loop

      NLLLoss → backward → SGD

      Update network parameters

      argmax for digit prediction

      • parameter update → 4. 784 → 128 → 64 → 10
    1. 1Noisy binary examples Perceptrontrain
    2. 2Perceptron Composed logic gatescompose
    3. 3ImageFolder + DataLoader 784 → 128 → 64 → 10flatten 28 × 28
    4. 4784 → 128 → 64 → 10 NLLLoss → backward → SGDpredicted log probabilities
    5. 5NLLLoss → backward → SGD 784 → 128 → 64 → 10parameter update
    The perceptron update is implemented by hand. The separate digit classifier uses PyTorch autograd and SGD; it is not a hand-written backpropagation engine.
    Read from the implementation
    • Perceptron.py
    • XOR.py
    • Classifier.py
    03

    A bridge to the browser

    The README proposed moving learning logic into interactive JavaScript visualizations. The later 3D neural-network experiment makes that direction visible. No evaluation score is claimed here; the historical training path also depends on its dataset and older Torchvision APIs.

    Python / Perceptron.py · learning rule
    self.weights[index] += (
    learning_rate
    * (target_output - output)
    * x
)
    CONTINUE EXPLORING

    Reinforcement Learning →

    Grid-world experiments in value iteration and Q-learning, with rewards, obstacles, and animated state histories.

    2021
    • Python
    Read Field Note Source