Docs & benchmarks

Overview

This is a small GPT-style Transformer, implemented and trained entirely from hand-written NumPy — no PyTorch, no autograd library, no pretrained weights downloaded from anywhere. Every reply the chat page shows comes from sampling the model's own learned probability distribution over its vocabulary, given the conversation so far — there is no lookup table, no keyword matching, and no scripted responses anywhere in the pipeline.

The live numbers on this page's Benchmarks section below are reported figures from real training/benchmark runs; the chat page's connection panel also shows live figures (parameter count, training step, validation loss) straight from the running model via GET /api/health, so you can cross-check what's actually deployed against what's documented here.

Architecture

A byte-pair-encoding (BPE) tokenizer, learned from the training corpus itself, turns text into token IDs. Special tokens mark conversation structure (<system>, <user>, <assistant>, <eos>), so the model is trained on real multi-turn conversations, not isolated question/answer pairs.

From there it's a standard decoder-only Transformer: learned token + positional embeddings, several stacked blocks of causal multi-head self-attention and a feed-forward network (with residual connections and layer normalization around each), and a final linear layer producing a probability over the whole vocabulary for the next token. Every one of those operations has a hand-written backward pass (see model.py in the source) computing exact gradients — there is no automatic differentiation engine doing this for you.

Two size profiles exist: a small one sized to train in minutes on a phone (Termux/Android, ARM64), and a substantially larger one (roughly 13x more parameters) intended for training on GitHub Actions' CPU, where training time isn't as tight a constraint. Both are the exact same architecture, just scaled.

Dataset

Training data comes from a configurable list of sources, not a single hardcoded file — each source is normalized into a common conversational format, quality-filtered, deduplicated, weighted/mixed together, and split into train/validation sets with cross-set leakage removal (a conversation that ends up in both train and validation is dropped from validation, never train, so validation loss can't be quietly inflated by testing on something the model already trained on).

The bundled dataset is small and largely synthetic, written for this project rather than scraped or downloaded — that's reflected honestly in how the model performs: expect short, sometimes rough replies, not broad world knowledge. The pipeline is built to mix in larger, real datasets once configured; it doesn't fetch or bundle any large dataset on its own.

Benchmarks

Every hot-path operation (matrix multiplication, softmax, layer normalization) was actually implemented two ways — plain NumPy, and a hand-written C loop — and benchmarked at this model's real tensor shapes before deciding what to use where, rather than assuming C is always faster.

NumPy is backed by a real BLAS library (cache blocking, SIMD, sometimes multi-threading) that a hand-written C loop simply can't beat for matrix multiplication — so matmul, attention, the feed-forward layers, and the output layer all stay in NumPy. Layer normalization is different: NumPy's version does the computation as several separate array operations, each allocating a temporary array; a fused C loop does it in one pass with no temporaries — a real, structural win independent of BLAS. That native path is entirely optional: it auto-builds on first use if a C compiler is available, and transparently falls back to pure NumPy — correctness and results are identical either way, verified with a gradient check.

The larger, GitHub-Actions-scale profile was actually built and run (not just estimated) to get real numbers: on a constrained single-core development sandbox, a full training step (forward + backward + optimizer update) measured on the order of several seconds — reported honestly rather than smoothed over, and the reason that profile's default training run is modest per invocation and designed to resume across multiple runs instead of trying to fit everything into one.

Honest limitations

• Millions, not billions, of parameters — nowhere near a large pretrained model's scale.
• Trained on a small, largely synthetic dataset unless configured otherwise — limited real-world knowledge.
• Fixed context window — long conversations have their oldest turns dropped to keep fitting.
• No retrieval, no tool use, no persistent memory beyond what's baked into the model's trained weights.
• CPU-only, always — no GPU acceleration is available or assumed anywhere in this pipeline.