AFTER HOURS / THE LAB

Ideas in
the making.

You found the side room. This is where I explore, test, and learn. Everything here is still in progress.

WHAT I’M EXPLORING

From text to predictions.

  • A reproducible pipeline for preparing the training corpus.
  • A custom byte-level byte-pair encoding tokenizer.
  • A hand-written decoder-only Transformer architecture.
  • Single-GPU training with mixed precision and gradient accumulation, targeting a 6GB VRAM budget.
NEXT ON THE BENCH

Train. Evaluate. Iterate.

Evaluate held-out perplexity and generated text samples, then document the results. The memory budget is a target, not a measured result.

There is no public repository or demo to share yet.

Return to the portfolio