Build, learn, and ship ML at UoG.
Onboard in Semester 1 by rebuilding GPT-2 (124M) with the NanoGPT playbook. In Semester 2 we ship a project. Hackathons run through the year.
In collaboration with the University of Gloucestershire. Access to 4× H200 GPUs for Society initiatives.
About
Our mission: provide a collaborative community to plan then produce large-scale projects, utilising our partnership with the University and access to compute. Experiments can be run on-site through this collaboration.
Onboarding — 3–4 months
Rebuild GPT-2 (124M) following Karpathy's walkthrough and learnings from NanoGPT, adapted for our cluster with 4x H200s.
What you'll actually do
- Model: GPT-2 124M; context length 1024 (see model card); tokenizer: byte-pair encoding — BPE overview.
- Objective: next-token prediction with cross-entropy loss.
- Data: train on a public FineWeb subset.
- Training: AdamW optimizer; cosine learning rate schedule with warmup; mixed precision (bfloat16); DDP for multi-GPU.
References: NanoGPT · Reproducing GPT-2 (124M)
Semester 2 Project
TBD, members will be involved heavily in this process.
Events
Hackathon-style sessions and talks run through the year.
Upcoming
To be announced. Watch Discord and this page.
Lab
Interactive lab support is in place. It will auto-detect device capability and offer a static explainer on low-power devices. For now, it’s switched off.
Device check pending…
Join
Membership is handled by the Students’ Union.
Outreach form (Welcome Fair & Induction Weeks): MS Forms link to be published here.