DEV Community: Jem Herbert-Rice

I Built an Interactive RL Playground to Help Me Learn Sutton & Barto

Jem Herbert-Rice — Thu, 16 Apr 2026 06:47:19 +0000

It's been a while since I've been able to get into this again - sometimes life just throws things your way. However, I am happy that the new chunk of my journey into machine learning has been so satisfying!

I've been working through Sutton & Barto's Reinforcement Learning: An Introduction and, if I'm honest, some of the maths took a while to click. Bellman equations, policy iteration, Beta posteriors — on the page they're coherent, but building a real intuition for why things behave the way they do is a different challenge entirely.

So I did what I always do when something isn't sticking: I got my hands dirty.

I built an interactive Streamlit app that implements the algorithms from scratch — no RL libraries, just NumPy — and lets you adjust parameters and watch what happens in real time. Nine pages covering bandits, dynamic programming, and Monte Carlo methods, with more to come as I work through the rest of the book.

Once it started helping me, I figured it might help someone else too. So I deployed it and put it online.

Try it here on Streamlit→

GitHub Repo this way →

I'll keep adding to it as I finish the book — there's already a Summary page that accumulates plain-English notes on each concept as I work through it, and TD methods, function approximation, and policy gradients are all on the list.

One thing I'll say: Streamlit makes it remarkably easy to turn an algorithm into something legible and explorable. If you need proof, here are a few screenshots — but honestly, you should just go and have a go yourself.

)

Built this in (delayed) Week 4 of my transition from operations manager to ML engineer. One concept at a time, one visualization at a time.

I Couldn't Quite Understand Transformers Until I Built One

Jem Herbert-Rice — Fri, 27 Feb 2026 04:25:06 +0000

The Problem

I read "Attention is All You Need" a couple of times and watched a few hours worth of Youtube (thanks 3Blue1Brown and Andrej Karpathy!) to try and wrap my head around multi-attention heads and transformers. However, it wasn't quite clicking.

So, I built a visualiser where I could watch it happen myself - and it turned out to be pretty useful! There's something about wading through all the error messages to get to a functional final product that is the most satisfying feeling in the world.

The Solution

Seeing how attention works in such a plain format was the 'A-Ha!' moment, and I thought it may be pretty help some others as well. So, I fleshed it out a little more, added some trained models and causal masking options, added some user friendly features and bundled it together in a Streamlit app.

So now, you can try my Transformer Attention Visualiser right here! Try it here →

What It Does

It let's you:

Build sentences (Mad Libs style or custom input)
Watch attention patterns form across 1-16 heads
Toggle trained vs random weights (see the difference training makes)
Enable causal masking (watch the matrix become triangular)
Read explanations tailored to what you're currently seeing

What's Next

I'm certainly not done learning, and I suspect that means I am not done creating hands on apps for all of the various topics I will encounter. If you're also a visual learner, or these help you in any way, let me know - especially if there are things missing or under-explained! I'd love to hear about it, and learn something in the process of getting it online.

If anybody knows how to get it to load faster on the first cold open, please reach out. I may be getting better at ML concepts, but this has stumped me for some time!

The GitHub Repo is below - this won't be the last you hear of me!

Code on GitHub →

Built this in Week 3 of my transition from operations manager to ML engineer. One concept at a time, one visualization at a time.

The Question That Changed How I Build ML Models

Jem Herbert-Rice — Tue, 17 Feb 2026 23:47:30 +0000

I was proud of my first machine learning model. Linear regression on a housing dataset, decent R² score, clean visualizations. I showed my mentor.

"What's the use case?" they asked.

I didn't have an answer.

The Problem With My First Model

I'd done what a lot of bootcamp grads do: grabbed a dataset from Kaggle, ran it through scikit-learn, got some metrics, called it done. The model worked. The code ran. The accuracy was... fine.

But I couldn't answer a simple question: Why does this exist?

"Well, it predicts house prices," I said.

"For who? When would someone actually use this?"

Silence.

I'd built a solution without understanding the problem. Classic mistake.

Starting Over With a Real Question

I scrapped the fabricated dataset and asked myself: What question am I actually trying to answer?

Here's what I landed on: I'm looking at properties in Sydney. I want to know what price range to expect for different suburbs and property types. I need something I can actually use when browsing real estate listings.

Not "build a machine learning model." That's the how, not the why.

The why: Give myself a realistic price estimate before I waste time on overpriced listings.

That's a real use case.

What Changed When I Had a Real Problem

1. The Dataset Mattered

Fabricated data → Real Sydney housing transactions (10,373 properties, 2016-2021)

I needed actual suburbs, actual price distributions, actual market segments. The model had to reflect reality, not just pass a training loop.

2. The Features Changed

Generic columns → Domain knowledge

I added:

Suburb clustering (5 market tiers from entry to luxury)
Price-per-sqm normalization (because a 50sqm apartment and 500sqm house shouldn't be in the same model unscaled)
Deviation from suburb median (is this property over/underpriced for its area?)
Inflation adjustment (2016 prices scaled to 2026 dollars)

These features came from asking: "What actually affects whether a property is expensive?"

3. The Accuracy Suddenly Mattered

R² = 0.62 → R² = 0.95

When it was a classroom exercise, 62% was fine. When I'm using it to evaluate actual properties, 62% means I'm off by $850k on a $1.5M house. That's useless.

I needed ±$146k error (8.7% MAPE). That's a range I can work with.

Getting there required:

Switching to XGBoost (non-linear relationships matter in real estate)
Proper feature engineering (not just throwing columns at the model)
Extensive hyperparameter tuning
Actually understanding why the model was making mistakes

4. Deployment Became Non-Negotiable

Jupyter notebook → Live Streamlit app

If I'm the only person who can use this, what's the point? I built a web interface where anyone can select a suburb, input property details, and get an instant estimate with an uncertainty range.

It's live. You can use it right now: house-price-predictor.streamlit.app

What I Actually Learned

Machine learning isn't about models. It's about problems.

The model is just the tool. If you don't know what problem you're solving, you're just doing math exercises.

Here's the framework I use now:

1. Start with the question

Who needs this answer?
When would they use it?
What decision does it inform?

2. Build for that question

What data actually answers this?
What features capture the real dynamics?
What accuracy is "good enough" for the use case?

3. Make it usable

Can someone else run this?
Does it work on real data, not just test sets?
Is the output actionable?

The Difference

Before: "I built a model that predicts house prices with 62% accuracy"

After: "I built a tool that tells me if a Sydney property is overpriced before I waste time inspecting it, with ±8.7% error on typical listings"

One is a portfolio project. The other is something I actually use.

What Changed for Me

I still build models. But now I start every project with: What question am I answering?

If I can't articulate that in one sentence, I don't write the code yet.

That one question from my mentor—"What's the use case?"—completely shifted how I approach machine learning. I'm not building models anymore. I'm solving problems that happen to need models.

Turns out that's what the job actually is.

The predictor is live if you want to try it: house-price-predictor.streamlit.app

Code on GitHub: github.com/JemHRice/house-price-predictor

Building in public as I transition from operations manager to ML engineer. Follow along on Dev.to or connect on LinkedIn.

From Non-Profit Ops Manager to Building Neural Networks: Week 1

Jem Herbert-Rice — Tue, 10 Feb 2026 05:16:14 +0000

From Non-Profit Ops Manager to Building Neural Networks: Week 1

Six months ago, I was managing operations for a basketball association. Scheduling, budgets, membership data, spreadsheets. Good work, meaningful work — but I kept looking sideways at what was happening in AI and feeling like I was watching the most important technological shift in history from the wrong side of the fence.

So I made a decision. I was going to get to the other side.

Where I Started

I'm not starting from zero. I completed a HyperionDev Data Science Bootcamp earlier this year, graduating first in my class. I can wrangle data, build basic ML models, and deploy them. But data analysis and actually working deep in AI are two very different things.

My goal isn't to become a data analyst who dabbles in machine learning. I want to be working at the frontier of AI development within the next 5-6 years. Building training environments. Working on agent systems. The kind of work that actually shapes where this technology goes.

That's the target. It's ambitious. It's probably going to take everything I have. I'm okay with that.

Week 1: What I Actually Built

I didn't spend Week 1 watching tutorials and feeling inspired. I built things.

House Price Predictor — a full end-to-end ML web app. Real dataset (545 housing records), data cleaning pipeline, model selection (tested Linear Regression vs Random Forest, documented why Linear won on this dataset), and deployed live on Streamlit Cloud. You can actually use it right now.

Sales Analytics Dashboard — A Streamlit-based sales analytics web app built with production-grade architecture. Real dataset (Superstore with 545 records), robust data validation pipeline (multi-delimiter/encoding support), 8 interactive visualization types, and dynamic filtering. 94.9% test coverage with unit, integration, and UI tests. Deployed live on Streamlit Cloud — upload your own CSV or explore the sample Superstore dataset in real-time.

Neural Network from Scratch — not using PyTorch, not using scikit-learn. Pure NumPy. I implemented a perceptron with sigmoid activation and gradient descent by hand, tested it on AND, OR, and NAND logic gates, and then threw it at a digit recognition task. Watching loss curves drop as the weights update is genuinely one of the most satisfying things I've experienced learning anything.

I also started Fast.ai's Practical Deep Learning course — which, by the way, is completely free and genuinely excellent — and got through Lesson 1 including training my first image classifier.

What Struck Me This Week

I knew AI was big. I didn't fully appreciate how big until I started pulling on the threads.

Deep learning alone is a rabbit hole with no visible bottom. Then there's reinforcement learning, multi-agent systems, distributed training infrastructure, interpretability research, alignment work — and these aren't shallow topics. Each one is a career's worth of depth. The field isn't just large. It's larger than any single field I've encountered, and it's expanding faster than anyone can fully track.

That's not intimidating to me. It's electric.

Six months ago I was building financial models in Excel. This week I implemented backpropagation by hand. The pace of what's possible when you commit fully to learning something is genuinely surprising, even to me.

The Honest Part

I'm at the tip of the iceberg. I know that.

I can build and deploy ML models, but I don't yet have the depth in reinforcement learning, distributed systems, or research methodology that serious AI work requires. I haven't published anything. I don't have a computer science degree. The people working at the frontier of this field are extraordinary, and I'm not there yet.

But I'm not trying to be there yet. I'm trying to be there in six years. And right now, Week 1 of that journey, I'm exactly where I need to be — building foundations, staying consistent, and moving forward every single day.

What's Next

Week 2 kicks off with finishing Fast.ai Part 1, going deeper on CNNs, and implementing a Transformer architecture from scratch. From there the plan moves into reinforcement learning fundamentals — the area I'm most excited about, and the direction I want to ultimately specialise in.

I'll be documenting the journey here as I go. The wins, the confusion, the errors I couldn't figure out for two hours, and the moments where something finally clicks.

If you want to follow along or check out what I've been building, my GitHub is here: github.com/JemHRice

The house price predictor, and the sales dashboard, are live if you want to poke around at it.

Once the fundamentals are locked in, I genuinely cannot wait to see what this skillset opens up. The projects that feel out of reach right now won't always be. That's what keeps me going.

Week 1 complete. See you next week.