asbedb.dev/ | Musings

Untangling Bureaucracy with RAG IM2026

Sat, 18 Jul 2026 00:00:00 GMT

It’s currently innovation month across the Australian Public Sector - as part of the festivities a challenge was set to “build a bureaucrat bot”.

I thought it apt to give the challenge a “fair shake of the sauce bottle” . It was a good excuse for me to dive into something new and untested, maybe learn a thing or two along the way. I wanted to summarise the journey - not necessarily in my usual deep dive technical style, but approachable enough to get an idea, with some relevant links to code.

If you want to poke at it yourself, the repo’s here with a live demo here and if you spot something I’ve done badly (very possible), I’d love to hear about it.

And a summary of what the bot does below (excuse my artistry.😊)

Challenge

Innovation month similar to a hackathon is a good way to platform and provide opportunity to showcase - dare I say it - innovation in delivering smarter outcomes in what I’m going to coin “good public service”. With the theme “Ready or not” the primary focus this year is around harnessing the capabilities of AI (in all forms) without losing sight of the goals aforementioned. The “Build a Bureaucrat Bot” event took this theme and provided a tangible category for all to participate.

“Can you build a helpful bot for a real public service problem? Use generative AI to create an interactive bot that helps public servants, senior leaders or citizens. No coding experience needed - no-code, low-code and prompt-based builds are welcome.”

In addition the bot could sit in primarily one of the below categories.

The Chief AI Officers Aide: Tools that support strategic decision-making, adoption planning, governance, maturity assessment, prioritisation, risk framing or executive briefings
The Workday Wingmate: Drafting, planning, summarising, navigating internal guidance, preparing meetings, improving correspondence, supporting teams, or making routine work less painful
The Cobber Bot: Citizen-facing or public-facing ideas that make services, obligations, forms, decisions, entitlements or processes easier to understand and engage with.
The Complexity Tamer: Tools for policy, research, modelling, simulation, evidence synthesis, system mapping, scenario testing or making trade-offs more visible
The AI Trailblazer: Move from curiosity to responsible practice: identifying opportunities, designing experiments, working through risks, building confidence, planning implementation, or scaling what works

Approach

The Complexity Tamer category instantly lit up for me. Untangling bureaucracy can be an art form, particularly when trying to administer and uphold laws.The immediate scenario which came to mind was navigating Law Administration Practice Statements (PS LAs).

A PS LA is an article that provides direction to Public Service workers on approaches to take when performing duties which involve the Superannuation and Tax laws being administered. Whilst the PS LAs themselves are not law or public ruling, they are used to help undertake technical direction in various matters. This “abstraction” layer above the law, certainly is in the category of taming complexity - but there were a few “bottle necks” in analysing a practice statement that I wanted to try and test.

The abstraction of law means that the language or legal ease is still going to be present in the directions provided - how can this become more accessible to someone less technical.
Drawing parallels between articles is currently difficult unless directly referenced in the statement itself - In some scenarios there may be multiple PS LAs covering various technical topics - “GST Rulings” can appear across three PS LAs with two of them withdrawn.
Searching and using the PS LAs as reference is currently relegated to a public facing website with the content being in HTML/Static Web - it works, but outside of reading the PS LAs themselves you would need additional uplift to use the data in other environments.

The Idea

My idea was to implement a retrieval augmented generation tool (RAG) to quickly sift through PS LAs based on a user’s prompt and provide a highly tailored response to the user. I’ll try and provide a simple step by step example of a RAG assisted result.

User prompts tool in simple language to try and gather information on making a technical decision.
Prior to our call going to our preferred generative model we ensure that our prompt is searched against a database of information housing all of our PS LA knowledge.
If a PS LA is found its crafted into a context payload.
The personality of our model is heavily guarded - it can only talk about PS LAs, it is given strict instructions to use the context provided and any request outside of PS LAs is rebuffed.
We stuff the context between our guard rails (prompt stuffing) and tell the model this is its primary subject.

And there you have it - a retrieved piece of information has augmented our prompt and resulted in a pointed generation!

Initially, I was imagining a Saturday and Sunday at best to build, test and deploy - leaving me plenty of time to write this piece. How wrong I was (apologies for any grammar or spelling mistakes in advance 🙃).

The Plan

The project is broken into two repositories - the ingestion engine and the presentation layer or “front end”.

For this project to show promise it was important to fine tune both - not only were they reliant on each other but also each in its own right drastically changed the user experience outcomes.

Ensuring good data stewardship is going to dictate the success of outcomes - and to some degree this holds true for any technology program in the not too distant future. I will go through and break out both the ingestion pipeline and front end into interesting challenges and learnings, however, if you want to review the full code its open source and available on GitHub here.

Below is an outline of the initial ingestion flowing into a full stack interactive web interface.

    Website with PS LA content (Public Facing Website)
                           │
                           ▼
             ┌───────────────────────────┐
             │ Ingestion Engine Pipeline │
             │  • Converts HTML to MD    │
             │  • Parses metadata        │
             │  • Sanitizes URLs/Dates   │
             └─────────────┬─────────────┘
                           │
                           ▼
       [ knowledge.json ] (DB Artifact)
                           │
                           ▼
             ┌───────────────────────────┐
             │     SvelteKit Backend     │
             │  (api/library/server.ts)  │
             │                           │
             │   ┌───────────────────┐   │
             │   │  • Context Search │   │
             │   │  • Filter Stops   │   │
             │   │  • Score Relevance│   │
             │   │  • Stuff Prompt   │   │
             │   └─────────┬─────────┘   │
             └─────────────┼─────────────┘
                           │
     [API Payload] (Stuffed Prompt to Model)
                           │
                           ▼
             ┌───────────────────────────┐
             │    SvelteKit Frontend     │
             │                           │
             │  ┌─────────────────────┐  │
             │  │  uiState.svelte.ts  │  │ (Global UI state)
             │  └──────────┬──────────┘  │
             │             │             │
             │  ┌──────────▼──────────┐  │
             │  │ • Display Chats     │  │
             │  │ • Search DB/Context │  │
             │  │ • Prompt            │  │
             │  │ • Detail & timeline │  │
             │  └─────────────────────┘  │
             └───────────────────────────┘

Ingestion

Ingestion has been broken down into a few parts - I want to thank whoever built and maintaned the website housing PS LAs, there are some key patterns I relied on throughout the ingestion phase and without them, I don’t think I would have finished this project in time.

Crawling

Each PS LA has a unique URL, lets take one as an example:

https://www.ato.gov.au/law/view/document?docid=PSR/PS19981/NAT/ATO/00001.

Removing the static parts of the URL we can review only this section:

PS{year}{sequence}/NAT/ATO/00001.

Rather then copy pasting each URL my first automation was to build a quick request resolver to go through a range of years to scan valid links within that year. If you are interested in the code you can review it here.

For each successfully resolved URL it was appended to a text document called verified_urls.txt in the /data folder.

A few things I learnt from this feature:

Not all websites expect a friendly header to let the web administrator know what your crawler is doing (I’m assuming some Cloudflare technology trend) just rejected my requests when I had something along the lines of Crawler-Innovation-Month in my header. Sorry admin.
Graceful delays and timeouts help reduce gaps in the data hammering a request isn’t the most efficient way to gather and validate evidence.
Reliable patterns make for good automation - and allow not only for an easier approach to administration but also for external parties to integrate and extend your product.

Scraping

I am of the firm belief that one of the top sought after skills in the next 5-10 years will be centred around the ability to write good documentation, in addition to being able to write it in markdown.

Generative models are good at this, yet when it comes to articles that are extremely nuanced or not in the public domain - it’s on us to build that layer to help get to outcomes faster.

Scraping a website isn’t ground breaking - however - translating a website from HTML markup to markdown is where it gets interesting. Thankfully the web administrators once again were gracious enough to clearly mark the PS LA webpage with easy to understand id’s tied to the various markup elements. It was on me to translate those elements into a markdown format - the python utility responsible for this can be reviewed here.

Doing this during ingestion made sense, we could have reserved it for the front end however that also complicates the prompt stuffing - LLM’s are a lot better at understanding markdown as context then they are HTML.

During this parsing stage I created a key value data structure to return which would be appended to a .json file. The return object looks like this:

    return {
        "doc_id": doc_id,
        "title": title,
        "category": category,
        "raw_text_content": markdown_content,
        "revision_history": revision_history,
    }

Learnings from this feature:

Stripping elements and cleaning the ingestion data was critical - we didn’t need images, contact details or scripts. Purely the content relevant to the task
By preserving the HTML in its markdown equivalent presenting this back in a front end was much easier to manage. i.e. a title denoted with # was easy to render as a <h1> using a markdown parser.
Breaking up the parsed data into metadata attributes allowed us to create faster searches across documents (and expand out capability to include other sources in the future).

Persistence/Storing

Finally should there be no data ingested, the application will create and store the results in two files, one which houses all the valid URLs, and one which will be the main source of truth artifact to our full stack web app (knowledge.json).

I built a persistence service to check if the files existed and only run the service if they didn’t. This was primarily to rapidly test but also not recrawl URLs if I had a fully valid list and only wanted to update the database with new tags/content.

The Application

With the ingestion engine reliably producing knowledge.json, the other half of the problem was giving that data somewhere to live that didn’t feel like reading a legal document with extra steps. I decided to go back to my tried and tested friend SvelteKit - if you are interested in my feelings about it you can read more here.

Similar to my reasonings for that project I wanted to have a quick and easy way to deliver both front and back end without splitting infrastructure and inflating cost.

The front end has two main responsibilities, I won’t go into the technical detail of the UI implementation as this post is more focussed on the “RAG” side of things however it provides the user with.

A way to communicate to the chatbot through a common chat interface
A way to browse all of the context available, and, prompt based on a PS LA as a starting point to a conversation.

Models

For the generative engine I did test both a local model using llama 3.1 4B and Google Gemini’s cloud model. I ended up choosing Gemini as it’s free and one less hosting problem to consider - that said the system is fully capable of slotting in and out models. Currently the hosted version is using gemini-3.1-flash-lite which I found perfect for a tech demo.

Context Search

Once knowledge.json existed, the job of the backend was fairly simple to state and much less simple to get right: take whatever a user typed, work out which PS LA (if any) it’s actually about, and only then let the model open its mouth. I’ll admit up front - there’s no embedding model or vector database sitting behind this thing. getContextByKeyword is doing keyword scoring, plain and simple.

Every document in the knowledge base gets a score based on a handful of signals:

Does the raw query appear verbatim in the text (+10).
Does it appear in the title or doc ID (+20).
Word-by-word matches against title, doc ID and body content at smaller weights.

There’s a small bit of nuance in there for our PS LA 1998/1 style references specifically - if a query word contains a slash and matches the doc ID (think someone pasting in PS 1998/1 or similar), it gets bumped harder than a generic word match, because that’s about as close to a direct citation as a user is going to give us.

Anything that doesn’t clear a minimum match ratio (at least a quarter of the query’s meaningful words showing up somewhere relevant) or a minimum score gets discarded, and I only ever pass the top three scoring matches through to the model.

Stop words get filtered out before any of this even starts, otherwise “what does the ATO say about GST” would spend most of its scoring budget matching “the” and “about” against every document in the library. We don’t want to score these words as they are just part of common lexicon and will appear everywhere.

The /api/chat is home to this functionality, specifically context.ts doing the scoring and searching.

Learnings from this feature:

Keyword scoring is unglamorous but it’s transparent - I can look at any result and explain in one sentence why it scored the way it did.
The slash-matching quirk exists purely because of how PS LAs are referenced in the wild (1998/1, 2011/27, etc.) - a generic RAG implementation wouldn’t have thought to weight that, but knowing your data’s shape earns you these small wins.
Capping at three matches was a deliberate guardrail on context size as much as anything else - Gemini doesn’t need six PS LAs shoved into a system prompt to answer a question about GST rulings, and neither does the person asking it.

Prompt Stuffing

The personality of the agent is fine tuned in this file. Here you can see its personality and instructions. I wrote a custom function which takes whatever came back from the context search and does one of two things.

If nothing matched, the model gets AGENT_INSTRUCTIONS plus a POLICY_NOT_FOUND_GAURDRAIL block which tells it, in no uncertain terms, that it doesn’t get to freelance an answer about tax law from its own training data.
If something did match, it gets AGENT_INSTRUCTIONS plus POLICY_FOUND_INSTRUCTION, followed by every matched document wrapped in its own clearly delimited block - source link, doc reference number, title, and the full markdown content, separated by a run of equals signs so the model has an unambiguous way of telling one PS LA from another when more than one comes back.

This is the “prompt stuffing” I mentioned earlier made real - the model’s personality and its subject matter are being assembled fresh on every single request, out of instructions I control and content that’s been vetted by the scoring step above.

Learnings from this feature:

Separating “found” and “not found” instructions turned out to matter more than I expected. Early on I was using one instruction set and telling the model to “use context if provided” - it would occasionally still have a go at answering from general knowledge when nothing matched. Splitting the guardrail language into its own explicit branch closed that gap.
Labelling each context block with a document reference number (#1, #2, #3) rather than just dumping raw text back to back gave the model something concrete to cite back to the user, which matters a lot when someone’s asking about a topic that spans a withdrawn PS LA and its replacement.

Model Calls

+server.ts is the thin layer that actually talks to Gemini - and deliberately thin. It pulls the API key, validates the incoming message array, pulls the last user message out as the query, and hands it to getContextByKeyword. From there it’s mostly plumbing: build the system instruction, reshape the conversation history into the user/model roles Gemini expects (SvelteKit and the frontend are happy calling it “assistant”, Gemini is not), and open a streaming connection with generateContentStream.

The response comes back chunk by chunk through a ReadableStream, which gets piped straight to the client as plain text rather than waiting for the full generation to finish - useful for anything constrained to an edge runtime, and it’s the difference between a user staring at a blank screen for a few seconds versus watching an answer arrive. One small addition I was glad I made early: an x-context-retrieved header on the response, set to whether any PS LA actually matched. It’s a cheap way for the frontend to know, before it’s even parsed a word of the streamed answer, whether it’s looking at a grounded response or a guardrail refusal.

Library Endpoint

Sitting alongside all of this is /api/library, a much simpler read-only endpoint that just flattens the entire knowledge base into an array - id, title, category, content, source URL and revision history per document. This is what lets the frontend browse the full PS LA library independently of any chat interaction, which turned out to matter once I started thinking about the “detail & timeline” view - being able to look at a PS LAs revision history directly, without needing to have asked the chatbot a question first.

Wrapping Up

I thought a “Saturday and Sunday” session would be all it took, here we are finishing this piece 2 days before the deadline, which is probably the most honest lesson from Innovation Month: taming complexity is itself complex.

If you want to poke at it yourself, the repo’s here with a live demo here and if you spot something I’ve done badly (very possible, see grammar disclaimer above), I’d love to hear about it.

CSS June Roundup

Tue, 02 Jun 2026 00:00:00 GMT

I’ve been lucky enough to present at and help organise the local CSS meetup. This article is companion to a talk I gave showcasing some community updates and fun features either experimental or widely available in CSS!

TLDR

If you just want to jump around to links see the high level recap below (best viewed with chrome to see all non baseline features). Please note the title link scrolls to the section and the external link will take you to the relevant page!

MelbCSS Community Themes - External Link - The MelbCSS website has had a fresh paint job and is now ready to accept community theme submissions! Head on over to the GitHub to find out more.
CSS 1D Chess - External Link - David Sojevic One of our awesome community members put together a puzzle game demo and blog post inspired by a Hacker News article in which the player solves a Chess puzzle on one dimension. The project showcases what’s possible with only CSS and HTML - It even has a 3D toggle!
corner-shape - External Link - A new property that specifies a box’s corners within the area specified by its border-radius property!
random() - External Link - Brace yourself random() is coming. At the moment you can only test this feature in Safari.
if() - External Link - The if() function is gaining traction, perhaps if() it’s ready soon conditional checkboxes are a thing of the past?
Lazy Loading - External Link - A HTML Attribute that helps us manage the load time of a site, but can we also make it look good!
Sprites - External Link - Using pixel art magic to have some fun!

MelbCSS Community Themes

During the last few MelbCSS meetups we talked about putting together a neat little project to bring everyone together - I’m happy to announce it’s now ready!

The aim is to provide a canvas being the MelbCSS Website as a playground for CSS creativity. We have tried to make it accessible to contribute and CSS focused, using GitHub as the platform to help manage submissions and deployment.

Our first official submission from Ronvoluted did not disappoint! Titled chaos the swirls of buttons and emojis certainly live up to Ron’s vision with some interesting SVG filter effects being used similar to ripples on a pond! - and of course it’s all CSS.

It’s been very exciting to see the pull requests coming in, as I was writing this piece up a new theme arrived! This one was from LakshmiChaitanya2008 titled Minecraft.

As I had to test some features, I did put together a Retro theme which was a homage to old terminals and operating systems.

Check out the contributing guide to get started! We are more than happy to help answer any questions in the discord as well.

CSS 1D Chess

Play Here

Perched on a single-line board this mini game challenges the player to “win at chess” with the constraint. It’s ironic because the developer who made this David Sojevic has won at CSS with the constraint of not using JavaScript! There are some neat in action techniques here including the classic checkbox hack .

Without spoiling too much - there’s also a 3D Easter egg purely written in CSS. I highly recommend reading the full blog post on how its done - lots of great little techniques.

corner-shape

If you have needed to style an element beyond a simple radius of a border you’ve probably reached up to grab the humble SVG. Whilst the SVG still sits in its rightful spot of extremely flexible but a pain to master, some new CSS flags might make our lives slightly easier in the not too distant future. Meet the corner-shape!

See the Pen corner-shape in action by dots (@dottys) on CodePen.

Unlike border-radius in which borders just curve the corner-shape property provides us with actual shapes to put into each corner! By manipulating the superellipse() value we are able to create some awesome corners in inverse or standard composition. The spec also has some keywords baked in allowing for you to quickly call values without guessing. As an example corner-shape: squircle;.

Below is a summary of the keywords available.

Keyword	Value
square	`superellipse(infinity)`
squircle	`superellipse(2)`
round	`superellipse(1)`
bevel	`superellipse(0)`
scoop	`superellipse(-1)`
notch	`superellipse(-infinity)`

random()

Computer scientists have been harping on about “true random numbers” for a while now. Whilst this isn’t necessarily a discussion about that I did want to add an interesting note for those interested. True randomness is observed through physical phenomena the famous “lava lamp wall” from Cloudflare is a great example of this.

With that out of the way, It looks like CSS may be getting its very own random() function! I don’t believe many would dare to use it for cryptographic/security purposes but what it can do is add some fun ways to make a front end feel alive!

Whether it’s cycling through colours in a range or animating edges to randomly oscillate, I think it will open a world of creativity without needing to rely on additional JavaScript!

If you would like to test this feature it is currently only supported on the Safari browser.

if()

The checkbox hack was a great way to bake some “logic” into the front end but what if() I told you (see what I did there), a new option is coming. The utility of an if statement in CSS will bring with it a host of functionality and standardisation previously obscured or reliant on frameworks/JavaScript.

See the Pen if() demo by dots (@dottys) on CodePen.

One thing I’m really excited about is the ability to use single variables to conditionally update styling. This is a big win as it allows for more performant and easier to manage code that dynamically manages themes.

Lazy Loading

With the loading attribute now baked into the HTML Spec there’s a fair amount of options available to make those skeletons pop! By combining the starting-style rule with a loading=“lazy” - there are some really neat ways to naturally transition in loading elements. Paired with an intersection observer things like infinite scroll and load time become less reliant on JavaScript!

The demo below gives a great example of what can be achieved (note - I am using JavaScript to randomly fetch 400 images to show the effect in real time!)

See the Pen Lazy Loading Demo by dots (@dottys) on CodePen.

Sprites

Earlier in the year I came across a beautifully written post from Josh w Comeau detailing the use of sprites and sprite sheets, it’s a great example of how web developers have always had neat little hacks and tricks web developers use to emulate functions we are familiar with. If you remember this UI.

Were getting old
You would know the pain of putting one frame after another to make an animation.

I decided to get out asperite to make this funky little animation as an example!

See the Pen Spritesheet Demo by dots (@dottys) on CodePen.

The linked article and external links in the TLDR have some examples of this but the premise is simple.

Get yourself a sprite sheet.
Use a cover div to effectively crop to your tile size, by setting the width and height to the sprite dimensions.
Create a stepped animation function sprite 1s steps(<number of frames>, jump-none) infinite;
Animate across or around your sprite sheet by manipulating either object-position or background-position’s from -> to.

I made my own Wordle

Tue, 21 Apr 2026 00:00:00 GMT

When my friends asked how easy it would be to make a version of Wordle based on characters, items and quests from the popular game RuneScape. My initial answer was “It should be easy!”, I didn’t expect the architecture decisions, algorithm optimisation, and insight into trade-offs it would provide. Whilst it was a shorter fun little project, there are a few pearls worth sharing in case you too want to make your own Wordle.

If you already know what RuneScape and Wordle is and you want to get straight into it - start here.

If you want to play the game you can check it out at Scapedle.com

This project is open to contributions and has been released fully open source under a GPL 2.0 license - you can find the repository and contributing guide here.

What is Wordle

Wordle is a guessing game in which you have to guess a five letter word in five guesses, each guess will provide you hints by displaying letters in three colours/shades.

Darker grey when they aren’t present in the word
Orange if they are present in the word but not in the correct spot
Green if they are present and in the correct spot

What is RuneScape

Released in 2001 RuneScape is a fantasy massively multiplayer online (MMO) game with a unique art style, point and click controls and a large (still growing) player base. When I began to look into the number of items in the game it was estimated to be close to 17,000 (that’s a lot of words!)

Scapedle

So now we mush Wordle and RuneScape’s characters, quests and items together and you have a word guessing game called Scapedle! There are a few minor differences to note.

Scapedle doesn’t have a five letter restriction, there are even multiple words to guess, due to this added difficulty ceiling there is an emoji hint to assist users.
Some words are unique to the world of RuneScape so having prior knowledge of the game is also essential.
Finally in addition to the emoji hints a user can “give up” one guess for an extra hint in the form of text.

The Architecture

At a high level a typical game of Wordle is comprised of the following functions and features.

Word of the day - a word is chosen after a nominated time period, these words seem to have a method around being chosen as they generally have a unique word number when being shared as solved.
Dictionary - when users guess words there is a dictionary lookup ensuring the word being guessed is in the English language.
Stateful - a game of Wordle can be played all day, a user can walk away and close the app/website and come back to play it.
Mobile Friendly - generally Wordle users are playing on a mobile device, it’s a fun little time passer and gets on average ~5-10 minutes screentime per day. The UI should be intuitive and touch friendly.

The architecture and design phase actually came to be the most surprising and fulfilling part of this project. I wanted to “think like the team behind Wordle”, a team that supports an estimated 10-14.5 million active players per day. This meant making choices that allowed for scale, flexibility, automation, integrity (anti-cheat) and accuracy. In hindsight I am extremely happy I approached it this way as choices were easier to make when you are constrained.

SvelteKit

After careful consideration and pre-empting the work required for processes like “generating a daily word” and “managing user and game state”. I settled on SvelteKit.

I wanted a framework that could fill the role of both client and server side logic, but also stay within the realms of reasonable performance for a simple single page application. My experience with frameworks like NextJS left me with an impression that choosing it would be “overkill” for this project. The routing and SSR features whilst great for larger, more complex applications, would bring with it a performance hit for a simple word guessing game.

Svelte was a great choice for this particular project and here’s why:

Global State management for Game and Player - I can just use runes to manage things across multiple components without having to worry about prop drilling
Simple HTML - I just wanted the UI to work Svelte has a similar feel to Astro where it leans into the HTML feeling like HTML!
Dynamic file-based routing - The routes of the API are defined by the file structures in codebase making it easier to manage functions.
On build server functions - Every rebuild of the static site required an automation of word processing to happen, Svelte type safe load functions provide a neat interface to manage this.

Theming and Design

I also needed to ensure theming and design was on brand to invoke that nostalgic RuneScape feeling, I found the OSRS Design System by milkbar designs as a great reference point helping keep colours, fonts and components consistent across my project.

Using tailwind 4 theme syntax, declarations means that the color property will be available across text, backgrounds and more. (see snippet below).

@theme {
    --font-osrs-small: 'RuneScape Small', ui-sans-serif;
    --font-osrs: 'RuneScape UF', ui-sans-serif;
    --color-osrs-yellow: #ffcf3f;
    --color-osrs-gold: #e6a519;
    --color-osrs-brown: #694d23;
    --color-osrs-dark-brown: #382d1a;
    --color-osrs-black: #0f0f0f;
    --color-osrs-green: #00ff00;
    --color-osrs-red: #ff0000;
    --color-osrs-blue: #0088ff;
    --color-osrs-chat: #00ffff;
    --color-osrs-background: #2e2c29;
    --color-osrs-border: #474745;
    --color-osrs-panel: #46433a;
}

Scaling

There are two ways to check a users input against the correct answer. We either ship the answer in the client and test against it or we host the answer on a server and use some sort of middleware to check if the encrypted guess matches the encrypted answer. Shipping it with the client comes at the cost of making it way easier to cheat, and hosting it with a server comes at the cost of added infrastructure, complexity and management.

I decided to keep the word of the day a client side feature and focussed on making it harder to cheat (It’s impossible to stop cheating client side).

This decision also means that theoretically the load of 14.5 million active users playing has significantly reduced as the majority of compute is client side. Sure bandwidth is a concern but the over the wire requirements are not as taxing as 14.5 million API requests to a back end.

Wordlist

Originally the wordlist was built with some help from those beautiful guessing machines we have come to love. The file houses an array of objects and within each object we have a itemName, emojiHint and wordHint. I am currently in the process of refining this list and pushing updates per skill.

server/wordList.ts

export const WORD_LIST = [
    // ============================================================
    // MINING
    // ============================================================
    { itemName: 'Clay', emojiHint: '🧱🏺⚒️', wordHint: 'ceramic' },
    { itemName: 'Rune Essence', emojiHint: '⚪⛏️🪨🪄', wordHint: 'Aubury' },
]

Nothing particularly groundbreaking here, just a simple structure to help house all of our words!

Routing

The fun part of choosing a word comes from utilising Svelte file based routing system to dynamically load words from this list on every re-build (more to come in automation).

Within the src folder which houses all of the logic that builds up the application, by nominating a routes folder you effectively tell Svelte to build pages based on folders and files.

As an example, If there was a folder called src/routes/lumbridge and within that there’s a file called +page.svelte. After building the website a user navigating to scapedle.com/lumbridge (not a real link), would be greeted with the content in that route. In addition to this a route can also have a +page.server.js or +page.server.ts this is telling Svelte router that the load function for this particularly page can only run on the server and not the client.

I am only shipping the daily word, hint and emojis to the client - I do not want the entire wordlist. The idea was to house the wordlist in a dedicated src/server (this route is a dedicated back end/build route) folder which tells Svelte to never include that in the client and access it through a server load function in +page.server.ts!

Daily Word

I needed a consistent way to seed the word, it needed to be random, within the constraints of the wordlist and seeded in a way that ties back to the date - the daily word number should correlate and be seeded with the date.

The wordlist itself has some sequential structure based on RuneScape’s skills so going in order would mean a user would experience a theme of words grouped together if we went through the list in order.

Unlike a Math.random() function in which every generated result would be unpredictable. I needed a helper that could take a seed and return consistent results. I came to realise these are called “Pseudo Random Number Generators” (PRNG).

const getSeededRandom = (seed: number): number => {
    let t = seed + 0x6d2b79f5
    t = Math.imul(t ^ (t >>> 15), t | 1)
    t ^= t + Math.imul(t ^ (t >>> 7), t | 61)
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296
}

The helper above is called a Mulberry32, the linked Stack Overflow article in the GitHub repository has many more examples of PRNGs to consider. I decided to go with Mulberry32 due to its speed since the PRNG was required once per build. With that I used the date of build as a seed to get consistent results!

export const load = () => {
    const now: Date = new Date()
    const formatter = new Intl.DateTimeFormat('en-AU', {
        timeZone: 'Australia/Sydney',
        year: 'numeric',
        month: 'numeric',
        day: 'numeric',
    })
    const parts = formatter.formatToParts(now)
    const dateMap = Object.fromEntries(parts.map((p) => [p.type, p.value]))
    const year = parseInt(dateMap.year)
    const month = parseInt(dateMap.month) - 1
    const day = parseInt(dateMap.day)
    const aestToday = new Date(year, month, day)
    const startOfYear = new Date(year, 0, 0)
    const dayOfYear = Math.floor(
        (aestToday.getTime() - startOfYear.getTime()) / 86400000
    )
    const seed: number = year * 1000 + dayOfYear
    const randomIndex: number = Math.floor(
        getSeededRandom(seed) * WORD_LIST.length
    )
    let wordCount: number = 0
    const dailyWord = WORD_LIST[randomIndex]
}

Notes:

All words are stripped of special characters like -, ', + and ( and converted to upper case to ensure any inconsistent names in the wordlist are able to be guessed fairly.
The return statement is cut off on purpose; you can find full code on GitHub below is for illustrative purposes only.
Australian time zone is hardcoded to AEST (Australia/Sydney) to ensure consistent daily word rotation aligned with my local schedule.

Anti-cheat

In game development, the more you serve via the client, the more you relinquish control over anti-cheat opening an application up to memory manipulation and cheating. I guess it’s not that different in web - looking at functions such as secure authentication and encryption, these rely on appropriate methods in middleware and back end.

So you decided to ship the answer for your game with the game client, what next?

Well we need to make it harder for Hackerman to find the answer! Not impossible, just harder.

I’ll be honest, beyond simple encoding, if someone is willing to make the effort to cheat and decode the word, that’s fine. Funnily enough for a long time Wordle also did the same thing and worked on an honour policy. These days I’m not too sure, essentially the simple things to do here is to ensure the returned objects key isn’t too obvious in suggesting the answer lies there and to wrap the word itself in a btoa() method which encodes the answer as a Base64 ASCII string from a Binary string. . This provides basic obfuscation but isn’t true encryption—anyone inspecting the code can decode it trivially.

Dictionary

The leetcode grind actually came in useful for this feature. I needed to create a dictionary of words which were derived from the initial wordlist, this included words which were specific to RuneScape, you wouldn’t find Guthix or Saradomin in the English lexicon.

I committed to client side which meant no fancy third party services or APIs deciding to instantiate a dynamic dictionary during build time which shipped with the client. The initial build is O(n) as we would iterate through every word in our wordlist, the lookups are O(1) on average as we are effectively building a HashMap that takes into account the length of a word and the word itself!

Here’s the dictionary code in action! I was also able to integrate a nice little UI feature that showed how many guessable words are available by tracking the word count when building the dictionary.

const dictionary = WORD_LIST.reduce(
    (acc, item) => {
        const individualWords = item.itemName
            .toUpperCase()
            .replace(/'/g, '')
            .split(/\s+/)
        individualWords.forEach((word) => {
            const cleanWord = word
            const len = cleanWord.length
            if (len > 0) {
                if (!acc[len]) {
                    acc[len] = []
                }
                if (!acc[len].includes(cleanWord)) {
                    acc[len].push(cleanWord)
                    wordCount++
                }
            }
        })
        return acc
    },
    {} as Record<number, string[]>
)

Svelte Runes

After the server pushed to the client the following items:

Encoded word of the day
Emoji hint
Word hint
Word number
Dictionary
Word count (to render on the UI number of words available to guess)

The functional logic of the game needed to be mapped. Inadvertently Svelte has this concept of Runes (what a coincidence!).

What are runes?

Runes are symbols that you use in .svelte and .svelte.js / .svelte.ts files to control the Svelte compiler. If you think of Svelte as a language, runes are part of the syntax — they are keywords.

Runes have a $ prefix and look like functions:

let message = $state('hello')

Source

Svelte provides a few of these runes for consideration - if you are familiar with reactive web UI libraries some of the terminology will be familiar.

$state - a rune which holds a variable data store which can be used to trigger re-renders, the UI reacts to this component as an example a state that holds a count could start at 0.
$derived- a rune which will conditionally update based on a $state rune within your component, as an example a component may always be $state + 2 so we can derive it accordingly.
$effect - A function which runs when a $state updates.
$props - Inputs to a component including it’s children

There’s a lot of additional nuance but at a high level these are the main runes to consider.

Global State

Disclaimer: The next section outlines a file placed in stores folder - this should not be confused with Svelte global stores prior to Svelte 5 - I only realised the naming convention may cause confusion after the fact and will refactor to remove doubts in future.

In a svelte project the lib folder within your src, is a unique path.

lib contains your library code (utilities and components), which can be imported via the $lib alias, or packaged up for distribution using svelte-package

Within this folder there are a few additional paths that are unique like server, outside of this you are free to manage and build out your structure in an easy to read and manageable format.

For the game state this looked a little something like this.

src/lib/stores/gameState.svelte.ts

export interface ExternalData {
    itemName: string
    emojiHint: string
    wordHint: string
    wordNumber: number
    dictionary: Record<number, string[]>
    wordCount: number
}

export interface GameStateType {
    guesses: string[]
    currentGuess: string
    status: 'playing' | 'won' | 'lost'
    hintUsed: boolean
    gameStarted: boolean
    invalidWord: boolean
}

let rawData = $state<ExternalData>({
    itemName: '',
    emojiHint: '',
    wordHint: '',
    wordNumber: 0,
    dictionary: {},
    wordCount: 0,
})

export const solution = {
    get value() {
        return rawData.itemName
    },
}

export const gameData = {
    get solution() {
        return rawData.itemName
    },
    get emojiHint() {
        return rawData.emojiHint
    },
    get wordHint() {
        return rawData.wordHint
    },
    get wordNumber() {
        return rawData.wordNumber
    },
    get dictionary() {
        return rawData.dictionary
    },
    get wordCount() {
        return rawData.wordCount
    },
}

export const gameState = $state<GameStateType>({
    guesses: [],
    currentGuess: '',
    status: 'playing',
    hintUsed: false,
    gameStarted: false,
    invalidWord: false,
})

export function initStore(data: ExternalData) {
    const isNewWord = data.wordNumber !== rawData.wordNumber
    if (isNewWord) {
        gameState.guesses = []
        gameState.status = 'playing'
        gameState.gameStarted = false
        gameState.hintUsed = false
    }
    rawData = data
}

The above in summary works on:

Setting up types for ExternalData and GameStateType
Instantiating a rawData object to manipulate.
Exporting functions to get our solution and gameData values
Exporting our gameState object
Exporting an initialisation function initStore that checks if a new word has been loaded prior to resetting the game state.

With the state loaded in our lib this allowed me to call the same consistent state no matter how deeply nested my folders were - this is crucial as I needed to track.

Guesses
Prompt usage
Letters in word/out of word already guessed
The word of the day, emojis and the dictionary

You can view the component code in more detail on the GitHub however here is a small example of how this came to be used in the project.

src/lib/components/HintButton.svelte

<script lang="ts">
  import { gameData, gameState } from '$lib/stores/gameState.svelte';
  import { BLANK_GUESS } from '$lib/constants';
  let { focusInput } = $props<{
    focusInput: () => void;
  }>();
  function useHint() {
    if (gameState.status !== 'playing' || gameState.hintUsed || !gameData.wordHint) return;
    gameState.guesses.push(BLANK_GUESS);
    gameState.hintUsed = true;
    focusInput();
  }
</script>
{#if gameData.wordHint}
  <div class="..styling">
    {#if !gameState.hintUsed}
      <button
        onclick={(e) => {
          e.stopPropagation();
          useHint();
        }}
        class="..styling">
        PROMPT HINT (-1 GUESS)
      </button>
    {:else}
      <div class="..styling">
        <p class="..styling">Hint Revealed</p>
        <p class="..styling">{gameData.wordHint}</p>
      </div>
    {/if}
  </div>
{/if}

In the script tag we import the gameData and gameState - note the path points to $lib, this means no drilling and mental math is required and the state is persistent across our shared components and page.

This also means pushing up changes across multiple components becomes much easier.

GitHub Pages + Daily Runner

The last piece to the puzzle was automating the publishing of the page with updates, whether it be a daily word or a new feature being merged. As the project is being managed through GitHub it made sense to stay within the ecosystem. The project is open source meaning other developers can contribute or fork the repository to build their own versions.

The way I solved this challenge was by hosting the main branch on GitHub Pages with a workflow file that automates a rebuild every 24 hours and on successful merge of pull requests to the main branch.

Managing the Game Loop

As the game state is now an accessible rune across all components, managing that state is the main task at hand with various functions. I am going to provide a quick fire insight into this with links to the full code for the curious.

Local Storage

Early on I elected to utilise local storage as a means to track progress on the word of the day, a simple yet effective browser API that lets us manage an otherwise overly complicated feature if we wanted a persistent user state via a database.

The local storage holds the bare minimum required for a user to come back and check in/make extra guesses until the game is up.

onMount(() => {
    const saved = localStorage.getItem(STORAGE_KEY)
    if (saved) {
        const data = JSON.parse(saved)
        if (data.wordNumber === gameData.wordNumber) {
            gameState.guesses = data.guesses || []
            gameState.hintUsed = data.hintUsed || false
            gameState.gameStarted = data.gameStarted || false
        } else {
            localStorage.removeItem(STORAGE_KEY)
        }
    }
})

$effect(() => {
    if (gameState.gameStarted || gameState.guesses.length > 0) {
        const stateToSave = {
            wordNumber: gameData.wordNumber,
            guesses: $state.snapshot(gameState.guesses),
            hintUsed: gameState.hintUsed,
            gameStarted: gameState.gameStarted,
        }
        localStorage.setItem(STORAGE_KEY, JSON.stringify(stateToSave))
    }
})

The cliff notes:

On initial mount/render of site the client checks against local storage to see if a new word is being guessed data.wordNumber === gameData.wordNumber - if it’s new removeItem is invoked and the $effect will instantiate a blank object for the new word.
An effect runs when making a guess, using a hint or starting a game. The primary purpose of this effect is to update guesses in local storage.
Svelte has an advanced feature called snapshots this allows us to de-proxy the state stored in local storage which can help remove issues around serialisation.

Starting the game

The gameboard is initialised and the user is greeted with a “Start Quest” button, this was a strategic choice as “input jacking” with a hidden input is generally frowned upon with mobile browsers.

Both Scapedle and Wordle provide the user an option to use normal onscreen keyboards (or physical on pc) to input guesses. I wanted the same feeling on mobile. I resorted to using a start button that tells the browser to focus on this hidden input and in turn brings up the onscreen keyboard!

This in turn also helps with initialising our local storage (used to track progress on the daily word) by putting the game in a playing state.

Making a Guess

Once the user makes a guess using the onscreen keyboard or the presented letter blocks that guess is checked against the word of the day. Each guess is updated in our gameState.guesses array and those entries are used on the keyboard component to provide user feedback on if the letter has been exhausted in the word (green), present but not in correct place (orange) or not in word at all (darker shade on keyboard).

There are a few techniques I used here that turned out to be really useful.

if (gameState.currentGuess.length === totalExpectedLength) {
  let isEntireGuessValid = true;
  let currentIndex: number = 0;
  for (const part of targetParts) {
    const cleanPart = part.replace(/[^A-Z]/g, '');
    const partLength = cleanPart.length;
    const guessChunk = gameState.currentGuess.slice(currentIndex, currentIndex + partLength);
    const validWordsForLen = gameData.dictionary[partLength] || [];
    if (!validWordsForLen.includes(guessChunk)) {
      isEntireGuessValid = false;
      triggerInvalidEffect();
      break;
    }
    currentIndex += partLength;
  }
  if (isEntireGuessValid) {
    gameState.guesses.push(gameState.currentGuess);
    gameState.currentGuess = '';
    gameState.invalidWord = false;
  }
}
} else if (gameState.currentGuess.length < totalExpectedLength && key.length === 1) {
  if (/[a-zA-Z]/.test(key)) {
    gameState.currentGuess += key.toUpperCase();
  }
}

The word of the day is broken into parts (split by spaces) this in turn is used when making a guess to check a single string against the dictionary because we can split the guess into parts - that’s how the valid words are identified (we even slide a window here!).
All special characters have been stripped to make this logic easier to follow.
The currentGuess is tracked and that’s how we determine if the user even has an option to “attempt to guess”. If the currentGuess doesn’t match the full word length then we don’t even attempt to look at the dictionary.
A special global constant is used if the user elects to use a “hint” that will remove one guess completely and hold it in memory as such (this comes handy when sharing results).
Finally once all guesses are exhausted or the correct word is guessed the game ends with a screen showing the word of the day and a share option.
a triggerInvalidEffect() function is dedicated to give the user a little flashup of red text letting them know the word they have guessed is not currently in the dictionary of words available to guess.

Winning or Losing

Winning or losing a game of Scapedle looks a little like this

Naturally the global state tracking our games progress is utilised to match our guesses to the word of the day and effectively return a win/lose/still playing result.

This is actually a fairly simple if statement being run as an $effect

$effect(() => {
    if (gameState.guesses.includes(targetAlpha)) {
        gameState.status = 'won'
    } else if (gameState.guesses.length >= MAX_GUESSES) {
        gameState.status = 'lost'
    } else {
        gameState.status = 'playing'
    }
})

by tracking our status - we can lock the gameboard, show modals or allow additional guesses making it very easy to manage the UI. The modal in the pictured screenshot renders conditionally when a user is no longer in a playing state, taking into account the win/loss position of the board. From there a share results option is presented - these are fun little emoji messages used to help spread the game and challenge friends!

Scapedles looks a little something like this.

Scapedle - An OSRS Guessing Game
Todays Word #111 - 3/5

🟨🟨⬛⬛🟨🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨⬛
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

Play at: https://scapedle.com/

With the code looking like this:

function generateEmojiGrid() {
    return gameState.guesses
        .map((guess) => {
            if (guess === BLANK_GUESS)
                return Array(targetAlpha.length).fill('⬛').join('')
            return guess
                .split('')
                .map((char, i) => {
                    if (char === targetAlpha[i]) return '🟩'
                    if (targetAlpha.includes(char)) return '🟨'
                    return '⬛'
                })
                .join('')
        })
        .join('\n')
}

Essentially a simple map of characters in each valid guess to represent its corresponding return in the original guess with a line break at the end of the word length.

Wrap it up

Building Scapedle taught me that “easy” projects often hide the most interesting challenges. What seemed like a simple Wordle clone turned into a whole lot more.

The best part? It’s all open source. Fork the repository, build your own variant, or just come play at Scapedle.com.

Whether you’re here for RuneScape nostalgia or insights, I hope you found something useful in this breakdown. Now go guess that word—you’ve got this.

sRGB to HSV - Science or Magic?

Sat, 17 Jan 2026 00:00:00 GMT

Update 21/04/26: You can find the accompanying talk to this topic by clicking this link

Recently, I was automating colour generation for a theme engine, which sparked a question

“Why do I instinctively lean towards a HSV Colour picker when trying to pick a colour on my computer?”

Investigating a little further into the “why” of how computers work, brings me immense joy. By the end of this piece, I hope to provide insight into what helped me understand colour space, colour transformations and what they can represent in digital form.

The Brief

From a programming standpoint, the problem I was looking to solve was fairly simple.

This is the high level summary:

Take seed colours (Primary, Secondary, Tertiary, Neutral and Semantic).
Ensure the colour set has an isDark flag to help deal with contrast and shades for a “Dark Mode” colour scheme.
Generate shades of each colour and label 100,300,500,700,900 with 100 being the lightest and 900 being the darkest (if you use tailwindcss this might already be familiar to you).
Compile our colours into a list of variables to be imported at the parent level of a CSS file (example `-primary-100: #222222;).

And my intuition immediately lead to:

Obtain our seed colour hex codes.
Convert to a HSV value.
Have a set of standard transforms based on the colour scheme flag (isDark so we can apply the right contrast ratios).
Apply transforms and reconvert to hex with results ready to append.

The reason for my intuition came from having played with HSV values (in a visual tool like a colour picker) in the past. I already knew they were the right choice for this exercise, I had just never looked into why.

Source: Material Design: Tools for Picking Colour

But before I do that we need to cover some of the introductory content to this space. If you want to jump straight into the meat and bones feel free to skip ahead to The RGB Colour Model.

Colour is Sensed and Perceived

Colour is the translation of light sensed by the human eye and perceived by the brain.

Colour literature has roots dating as far back as 322 BCE (Aristotle’s - On Colours). Many of the concepts and standards we will come to discuss are tied to Isaac Newton’s extensive work on colour theory stating that white light is composed of a spectrum of colours, and colour is not intrinsic to objects, but rather arises from the way an object reflects or absorbs different wavelength.

Newton also discovered that sunlight is composed of 7 primary colours which are visible when passed through a prism: Red, Orange, Yellow, Green, Blue, Indigo and Violet. (fun fact: Newton added Indigo to ensure the 7 colours were aligned to musical notes Do, Re, Mi, Fa, Sol, La, Ti)

An average healthy human eye can see around 100 million different colours.

This is due to the three cone cell types (photoreceptors) found in the retina which can process ~100 different colour shades, amounting to ~one million combinations.

Fun Fact: The same biological factors can be observed in the animal kingdom, with the number of cone cells being varied. As an example some species of fish are considered to have double cones where as others might have four cones. The result of less or more cell types changes the way colours are perceived.

Source: American Academy of Ophthalmology

The human eye has light-detecting neurons in the retina (rods and cones) that convert light into electrical signals for vision. The rods handle dim light and black/white vision and the cones are responsible for colour and sharp daytime vision. The human eye can perceive more variations in warmer colours rather than cooler ones due to a higher mix of cones that process longer light wavelengths (reds). These cone cells are classed as L, M and S (Long, Medium and Short)

The estimated split of cone cells in a human eye are:

60% Red Sensing L (these cones technically peak in the yellow-green part of the spectrum)
30% Green Sensing M
10% Blue Sensing S

Variations in this make up means a different perception of colour with a deficiency being considered a root cause of colour blindness.

In summary the process looks like this:

Light hits photopigments in the photoreceptors outer segment
The absorption starts a biochemical chain reaction (transduction)
This changes the cell’s membrane potential and creates an electrical signal
The signal is passed along through cells to the optic nerve and then brain

With this some of the biology out of the way lets dive a little more into computers and colour. Because our perception is so tied to these biological quirks—like our heightened sensitivity to green and red—computer programmers needed a way to translate the ‘sensing’ into reproducible data. This is where standards come in.

Standards, Standards, Standards

In doing my research an article I discovered had three poignant quotes that are worth repeating (as I am a fledging in this space and can see the depths to which this topic can lead).

CIE colorimetry isn’t even half the story of colour science, it’s a tiny piece of the huge puzzle of human perception and engineering that is colour science. Don’t confuse understanding it with understanding colour perception.

Colorimetry is essentially linear algebra, human perception is much more complex. CIE colorimetry isn’t the whole story of colorimetry. It’s the modern standard but the theoretical work was done by James Maxwell in the 1850s.

Differentiating theoretical colorimetry from the CIE’s implementation of it will be a milestone in your understanding. I recommend : Jan Koenderink’s Color For the Sciences

Source: A Beginner’s Guide to (CIE) Colorimetry

The CIE (Commission internationale de l’éclairage) an over 100 year old organisation is the peak body providing a forum, developing international uniform standards and cooperating with interwoven fields related to the following categories (within the context of light and colour):

Light
Lighting
Photobiology
Vision and Colour
Image Technology
Measurement

Specifically around Image Technology they focus on bridging the gap between how we can make printed and display images match what we see. This includes methods, measurements and quantifiable ways of processing analogue or digital reproductions of colour.

CIE 1931 Colour system

The CIE 1931 Chromaticity Diagram (below) “represents” the full gamut of human colour perception. Funnily enough depending on how the chart is presented you are also going to be limited by the colour space that it was rendered in (more on this shortly). It is a mathematical model and is a “static idealisation” i.e. what is theorised to be the colour vision of a normal human. The diagram has a few elements to consider:

The outer curved boundary is the spectral (or monochromatic) locus - it represents the physical measurement of light oscillation represented in wavelengths.
The bottom of the chart omits this outer boundary (The purple line) These colours don’t exist as a single wavelength in nature; they are only “sensations” created by our brain when it receives both short (blue) and long (red) wavelengths simultaneously.
The diagram adopts colour names from the Munsell colour system.
The solid curved line with dots through the middle (Planckian locus) is the path an incandescent black body would take in this colour space as its temperature changes. (Think the colour of heat).
Near the centre of the curve (denoted as either D65 or Point E) is usually known as the “white point” or a “defined white”.
And just above the x-axis those temperatures on the locus are represented in Kelvin.

Colour Spaces, what’s that about?

A colour space is a digital device translation tool or dictionary used to define how colours are coded and organized.

Below I’ve summarised a few colour spaces to help give an understanding on how different output devices may display colour.

Name	Primary Medium	Key Characteristic
NTSC 1954	Analogue Television	The first colour standard for broadcast; notable for using YIQ colour space to remain backwards-compatible with black-and-white television. Some great information here
sRGB	Web & Standard Monitors	The universal standard; smallest gamut but highest compatibility.
Adobe RGB	Print & Photography	Extended greens/cyans to match CMYK printing capabilities.
DCI-P3	Cinema & Modern Smartphones	High-saturation standard required for HDR content.
Rec.2020	UHD / 8K Television	A massive gamut representing the future of display technology.

The RGB Colour Model

The RGB Colour model is considered an additive colour model in which the combination of red, green and blue primary colours of light are added together to reproduce a broad array of colours. The RGB model is for sensing, representation and display of images in electronic systems. The model aligns to our perception of colours (Trichromacy).

Source: Figma’s article “What is RGB”

To form a colour we superimpose three light beams (red, green and blue) either by, emission from black, or reflection from white. Each beam is labelled a component of that colour. The reason the RGB space is considered additive is because if light beams of differing colour (frequency) are superposed in space, their light spectra adds up, resulting in total spectrum.

To map a RGB triplet the component values are treated as positive Cartesian coordinates with a double value clamped between 0 and 1. If you could visualize a cube holding the bounds of the colour space a point with an X (Red), Y (Blue) and Z (green) “coordinate” will represent a colour (more on this shortly).

With this geometric representation additional computations can be applied such as colour similarity by mapping the distance between two given RGB colours.

sRGB Colour Space (and Transfer Function)

On November 5, 1996 Michael Stokes (Hewlett-Packard), Matthew Anderson (Microsoft), Srinivasan Chandrasekar (Microsoft) and Ricardo Motta (Hewlett-Packard) submitted a joint proposal to the W3C titled A Standard Default Color Space for the Internet - sRGB.

Jointly written by HP and Microsoft the paper plays an instrumental role in shaping how we sense colour on computers today.

The tech industry was facing a challenge. Hardware, applications and output devices were all displaying colour information with different standards applied. High end users who needed precision had access to hardware and software which supported a wider gamut whilst lower end entry level devices would be limited in support for such colour spaces. If the problem wasn’t addressed there would be a considerable effort required to maintain the colour display ecosystem.

To address this, the paper put forward a standardised colour space (Standard RGB or sRGB) with the intention to:

Improve colour fidelity in the desktop environment
Create a uniform standard for software development
Confidently communicate colours with minimal overhead

Essentially sRGB is a communication ruleset, a “lingua franca” (common language), made possible by constant parameters adopted to create a common colour language.

To achieve this there are a couple of important factors to consider:

Colorimetric RGB

sRGB uses precise standardised definition of anchor colours, specifically the Rec.709 values for Red, Green Blue and White (D65). The reason REC.709 was chosen was because it matched the phosphors used in the consumer CRT monitor and Television set of the 1990’s ensuring backwards compatibility.

Gamma (Transfer Function)

The proposal provides a a bridged target of gamma suitable for most devices and breaks up the requirement into various subsets.

Viewing Gamma - Computed Gamma we want to obtain
- $\text{Viewing Gamma} = \text{Camera Gamma} * \text{Display Gamma}$
Camera Gamma - Derived characteristic from the image sensors standard transfer function.
CRT Gamma - The gamma of the physical CRT (display).
LUT Gamma - The gamma of the frame buffer lookup table
Display Gamma - Computed downstream “display system” gamma
- $\text{Display Gamma} = \text{CRT Gamma} * \text{LUT Gamma}$

The four specific numbers the proposal settles on

1.125 - Target Viewing Gamma (The desired boost for dim environments)
$\frac{1}{1.956}$ - Rec.709 Power Function (The mathematical behaviour of the 709 transfer function)
2.5 - Physical CRT Gamma (Baseline characteristic for CRT displays in the 90’s)
2.2 - The magic number or “New Standard” proposed as the ideal target for the sRGB Display Gamma

To simplify further:

1.125 is our goal
$\frac{1}{1.956}$ is a constraint of Rec.709
2.2 is used to bridge our gap

The solution results in the simple formula:

$1.1247 = 2.2 * \frac{1}{1.956}$ (This rounded would resolve to 1.125 our target viewing gamma)

The reasoning behind 1.125 is clearly laid out in the proposal and well beyond the scope of what we are discussing here, in simpler terms they understood the colour space would be used in a multitude of environments which still required images to look correct and not washed out.

Definition

The final piece I will touch on at a high level is definition. The key components that were defined as standard:

The Standard Observer: While based on the CIE 1931 Standard Observer, the sRGB proposal specifically optimised for the “average” human’s response to the phosphors of a CRT monitor. It effectively turned a biological variable into a mathematical constant.
Reference Viewing Environment: This defined the “room” the observer is in, assuming a dimly lit office and a specific D50 ambient white point. This is why the $1.125$ viewing gamma was necessary (to compensate for the “darker” office surroundings).
The sRGB Gamut: This is the “triangle” of colours within the CIE 1931 space. Any colour falling outside this triangle is considered “out of gamut” for sRGB and must be re-mapped.
Render Intent: The inclusion of render-intent was a forward-thinking attempt to handle those “out of gamut” colours. It instructed the browser on how to squeeze a wide-gamut image (like a professional photograph) into the smaller sRGB space without losing detail in the highlights or shadows.

Some Cube Code

To visualise the sRGB transformation think of a cube (see above) and now in code (below):

const points = []
for (let r = 0; r <= 1; r += 0.1) {
    for (let g = 0; g <= 1; g += 0.1) {
        for (let b = 0; b <= 1; b += 0.1) {
            points.push(r, g, b)
        }
    }
}

Mapping in increments of 0.1 an 11x11x11 cube which would equate to 1331 unique combinations. Whilst this works for a cube with a single factor of consideration, It is not taking into account the nature of our above pictured cube having corners starting at Red, Green, Blue, White, Magenta and Black.

True sRGB works in 8-bit space providing over 16.7 million unique combinations and takes into account the tracking of index in relation to each other.

$256×256×256=256^{3}=16,777,216$

(don’t run the below code it generates over 50 million entries)

const points = []
const steps = 255

for (let r_i = 0; r_i <= steps; r_i++) {
    const r = r_i / steps
    for (let g_i = 0; g_i <= steps; g_i++) {
        const g = g_i / steps
        for (let b_i = 0; b_i <= steps; b_i++) {
            const b = b_i / steps
            points.push(r, g, b)
        }
    }
}

The above “cube code” with the additional parameter in mind, return coordinates to the value of $256^{3}$ .

The axis of the cube is represented as such:

X-axis = Red (R)
Y-axis = Green (G)
Z-axis = Blue (B)

Charting the colour cube is as simple as plugging in our coordinates and setting sail - see some example coordinates below:

Coordinate (x,y,z)	Resulting Color	Description
$(0, 0, 0)$	Black	The Origin (No light)
$(1, 0, 0)$	Red	Max Red, no Green or Blue
$(0, 1, 0)$	Green	Max Green, no Red or Blue
$(0, 0, 1)$	Blue	Max Blue, no Red or Green
$(1, 1, 0)$	Yellow	Red + Green (Additive)
$(0, 1, 1)$	Cyan	Green + Blue (Additive)
$(1, 0, 1)$	Magenta	Red + Blue (Additive)
$(1, 1, 1)$	White (D65)	All primaries at max

An important distinction I needed to wrap my head around:

The mapping of the colour space (setting the coordinates) is linear, the issue is the adjustment of colour.

Remember all that stuff about gamma? Now that the coordinates are mapped I found that the relationship between coordinates and gamma to be non-linear. This was the main reason why the sRGB transformation was not suitable for my computed colour scheme exercise (I’ll show a visual example shortly).

When you adjust only the Red, Green or Blue sliders on an sRGB colour picker the results do not necessarily meet expectation.

Side note: There is so much more to explore around gamma specifically, maybe another time.

Vector Manipulation

If it wasn’t already implied the combination of vectors in an additive fashion allow for us to come up with a colour. As an example adding Pure Red and Pure Green should give us Yellow.

$[1,0,0] + [0,1,0] = [1,1,0]$

As it was aptly put by Troy from hg2dc

What the F$%k Happens When We Change an RGB Slider Value?

You are increasing or decreasing the emission intensity of one of the three RGB lights

The transformation lends itself to great precision providing a very large number of combinations, however, the intuitive expectations become difficult to meet.

As an example let’s take Red $[1,0,0]$ If we wanted to make this a “Deeper Red” programmatically we would need to have either reviewed and memorised the coordinates of the red we are seeking or manipulated our RGB values and reviewed incrementally until we were happy this could land us at $[0.9098,0.0353,0.0353]$ (a slightly darker red).

Even with clamped helper values in the RGB space we would still need to manipulate our green and blue values to find the output we desire.

Because of the biological quirks ([discussed earlier](#Colour is a Sensation)) — like our heightened sensitivity to Green —moving a coordinate by 0.1 on the Green axis “feels” much more dramatic than a 0.1 move on the Blue axis.

This “overhead” in managing our perception and sense is why It is harder to apply a programmatic approach to computed Themes in the sRGB space. That’s why I needed to look at alternatives like HSV and HSL.

HSV and HSL

HSV and HSL comes from the desire to have a natural system in which it’s easier to adjust a colour scheme, providing more intuitive functions familiar to the standard observer (humans). The transformations are inspired from art where a base colour (hue) is picked adding white (tint), black (shade), or grey (tone/saturation).

Source: Wikipedia Article on HSV and HSL

sRGB is hardware-focussed valuing precision, where as HSL/HSV accommodates human intuition. The transformations for HSL/HSV are implemented by mapping cartesian RGB values into a cylindrical co-ordinate system.

To apply the transformation (assuming RGB values are normalized to the range $[0,1]$ ) a pre-calculation determines the peak and floor of each RGB triplet with:

$M = max(R,G,B)$
$m = min(R,G,B)$
$C = M - m$ (This is the Chroma , or the distance from the neutral grey axis)

We use the above values in both Value and Saturation so it’s important to remember them.

Source: Wikipedia Article on HSV and HSL

Visualising the cylinder with a colour space mapped to it (like below) - HSL and HSV provide a transformation option where specific choices for Lightness (or Brightness in the case of HSV), Hue and Saturation become special cases, unified by adjustable parameters - also known as a parametric framework.

Source: Wikipedia Article on HSV and HSL

I will try to summarise Hue, Saturation and Value (or lightness/intensity) in its transformative context. I did note through a lot of my readings that definition for these terms is going to have variance based on context - so keep that in mind when exploring other areas.

Hue

TLDR: Hue is an angle on our unwrapped colour cylinder normalised to $360\degree$ to represent a perceived colour.

Source: Wikipedia Article on Hue

I came to find this particular definition from Color Appearance Models by Mark D. Fairchild as a great starting point for my understanding of hue.

Hue - Attribute of a visual sensation according to which an area appears to be similar to one of the perceived colors; red, yellow, green and blue or to a combination of two of them.

In a HSV/HSL model both hue and chroma are parameters which when applied navigate our coordinates with different effect.

To have a reference for hue first the RGB cube tilted with the black vertex on the bottom is projected to a hexagonal shape.

Each projected vertex which constitutes points between edges in our hexagon become the anchor vectors with the definition of hue being roughly the angle of the vector to a point in the projection, with red at 0°

As such Hue is the angular component and is typically measured in degrees. As an example from the above image a yellow hue would sit at roughly $60\degree$ - if H is less then 0 then it would add $360\degree$ to normalise it.

Saturation

TLDR: In HSV saturation is simply the chroma $C$ divide by the maximum value $M$ .

Saturation is the “colourfulness of an area judged in proportion to its brightness” (17-1136)

Source The Difference Between Chroma and Saturation*

The math behind the generalised saturation functions is certainly a little above my paygrade however I will try to keep it simple.

Saturation is traditionally calculated at the length of a vector $L(c)$ (calculation varied based on weightings - see lightness below) divided by the length of the extension of this vector to the surface of the RGB cube. Dependant on the brightness function creating a parametric system which allows us to keep a conic or bi-conic form of transformation space, intimately utilises brightness functions and takes into account specific co-ordinates on the brightness slope. Those poitns are then used in a calculation to divide our length accordingly.

Brightness or Lightness Value (ft. Intensity)

TLDR: In HSV = Value = Max of RGB Triplet - the below gives more detail and insight and can be skipped if it’s not of interest.

HSL, HSV and HSI all use a form of luminous intensity ( $L(c)$ ) as a subjective measure of brightness - this is derived from the Generalized Lightness Hue and Saturation (GLHS) colour model.

The equation below provides insight into how the brightness is calculated:

Assuming c as a colour $[R, G, B]$ value.
$w_{min}$ , $w_{mid}$ and $w_{max}$ as user set weights (generally 0 or 1)
$min(c)$ represents the minimum value amongst the $[R, G, B]$ values
$mid(c)$ represents the median of the $[R, G, B]$ values
$max(c)$ represents the maximum value amongst the $[R, G, B]$
Finally a few constraints - $w_{max} > 0$
and $w_{min} + w_{mid} + w_{max} = 1.$

$L (c) = w_{min} · min (c) + w_{mid} · mid (c) + w_{max} · max (c)$

Specific values of the weights give the brightness functions used by the common cylindrical colour spaces:

Model	$w_{min}$ , $w_{mid}$ , $w_{max}$	Description	Resulting Formula
HSV	$0 , 0, 1$	Value determined by strongest colour component	$L = max(c)$
HSL	$\frac{1}{2} , 0 , \frac{1}{2}$	Lightness is the average of the most and least saturated components	$L = \frac{min(c) + max(c)}{2}$
HSI	$\frac{1}{3}, \frac{1}{3}, \frac{1}{3}$	Intensity is the true average of all three RGB components.	$L = \frac{R+G+B}{3}$

Why HSV?

So now that the RGB space is clearly laid out, we need to consider colour matching in a very basic sense, if my theme engine was to take some seed colours for Primary, Secondary, Tertiary and Neutral how do I expand this system with multiple derivative shades in a formulaic fashion.

Lets take hex code #4287f5 as our seed colour - RGB 66, 135, 245

And here is a subjectively lighter shade of blue #7cadfc - RGB 124, 173, 252

As the nature of travelling between colours (adjusting gamma) in sRGB is non linear there is no defined step change across all three vectors that will yield a consistent results. If we were to take the above logic then a stepped change of $[+58, -11, +7]$ across our vectors should have a similar result on a green colour. Lets try it!

Here is our green seed hex #30d42a - RGB 48, 212, 42

Here if we apply the same modifiers to our RGB coordinates we map to RGB 106, 201, 49 which results in hex #6ac931 In comparison to our blue shade variant the green looks washed out and “not quite right”.

You may also notice that the green variation feels different compared to blue. This is because the standard observer (human being) has an uneven sensitivity across the spectrum: we are most sensitive to changes in Green, followed by Red, and least sensitive to Blue. This is why a fixed mathematical shift in RGB results in wildly different perceptual shifts depending on the starting colour.

To demonstrate this point I will pick two seed colours and apply the same step change of $[+58, -11, +7]$ four times and put them side by side for comparison. (The top is the seed colour picked at random).

The results tend to feel washed out, and the uniform intuitive option will not resonate across a swatch.

Now let’s apply the same exercise in the HSV space. First and foremost we are not going to play with Hue or Saturation only Value (Brightness Factor!).

Now one thing I didn’t mention is this project is being done in Java (sorry in advance) so the code here is going to be a bit verbose but should be easy to follow:

public record ColorShade(String name, String hex, Color color) {};

private List<ColorShade> generateShades(String baseColorName, String hexColor) {
  Color baseColor = Color.web(hexColor);
  List<ColorShade> palette = new ArrayList<>();
  double[] brightnessFactors = {1.5, 1.2, 1.0, 0.8, 0.5};
  String[] labels = {"100", "300", "500", "700", "900"};
  for (int i = 0; i < brightnessFactors.length; i++) {
    Color shadeColor = baseColor.deriveColor(0, 1.0, brightnessFactors[i], 1.0);
    String shadeName = baseColorName + "-" + labels[i];
    ColorShade shadeToAppend = new ColorShade(
      shadeName,
      toHexString(shadeColor),
      shadeColor
    );
    palette.add(shadeToAppend);
  }
  return palette;
}

By applying a standardised multiplier to our base colours brightness we get some interesting results! lets take our colours we used in our sRGB approach and apply the same multipliers:

Lets take our original purple hex #FF1BB4 in our function the 1.0 or “500” shade is considered the centre and the results show a noticeable difference.

and now the same with the #FF714E coral orange.

The immediate uniformity in shade application and the effect of having a parameter that gives us consistent results becomes apparent!

Conclusion

This was a really long way of saying if you want to automate theming a parametric system like HSV/HSL or HSI and now the new standards in CSS like OKCLH (which supports a wider gamut of colours in the p3 space) are going to allow for you to have consistent automated results.

Until next time!

Reading Sources

sudo apt-get it on an old Pixel 3xl

Mon, 08 Dec 2025 00:00:00 GMT

TLDR

This is a high level guide on some fun I had with an old Google Pixel 3xl with the following goals in mind.

De-google and run a Linux distro
Glorious SSH
Install and run a developer environment
Potentially host and access services from another device through incoming connections

If you want to skip the why, and learn more about the how, feel free to go straight to LineageOS

Lets go!

We recently hit spring (supposedly) which means its that time of the year to clean out the tech closets. After a lifetime of advocating to reduce “tech debt”, I always find it funny opening my closet to find decades of technology just sitting there dormant. Like warriors of past the closet doubles as hall of fallen heroes, each with it’s own tale of conquest and heroism from a simpler time.

So as one does, I come back to clean this space every few months and each time a new seemingly simple, but usually complicated, idea pops into my mind. Things like “I wonder if I can run RAID across 5 old laptop hard drives?” or “I should build a cyberdeck out of all these extra parts”. To be fair, one of these ideas resulted in installing Arch on an old ThinkPad - a decision I am extremely happy with. (I use Arch btw).

So this time when the cogs started to turn there were two things in the back of my mind that really helped speed things along.

The first was the concept of ownership as opposed to being a hostage. Whilst this isn’t a piece dedicated to that, and many before me have done it justice the crux of it came down to.

Right to repair
Freedom to modify for personal use
Unrestricted access to features

The second came from an odd place.

Long time content creator recently turned 10x engineer PewDiePie has gone on an interesting journey building fun projects and solving tech problems. He has, probably like me, opened the tech closet and gazed upon the titans of past thinking of giving them purpose once more. One project that stood out was self hosting using a Steam deck.

So with an optimistic glimmer in my eye and my gaze affixed on a retired Google Pixel 3xl from 2018, I set forth to conquer what I thought would be a few simple milestones.

De-google and run a Linux distro on my Pixel 3xl
Glorious SSH
Install and run a developer environment
Potentially host and access services from another device through incoming connections

LineageOS

The first step was to “unshackled the Google from Google” a task that was surprisingly accessible. I know its easy to pick at issues with big tech companies - but for what its worth, at least there’s a way to do this with relative ease.

I thoroughly enjoyed this process because it was reminiscent of installing an operating system back in the “good old days”. A time where taking careful note of hardware compatibility and kernels could be the difference between a short simple process or hours of troubleshooting.

So after some research I found there weren’t many choices for my relic and the best suited one would be LineageOS an android distribution that has an impressive amount of devices it supports.

This isn’t going to be a step by step guide on how to install LineageOS as the documentation does a good job in explaining the process there are however key things when tinkering with an android phone that were critical to installing LineageOS and subsequently rooting the phone. These being Android Developer Options, Android Debug Bridge (adb) and Fastboot.

Android Developer Options

To access and run debugging via a USB Cable the phone required Android developer options enabled. To enable this

Go to the phones Settings
PressAbout Phone
Tap the Build Number seven times
Enter your pin or password if prompted.

You can then find the Developer Options menu in your main settings, often under System. This menu will be crucial to enabling USB Debugging which will be used through out the process.

Android Debug Bridge (ADB) and Fastboot

As part of installing LineageOS the process will require you to reboot into a bootloader by using a command line interface. ADB is a command line tool which allows you to communicate with the device and perform a number of privileged actions after enabling developer access.

If you have used Android Studio or an Android SDK chances are you already have adb installed to check this you can run the command in your preferred command line interface. The communication between PC and Phone is usually done over a USB cable, but Wi-Fi connections are also supported.

adb version

Using fastboot (if your device supports it) should simply involve rebooting to fastboot mode. While in fastboot mode, you can type fastboot devices to verify that your device is being detected

To run adb and fastboot you will need to consider your OS and follow the instructions here

Su (magisk)

Even after installing LineageOS there still will be an issue when trying to run processes or commands requiring elevated privileges. Simply put without rooting the device the su command is non existent. To do this you can follow along here

The shortened guide below.

Download and install the Magisk mobile app
Within the app get a copy of one of the following images boot.img, init_boot.img or recovery.img. (If you don’t see either of these see the detailed instruction linked in the first paragraph for more details).
Copy the boot/init_boot/<recovery_image.img> to your phone
Press install in the Magisk section (not the app section).
Choose Select and Patch a File followed by your recovery image.
Once installation is done copy your new img file via adb. (Your device might have a different folder structure instead of sdcard) Warning: Never download and use a random img file online - always patch your own file to avoid any security or configuration issues.

    adb pull /sdcard/Download/magisk_patched_[random_strings].img <your_pc_directory>
    # You can use the . symbol to download into the same directory

Reboot phone into recovery mode using adb -d reboot bootloader
Use fastboot and point your device to your new image (take note of boot type based on your img file type) and then turn on your phone.

fastboot flash boot /path/to/magisk_patched_[random_strings].img
# OR
fastboot flash init_boot /path/to/magisk_patched_[random_strings].img
# OR
fastboot flash recovery /path/to/magisk_patched_[random_strings].img`

Launch the Magisk app and select Direct Install followed by another reboot
The phone should now be rooted successfully.

F-Droid and Termux

Now that we have a free and opensource operating system we also need a catalogue of free and opensource applications available for the android platform, and that’s exactly what F-Droid does. Installation of F-Droid is fairly simple you navigate to the main page on the phone hit the big download button and install via the APK.

Going back to our goal of SSH and emulating Linux the first step was to ensure we had a terminal we could play with. I decided to go with Termux as it provides an extensive Linux package collection. Heck you could even compile code with gcc and clang.

Now the old Pixel 3XL not only is running a brand new de-Googled OS but it also has a nice terminal emulator waiting for root access. All that’s left is to ensure we allow for Termux to have said access as a super user.

Open Termux
run the su command (inside Termux on the phone)
Grant access via Magisk app (approve prompts on your phone)
execute the adb shell on your PC - whilst the phone is plugged into your pc (this is akin to having a direct wired connection to your terminal)
type in su within the adb shell terminal on your pc and then approve/verify the access on the phone.

Glorious SSH

Alas, the last step is to provide remote SSH access (at this stage via local network). Now that we have an elevated terminal and can technically access it straight from our PC via a USB cable it becomes no different to a regular Linux process! You can elect to do this via adb shell or directly on the phone via Termux

Open Termux / or adb shell
Run the command pkg upgrade && pkg update
Run the command pkg install openssh
Set a SSH password using the passwd command
Run the sshd command to start the SSH daemon
Check your phones local IP address using the ifconfig command
Check your terminal username using the whoami command (this needs to be done on Termux directly on the phone)

(Optional but recommended) With this setup if the device goes to sleep it will automatically sever the SSH connection - you can stop this behaviour by running the command termux-wake-lock in Termux on the phone.

This stops your device from going into “deep sleep” ensuring access even if the screen is “asleep”.

So by now you should have:

Phones Local IP
Termux Username
SSH Password

The final check is ensuring you can SSH into the phone to do this open up your favourite CLI which can of course provide you with glorious SSH and type in the following:

ssh -p 8022 <username>@<phone_IP_Address>

If everything has been done correctly you should be greeted with something similar to the below image:

proot-distro

Termux provides a package called proot-distro which takes care of managing Linux distributions inside Termux. You can install this utility by executing.

pkg install proot-distro

By using proot we open the device up to so much more fun! In my example I went with ubuntu - this was as simple as executing the following commands:

proot-distro install ubuntu
proot-distro login ubuntu

Proot supports the following distros:

Alpine Linux (edge)
Arch Linux / Arch Linux 32 / Arch Linux ARM
Debian (stable)
Fedora 35
Manjaro AArch64
OpenSUSE (Tumbleweed)
Ubuntu (22.04)
Void Linux

hey look, my phone now runs a dev environment

So this was my last goal really, could I run a node server and access it on my local network. Sadly I was able to only get it half done.

There were a few reasons for this but let me start by providing a quick overview on the Node and NPM experience .I took a bit of a different route to get it running because we needed to ensure all the binaries were available a luxury that you overlook when using a desktop OS.

Below you will find the commands in order, I also opted to use Node Version Manager (NVM) to make it easier to manage multiple node packages in the future.

apt update && apt upgrade
apt install nodejs npm build-essential
apt install curl -y
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm --version
nvm install --lts
nvm use --lts

Magic

With what I have just demonstrated you can imagine the amount of possibilities that open up, I have seen people turn old phones into a bunch of things including a mobile pen testing tool running Kali Linux.

Here’s an Astro site running locally as an example: Sadly there was one thing I didn’t account for.

Nobody escapes the Google

Just when I thought I was out Google pulled me right back in. Compared to it’s fruit variety the freedom is unmatched however there was still some restriction I had not accounted for. The NIC is extremely locked down in most mobile phones (for good reason!) that certainly is no different in the case of the pixel.

After trying almost every option to fix the error A system error occurred: uv_interface_addresses returned Unknown system error 13 (Unknown system error 13). I staggered off to lick my wounds - Whilst I think there still might be a way for me to beat the final challenge - perhaps it’s a tale for another day.

In retrospect the issue stems from permissions as the libuv library tries to call the function os.networkInterfaces(). This function triggers an Access denied on modern Android (11+) as querying network interface details are restricted.

This was a fun exercise and it has given me something to do with an otherwise dormant relic of the past. I would always recommend finding ways to give old tech a new purpose - you may even surprise yourself with the results.

Do you even grep? Disko 1 picoCTF

Thu, 11 Sep 2025 00:00:00 GMT

“In the Microsoft eco-system, the people are not represented by two separate, yet equally important, groups: those who grep, investigating file content using simple commands; and those with beautiful window managers, effortlessly floating between screens rarely touching the mouse.”

This all started with a CTF (Capture The Flag) challenge over at picoctf.

For those who are interested in trying the challenge on their own.

Sorry but the title of this piece might be a major clue.
Stop reading this RIGHT NOW - Go try it first (and if you can be bothered) try figuring it out on windows.

For the uninitiated a CTF is a technical challenge trying to find a hidden string by completing either some digital forensic examination, real vulnerability exploits (on fake and intentionally vulnerable products), and sometimes even fake social engineering. They are really fun and a good way to get the neurons firing. The skills you gain from them are practical and can make you a better technology professional.

I don’t want to be one of those people but…

I’ve been daily driving Linux on my laptop and every time I want to do a Linux thing on my windows machine - A little bit of me cries.

Disko 1

This CTF is meant to be a digital forensic exercise analysing a dump of a disk (dd File) trying to find a hidden string. So really we need to do two things.

Get all the strings from the dd File
Search that string to find a pattern that will match our CTF Flag.

Thankfully we know what the CTF Flag looks like with this handy greyed out input box showing us what we are probably looking for.

and if you are on Linux - well you can do this with one line.

strings disko-1.dd | grep "picoCTF{"

the strings command displays all listable strings inside our disko-1.dd the pipe | redirects our output to a second command feeding the output directly into grep "<search term>" which effectively searches our output for the search term.

If you did this CTF with Linux - congrats. You did it in 5 minutes.

But what about Windows?

Is it winning if it’s not WIN-dows?

This really bugged me. I started this CTF in Windows. Why can’t I do it with one line in Windows. Surely its not that bad is it?

The first challenge I had was dealing with a .dd file - the file size wasn’t overly large (50mb), maybe I could just open this in notepad and ctrl+f the string?

Sounds silly but isn’t that what the grep command is doing in a neat little CLI anyway? That was not a good move, notepad my good friend for many years had been checkmated into a “not responding”. This of course was due to the strings command in Linux extracting all printable strings restricting our search and return to the correct type we were looking for.

There’s something hallmark about that signature grey/white UI with emboldened black text and data being presented in tabular format. It screams .NET core, those who win-amped remember aero styling, those who used Visual Basic remember grey square buttons.

I say all this because the first popular choice that came up to open and display content from a .dd file was a tool from PassMark called OSFMount a software company I used in high school to prove my computer could run the Age of Empires disc that came with my cereal. Credit where credit is due - once I had OSFMount installed and opening the .dd file I could now peak into the structure and contents of the folder. I was no closer to the one liner; however now that I had a “Folder” maybe our good friend PowerShell could come in to save the day.

PowerShell had it’s own comic-book.

I spoke about daily driving Linux earlier (I use Arch btw). I think every Linux user at one point or another has used a command similar to sudo apt-get, after slowly realising how easy it is to install update and manage packages from the command line interface (CLI) not only do these commands become second nature but the CLI itself becomes a “friendly(er)” to the user.

To be fair I think most Windows users have seen Command Prompt - or so I hope. PowerShell on the other hand actually needed its own brand mascot.

Anyway I digress - I’ve been trying to slot in showing people the PowerShell mascot without being too obnoxious about it.

So can we “grep” now? To review I have a mounted drive full of files a file path to be able to search contents of said files, granted the files are also nondescript and might have the same issues perhaps breaking down the operations to per file will yield results as my direct attempts on the .dd file had mostly failed

(The direct attempts below)

This one timed out

$fileContent = Get-Content -Path "C:\disko-1.dd"
Select-String -InputObject $fileContent -Pattern "picoCTF"

and this one returned something however the output was not correct

Get-ChildItem -Path "*" | Select-String -Pattern "pico"

I felt the last command was close - It was at this moment I realised something - perhaps i was doing this all wrong and there was a simple way. It wasn’t going to be as pretty as a strings command but maybe - if I just passed my .dd file as a variable and piped a Select-String it would work. and so:

$fileContent = Get-Content -Path "c:\disko-1.dd"
$fileContent | Select-String "picoCTF"

I couldn’t believe my eyes - after quite a long journey the answer was lying there - now formatting wasn’t nice and i got some garble with it. But we did it. we finally “won” on windows.

picoCTF{1t5_ju5t_4_5tr1n9_be6031da}

Interestingly - if you tried this command as a single lined pipe declaring the Get-Content on the left you would have a blank result.

Something that if I knew early on might have stopped me exploring OSFMount - but where’s the fun in that.

angOS- Part One

Mon, 04 Aug 2025 00:00:00 GMT

** Update 11/09 - Someone told me the name might be confused with another project out in the wild so updating to reflect my working title **angOS** (I just mushed angular and OS together - nothing fancy)

My earliest memory of a Windows PC dates back to when I was five. The information age had started to pick up steam and Microsoft was sporting it’s latest operating system, Windows 95 - a powerhouse for business, connectivity and leisure marketed as a must-have tool for the modern family.

My parents who were operating a small business and raising a young family certainly thought so and bought home the Compaq Presario 4160, a technological marvel for its time. This machine not only served me throughout most of my adolescent life, but also sparked a joy for technology that would be lifelong.

I reference this because even at that young age, wanting to understand the ins and outs of my machine, how all the software operated was something that excited me. That’s why I recently decided to go deeper and attempt to simulate an operating system using a modern web language - these projects are often described as a webOS and are generally used to learn and grow as a developers.

This project has been very rewarding for me, it has taught me a great deal on how larger teams make various decisions. For those who are at an intermediate level and looking to branch from one type of development to another (in my case it was web to Systems/OS) - this is a great way to think outside the box.

Conception

At the end of June 2025 I came across whispers that the Windows 11 Start Menu was built with React Native. After further research, it seemed to be some casual “clickbait” that surfaced every few months and nothing of note (the Start Menu itself is built with C++ and XAML with the recommendations feature using React Native). It was a nice little rabbit hole at the time however it did spark a small proof of concept.

The nostalgia of older Microsoft operating systems notably 95, 98 and XP brings me joy, reminiscing about aero icons and MSN Messenger dings after school I set myself a goal of visually recreating a Windows XP desktop environment in React - not the hardest front end styling exercise but one I came to enjoy.

It was around this time that my curiosity and learning topics started to mingle - I had recently started reading Clean Architecture by Robert C. Martin (Uncle Bob), alongside working through some courseware from Princeton around Data Structures and Algorithms. which was taught in Java. Putting the two together I had fully bought into the OO paradigm and was itching to practice my knowledge and theory - as I started crafting my UI design, I asked myself some fundamental questions.

“How does Windows get to the front end?”
“Is Windows different to other operating systems in how it achieves this?”
“How do we manage themes, preferences, start menu applications and other users?”

The more I asked these questions the more I realised this project was going to go beyond the scope of just a front end recreation.

New Goals

I had made a decision to turn this small project into a “learning companion” giving me a higher level understanding of core components that go into an Operating System, to keep it simple it would be written in a language I am currently familiar and comfortable in (TypeScript). I wanted the project to push me to think outside of the box with my current language whilst introducing me to concepts found in different development practices (Kernel Development, Drivers, Firmware, OS etc. ).

I first needed to set myself some rules - after careful consideration these were the main ones.

Pick a reference architecture.
Choose a Paradigm.
Stick to SOLID principles .
Modularity and expansion prioritised in design and structure.
Constrain realism to limitations of language and tools we are building in.
Be open to changing these rules if it meets the goals of simulating our reference architecture.

Opinionated Early

These rules resulted in some early decisions. I first elected to align with the Object Oriented Paradigm for this project, this would mean some changes to frameworks and structure (more on that shortly). I also decided to choose the NT Architecture as my reference architecture considering I was already working on a Windows front end.

Some other “standards” I set for myself.

kebab-case naming convention with dot separators to identify our files with ease etc: example-file.class.ts
Separation of concerns including modularised type.ts and const.ts
Test driven development to milestone major functionality implementations with unit testing methodology.

I believe the above to be more foundational, I realised the more important decisions were paradigm and reference architecture which helped shape and grow these standards organically.

In addition I chose some technologies to assist with the developer experience - the main ones being jest, prettier, tailwind, angular, ts-jest and jsdom.

jest with js-dom: Is the main testing suite being used as it allows for both functional unit tests and DOM related output validation.
prettier: I used to ensure consistent formatting across our files.
tailwind: Is a utility class based CSS library allowing for us to have simple inline styling across our html components
angular: Is an OO front end framework which we will discuss in more detail in the next section.

Functional vs OO

I had done a considerable amount of work making a functional front end using vite. I had built a replica of the Windows XP taskbar which visually matched the old Start Menu. I had even built multiple themes being served through a contextProvider, originally the vision was to allow for users to make their own themes to replicate other popular operating systems - the only issue, vite and React (whilst no issue mixing in OO principles after all everything is an object in javascript) are primarily better suited to the functional paradigm.

I had my first decision to make - one that I now believe to be the right one.

I elected to port my project in full to Angular. This was a great experience overall and my approach helped speed things up, if you are ever looking to port a project from one framework to another - my suggestion is to speak in the language of the Framework you are familiar with to an LLM (my current choice is Gemini for “pair programming ” purposes) - I’m not suggesting the responses are gospel however, questions like.

“What is the React useState equivalent in the Angular framework?”

Helped speed up finding the right reference materials for me to learn the basics of the framework, other more nuanced techniques were reserved to reading documentation and self research.

An example of change

Here I want to show how different the two frameworks are, let’s take the Windows XP Start Menu (see reference picture from my frontend).

The original functional component code (vite) looked like this:

StartMenu.tsx

src/applications/system/desktop-environment/components/StartMenu.tsx

import Divider from '../divider/Divider'
import PowerPanel from './PowerPanel'
import StartMenuLeftPanel from './StartMenuLeftPanel'
import StartMenuRightPanel from './StartMenuRightPanel'
import { forwardRef } from 'react'
import { useTheme } from '../../themes/ThemeContext'

interface StartMenuProps {
    isOpen: boolean
}

const StartMenu = forwardRef<HTMLDivElement, StartMenuProps>(
    ({ isOpen }, ref) => {
        // 'ref' is the second argument provided by forwardRef
        const theme = useTheme()
        const startMenuTheme = theme.startmenustyles
        const startMenuWrapperClasses = [
            startMenuTheme.wrapper.border,
            startMenuTheme.wrapper.color,
            startMenuTheme.wrapper.display,
        ]
            .filter(Boolean)
            .join(' ')

        const startMenuHeaderClasses = [
            startMenuTheme.startMenuHeader.border,
            startMenuTheme.startMenuHeader.color,
            startMenuTheme.startMenuHeader.display,
        ]
            .filter(Boolean)
            .join(' ')

        const innerMenuClasses = [
            startMenuTheme.innermenu.border,
            startMenuTheme.innermenu.display,
        ]
            .filter(Boolean)
            .join(' ')

        return (
            <div
                className={`${
                    isOpen ? 'absolute' : 'hidden'
                } ${startMenuWrapperClasses}`}
                ref={ref}
            >
                       {' '}
                <div className={startMenuHeaderClasses}>
                             {' '}
                    <img
                        src={theme.startmenustyles.profilepicture.iconSrc}
                        className={
                            theme.startmenustyles.profilepicture.iconStyle
                        }
                        alt={theme.startmenustyles.profilepicture.iconAlt}
                    />
                             {' '}
                    <span className={theme.startmenustyles.username.text}>
                                    Administrator          {' '}
                    </span>
                           {' '}
                </div>
                       {' '}
                <div className={innerMenuClasses}>
                              <Divider dividerColor="orange" />         {' '}
                    <div
                        className={theme.startmenustyles.pannelwrapper.display}
                    >
                                    <StartMenuLeftPanel />
                                    <StartMenuRightPanel />         {' '}
                    </div>
                              <PowerPanel />       {' '}
                </div>
                     {' '}
            </div>
        )
    }
)

export default StartMenu

And the ported object oriented component (Angular) looks like this:

StartMenu.ts

src/applications/system/desktop-environment/components/StartMenu.ts

import { Component, inject, computed, Input } from '@angular/core'
import { CommonModule } from '@angular/common'
import { NgOptimizedImage } from '@angular/common'
import { ThemeContextService } from '../../themes/theme-context'
import Divider from '../divider/divider'
import StartMenuLeftPanel from './start-menu-left-panel'
import StartMenuRightPanel from './start-menu-right-panel'
import StartMenuPowerPanel from './power-panel'
import { StyleHelpers } from '../../style-helpers'

@Component({
    selector: 'start-menu',
    standalone: true,
    imports: [
        CommonModule,
        NgOptimizedImage,
        Divider,
        StartMenuLeftPanel,
        StartMenuRightPanel,
        StartMenuPowerPanel,
    ],

    template: `

        <div [ngClass]="startMenuWrapperClasses">

            <div [ngClass]="startMenuHeaderClasses">

                <img
                    [ngSrc]="profilePicture.iconSrc"
                    [ngClass]="profilePicture.iconStyle"
                    [alt]="profilePicture.iconAlt"
                />

                <span [ngClass]="startMenuUsernameClasses">
                    Administrator
                </span>

            </div>

            <div [ngClass]="startMenuInnerMenuClasses">
                        <divider color="orange" />
                <div [ngClass]="startMenuPanelClasses">
                              <start-menu-left-panel />
                    <start-menu-right-panel />
                </div>
                        <start-menu-power-panel />
            </div>

        </div>

    `,
})
export default class StartMenu {
    @Input() isOpen: boolean = false
    private theme = inject(ThemeContextService)
    styles = computed(() => this.theme.theme().startmenustyles)
    get profilePicture() {
        return StyleHelpers.returnIconValues(this.styles().profilepicture)
    }
    get startMenuPanelClasses() {
        return StyleHelpers.joinStyleValues(this.styles().pannelwrapper)
    }

    get startMenuInnerMenuClasses() {
        return StyleHelpers.joinStyleValues(this.styles().innermenu)
    }
    get startMenuHeaderClasses() {
        return StyleHelpers.joinStyleValues(this.styles().startMenuHeader)
    }
    get startMenuUsernameClasses() {
        return StyleHelpers.joinStyleValues(this.styles().username)
    }
    get startMenuWrapperClasses() {
        const classes = [StyleHelpers.joinStyleValues(this.styles().wrapper)]
        if (!this.isOpen) {
            classes.push('hidden')
        } else {
            classes.push('absolute')
        }
        return classes.filter(Boolean).join(' ')
    }
}

Two completely different approaches ultimately leading to the same rendered component - I had moved from useContext to inject (and compute); props to @decorators; state to signals and of course was now returning classes instead of functions.

Whilst the conscious choice to port to Angular was made early this wasn’t necessarily due to one framework being better then the other. It aligned more to those opinions and rules I had set - in particular adhering to an OOP approach upfront.

Starting at the Hardware

A computer is made up of components, those components put together make it compute. So how do we simulate those components without losing focus on the overall project and its learning outcomes. My first task was to validate a configuration with some simulated core hardware components.

CPU

During my initial research I made a decision to forgo simulating a CPU at this stage - I made this decision as I saw the task as one that would consume a significant amount of early stage development that could cause the project to lose focus and fail to meet its initial learning goals.

Should I have included the CPU I would need to consider, threading, concurrency, logic gate functionality, orchestrating kernel executions and much more. My approach was simple, allow for the structure to “plugin” a CPU at a later date and for now continue with the main goals.

Storage, RAM and GPU

Memory from a functionality perspective is fairly simple with the two main functions being read and write. (technically it is storing and releasing but for now let’s relegate to read and write).

Accessing memory has no linearity, allowing for non-sequential reads and writes to ensure fast access. As this is a simulation we were not going to get into the granularity of control logic, address decoders (at least in detail) or multiplexers. The function of our various memory modules was to meet and address a few points.

A way to simulate and limit concurrent running processes
Abstracted pointers to Hexadecimal values (Uint8Array was used for this).
Simple read and write that could eventually be accessed through our win32API (More on this in part two of this article in the near future).

The first technical decision made was to limit the simulated storage parameters to be kilobytes (kb/ $*1024$ ) noting I would scale this up in the front end environment for rendering/visual purposes. The reason was due to the constraints for object creation lengths in JavaScript. Having a Uint8Array with a length that equated to the amount of bytes in a GB would not be acceptable or even required for this project.

As mentioned I made a choice of a Uint8Array to represent our “address decoder” (using the term very loosely) because it supports the working with arbitrary binary data. Should I decide to dip my toes into WASM and decoding binary data then we have a good typed array to support this in the future!

I also decided to not implement GDI level simulation which meant limiting the GPU to memory functionality, this means that both the GPU and RAM classes would be identical for now separated in name.

ram.class.ts

src/simulated-hardware/ram/ram.class.ts

// GPU and RAM have the same functions at this stage.
// technical decision to use kb instead of mb to simulate actual memory reads within javascript
export class RAM {
    public totalMemoryKB: number
    public memoryContent: Uint8Array
    public usedMemoryKB: number = 0
    public availableMemoryKB: number = 0
    public totalBytes: number

    constructor(memoryKB: number) {
        this.totalMemoryKB = memoryKB
        this.availableMemoryKB = this.totalMemoryKB
        this.totalBytes = this.totalMemoryKB * 1024
        this.memoryContent = new Uint8Array(this.totalBytes)
    }

    public allocateMemory(expectedMemoryUsageKB: number): number | null {
        if (expectedMemoryUsageKB <= 0) {
            console.warn('RAM: Cannot allocate 0 or negative memory')
            return null
        }
        if (expectedMemoryUsageKB > this.availableMemoryKB) {
            console.warn(
                `RAM: Not enough RAM Memory available. Requested ${expectedMemoryUsageKB}KB, but only ${this.availableMemoryKB}KB remaining.`
            )
            return null
        } else {
            const allocatedAddressBytes = this.usedMemoryKB * 1024
            this.availableMemoryKB -= expectedMemoryUsageKB
            this.usedMemoryKB += expectedMemoryUsageKB
            console.log(
                `RAM: Allocated ${expectedMemoryUsageKB}KB of memory. Used: ${this.usedMemoryKB}KB, Remaining: ${this.availableMemoryKB}KB`
            )
            return allocatedAddressBytes
        }
    }

    public readMemory(addressBytes: number, lengthBytes: number): Uint8Array {
        const totalBytes = this.totalMemoryKB * 1024
        if (addressBytes < 0 || addressBytes + lengthBytes > totalBytes) {
            console.error(
                `RAM: Memory read out of bounds. Address: ${addressBytes}, Length: ${lengthBytes} bytes. Total Size: ${totalBytes} bytes.`
            )
            return new Uint8Array(0)
        }
        return this.memoryContent.subarray(
            addressBytes,
            addressBytes + lengthBytes
        )
    }

    public writeMemory(addressBytes: number, data: Uint8Array): void {
        const totalBytes = this.totalMemoryKB * 1024
        if (addressBytes < 0 || addressBytes + data.length > totalBytes) {
            console.error(
                `RAM: Memory read out of bounds. Address: ${addressBytes}, Data Length: ${data.length} bytes. Total Size: ${totalBytes} bytes.`
            )
            return
        }
        this.memoryContent.set(data, addressBytes)
    }

    public getAvailableMemory(): number {
        return this.availableMemoryKB
    }

    public getUsedMemory(): number {
        return this.usedMemoryKB
    }
}

Storage Devices

Whilst the storage device would also take the same functions a small addition was added to the constructor to ensure I could adapt the class to have different type properties. I did this by using a StorageType export. I was also initially considering how to check “port compatibility” and this structure would help me address that across components.

storage/types.ts

src/simulatedHardware/storage/types.ts

export const StorageTypes = {
    DISC_DVD: 'DISC_DVD',
    IDE_HDD_ONE: 'IDE_HDD_ONE',
    IDE_HDD_TWO: 'IDE_HDD_TWO',
    IDE_HDD_THREE: 'IDE_HDD_THREE',
    USB_FLASHDRIVE: 'USB_FLASHDRIVE',
}

export type StorageType = (typeof StorageTypes)[keyof typeof StorageTypes]

storage-device.class.ts

src/simulated-hardware/storage/storage-device.class.ts

constructor(storageType: StorageType, memoryKB: number) {
  if (memoryKB <= 0) {
    console.error('STORAGE: Cannot create a storage device with no memory.');
    this.availableMemoryKB = null;
    this.totalMemoryKB = 0;
    this.totalBytes = 0;
    this.memoryContent = new Uint8Array(0);
    this.hardwareProfile = storageType;
    return;
  }
  this.totalMemoryKB = memoryKB;
  this.totalBytes = this.totalMemoryKB * 1024;
  this.availableMemoryKB = this.totalMemoryKB;
  this.memoryContent = new Uint8Array(this.totalBytes);
  this.hardwareProfile = storageType;
}

Motherboard

The motherboard serves as a I/O board that verifies if its slots are occupied with valid devices and returns a class with “what’s plugged in”. I decided to use custom interfaces to construct types that would ensure the compatibility requirements of slots were met.

This approach allows me to come back to this module and expand on the idea in the future. For now the pcieOne and pcieTwo ports as per pcConstructorProperties take a pcieSlotConfig type which would offer us with either a null value (nothing plugged in) or an interface that’s gpu or storage.

The motherboard can ultimately have anything plugged in (assuming compatibility requirements are met), at a minimum it needs GPU, Storage and RAM to be a valid construction of a motherboard otherwise returning a configurationError.

motherboard/types.ts

src/simulated-hardware/motherboard/types.ts

import { GPU } from '../gpu/gpu.class'
import { RAM } from '../ram/ram.class'
import { StorageDevice } from '../storage/storage-device.class'

interface GpuPcieProperties {
    type: 'gpu'
    profile: GPU
}

interface StorageProperties {
    type: 'storage'
    profile: StorageDevice
}

export type PCIeDevice = GPU | StorageDevice | null
export type pcieSlotConfig = GpuPcieProperties | StorageProperties | null

export interface pcConstructorProperties {
    dimSlotOne?: RAM
    dimSlotTwo?: RAM
    dimSlotThree?: RAM
    dimSlotFour?: RAM
    ideOne?: StorageProperties
    ideTwo?: StorageProperties
    sataOne?: StorageProperties
    pcieOne?: pcieSlotConfig
    pcieTwo?: pcieSlotConfig
}

motherboard.class.ts

src/simulated-hardware/motherboard/motherboard.class.ts

import { RAM } from '../ram/ram.class'
import { StorageDevice } from '../storage/storage-device.class'
import type { pcConstructorProperties } from './types'
import type { PCIeDevice } from './types'

export class MotherBoard {
    public dimSlotOne: RAM | null = null
    public dimSlotTwo: RAM | null = null
    public dimSlotThree: RAM | null = null
    public dimSlotFour: RAM | null = null
    public ideOne: StorageDevice | null = null
    public ideTwo: StorageDevice | null = null
    public sataOne: StorageDevice | null = null
    public pcieOne: PCIeDevice | null = null
    public pcieTwo: PCIeDevice | null = null

    constructor({
        dimSlotOne,
        dimSlotTwo,
        dimSlotThree,
        dimSlotFour,
        ideOne,
        ideTwo,
        sataOne,
        pcieOne,
        pcieTwo,
    }: pcConstructorProperties) {
        if (!dimSlotOne && !dimSlotTwo && !dimSlotThree && !dimSlotFour) {
            this.configurationError(`Missing RAM`)
        }
        if (pcieOne?.type !== 'gpu' && pcieTwo?.type !== 'gpu') {
            this.configurationError(`Missing GPU`)
        }
        if (
            !ideOne &&
            !ideTwo &&
            !sataOne &&
            pcieOne?.type !== 'storage' &&
            pcieTwo?.type !== 'storage'
        ) {
            this.configurationError(`Missing Storage`)
        }
        if (dimSlotOne) this.dimSlotOne = dimSlotOne
        if (dimSlotTwo) this.dimSlotTwo = dimSlotTwo
        if (dimSlotThree) this.dimSlotThree = dimSlotThree
        if (dimSlotFour) this.dimSlotFour = dimSlotFour
        if (ideOne && ideOne.type == 'storage') this.ideOne = ideOne.profile
        if (ideTwo && ideTwo.type == 'storage') this.ideTwo = ideTwo.profile
        if (sataOne && sataOne.type == 'storage') this.sataOne = sataOne.profile
        if (pcieOne) this.pcieOne = pcieOne.profile
        if (pcieTwo) this.pcieTwo = pcieTwo.profile
    }

    private configurationError(errorMessage: string) {
        throw new Error(`PC: ${errorMessage}`)
    }

    public currentConfig() {
        return {
            dimSlotOne: this.dimSlotOne,
            dimSlotTwo: this.dimSlotTwo,
            dimSlotThree: this.dimSlotThree,
            dimSlotFour: this.dimSlotFour,
            ideOne: this.ideOne,
            ideTwo: this.ideTwo,
            sataOne: this.sataOne,
            pcieOne: this.pcieOne,
            pcieTwo: this.pcieTwo,
        }
    }
}

Firmware

I introduced the concept of firmware when I was separating concerns related to verifying an option ROM and the master boot record. In a normal scenario an operating system would initially be installed off external boot media with a valid boot signatures however in this scenario the simulation would assume that the operating system was “pre-installed” and these installation steps had been completed. I made a choice to relegate the master boot record to a “firmware” function, this would allow for me to create different signatures, should I for example want to dynamically load a Linux inspired MBR onto a Storage device in the future.

Storage Firmware (Master Boot Record)

To create the master boot record the approach I took wasn’t dissimilar to how it would work in a real operating system. The first step was to set some variables.

diskSectorSize - This would be the smallest size allocated for each sector, similar to a real device we chose 512 as the smallest divisible number.
pointerVariables - These variables are the memory addresses reserved to identify characteristics of the the boot record we have:
- 446: Pointer which returns if partition is Active or Inactive.
- 450: Partition type (NTFS, FAT etc.)
- 510 and 511: These pointers are reserved to let the BIOS know if the partition has a boot signature.
- 328 and 329: For the simulation I reserved pointer 328 and 329 to be our OS Signature, in a real operating system unreserved bytes in the MBR would store the bootloader software. As I was going straight into kernel mode depending on signature found I simplified the logic to match - it also means we can make different operating systems without needing to build a bootloader for each one.
partitionVariables: To accompany the above memory addresses the partition variables are set into two categories.
- Storage Specific - File Type, Active/Inactive
- OS Specific - Boot Signature Values, OS Signature Values (For our simulation we only have the Windows ones for now.)

const.ts

src/firmware/storage/const.ts

// MBR Constants
export const diskSectorSize = 512

// MBR - Pointers
export const pointerBootablePartition = 446
export const pointerPartitionType = 450
export const pointerBootSignatureOne = 510
export const pointerBootSignatureTwo = 511
export const pointerOSSignatureOne = 328
export const pointerOSSignatureTwo = 329

// MBR - Flags/Data
export const partitionActive = 0x80
export const partitionInactive = 0x00
export const partitionTypeNTFS = 0x07
export const partitionTypeFat32 = 0x0c

// MBR - OS Sigantures
export const osBootSignatureOne = 0x55
export const osBootSignatureTwo = 0xaa
export const osSignatureBindowsXDOne = 0xff
export const osSignatureBindowsXDTwo = 0xee

With the values set now it was time to translate this to a class structure - I opted to take four variables for the constructor as captured by the types file:

partitionType: a simple union type that would translate different filesystem types.
sectorSize: a number representing the number of bytes per sector.
osSingatureOne and osSignatureTwo: our unique system implemented to invoke the correct operating system from BIOS.

types.ts

src/firmware/storage/types.ts

type MasterBootRecordPartitionType = 'ntfs' | 'fat32'

export interface MasterBootRecordTypeConstructorProperties {
    partitionType: MasterBootRecordPartitionType
    sectorSize: number
    osSignatureOne: number | null
    osSignatureTwo: number | null
}

Finally I combined the variables and put together the MasterBootRecord class when constructed would return a Uint8Array which could be directly written into the previous StorageDevice class.

The class when constructed would first ensure the constructor arguments adhered to the original architecture specifications (512 byte length, active partition, valid OS signature and valid filesystem type). Once checks are complete it sets the this.masterBootRecord property to the typed array with the crucial pointers having the correct corresponding data.

I also included a validateMasterBootRecord function which would return true if all specifications were met else returning false.

master-boot-record.class.ts

src/firmware/storage/master-boot-record.class.ts

import type { MasterBootRecordTypeConstructorProperties } from './types'
import {
    pointerBootSignatureOne,
    pointerBootSignatureTwo,
    pointerBootablePartition,
    pointerOSSignatureOne,
    pointerOSSignatureTwo,
    pointerPartitionType,
    partitionActive,
    partitionInactive,
    partitionTypeFat32,
    partitionTypeNTFS,
    osBootSignatureOne,
    osBootSignatureTwo,
} from './const'

export class MasterBootRecord {
    public masterBootRecord: Uint8Array
    constructor({
        partitionType,
        sectorSize,
        osSignatureOne,
        osSignatureTwo,
    }: MasterBootRecordTypeConstructorProperties) {
        this.masterBootRecord = new Uint8Array(sectorSize).fill(0x00)
        if (sectorSize !== 512) {
            this.masterBootRecord[pointerBootablePartition] = partitionInactive
            throw new Error('Master Boot Record: Sector Size not supported.')
        }
        this.masterBootRecord[pointerBootablePartition] = partitionActive
        if (!partitionType) {
            this.masterBootRecord[pointerBootablePartition] = partitionInactive
            throw new Error('Master Boot Record: Missing Partition Type')
        }
        if (partitionType === 'ntfs') {
            this.masterBootRecord[pointerPartitionType] = partitionTypeNTFS
        } else if (partitionType === 'fat32') {
            this.masterBootRecord[pointerPartitionType] = partitionTypeFat32
        }
        if (
            osSignatureOne !== osBootSignatureOne ||
            osSignatureTwo !== osBootSignatureTwo
        ) {
            this.masterBootRecord[pointerBootablePartition] = partitionInactive
            throw new Error(
                'Master Boot Record: Invalid boot signature values provided'
            )
        }
        this.masterBootRecord[pointerBootSignatureOne] = osBootSignatureOne
        this.masterBootRecord[pointerBootSignatureTwo] = osBootSignatureTwo
        if (!osSignatureOne || !osSignatureTwo) {
            this.masterBootRecord[pointerBootablePartition] = partitionInactive
            throw new Error('Master Boot Record: No OS Signature provided.')
        }
        this.masterBootRecord[pointerOSSignatureOne] = osSignatureOne
        this.masterBootRecord[pointerOSSignatureTwo] = osSignatureTwo
    }

    public validateMasterBootRecord(): boolean {
        if (!this.masterBootRecord) return false
        return (
            this.masterBootRecord.length === 512 &&
            this.masterBootRecord[pointerBootablePartition] ===
                partitionActive &&
            (this.masterBootRecord[pointerPartitionType] ===
                partitionTypeNTFS ||
                this.masterBootRecord[pointerPartitionType] ===
                    partitionTypeFat32) &&
            this.masterBootRecord[pointerBootSignatureOne] ===
                osBootSignatureOne &&
            this.masterBootRecord[pointerBootSignatureTwo] ===
                osBootSignatureTwo &&
            this.masterBootRecord[pointerOSSignatureOne] !== 0x00 &&
            this.masterBootRecord[pointerOSSignatureTwo] !== 0x00
        )
    }
}

Finally we can create a custom const for a “Windows” master boot record (if you haven’t noticed yet my simulated OS working title is Bindows XD).

windows-master-boot-record.ts

src/firmware/storage/windows-master-boot-record.ts

import { MasterBootRecord } from './master-boot-record.class'
import { osSignatureBindowsXDOne, osSignatureBindowsXDTwo } from './const'

export const windowsMasterBootRecord = new MasterBootRecord({
    partitionType: 'ntfs',
    sectorSize: 512,
    osSignatureOne: osSignatureBindowsXDOne,
    osSignatureTwo: osSignatureBindowsXDTwo,
})

GPU Firmware (Option ROM)

Now that I have explained the logic behind how pointers and data is set when dealing with storage - I can expand on the GPU’s role. For now I have decided to use Angular as a “Graphical Device Interface Service” rather then building something custom to simulate the OS GDI Layer. There is however a crucial role a GPU can play in the boot process - mainly housing the option ROM functionality.

An option ROM allows you to use basic peripherals prior to your device loading an operating system (as an example display and Input support when configuring your BIOS). Like a Master Boot Record, an Option Rom also has a unique signature noted by the first two bytes accessed on the devices onboard storage. For our simulation the GPU houses the option ROM and the option ROM’s function is merely to be validated during POST (nothing more at this stage). This means I can come back later and expand on this idea should I decide to expand on the BIOS functionality or work on a GDI service/expand the GPU’s role and functions.

const.ts

src/firmware/gpu/const.ts

export const optionRomBootSignatureOne: number = 0x55
export const optionRomBootSignatureTwo: number = 0xaa
export const pointerOptionRomBootSignatureOne = 0
export const pointerOptionRomBootSignatureTwo = 1

option-rom.class.ts

src/firmware/gpu/option-rom.class.ts

import {
    pointerOptionRomBootSignatureOne,
    pointerOptionRomBootSignatureTwo,
    optionRomBootSignatureOne,
    optionRomBootSignatureTwo,
} from './const'

export class OptionRom {
    public optionRomData: Uint8Array

    constructor() {
        this.optionRomData = new Uint8Array(2).fill(0x00)
        this.optionRomData[pointerOptionRomBootSignatureOne] =
            optionRomBootSignatureOne
        this.optionRomData[pointerOptionRomBootSignatureTwo] =
            optionRomBootSignatureTwo
    }

    public validateOptionRom(): boolean {
        if (!this.optionRomData) return false
        return (
            this.optionRomData.length === 2 &&
            this.optionRomData[pointerOptionRomBootSignatureOne] ===
                optionRomBootSignatureOne &&
            this.optionRomData[pointerOptionRomBootSignatureTwo] ===
                optionRomBootSignatureTwo
        )
    }
}

Assembled PC

Now we have our building blocks to build a PC ready to be inspected by a BIOS function. Below is what all the parts together look like. First I created a type interface which allocated compatibility for each slot available in a fully constructed PC

types.ts

src/simulated-hardware/pc-builds/types.ts

import { RAM } from '../ram/ram.class'
import { StorageDevice } from '../storage/storage-device.class'
import { PCIeDevice } from '../motherboard/types'

export interface FullSystem {
    dimSlotOne: RAM | null
    dimSlotTwo: RAM | null
    dimSlotThree: RAM | null
    dimSlotFour: RAM | null
    ideOne: StorageDevice | null
    ideTwo: StorageDevice | null
    sataOne: StorageDevice | null
    pcieOne: PCIeDevice | null
    pcieTwo: PCIeDevice | null
}

Following this a helper function was created to return a default system spec - for now this is functional. I will move this to a class at a later stage with the ability to choose multiple builds for testing.

src/simulated-hardware/pc-builds/pc-builds.ts

import { GPU } from '../gpu/gpu.class'
import { RAM } from '../ram/ram.class'
import { StorageDevice } from '../storage/storage-device.class'
import { MotherBoard } from '../motherboard/motherboard.class'
import { windowsMasterBootRecord } from '../../firmware/storage/windows-master-boot-record'
import { OptionRom } from '../../firmware/gpu/option-rom.class'
import type { FullSystem } from './types'

export function defaultSystemSpecs(): FullSystem {
    const ram1 = new RAM(64)
    const gpu = new GPU(64)
    const ideone = new StorageDevice('IDE_HDD_ONE', 64)
    const windowsMBR = windowsMasterBootRecord
    const gpuOptionRom = new OptionRom()
    if (windowsMBR.validateMasterBootRecord()) {
        // represent kb as bytes for memory allocation
        ideone.allocateMemory(windowsMBR.masterBootRecord.length / 1024)
        ideone.writeMemory(0, windowsMBR.masterBootRecord)
    }
    if (gpuOptionRom.validateOptionRom()) {
        // represent kb as bytes for memory allocation
        gpu.allocateMemory(gpuOptionRom.optionRomData.length / 1024)
        gpu.writeMemory(0, gpuOptionRom.optionRomData)
    }
    const pc = new MotherBoard({
        dimSlotOne: ram1,
        pcieOne: { type: 'gpu', profile: gpu },
        ideOne: { type: 'storage', profile: ideone },
    })
    return pc.currentConfig()
}

As you can see the RAM, GPU, Storage, Option ROM and Windows MBR classes are instantiated. A validation of the option ROM and MBR then allocate and write memory to storage and GPU. Resulting in the return of a constructed PC via the MotherBoard class.

Unit Testing Examples

I mentioned earlier following a test driven development model for each of these components iterating on functionality by building to pass tests. This methodology has not only been useful in accounting for edge cases early but it has also been an invaluable tool when I refactored code or separated concerns helping me rapidly find variables/changes that may have causes chain reactions across my project where there were joint dependencies. Below I have included some simulated hardware tests as an example. I have made it a a point to go back and ensure these tests are updated to reduce any burdens caused by refactoring at a later stage.

gpu.test.ts

src/simulated-hardware/__tests__/gpu.test.ts

import { GPU } from '../gpu/gpu.class'
/// <reference types="jest" />

describe('GPU', () => {
    let simulatedGPU: GPU
    beforeEach(() => {
        simulatedGPU = new GPU(64)
    })

    it(`Should return the hardware specifications as per it's constructor correctly`, () => {
        expect(simulatedGPU.totalMemoryKB).toBe(64)
        expect(simulatedGPU.availableMemoryKB).toBe(64)
        expect(simulatedGPU.memoryContent.length).toBe(64 * 1024)
    })
    it('Should allocate and return the first "byte address" being used', () => {
        expect(simulatedGPU.allocateMemory(10)).toBe(0)
        expect(simulatedGPU.allocateMemory(20)).toBe(10240)
        expect(simulatedGPU.allocateMemory(30)).toBe(30720)
    })
})

motherboard.test.ts

src/simulated-hardware/__tests__/MotherBoard.test.ts

/// <reference types="jest" />
import { MotherBoard } from '../motherboard/motherboard.class'
import { GPU } from '../gpu/gpu.class'
import { RAM } from '../ram/ram.class'
import { StorageDevice } from '../storage/storage-device.class'

describe('PC', () => {
    let simulatedPC: MotherBoard
    beforeEach(() => {})

        const ideone = new StorageDevice('IDE_HDD_ONE', 64)
        const ram1 = new RAM(64)
        const gpu = new GPU(64)
        const ideone = new StorageDevice('IDE_HDD', 64)
        const pc = new MotherBoard({
            dimSlotOne: ram1,
            pcieOne: { type: 'gpu', profile: gpu },
            ideOne: { type: 'storage', profile: ideone },
        })

        expect(pc.currentConfig()).toStrictEqual({
            dimSlotOne: ram1,
            dimSlotTwo: null,
            dimSlotThree: null,
            dimSlotFour: null,
            ideOne: ideone,
            ideTwo: null,
            sataOne: null,
            pcieOne: gpu,
            pcieTwo: null,
        })
    })

    it('should throw an error if no RAM is provided', () => {
        const pcieGPU = new GPU(64)
        const bootDriveIDE = new StorageDevice('IDE_HDD', 64)

        expect(() => {
            new MotherBoard({
                pcieOne: { type: 'gpu', profile: pcieGPU },
                ideOne: { type: 'storage', profile: bootDriveIDE },
            })
        }).toThrow('PC: Missing RAM')
    })

    it('should throw an error if no GPU is provided in PCIe slots', () => {
        const slotOneRam = new RAM(64)
        const bootDriveIDE = new StorageDevice('IDE_HDD', 64)

        expect(() => {
            new MotherBoard({
                dimSlotOne: slotOneRam,
                ideOne: { type: 'storage', profile: bootDriveIDE },
            })
        }).toThrow('PC: Missing GPU')
    })

    it('should throw an error if no storage device is provided', () => {
        const slotOneRam = new RAM(64)
        const pcieGPU = new GPU(64)

        expect(() => {
            new MotherBoard({
                dimSlotOne: slotOneRam,
                pcieOne: { type: 'gpu', profile: pcieGPU },
            })
        }).toThrow('PC: Missing Storage')
    })

    it('should allow multiple RAM and storage devices', () => {
        const ram1 = new RAM(16)
        const ram2 = new RAM(16)
        const sataDrive = new StorageDevice('SATA_SSD', 128)
        const gpu = new GPU(128)

        const pc = new MotherBoard({
            dimSlotOne: ram1,
            dimSlotTwo: ram2,
            sataOne: { type: 'storage', profile: sataDrive },
            pcieOne: { type: 'gpu', profile: gpu },
        })

        expect(pc.currentConfig()).toStrictEqual({
            dimSlotOne: ram1,
            dimSlotTwo: ram2,
            dimSlotThree: null,
            dimSlotFour: null,
            ideOne: null,
            ideTwo: null,
            sataOne: sataDrive,
            pcieOne: gpu,
            pcieTwo: null,
        })
    })
})

BIOS

At this stage I felt ready to start tackling the BIOS - at a high level my plan was to:

Take an assembled PC in its constructor
Detect components in PC at a BIOS level.
Conduct a Power on Self Test (POST) to ensure the PC would boot
Check and validate a boot signature and option ROM signature
Invoke an interrupt (19h) to boot into Kernel

Class Properties and Constructor

The BIOS class would mirror the motherboard when it comes to slot properties alongside a few tracking variables.

  public eventLog: Array<string> = [];
  public postResults: boolean = false;

  private dimSlotOne: RAM | null = null;
  private dimSlotTwo: RAM | null = null;
  private dimSlotThree: RAM | null = null;
  private dimSlotFour: RAM | null = null;
  private ideOne: StorageDevice | null = null;
  private ideTwo: StorageDevice | null = null;
  private sataOne: StorageDevice | null = null;
  private pcieOne: PCIeDevice | null = null;
  private pcieTwo: PCIeDevice | null = null;
  private bootTable: Array<StorageDevice> = [];
  private totalMemory: number = 0;
  private totalStorage: number = 0;
  private totalGpuMemory: number = 0;

The two public properties would be the eventLog and postResults with the rest being reserved for the internal state of the class.

The constructor takes an assembled PC, runs a detectDevices function followed by a postResults (Power on Self Test), if the POST fails it returns an error, if it passes it invokes Interrupt 19h (further explained below).

  constructor(assembledPC: FullSystem) {
    if (!assembledPC) {
      this.updateEventLog(`PC: Assembled PC Not Found`, true);
      return;
    }
    this.detectDevices(assembledPC);
    this.postResults = this.powerOnSelfTest();
    if (!this.postResults) {
      this.updateEventLog(
        'BIOS: POST FAILED During boot-up. System cannot proceed.',
        true,
      );
    }
    this.interrupt('19h');
  }

Not much commentary required for the detectDevice Function; it simply translates the slots that are occupied from our assembledPC to the private BIOS variables.

  private detectDevices(assembledPC: FullSystem) {
    if (assembledPC.dimSlotOne) this.dimSlotOne = assembledPC.dimSlotOne;
    if (assembledPC.dimSlotTwo) this.dimSlotTwo = assembledPC.dimSlotTwo;
    if (assembledPC.dimSlotThree) this.dimSlotThree = assembledPC.dimSlotThree;
    if (assembledPC.dimSlotFour) this.dimSlotFour = assembledPC.dimSlotFour;
    if (assembledPC.ideOne) this.ideOne = assembledPC.ideOne;
    if (assembledPC.ideTwo) this.ideTwo = assembledPC.ideTwo;
    if (assembledPC.sataOne) this.sataOne = assembledPC.sataOne;
    if (assembledPC.pcieOne) this.pcieOne = assembledPC.pcieOne;
    if (assembledPC.pcieTwo) this.pcieTwo = assembledPC.pcieTwo;
  }

BIOS Event Logging

I included a utility function to track BIOS level events. The intent is to take an event object to lift up to the Kernel (and eventually Event Manager). For now the function takes the arguments of a message and isError. Pushing to our public Array<string> typed eventLog a message entry.

When the function is called a new Date object with the current system time is broken into a string representation of hours, minutes and seconds, padding the start with two zeros to create a time stamp. We then concatenate our timestamp with the event message to update our event log entry. The isError is used to display our message in the DOM as either a console.error or console.log.

  public updateEventLog(message: string, isError?: boolean): Array<string> {
    const date = new Date();
    const hours = String(date.getHours()).padStart(2, '0');
    const minutes = String(date.getMinutes()).padStart(2, '0');
    const seconds = String(date.getSeconds()).padStart(2, '0');
    const timeStamp = `[${hours}:${minutes}:${seconds}]`;
    const concatEventMessage = `${timeStamp} ${message}`;
    this.eventLog.push(concatEventMessage);
    if (isError) {
      console.error(concatEventMessage);
    } else {
      console.log(concatEventMessage);
    }
    return this.eventLog;
  }

Power on Self Test (POST)

The Power on Self Test is responsible for verifying if the hardware is “functioning”. The function is private to the class and starts with an updateEventLog function call.

this.updateEventLog('BIOS: Starting Power-On Self-Test (POST)...')
let overallPostPassed = true
let atLeastOneGPUFound = false
let atLeastOneBootDriveFound = false

For the simulation the POST begins with a overallPostPassed boolean set to true, I also created a variable for atLeastOneGPUFound and atLeastOneBootDriveFound. Whilst technically both of these flags are not required for an actual POST function. I went with integrating these two “checks” into the POST to simulate a verifying an option ROM and bootloader. Should I decide to research bootloaders in the future I can easily create a class that sits between our BIOS and Kernel that can act as an additional interface.

Setting some constants helped maintain modularity when testing various slot without having to repeat code. Using the structure of an object entry inside an array I set up storageSupporingSlots, gpuSupportingSlots and ramSupportingSlots.

I also accounted for scenarios where a GPU or a StorageDevice could be using a PCIE slot by using typescripts instanceof type check to determine, and return, the matching class ensuring edge cases were covered.

const storageSupportingSlots = [
    { name: 'IDE ONE:', device: this.ideOne },
    { name: 'IDE TWO:', device: this.ideTwo },
    { name: 'SATA ONE:', device: this.sataOne },
    {
        name: 'PCIE ONE:',
        device: this.pcieOne instanceof StorageDevice ? this.pcieOne : null,
    },
    {
        name: 'PCIE TWO:',
        device: this.pcieTwo instanceof StorageDevice ? this.pcieTwo : null,
    },
]
const gpuSupportingSlots = [
    {
        name: 'PCIE ONE:',
        device: this.pcieOne instanceof GPU ? this.pcieOne : null,
    },
    {
        name: 'PCIE TWO:',
        device: this.pcieTwo instanceof GPU ? this.pcieTwo : null,
    },
]
const ramSupportingSlots = [
    { name: 'DIM SLOT ONE:', device: this.dimSlotOne },
    { name: 'DIM SLOT TWO:', device: this.dimSlotTwo },
    { name: 'DIM SLOT THREE:', device: this.dimSlotThree },
    { name: 'DIM SLOT FOUR:', device: this.dimSlotFour },
]

RAM Testing

I was now able to construct a for loop that would take the array and run the right battery of tests based on “slot type” (this pattern is repeated for GPU and Storage). As the class slot properties are constructed with either null or a valid object the property would be mirrored for slotInfo.device reducing operations and only testing on slots that are “occupied”.

For RAM a postTestMemory is first called returning a boolean value followed by adding to the totalMemory of the bios class alongside an update to the overallPostPassed based on a combination of the existing boolean and ramTestPassed.

for (const slotInfo of ramSupportingSlots) {
    if (slotInfo.device) {
        const ramTestPassed = this.postTestMemory(
            slotInfo.name,
            slotInfo.device
        )
        this.totalMemory += slotInfo.device.availableMemoryKB
        overallPostPassed = overallPostPassed && ramTestPassed
    }
}

The postTestMemory function is used in both for loops for ramSupportingSlots and storageSupporingSlots. It initially logs an event message followed by a isFunctioning check. Once complete it ensures the memory modules available memory is above 0 before returning true for the test being passed.

  private postTestMemory(slot: string, detectedMemoryModule: RAM): boolean {
    this.updateEventLog(`POST: ${slot} Testing Memory.`);
    if (!this.isFunctioning(detectedMemoryModule, slot)) {
      return false;
    }
    if (detectedMemoryModule.totalMemoryKB <= 0) {
      this.updateEventLog(
        `POST: ${slot} ${detectedMemoryModule.constructor.name} Zero Memory  Detected.`,
        true,
      );
      return false;
    }
    this.updateEventLog(
      `POST: ${slot} ${detectedMemoryModule.constructor.name}: ${detectedMemoryModule.constructor.name} Initialised.`,
    );
    return true;
  }

isFunctioning is purely decorative - we don’t really need this in typescript. We are just simulating that the slot is powered and returning a 1 or a 0 basically.

  private isFunctioning(
    testModule: RAM | StorageDevice | GPU | PCIeDevice | null,
    slotName: string,
  ): boolean {
    if (!testModule) {
      this.updateEventLog(`POST: ${slotName} Hardware Detection issue.`, true);
      return false;
    }
    this.updateEventLog(
      `POST: ${slotName} ${testModule.constructor.name} detected.`,
    );
    return true;
  }

GPU Testing

Again the for loop cycles through gpuSupporingSlots this time calling the postTestGPU function.

for (const slotInfo of gpuSupportingSlots) {
    if (slotInfo.device) {
        const gpuTestPassed = this.postTestGPU(slotInfo.name, slotInfo.device)
        overallPostPassed = overallPostPassed && gpuTestPassed
        this.totalGpuMemory += slotInfo.device.availableMemoryKB
        if (gpuTestPassed) {
            atLeastOneGPUFound = true
        }
    }
}

As relayed earlier to reduce complexity around deploying a full bootloader the POST relies on a GPU not only being loaded but also including a valid option rom signature within its first two bytes.

  private postTestGPU(slot: string, gpuModule: GPU): boolean {
    this.updateEventLog(`POST: ${slot} Testing GPU.`);
    if (!this.isFunctioning(gpuModule, slot)) {
      return false;
    }
    const gpuSignature = gpuModule.readMemory(0, 2);
    if (
      gpuSignature[0] !== optionRomSignatureOne ||
      gpuSignature[1] !== optionRomSignatureTwo
    ) {
      this.updateEventLog(
        `POST ERROR: ${slot} GPU Option ROM Signature Not Found.`,
        true,
      );
      return false;
    } else {
      this.updateEventLog(`POST: ${slot} GPU: Option Rom Signature Found.`);
    }
    this.updateEventLog(`POST: ${slot} GPU: GPU Initialised.`);
    return true;
  }

Storage Testing

The final for loop cycle continues with the same tests on storage with an additional check to add devices to a bootTable if the check returns true on a bootSignature

for (const slotInfo of storageSupportingSlots) {
    if (slotInfo.device) {
        const storageTestPassed = this.postTestStorage(
            slotInfo.name,
            slotInfo.device
        )
        if (slotInfo.device.availableMemoryKB) {
            this.totalStorage += slotInfo.device.availableMemoryKB
        }
        overallPostPassed = overallPostPassed && storageTestPassed
        if (storageTestPassed) {
            const isBootable = this.checkBootSignature(
                slotInfo.name,
                slotInfo.device
            )
            if (isBootable) {
                this.bootTable.push(slotInfo.device)
                atLeastOneBootDriveFound = true
            }
        }
    }
}

and the checkbootSignature function similar to our GPU check returns true if the MBR sectors last two bytes return a boot signature (0x55, 0xaa).

  private checkBootSignature(
    slot: string,
    detectedStorageModule: StorageDevice,
  ): boolean {
    this.updateEventLog(
      `POST: ${slot} ${detectedStorageModule.hardwareProfile} Drive: Searching for Boot Signature in Storage Device.`,
    );
    const bootSignature = detectedStorageModule.readMemory(0, 512);
    if (
      bootSignature[510] !== bootSignatureOne ||
      bootSignature[511] !== bootSignatureTwo
    ) {
      this.updateEventLog(
        `POST: ${slot} ${detectedStorageModule.hardwareProfile} Drive: Boot Signature not found.`,
      );
      return false;
    }
    this.updateEventLog(
      `POST: ${slot} ${detectedStorageModule.constructor.name}: Boot Signature Found.`,
    );
    return true;
  }
}

Finalising POST

The last step in POST is a boolean return with a final update to our event log assuming all the tests have passed. This marks the conclusion of the POST function with our Interrupt19h being called next in the constructor should POST be successful.

overallPostPassed =
    overallPostPassed && atLeastOneBootDriveFound && atLeastOneGPUFound
this.updateEventLog('BIOS: POST Completed successfully...')
return overallPostPassed

Interrupts

Currently the only Interrupt I have implemented in 19h. Interrupt calls are invoked by operating systems and application programs as an interface to firmware. For our example we are using the 19h call to load our “bootloader”. As mentioned earlier as we are going straight from BIOS to Kernel our 19h call in this example will instantiate a new Kernel object. Using a switch case I have set the foundation for future interrupts with relevant event logging and comments as a reminder of the functionality of each call.

  private interrupt(
    interuptVector: string,
    device?: StorageDevice | GPU | RAM | null,
  ): void {
      case '12h':
        this.updateEventLog('Interrupt 12h invoked - Returning Memory Size...');
        break;
      case '13h01h':
        this.updateEventLog('Interrupt 13h01h invoked - Check Drive Status...');
        break;
      case '13h02h':
        this.updateEventLog('Interrupt 13h02h invoked - Read Sectors...');
        break;
      case '13h03h':
        this.updateEventLog('Interrupt 13h03h invoked - Write Sectors...');
        break;

      // Implemented Interrupt to find bootloader.
      case '19h':
        this.updateEventLog(
          'Interrupt 19h: Invoked - Searching for Boot Table...',
        );
        if (this.bootTable.length > 0) {
          this.updateEventLog(
            'Interrupt 19h: Boot Table Found - Searching for OS...',
          );
          this.interrupt19h();
        }
        break;

      default:
        break;
    }
  }

Finally the interrupt19h function will check the OS Signature and return a new kernel object accordingly (The return has not been implemented yet). This structure allows for me to create support for other operating systems in the future.

  private interrupt19h() {
    for (const bootDrive of this.bootTable) {
      this.updateEventLog(
        `Interrupt 19h: Checking OS Signature on ${bootDrive.constructor.name}`,
      );
      const osSignatureAddress = bootDrive.readMemory(328, 2);
      if (
        osSignatureAddress[0] === osSignatureOneBindows &&
        osSignatureAddress[1] === osSignatureTwoBindows
      ) {
        this.updateEventLog(
          `Interrupt 19h: OS Signature on ${bootDrive.constructor.name} matches BindowsXD`,
        );
      }
    }
  }

Conclusion and Next Phase

I hope you enjoyed part one of my journey so far. This project has helped me really think outside the box, it has driven me to change and adapt things to match a paradigm and constraints. At this stage I am working on a NT Architecture kernel splitting objects and functionality between User and Executive mode with various services which will make up the core of the operating system being orchestrated accordingly. In part two (once I finish most of the kernel level structure and code) I will provide a lot more insight into the NT Architecture and the challenges I faced implementing my simulated approach.

React Router, Vite and JWT Authentication

Tue, 18 Mar 2025 00:00:00 GMT

This is meant to act as a summary and reference for implementing React Router with hashed authentication in your Vite project. The project is made as a Single User Login using an .env file as basis to store credentials. It can be modified for multiple use cases.

What is React Router?

Well according to React Router themselves.

React Router is a multi-strategy router for React bridging the gap from React 18 to React 19. You can use it maximally as a React framework or as minimally as you want. Source

The site goes on to explain there are three modes available. For the purposes of this exercise we are primarily focused on declarative components.

I recommend reading the full documentation on React Router should additional requirements pop up.

Important Lessons

As part of this exercise, the most important lesson to remember is the delineation between client and server. As we are using stored credentials, any interaction with those credentials needs to:

Ensure that the communication of those credentials is encrypted.
Those credentials are not validated or exposed at a client level.
Restrict access to the sever when it comes to matters related to credentials is critical.

Prerequisites

This article assumes you are familiar with Node, React, and have deployed a Vite project as the basis for your web application. The code provided is written in TypeScript.

File Setup

Below are the main files we will be working with in your pre installed Vite folder structure.

/root-project
├── /server
│   ├── server.js
│   ├── .env
├── /src
│   ├── App.tsx
│   ├── /utils
│   │   └── ProtectedRoutes.tsx
│   ├── /components
│   │   ├── Dashboard.tsx
│   │   ├── Login.tsx
│   │   └── Home.tsx

Dependencies

The following dependencies are used in this project across both client and server (break down to follow):

"vite": "^6.2.2",
"typescript": "~5.7.2",
"react-dom": "^19.0.0",
"react-router-dom": "^7.3.0",
"cors": "^2.8.5",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"jose": "^6.0.10"

Client

These modules should be targeting your Vite project folder, ensure relevant package.json folders are updated with the correct dependencies.

The below NPM command’s can be utilised to install relevant dependencies client side:

Vite To create a new Vite project with React and TypeScript template:

npm create vite@latest <YOUR_PROJECT_NAME> -- --template react-ts

react-dom To install react-dom:

npm install react-dom

react-router-dom To install react-router-dom (for routing):

npm install react-router-dom

Server

These modules should be targeting your server folder, to have a separate set of modules within your Vite project structure it is recommended to run the npm init -y command in your server’s root directory prior to executing additional install commands.

The below NPM command’s can be utilised to install relevant dependencies server side:

dotenv To install dotenv (hold our server side secrets):

npm install dotenv

cors To install cors (allows for cross origin traffic filterting on our server):

npm install cors

express To install express (to host our server):

npm install express

jose To install jose (JWT utility):

npm install jose

Summary

Client Dependencies: Make sure you’re installing the relevant modules (vite, react-dom, and react-router-dom) in the /src folder of your Vite project.
Server Dependencies: Use npm init -y to initialize a package.json in your /server folder. Then, install the necessary server-side modules (dotenv, cors, express, and jose) there.
Environment Configuration: Ensure that your .env file is placed in the /server folder for storing sensitive information like SECRET_KEY and other configuration details.

Server Configuration

/server/.env

The user management side of things will be extremely simple, whilst a more complex system can be applied to manage multiple users authenticating. This article will focus on a single-user login using an .env file. The methodology applied is still very relevant as we are managing an Authenticated user with local storage authentication and routing accordingly. The same logic can be applied to a validated SQL Row as an example.

If you are unfamiliar with .env files and the significant role they may play in your application, I would recommend reading this article alongside doing some additional research.

Below is an example .env file to examine:

ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin
SECRET_KEY=changeme

We have three variables being stored: a username, password and secret which our application will interface with to verify a successful login.

server/server.js

Let’s dive into our server.js code:

The comment’s should provide a high level insight.

// We are importing the node installed components as a first step.
const express = require('express')
const { SignJWT, jwtVerify } = require('jose')
const cors = require('cors')
require('dotenv').config()
/* Setting our parameters for the server including variables assigned
to run our application.*/

const app = express()
/* Whilst this step is optional, it is highly recommended to set cross-origin
rules to control which origins can access your API and prevent unauthorised
access from untrusted networks.

Please note the origin and configure it accordingly for production builds.
It’s recommended to use an environment variable to store your project’s root
domain, ensuring consistency across different environments and potential
variables. */
const corsOptions = {
    origin: 'http://localhost/',
    methods: ['GET', 'POST'],
    allowedHeaders: ['Content-Type', 'Authorization'],
}
// Advising the application to use CORS and express
app.use(cors(corsOptions))
app.use(express.json())
// Setting our variables from our .env file
const SECRET_KEY = process.env.SECRET_KEY
const ADMIN_USERNAME = process.env.ADMIN_USERNAME
const ADMIN_PASSWORD = process.env.ADMIN_PASSWORD
// This is our login API request (more details below)
app.post('/api/login', async (req, res) => {
    const { username, password } = req.body
    if (username === ADMIN_USERNAME && password === ADMIN_PASSWORD) {
        const payload = { username }
        try {
            const token = await new SignJWT(payload)
                .setProtectedHeader({ alg: 'HS256' })
                .setIssuedAt()
                .setExpirationTime('1h')
                .sign(new TextEncoder().encode(SECRET_KEY))
            res.json({ token })
        } catch (err) {
            console.error(err)
            res.status(500).json({ message: 'Internal server error.' })
        }
    } else {
        res.status(401).json({ message: 'Invalid username or password.' })
    }
})
/*This is our JWT token verification middleware function (it checks the
validity of the token for accessing protected routes).*/
const verifyToken = async (req, res, next) => {
    const token = req.headers['authorization']?.split(' ')[1]
    if (!token) {
        return res.status(403).json({ message: 'No token provided.' })
    }
    try {
        await jwtVerify(token, new TextEncoder().encode(SECRET_KEY))
        next()
    } catch (err) {
        console.error('Token verification failed:', err)
        res.status(401).json({ message: 'Invalid or expired token.' })
    }
}
// These are our protected routes
app.get('/api/protected', verifyToken, (req, res) => {
    res.json({ message: 'This is a protected route' })
})
/* Finally, we initialise our app by listening on port 3000 (Port can
change or be removed for cloud instances like Heroku)*/
app.listen(3000, () => {
    console.log('Server running on port 3000')
})

/api/login

app.post('/api/login', async (req, res) => {
    const { username, password } = req.body
    if (username === ADMIN_USERNAME && password === ADMIN_PASSWORD) {
        const payload = { username }
        try {
            const token = await new SignJWT(payload)
                .setProtectedHeader({ alg: 'HS256' })
                .setIssuedAt()
                .setExpirationTime('1h')
                .sign(new TextEncoder().encode(SECRET_KEY))
            res.json({ token })
        } catch (err) {
            console.error(err)
            res.status(500).json({ message: 'Internal server error.' })
        }
    } else {
        res.status(401).json({ message: 'Invalid username or password.' })
    }
})

Below is a break down of the above code:

Defining the Login Route:

A POST request is set up at the /api/login endpoint to handle login requests.

Request Body Validation:

The incoming request body is de structured to extract username and password.
These are compared with the ADMIN_USERNAME and ADMIN_PASSWORD environment variables, effectively checking credentials for a single user login.

JWT Token Creation:

If the credentials match, a JWT (JSON Web Token) is created using the username as the payload.
The token is signed using the HS256 algorithm, with the SECRET_KEY encoded to ensure secure hashing.

Token Metadata:

The token is configured with:
- setIssuedAt(): Marks the time the token was issued.
- setExpirationTime(‘1h’): Sets the token to expire after one hour.
- setProtectedHeader({ alg: ‘HS256’ }): Specifies the signing algorithm used.

Returning the Token:
- The generated JWT token is returned to the client in the response as a token object.
- This token is then used for authenticating subsequent requests via middleware to access protected routes.

verifyToken

const verifyToken = async (req, res, next) => {
    const token = req.headers['authorization']?.split(' ')[1]
    if (!token) {
        return res.status(403).json({ message: 'No token provided.' })
    }
    try {
        await jwtVerify(token, new TextEncoder().encode(SECRET_KEY))
        next()
    } catch (err) {
        console.error('Token verification failed:', err)
        res.status(401).json({ message: 'Invalid or expired token.' })
    }
}

Below is a break down of the above code:

Extracting the Token from the Authorization Header:

The middle ware first checks the Authorization header and splits it by spaces. Typically, the header will have the format Bearer <token>, so we extract the token portion after the space.

Checking if the Token Exists:

We ensure that the token exists. If it’s missing or null, the middle ware responds with a 403 Forbidden status and an error message saying “No token provided.”

Verifying the Token’s Validity:

The token is then verified using jwtVerify function, which checks the token against the SECRET_KEY. If the token is valid and the signature matches, the middleware allows the request to proceed by calling next().

Error Handling:

If there’s any issue with the token (e.g., invalid, expired, or tampered with), an error is thrown and caught returning a status of 401 Unauthorized denying access to the requested resource.

Summary

At this point, we have a server with endpoints capable of handling authentication via JWT tokens. The client can request a token by providing valid credentials, and the server will issue a signed JWT. Subsequent requests to protected routes will require the token for access, ensuring that only authenticated users can reach sensitive endpoints.

Client

App.tsx

The App.tsx file serves as a “gateway” to the different routes available in your application. Traditionally, this file contains the core logic for how a user can navigate to various features of your app. For the purposes of this article, we will implement exactly that. As suggested by the file structure, our three main landing pages will be the Home Page, Login Page, and Dashboard Page.

Let’s have a look at the file and break it down.

App.tsx

import './App.css'
import { BrowserRouter as Router, Routes, Route } from 'react-router-dom'
import Home from './components/Home'
import Login from './components/Login'
import Dashboard from './components/Dashboard'
import ProtectedRoutes from './utils/ProtectedRoutes'

function App() {
    return (
        <Router>
            <Routes>
                <Route path="/" element={<Home />} />
                <Route element={<ProtectedRoutes />}>
                    <Route path="/admin/dashboard" element={<Dashboard />} />
                </Route>
                <Route path="/admin" element={<Login />} />
            </Routes>
        </Router>
    )
}

export default App

Import Statements

import './App.css'
import { BrowserRouter as Router, Routes, Route } from 'react-router-dom'
import Home from './components/Home'
import Login from './components/Login'
import Dashboard from './components/Dashboard'
import ProtectedRoutes from './utils/ProtectedRoutes'

Here we are doing three main imports:

Global CSS: The App.css file for styling, which is standard in Vite’s React TypeScript setup.
React Router Components: We import Router, Routes, and Route from react-router-dom to enable routing functionality in the application.
Custom Components: We import the Home, Login, and Dashboard components from the components folder to render the respective pages.
ProtectedRoutes: This is a custom utility component that handles protecting certain routes, ensuring only authenticated users can access them.

Understanding BrowserRouter

Router

The <Router>is the parent element for our <Routes>, aligning location context (Determining the URL Path). It is a declarative element that can either be a StaticRouter (used for server-side rendering) or a BrowserRouter (used for client-side rendering). In this case, we have declared our router to be a BrowserRouter, as Vite is optimised for client-side rendering.

Routes

The <Routes> component serves as the parent container that holds all of our <Route> logic. It allows the <Router> to match the current URL and render the appropriate components based on the matching route.

Route

All of our customisation will be done at this level. The <Route> is pointing our router to its final destination through the path property, returning the correct component based on the element property.

JSX Return

function App() {
    return (
        <Router>
            <Routes>
                <Route path="/" element={<Home />} />
                <Route element={<ProtectedRoutes />}>
                    <Route path="/admin/dashboard" element={<Dashboard />} />
                </Route>
                <Route path="/admin" element={<Login />} />
            </Routes>
        </Router>
    )
}

export default App

Here’s a breakdown of the App.tsx JSX return:

Router Component: The <Router> wraps all of the routes to provide routing functionality based on the URL.
Routes Component: Inside the <Router>, the <Routes> component holds all the <Route> definitions.

Paths and Components:

Home Page: The first route (<Route path="/" element={<Home />} />) corresponds to the root URL (/). When a user visits the root path, the Home component is rendered.
Login Page: The second route (<Route path="/admin" element={<Login />} />) is for the login page. It matches the /admin path and renders the Login component.
Dashboard Page (Protected): The third route (<Route path="/admin/dashboard" element={<Dashboard />} />) is nested inside a ProtectedRoutes wrapper. This ensures that only authenticated users can access the dashboard page at /admin/dashboard.

ProtectedRoutes Component: The ProtectedRoutes component is used to wrap the routes that require authentication. This ensures that only authenticated users can access the /admin/dashboard route. If the user is not authenticated, they will be redirected to the login page or shown an appropriate error message.

Summary

The App.tsx file is responsible for setting up routing in our application. It uses React Router’s BrowserRouter, Routes, and Route components to render different views based on the current URL:

Home: Accessible via the root URL (/).
Login: Available at /admin.
Dashboard: A protected route at /admin/dashboard that requires authentication.

/utils/ProtectedRoutes.tsx

The ProtectedRoutes.tsx component acts as a wrapper for routes that require authentication. It checks if the user has a valid authorization token before allowing access to the child routes inside it. If the token is not valid or missing, the user is redirected to the login page.

Let’s have a look at the file and break it down:

import { Outlet, useNavigate } from 'react-router-dom'
import { useEffect } from 'react'

const ProtectedRoutes = () => {
    const token = localStorage.getItem('authToken')
    const navigate = useNavigate()
    useEffect(() => {
        const verifyToken = async () => {
            if (token) {
                try {
                    const response = await fetch(
                        'http://localhost:3000/api/protected',
                        {
                            method: 'GET',
                            headers: {
                                Authorization: `Bearer ${token}`,
                                // Send the token in the Authorization header
                            },
                        }
                    )
                    if (!response.ok) {
                        localStorage.removeItem('authToken')
                        navigate('/admin', { replace: true })
                    }
                } catch (error) {
                    console.error('Token verification failed:', error)
                    localStorage.removeItem('authToken')
                    navigate('/admin', { replace: true })
                }
            } else {
                navigate('/admin', { replace: true })
            }
        }
        verifyToken()
    }, [token, navigate])

    return token ? <Outlet /> : null
}

export default ProtectedRoutes

Import Statements

import { Outlet, useNavigate } from 'react-router-dom'

We are not introduced to two new imports from react-router-dom:

useNavigate: The useNavigate function sends a navigation request when triggered, as of writing this article this is the preferred method of navigation suggested by React Router.
Outlet: The Outlet element returns nested Routes of the parent component.

ProtectedRoutes() function

const ProtectedRoutes = () => {
    const token = localStorage.getItem('authToken')
    const navigate = useNavigate()
    useEffect(() => {
        const verifyToken = async () => {
            if (token) {
                try {
                    const response = await fetch(
                        'http://localhost:3000/api/protected',
                        {
                            method: 'GET',
                            headers: {
                                Authorization: `Bearer ${token}`,
                            },
                        }
                    )
                    if (!response.ok) {
                        localStorage.removeItem('authToken')
                        navigate('/admin', { replace: true })
                    }
                } catch (error) {
                    console.error('Token verification failed:', error)
                    localStorage.removeItem('authToken')
                    navigate('/admin', { replace: true })
                }
            } else {
                navigate('/admin', { replace: true })
            }
        }
        verifyToken()
    }, [token, navigate])
    return token ? <Outlet /> : null
}

Below is a break down of the above code:

Setting Token and Navigation Variables:

The function will be relying on a locally stored token from localStorage to determine the users authorisation status.
The function needs a navigate variable to trigger useNavigate

Token Validation:

The useEffect hook is responsible for the validation based on changes to token or navigate
We ensure that the token exists prior to validation attempts.
The function then try’s the ap/protected endpoint where we have written our token verification function server side.
Should the signatures match and the token is deemed valid the router allows for children via the <Outlet/> to be rendered

Error Handling:

If there’s any issue with the token (e.g., invalid, expired, or tampered with), an error is thrown any instance of the local token is deleted and the user is returned to the unprotected /admin page.

The Dashboard and Home components are quite open-ended. You can include whatever JSX you are comfortable with. The only suggestion I would make is including a <button> component (or something similar) in the Dashboard to allow the user to log out.

/components/Login.tsx

import { useState } from 'react'
import { useNavigate } from 'react-router-dom'

function Login() {
    const [username, setUsername] = useState('')
    const [password, setPassword] = useState('')
    const [loading, setLoading] = useState(false)
    const [error, setError] = useState('')
    const nav = useNavigate()
    const checkLogin = async (e: React.FormEvent) => {
        e.preventDefault()
        setLoading(true)
        try {
            const response = await fetch('http://localhost:3000/api/login', {
                method: 'POST',
                headers: {
                    'Content-Type': 'application/json',
                },
                body: JSON.stringify({ username, password }),
            })
            if (response.ok) {
                const { token } = await response.json()
                localStorage.setItem('authToken', token)
                nav('/admin/dashboard', { replace: true })
            } else {
                const { message } = await response.json()
                setError(message || 'Invalid username or password.')
            }
        } catch (err) {
            setError(
                `An error occurred while trying to log in. Please try again later. ${err}`
            )
        } finally {
            setLoading(false)
        }
    }
    return (
        <div>
            <h1>Login</h1>
            <form onSubmit={checkLogin}>
                <label>Username:</label>
                <input
                    type="text"
                    onChange={(e) => setUsername(e.target.value)}
                    required
                />
                <label>Password:</label>
                <input
                    type="password"
                    onChange={(e) => setPassword(e.target.value)}
                    required
                />
                <button type="submit">
                    {loading ? 'Signing in...' : 'Sign In'}
                </button>
            </form>
            {error && <span className="text-red-500">{error}</span>}
        </div>
    )
}
export default Login

The above code should now be familiar and fairly easy to follow. If you have worked with React Query (Tanstack) before or any other API try/catch style requests, then it is clear we are pinging our API endpoint and validating a login form, returning a local storage item.

/components/Dashboard.tsx

import { useNavigate } from 'react-router-dom'
function Dashboard() {
    const navigate = useNavigate()
    const handleLogout = () => {
        localStorage.removeItem('authToken')
        navigate('/admin', { replace: true })
    }
    return (
        <div>
            Dashboard
            <button onClick={handleLogout}>Logout</button>
        </div>
    )
}

export default Dashboard

As previously mentioned, including a handleLogout function will allow the user to return to a logged-out state. Here, you can see we are removing the authentication token from local storage, causing the user to be returned to an unauthenticated state.

Conclusion

Overall, this was a great exercise with a lot to learn about how routing works. It taught me some great techniques in dealing with the security of communicating credentials and has given me some topics to research further. I’m excited to see where this journey leads me next.

What I learned Deploying NGINX Proxy Manager

Sat, 15 Mar 2025 00:00:00 GMT

It was a beautiful morning in Melbourne. I had just gotten up preparing for my day. Today’s task was continuing a front end Vite project and reviewing some content on Data Structures and Algorithms (so, hot right now). When suddenly, a notification from one of my learning adjacent communities, unfurled a chain of events eventuating to the deployment of NGINX Proxy Manager.

Along the journey there were a few moments oh which I felt were worthy to share, not only for those who are discovering the technology for the first time, but for myself also, a place to come back and reference should I need to revise.

Before we start with the basics — throughout this process, Cloudflare as an organisation really stood out for me. It was my first time interacting with their products and I have to say kudos where kudos is due.

I am sure there are some valid criticisms out there, however based on my personal use case, there were stark contrasts compared to the alternatives.

In addition, they do a tremendous job in providing documentation and education on networking related topics (a page taken out of the book of Cisco no doubt.)

What is a proxy Server?

The below image is a great representation of how a proxy server helps collate and add services related to traffic flowing out of a private network.

As an example, this could be a school using web filtering to block access to content, an anonymiser to protect the identity of a user, or even a tool to help circumvent institutional browsing restrictions.

Image Source: https://www.cloudflare.com/en-au/learning/cdn/glossary/reverse-proxy/

“A forward proxy, often called a proxy, proxy server, or web proxy, is a server that sits in front of a group of client machines. When those computers make requests to sites and services on the Internet, the proxy server intercepts those requests and then communicates with web servers on behalf of those clients, like a middleman.” — Source

What is a reverse proxy?

As you may have guessed, a reverse proxy sits in front of web servers and controls how client requests reach them, rather than controlling outbound client traffic like a forward proxy. (the two are not mutually exclusive and can work in tandem).

Image Source: https://www.cloudflare.com/en-au/learning/cdn/glossary/reverse-proxy/

“A reverse proxy is a server that sits in front of web servers and forwards client (e.g. web browser) requests to those web servers “ – Source

Why have a reverse proxy can’t you just link directly to a server?

Reverse proxies are typically implemented to help increase security, performance, and reliability. Yes, you can link directly to a server however this practice comes with many risks including (and not limited to) making it easier for threat actors seek holes in your server’s security.

NGINX

As the title of this article suggests we are primarily looking at NGINX Proxy Manager — NGINX (Pronounced Engine X [or _Ehn Jinx if you are cool like me and mispronounce GIF as well]_). Is one of the many unsung heroes of web, helping serve you web content since 2004.

Whilst most of us browse the internet with a Windows device. A lot of the servers are running on open-source Linux based applications and operating systems.

One thing I do want to note, which is important to understanding the nature of proxy managers and effectively can be applied to any application, a mantra I regularly chant for software I’m developing, implementing or running.

“Use the right tool for the right job”

NGINX Proxy Manager is primarily used for web proxy filtering and web traffic. It is not a tool to manage proxies into non web traffic servers (gaming servers as an example) — There are other ways to achieve that.

NGINX is a collection of web server applications which includes reverse proxy, load balancing, caching and HTTPS. Like most things popular and open-source, it has a passionate community of contributors maintaining and managing the repository inspiring over 7,000 forks (potential offshoots) and over 26,000 stars (the GitHub term for likes). It also serves as the core to the NGINX Proxy Manager.

What are the prerequisites (loosely).

There are a few ways to approach the next steps — so I won’t go into detail, rather I will provide guidance and general advice on how it may work for you.

Hardware — As the images in the introduction suggest, whether it’s in the public cloud or you are self-hosting. To set up a proxy server effectively you need a minimum of two servers. One acting as your reverse proxy device and one that is hosting your web services/applications. You can technically utilise the same server to run a proxy manager and web apps however this will detract from the security benefits provided.

2. The ability to run Linux on the hardware.

3. Some understanding of Docker technology — will go a long way in simplifying this process. My deployment utilised Docker to deploy a container running NGINX Proxy Manager which had the service up and running in no time. (See docker documentation if you are unsure on what Docker)

4. A domain — that will act as the address to your server.

5. Some (Not necessarily in depth) networking knowledge — At a minimum understanding what purpose IP Addresses, Ports and Firewalls serve and how you could configure those particularly on your host machine.

6. If you are self-hosting — policies you should implement on your host when it comes to tunnelling to your proxy server along side firewall rules, this is very important in managing what is exposed to the rest of your network.

How I implemented the stack to work for me.

Again the reason I have titled this section this way is because everyone’s configurations will differ slightly, in my scenario I was hosting my proxy manager on a VPS (Virtual Private Server) and pointing it to another VPS. More commonly the manager is used to tunnel to a server that is self-hosted to ensure host IP is obfuscated reducing risk to an administrator’s home network.

Virtual Private Server

Screenshot of the Binary Lane Website

The first thing I needed was a VPS. This wasn’t my first time requiring such a service and as I am running a developer environment where things aren’t necessarily going to be permanent, I decided to use a great Australian service provider I have used in the past called BinaryLane.

With some of the VPS options starting at $4 a month it’s just a no brainer to me. I’m sure there’s plenty out there this just comes down to personal choice and what features are important for you. As a side note the proxy manager on it’s own has very minimal hardware requirements and can even run on a Raspberry Pi.

Docker

Once our Ubuntu server was up and running in the cloud, we immediately had the option to remotely connect and work on it just like you would a virtual or local server. I ran your regular Linux upgrade, install get commands to ensure everything was up to date and proceeded to install docker as it would be the primary method to install/manage/maintain and deploy images for me.

A screenshot of Docker Documenation

From here there are multiple routes available — for those more experienced with Docker following the installation guide for NGINX Proxy Manager should be self-explanatory . That said there is a more guided approach available if the task is daunting.

I know there are plenty of CLI (Command Line Interface) enthusiasts out there that despise the idea of a Graphical User Interface (NeoVIM advocacy intensifies) thankfully there are also developers who understand humans can also process things in layouts and interfaces and sometimes (hear me out), it makes it easier for us to complete/achieve things if they are laid out a certain way.

Portainer

Screenshot of Portainer

Portainer is a tool/Docker application (with a free option available) allowing you to manage/visualise and deploy docker containers through a Web GUI, you deploy it as a docker app and proceed to browse to your serverip:port to complete the setup. From there you can run the rest of your docker experience through the browser with minimal CLI interaction.

Screenshot of Portainer Settings Where App templates can be found

One of the handy features Portainer brings to the table is the integration of application templates (App Templates) making deploying your containers a breeze. Communities maintain and share beneficial templates, typically as JSON files on GitHub, allowing others to use them.

For my deployment I used the pi-hosted template which can be found here, depending on the version of NGINX Proxy Manager you seek, there may be a need for a different application template that includes your version.

Screenshot of NGINX Proxy Manager v2 with Sqlite in App Templates

Odd Portainer Issue

Side Note: As with a lot of things which are free and highly customisable I hit an “odd” issue here — depending on when you are reading this either this method is outdated, or the issue has been resolved.

That said using the above JSON (and in general any other JSON I tried) I found that there was a visual glitch which re renders my search results rather than letting me proceed with the deployment of the container — the band aid solution to this is.

1. Ensure your search results are correct, and your required container is displayed.
2. Go back to your app settings and modify your JSON address slightly (I just changed v2 to v3). This technically is not a real file from what I have gathered it removes the polling to render the app list.
3. Return to your search results and you should be able to click through to deploy container.

Screenshot of a Container Ready to install in Portainer

From here you are free to deploy the proxy manager accordingly — ensure to select the correct network when deploying. By default, the container is deployed to ports 443, 80 and 81.

Finalising NGINX Proxy Manager Installation

You will now have access to your NGINX Proxy Manager through a Web GUI on port 81 (if you are using default ports). Your initial login will be set to the default of

Email: admin@example.com
password: changeme

The first login will prompt you to change these settings immediately.

Domain Management (Cloudflare)

Image Source: cloudflare.com

Here is where our path’s may split — My scenario required me to take some additional steps to move my name servers to Cloudflare yours could be different.

If for some reason you are having trouble with your domain name service provider, whether it’s revoked access to API’s (issue I had) or something else. The good news is Cloudflare at its free tier gives you everything you need to do this step.

The domain itself is of course going to be pointed to your proxy manager. The Domain Name System (DNS) layer provided is critical to ensuring you can resolve to these services with a layer of security through the browser (Secure Sockets Layer — SSL). Yes I know it’s TLS now, trying to keep it simple.

NGINX Proxy Manager creates and authorises these SSL certificates through a service called Let’s Encrypt and if your DNS provider allows it, you can automatically update those certificates without the need to manually intervene. This would be through an application programming interface (API) managed by the provider.

Why automate? — for arguments sake let’s say your domain is foo.com, browsing to this page resolves to a basic home page running on a standard web server you are hosting.

You decide to run another web server for your blog but now you need an address to point to that server.

If the environment is setup correctly you will have the ability to make browsing to multiple routes predictable utilising an automated system that creates subdomains (blog.foo.com as an example), generates SSL certificates and manages those for the life of the configuration — steps that would be extremely tedious to manage on a regular basis as your infrastructure/services grow and change.

So, what did I do with Cloudflare?

As mentioned, I quickly realised I needed to move my nameservers to Cloudflare as I had an issue with the original provider revoking access to API’s (needed for certificate automation).

I made an account, selected the free tier and used an automated service to transfer the existing DNS records across. In my scenario I needed to update the name servers to Cloudflare for the transfer to take place. (the wizard they provide is self-explanatory).

From here there are two DNS Records I was primarily interested in — again this may be different for you.

A screenshot of the cloudflare DNS Record Management System

A Record — A stands for Address and is the most fundamental type of DNS it contains the IP address of a given domain. example, if you pull the DNS records of cloudflare.com, the A record currently returns an IP address of 104.17.210.9 (Note: A records only hold IPV4 Addresses for IPV6 you need AAAA).

CNAME Record — CNAME stands for canonical name in short these are used as an alias and commonly provided as a record for a sub domain.

Additional note in DNS records to consider — naming conventions utilise the * character as a wildcard helping route anything that might be utilised as input in place. In the below example we use the wildcard to cover sub domains.

I have put together a table further explaining how this can be implemented:

Example Domain: foo.com

With the below DNS Records including the wildcard for the second CNAME.

Example DNS Table

We would reroute any subdomain.foo.com alongside the foo.com domain itself to the IP Address of our reverse proxy server thus having all traffic to that address now managed by our reverse proxy server.

Image Source: https://www.behance.net/gallery/10500093/Dinosaur-Car-Named-Dinobeelte

One of the things I have become a lot more appreciative of is organisations that don’t maximise their profits at the cost of those who make their products more desirable. I’ll give you an example.

Let’s make up a fictitious car make and model for this analogy. The Hundord Raploster — (why not) — and let’s say the Raploster on launch had an average at best reception.

One day, a group of clever engineers realised that the Raploster had a system or mechanism that if modified, could save the average consumer half a tank of fuel every week. They build this system — believe in a philosophy that allows everyone to have access to this great technology and push it out there for people to improve on modify and use.

Hundord in it’s great wisdom sees a potential to maximise profits, the next iteration of the car now has a padlock on the mechanism that allows for you to implement this modification, They advise Raploster owners that the padlock can only be unlocked with a key magically provided by Hundord (With the payments usually being a subscription model in which Hundlord can change the padlock and make you pay more for different tiers of padlock keys!).

I think the analogy should resonate with now — simply put, don’t alienate a community of enthusiasts who make your product better for small profit gains. These communites grow demand, trust and market presence. Alienating them means losing sales as they will go straight to your competitor and give them the free hype and expertise.

On that note…

Cloudflare API (and Certbot)

Screenshot of the Cloudflare API Menu

Yes, the Cloudflare API is available to use on the free tier. It’s super simple to setup and through a few clicks you can get your API key. You can also segregate your keys to ensure interfaces are only accessible to the zones you have approved should you have multiple zones being managed.

This will serve as the interface between your proxy server and Cloudflare.

NGINX Proxy Manager

A quick reminder before we continue: As mentioned the reverse proxy method is one of many ways to add a security layer between your server and client. It does not replace other techniques such as firewall management and traffic segregation, a robust security strategy must always be considered on the host itself — food for thought when you have nodes publicly accessible.

Screenshot of the NGINX Proxy manager homepage

And with that lets return to NGINX Proxy Manager ready to finalise our routes. This journey is near its completion and if you have made it this far the good news is the last few steps are simple.

This is in no way an in-depth guide to the dashboard or the various options available, my use case was a simple redirect. That said, there are plenty of features you can explore outside of this context highly useful for different scenarios.

Screenshot of NGINX Proxy Manager SSL Certificates Page

The first step to take was to ensure we had SSL Certifications set up with Let’s Encrypt alongside a valid connection to our Cloudflare API — by now we have a key and all relevant details ready to go.

Screenshot for adding Encryption Certificates

The menu presented asks for your domains, a valid email alongside your DNS provider (in our case Cloudflare). For the domain field it is important to include both base domain (foo.com) along with your wildcard domain (*.foo.com) you must press enter after typing each domain.

I will say the UI experience is not overly intuitive the first time you use it, so these small mistakes are likely to happen.

Once you have completed this step and everything is green on your dashboard. You can now navigate to your base domain, and it should return a congratulations page with a valid SSL certificate.

The contents of this page do acknowledge that the domain points to a proxy server. This might not be something you want to advertise, thankfully within the dashboard you can change the default site through the settings tab — if setup correctly any non-routed sub domain would redirect to this default site.

Now that certifications are checked off we are ready to route some hosts!

Screenshot of proxy host popup

Head on over to your Hosts Tab>Proxy Hosts and hit the Add Proxy Host button — from here you can assign the following settings:

· A domain or subdomain — for example blog.foo.com (don’t forget to hit enter).

· The forward hostname — Address to your hosted server.

· Scheme of your host — HTTP or HTTPS.

· Security Settings provided through NGINX Proxy Manager around exploits and websockets.

· Access Lists — Restrict who can access the host.

Additionally SSL is important to configure (found in the pop up upper tabs) to avoid browser presented errors — Here you can assign your generated SSL certificate to your domain/subdomain

🎉**And with that final step completed and the save button clicked, your host should now have a reverse proxy address that can be accessed in accordance with how you have set it up!**🎉