|
12498
|
Ferrite by @MarioS271 | Stardance
|
MarioS271
|
1
|
129.0
|
0
|
[{"ship_num": 1, "ts": "2026-1 [{"ship_num": 1, "ts": "2026-10-28T15:21:25Z", "hours": 129.0, "multiplier": null, "payout": 0, "blessing": "neutral", "text": "Ferrite An OS kernel, build from the ground up to be modern, clean, optimized and do it right. Written in Rust. Ferrite aims to be a modern and optimized kernel as well as provide some advanced capabilities such as intent-aware scheduling, real-time scheduling/scheduling hints (“don’t preempt me for this long”), a namespaced VFS and more. Furthermore, Ferrite aims towards a clean design, fixing a lot of design mistakes in other kernels. How to try it out FerriteOS is a UEFI x86_64 kernel. The easiest way to try it is in QEMU. Linux Download ferrite_os-debug.iso (or ferrite_os-release.iso , but I recommend the debug one as it gives way more output) and ferrite_os-scripts.zip from the assets in the github release. Unzip the scripts and go to the Linux folder: unzip ferrite_os-scripts.zip -d ferrite_os-scripts cd ferrite_os-scripts/linux Install the dependencies. This installs QEMU system-wide (needs sudo) and downloads OVMF next to the script: ./install-deps.sh Start the ISO: ./run.sh Put the ISO next to run.sh . It picks up ferrite_os-release.iso first, then any other ferrite_os-*.iso . To choose one explicitly, pass the path, for example ./run.sh ferrite_os-debug.iso . Kernel output is printed to the terminal via the serial port. Windows Download ferrite_os-debug.iso (or ferrite_os-release.iso , but I recommend the debug one as it gives way more output) and ferrite_os-scripts.zip from the assets in the github release. Unzip the scripts and open the windows folder. Double-click install-deps.cmd . This installs QEMU via winget (needs admin approval) and downloads OVMF next to the script. Put the ISO next to run.cmd and double-click run.cmd . It picks up ferrite_os-release.iso first, then any other ferrite_os-*.iso . To choose one explicitly, run run.cmd ferrite_os-debug.iso from a terminal. Other platforms Scripts for other platforms are not available yet. You can run the ISO manually with qemu-system-x86_64 , -cdrom ferrite_os-release.iso , -m 4G and the OVMF code.fd and vars.fd files from ovmf-prebuilt attached as pflash drives. Thanks for trying out my project and have fun! :)", "repo": "https://github.com/MarioS271/ferrite_os", "demo": "https://github.com/MarioS271/ferrite_os/releases/latest"}]...
|
[{"post_id": 83748, "ts": "202 [{"post_id": 83748, "ts": "2026-10-28T14:52:36Z", "hours": 0.5033, "text": "Github Actions and try-it-out scripts I’m now finished preparing for shipping the kernel on stardance! :) Github Actions I’ve added a workflow which runs on tagged pushes in the v* format (anything starting with v ), so that a tag like v0.0.1 is recognized as release version 0.0.1. The workflow builds a debug and release ISO, and also ships scripts to easily try the kernel out. (and yes, the workflow and scripts were written with claude code, i really don’t wanna spend hours on that kinda stuff)"}, {"post_id": 83711, "ts": "2026-10-28T13:09:32Z", "hours": 1.4428, "text": "Exception Table done! Finally, bad userspace pointers don’t bring down the kernel anymore. What the Exception Table does Only a few instructions in a few places in kernel code can actually produce a pagefault which is the user’s fault, not the kernel’s. If we handle those normally, we’d panic. The exception table however gets built at compile time and contains every place in the kernel that could produce userspace-caused page faults. We simply check if the page fault happened at one of these instructions, and if it did, skip the instruction and return a BadAddress error instead."}, {"post_id": 83421, "ts": "2026-10-27T22:53:10Z", "hours": 2.5444, "text": "Memory Mapping and Paging Stuff With the stack no longer exploding, I finally got to refactoring the page fault handler. Before, it operated based on whether the fault happened in ring 3 or ring 0, when it should actually mainly distinguish between kernel and user addresses. Why distinguish between addresses instead of rings? Imagine this: we’re handling a syscall, attempting to read userspace-provided memory, and we page fault. The fault happens in ring 0 but we’re actually working with userspace memory. Now, if we handle it like a kernel page fault, where our mindset is “Either there’s a valid VMA and we can map this or there’s a kernel bug so we panic”, a malicious userspace process could just provide a wrong address and panic the whole system. Even worse: a valid pointer also brings down the system if it points into lazily allocated memory. Now, the page fault handler looks at the address (and for user address page faults also at the current ring) and decides based on that what to do. As proof that this works, the user binary now uses the stack for printing a text smiley, which works, even with the userspace stack now being lazily allocated. What’s still left to do Currently, bad user programs can still bring down the system because the page fault handler is among others still missing the so called “Exception Table” but more on that next devlog."}, {"post_id": 81884, "ts": "2026-10-22T20:46:35Z", "hours": 2.2778, "text": "whoops Turns out, when testing the new debug write syscall, it was slightly broken when passing wrong params, specifially when passing an unmapped or invalid address as a pointer to the string to write. Because of that, a page fault was triggered in the syscall, which blew up the stack, which was for some reason already filled more than halfway (around 9 KiB)?!? Side note: in a syscall handler, there are only 16 KiB of stack memory available, and therefore 9 KiB before any error handling is already kind of a lot (normally it should be less than half in this application) What was actually wrong Turns out, the error was the freaking kprint stuff. Yes, the funny kdebug!() and knotice!() logging macros.. The main issue is that they’re macros, which expand in place. Why that’s a problem In Syscall::dispatch , there’s two kdebug!() calls. That blows the method’s needed stack to almost 5 KiB for a simple dispatcher function with 10 lines, which is unacceptable. The problem is that the stack memory for all the kdebug!() writes gets reserved as soon as the method is called, and each kprint macro call has a 2 KiB buffer and more behind it that it needs. The Solution To fix this, I replaced the internal __kprint macro with a function. This means that now the needed stack for the logging action is only reserved during the logging, not during the whole parent function. By doing this, I managed to cut the debug write system call’s memory footprint from around 8 KiB to just 1.5 KiB, which is a huge improvement."}, {"post_id": 72380, "ts": "2026-09-22T19:58:13Z", "hours": 3.6453, "text": "DebugWrite syscall! The userspace process is now able to ask the kernel to write log messages! This means that finally, you can actually properly do something from the userspace program. Next, I’ll be implementing userspace page fault handling"}, {"post_id": 71895, "ts": "2026-09-21T18:32:02Z", "hours": 4.145, "text": "Syscall Dispatching, Syscall returning and Exit Syscall Finally, after all this time: User programs can now interact with the kernel!! :) Changelog Added Syscall enum which represents all possible syscalls and their respective IDs Added SyscallArgs whichs contains syscall number and args in an architecture-independent format Added SyscallResult and SyscallError Added syscall dispatching logic Fixed kernel data leaking into userspace via initial_userspace_jump by clearing CPU registers before the jump the end of stardance is approaching… i wish all of you a lot of fun for the last 9 days."}, {"post_id": 71226, "ts": "2026-09-20T14:09:53Z", "hours": 7.2311, "text": "Syscalls! Changes so far: Added gs_info struct, which saves the user stack and holds a pointer to the kernel stack Added syscall::init which enables the syscall / sysret instructions and sets up the STAR , LSTAR and SFMASK registers Added syscall_entry which builds a UserFrame (a snapshot of the CPU registers on syscall entry) and calls syscall_dispatch Added syscall_dispatch , which currently logs the received syscall number and halts (the system call return path is not implemented YET) Added UserFrame which holds a CPU register snapshot which gets taken upon entering the kernel"}, {"post_id": 68847, "ts": "2026-09-15T21:08:45Z", "hours": 7.6578, "text": "ELF Loading Refactor and improved build system After a week of basically no progress, I’ve finally finished moving the ELF init sequence from main.rs into the proper methods under the sched subsystem. I’ve also simplified and improved the build system ( build.py / docs.py ). They now use a shared lib.py to avoid code duplication. Also, build.py got a few changes: Removed build cache (cargo already does it) Swapped console restore for proper serial control char output filtering Bug Fixes The docker permissions have been fixed, as under Linux, root would own parts of the build files which is of course annoying as you can’t easily interact with those files then. The github-hosted rustdoc was failing to build as it could not find USER-BINARY , so I just faked user-binary via touch user-binary . time to go to bed yippeeeee :D"}, {"post_id": 63879, "ts": "2026-09-07T21:37:33Z", "hours": 6.1186, "text": "Finally… after all that work, we’re back!! After almost 30 hours of work over the last week, I’ve finally finished the big refactor . Changelog of the refactor Rewritten kprint backend Restructured KSTATE Hit my head on the wall A LOT Added a few new datatypes and macros Optimizations all over the place Added quite a few TODOs (more work 🥲) Restructured the source tree Changed the entrypoint (kernel now starts in an arch-specific kernel_entry , which builds a BootInfo and hands to kernel_main ) Moved the userspace/ELF stuff from main.rs into the sched subsystem Rewrote the UART driver … and so much more. and of course i organized it all into one slightly large 144 file commit :)"}, {"post_id": 62991, "ts": "2026-09-06T15:43:08Z", "hours": 9.2244, "text": "Big changes are still happening oh boy this is taking way too long Rewritten kprint kprint , the kernel print macro has been fully rewritten. The public macros have not changed, but internally, the macros now: Construct a kernel writer Call core::fmt::write on the kernel writer The kernel writer collects the text of all calls made to it using core::fmt::write in an internal buffer, and on drop, it pushes a new entry into the kernel log ringbuffer in a format that is (currently only partially) syslog-compatible. Then, the kernel writer dispatches a log header and the full log text to all active log recievers (like UART or the basic framebuffer). New Types PackedU8 : A type which wraps all the bitwise logic for packing multiple pieces of data which are less than one byte into a byte UncheckedCell : A datatype with interior mutability, which can be uninitialized via MaybeUninit . When the cargo feature debug-checks is enabled, this type also checks for double initialization and uninitialized access NicheCell : A datatype with interior mutability, which can be uninitialized via Option . This datatype is designed to be used with values that can take advantage of niche optimization (where the datatype T has a value which is guaranteed to be invalid, like a NonZeroU64 being zero) and oh boy, the big refactor: redid kernel source tree, ... commit is gonna be one hell of a mess…"}, {"post_id": 61699, "ts": "2026-09-04T18:57:23Z", "hours": 7.995, "text": "Big changes upcoming… I’ve decided to change what the kernel aims to be and how it will work. I’ve dropped full linux drop-in compatibility in favour of creating a clean, modern and optimized kernel which tries to fix design mistakes other kernels have already made. New Design The kernel will be POSIX-compatible, feature a namespaced VFS to seperate virtual files or devices like /dev/sda , /sys/class/hwmon , /proc/self/ and more from actual files. It will have namespaces like fs:/ for the actual file system, dev:/ for device files (like dev:/sda ), vdev:/ for virtual devices (like vdev:/urandom ) and more, including hw:/ for a modern hardware info interface, log:/ for a standardized logging interface and reg:/ for a global registry-like configuration system. Internally, the kernel will also feature KSTATE , a central static struct for storing everything the kernel uses. Also, ther kernel will move from strictly limine to a dynamic bootloader format, where each architecture has its own kernel_entry , which prepares the CPU and kernel into a common state (long mode, paging, …) and builds a standardized BootInfo struct, which then gets handed to kernel_main when it is called. New Source Tree I’m also currently working on massively reorganizing and cleaning up the kernel source, by for example standardizing where arch-specific code lands, categorizing and organizing the code better and more. Also, the kernel print system ( kprint ) is currently being rewritten to improve performance and integrate it into KSTATE ."}, {"post_id": 60495, "ts": "2026-09-02T21:21:59Z", "hours": 5.7089, "text": "Refactoring hell is here again! Before I start on syscalls for the userspace binary, I’ve decided to first get most TODOs done, like updating the safety comments and rustdoc, refactoring the ELF loading stuff to properly utilize KSTATE::procs , proper error delegation and more. I’ve finally finished refactoring the TSS, GDT and IDT properly by removing the struct wrapper for GDT and IDT, getting rid of all the Once s and more. Additionally, I refactored KSTATE::cpu to be an arch-independent container of arch-specific structs. holy crap ive got so much left to refactor… and the new bodycam update just came out i wanna play but am stuck doing kernel stuff and some other stuff :’)"}, {"post_id": 59290, "ts": "2026-09-01T00:14:55Z", "hours": 6.2725, "text": "@everyone … WE HAVE USERSPACE!!! 🎉 :D This is something I’ve been waiting for sooo long… a binary is actually executing in USERSPACE now (ring 3 and stuff)!!! How it works Currently, the code is extremely hacked together, or “gepfuscht” as we like to call it in Austria ;). Basically, what should be cleanly integrated with KSTATE::procs , should use a proper userspace and kernel stack and so on, doesn’t. All of this of course won’t stay like that, but it currently is like that to just get something in userspace running. The one thing that is actually done cleanly is the ELF validation and definitions, which live under src/elf . There is a struct representing the ELF header, an ELF PHDR and also an enum for ELF validation errors. All other logic was just thrown into main for now, like creating the processes address space, mapping its memory, copying the binary into the mapped memor, setting up its stack and actually jumping to it. The current testing binary Written in good old assembly of course, the test binary currently has the job to cause some sort of CPU exception, as syscalls or any other way to interact with the kernel or output something doesn’t even exist yet. The current program manages that via just running the hlt instruction, which, when run from userspace (Ring 3) causes a general protection fault ( #GP ). This is because a userspace program shouldn’t just be able to halt the CPU or disable interrupts (via the cli instruction). That would defeat the whole purpose of rings and kernel/userspace. The userspace binary currently lies under src/user-binary.asm , next to src/kernel/ . It is build via running docker exec ferrite_os /ferrite_os/build-user-binary.sh . NASM was also added to the Dockerfile to make building the assembly file possible."}, {"post_id": 58145, "ts": "2026-08-30T14:31:31Z", "hours": 4.1728, "text": "Virtual Memory Management is complete! (for now) I’ve added methods for mapping pages and VMA together. I’ve also improved the kernel VMA granularity and added a kernel paging remap step to mm_init() which applies the kernel VMA permissions to the kernel pages. This is only intermediate until I switch to the linux boot protocol. Next up, I’ll be adding a (very basic) ELF loader!"}, {"post_id": 57670, "ts": "2026-08-29T21:52:52Z", "hours": 7.9519, "text": "Lots of stuff Changelog for the last two days Fixed the clean option in build.py Add kernel VMA setup method Update rustdoc in several places Add panic types to kernel_panic output Move SIMPLE_STATE out of main Improve per-arch code layout … and an INSANE amount of optimizations, including: Marking certain code paths as cold (which signals the compiler to expect that the code path is unlikely to occur and optimize accordingly) Remove a lot of overhead introduced by Once and similar by swapping them for UnsafeCell<MaybeUninit<T>> and others and taking advantage of things like niche optimization Using bit-packing instead of seperate booleans for state tracking Next up, I’ll be doing some more work on the VMM and finally add the full mapping methods, which map pages and VMAs together."}, {"post_id": 55494, "ts": "2026-08-27T11:46:16Z", "hours": 4.5481, "text": "AddressSpace and VMA data structs done! I’ve introduced a struct called AddressSpace which contains the virtual address of the root page table (the PML4 on x86_64) and a BTreeSet of VMAs (virtual memory areas). A BTreeSet is basically a type of set which organizes data in a tree-like structure (in a B-Tree structure to be exact) with a time complexity of O(log n) for inserting, removing and searching for elements. A VMA (virtual memory area) is an area in memory with specific flags, such as READ , WRITE , EXEC or USER . One of their main uses is determining what to do on a #PF (page fault): should we allocate a page here (lazy-mapping)? Or was this truly a faulty operation like trying to execute READ -only memory? Also, I briefly attempted to switch over to the linux boot protocol, but I decided to do that sometime later, because it is slightly complicated and definitely did not break everything ;)"}, {"post_id": 54689, "ts": "2026-08-25T19:03:42Z", "hours": 1.0014, "text": "Finally finished fixing the PMM free method! After needing to quickly relearn the logic of the free method because I somehow forgot it, it finally works again! Additionally, I’ve added a little safeguard to the bootloader memory reclamation. Next, I’ll be continuing on the VMM and refactoring it to support userland page tables too. Also, I’ve added a few core::fmt traits to PhysAddr and VirtAddr and also added a constructor which creates a new address type ( PhysAddr or VirtAddr ) with an address of zero. In addition, PhysAddr got a few new getters like as_ptr or as_mut_ptr ."}, {"post_id": 54565, "ts": "2026-08-25T16:11:33Z", "hours": 8.2356, "text": "Memory Stuff and General Fixes Currently working on the PMM and VMM a little more, also cleaned up the main for better readability. I also moved the PMM and VMM into the mm subsystem of the KState struct. Additionally, I switched from rust 2021 to rust 2024, which also needed a little fixing to get working again. Why? Well to use let chains in if statements of course cause its like a little bit nicer to write! :) Also, I partially upgraded scripts/x86_64/build.py to log to a log file and restore your terminal config on termination. Currently working on getting a bug in the PMM fixed, getting bootloader memory reclaim to work and then I’ll do the VMM’s VMAs."}, {"post_id": 45979, "ts": "2026-08-12T20:18:08Z", "hours": 2.7475, "text": "VMM: Major Refactor aswell as Huge Pages! I’ve refactored the VMM, aswell as added support for huge pages (2 MiB & 1 GiB) to it. Also, I’ve added vmm-debug-logging . Also, I’ve updated the README to reflect the most up-to-date information and to look nicer."}, {"post_id": 44829, "ts": "2026-08-10T21:04:45Z", "hours": 2.47, "text": "Memory Management Updates! (oh and a logo too) VMM Changes I’ve added remap_page aswell as translate to the VMM (virtual memory manager). Also, I’ve added alignment-checking to ensure the VMM doesn’t corrupt memory when an incorrect virt/phys address is given to one of it’s methods. Additionally, I’ve cleaned up the code a little bit by making things like the raw pointer stuff more readable. LOGO!!! :) I’ve painstakingly (hope i spelled it right) put together a handmade logo for the kernel. It’s pretty simple but I still think it looks pretty nice."}, {"post_id": 42930, "ts": "2026-08-07T22:15:35Z", "hours": 3.6658, "text": "PMM Buddy Allocator done! The PMM now uses a buddy allocator which gives us many advantages, including: Tracking overhead doesn’t grow with RAM size Faster (bitmap is always O(n), buddy allocator is O(1) most times, worst case O(log n)) Order-based allocation (group a lot of free space together into one 4 MiB region for order 10 or 16 KiB for order 2 for example) Next is adding a few methods to the VMM, and then I’ll work on the heap allocator to finally be able to work on the kstate struct :)"}, {"post_id": 42031, "ts": "2026-08-06T16:07:23Z", "hours": 1.3044, "text": "funny stuff I’ve added a marker byte to the ring buffer that makes it possible to later properly parse log levels from the buffer. Also, I’ve finally figured out how buddy allocators work meaning I can start work on redoing the PMM :)"}, {"post_id": 36746, "ts": "2026-07-29T00:21:27Z", "hours": 5.9975, "text": "The refactoring hell continues… I’ve refactored PMM and VMM into SIMPLE_STATE , updated the build and docs scripts, enhanced kernel logging via log levels and more, added and fleshed out a Claude Code Skill for writing rustdoc and let Claude re-do a lot of the rust doc to reduce the over-documentation that was done by none other than… Claude. also look at this fancy new fb logging :D"}, {"post_id": 36151, "ts": "2026-07-28T01:41:46Z", "hours": 6.1836, "text": "Refactoring into oblivion! Currently working on improving the kernel a lot to make it robuster for its later stages. What was so far improved Tooling Replace loose markdown docs with proper GitHub Pages hosted rustdoc Add docs.py , refactor build.py to work on linux too Kernel Serial Logging abstracted into trait + per-arch impls Improve panic handlers to give more info and dynamically use what is avail for outputting info (basic fb, serial) Refactor basic framebuffer and PSF2 font into cleaner struct+impl approach Remove text.rs and move its logic into kprint (which has also been refactored to be concurrency-safe) Other Stuff I also tested the kernel on real hardware !! To my suprise, grabbing the ISO from the build folder, flashing it onto a USB drive and plugging it into a laptop just… WORKED?!? well i guess my kernel is real hardware certified now :) I attached a video of the kernel booting on real hardware and IRQ0 (the BIOS-configured 18.2 Hz hardware timer) printing “a” repeatedly. :)"}, {"post_id": 34885, "ts": "2026-07-25T23:28:26Z", "hours": 7.9892, "text": "Big Update :) (sorry for not doing much the past weeks had to get 4th place at the botball competition real quick) Changelog Implemented heap allocator (partially and broken) Implement PIC (Programmable Interrupt Controller) Init + get basic IRQ stuff working Fix GDT (was missing a few entries) Fix TSS (IST1-4 interrupt stacks were missing) Implement interrupts for vectors #0 through #31 Update Panic for most interrupts to give way more details (example: pagefault in attached image) Replace Mutex with IrqMutex for text writing and holding CHAINED_PICS Current State I’ve noticed that my so far work has been kinda sloppy, so I’m currently working on improving everything (as u can see on the changelist above ive already started). I’m doing this because when I started with IRQs, stuff started breaking a lot. My goal is to get everything thread-safe and ready in a way that it’ll work in later stages of the kernel too. Next up? Next, I’ll be working on fixing logging (add a ringbuffer to prevent interrupts creating half-printed or partially drawn logs)"}, {"post_id": 6510, "ts": "2026-06-10T22:49:31Z", "hours": 6.0211, "text": "Had a little fight with github actions and qemu wasm stuff. Tried to implement a live web demo using a wasm implementation of qemu with UEFI and more, but couldnt get it to run. Also, I heavily improved documentation and finished up the virtual memory manager (for now…). Plans? Next, the heap allocator is coming then I’ll finally be able to use dynamic types! was worth it i guess (it was not i gotta get up in 5h 🥲)"}, {"post_id": 5062, "ts": "2026-06-08T18:47:11Z", "hours": 1.5903, "text": "Hi! This is my “main” project, ✨FerriteOS✨! (Well, idk if you can call it a whole OS if im only planning to do a kernel, but anyways) Ferrite is a Linux-ABI compatible kernel, written entirely in Rust, from scratch. My goal My end goal is for it to be able to run systemd, GNOME and more. But thats a very, VERY distant goal. For now, im focusing on getting a statically linked C binary running in userspace. Current State I’ve already managed to get some basic logging via serial and a basic framebuffer going, aswell as x86_64 GDT, IDT, TSS and a physical aswell as a very slim virtual memory manager. Whats next? Next, I’ll be “finishing” up the VMM a little to be able to start work on the heap allocator, which will enable use of datatypes like Vec, Box and more. nerd section For anyone interested in the more technical stuff: The build toolchain works through python scripts, which use docker to build and then QEMU to run the kernel. I’m mainly targeting x86_64, but aarch64 support is also my plan (but I’ll be focusing on x86_64 during this challenge)."}]...
|
Title: Ferrite by @MarioS271 | Stardance
Devlogs: Title: Ferrite by @MarioS271 | Stardance
Devlogs:
Hi! This is my “main” project, ✨FerriteOS✨! (Well, idk if you can call it a whole OS if im only planning to do a kernel, but anyways) Ferrite is a Linux-ABI compatible kernel, written entirely in Rust, from scratch. My goal My end goal is for it to be able to run systemd, GNOME and more. But thats a very, VERY distant goal. For now, im focusing on getting a statically linked C binary running in userspace. Current State I’ve already managed to get some basic logging via serial and a basic framebuffer going, aswell as x86_64 GDT, IDT, TSS and a physical aswell as a very slim virtual memory manager. Whats next? Next, I’ll be “finishing” up the VMM a little to be able to start work on the heap allocator, which will enable use of datatypes like Vec, Box and more. nerd section For anyone interested in the more technical stuff: The build toolchain works through python scripts, which use docker to build and then QEMU to run the kernel. I’m mainly targeting x86_64, but aarch64 support is also my plan (but I’ll be focusing on x86_64 during this challenge).
Had a little fight with github actions and qemu wasm stuff. Tried to implement a live web demo using a wasm implementation of qemu with UEFI and more, but couldnt get it to run. Also, I heavily improved documentation and finished up the virtual memory manager (for now…). Plans? Next, the heap allocator is coming then I’ll finally be able to use dynamic types! was worth it i guess (it was not i gotta get up in 5h 🥲)
Big Update :) (sorry for not doing much the past weeks had to get 4th place at the botball competition real quick) Changelog Implemented heap allocator (partially and broken) Implement PIC (Programmable Interrupt Controller) Init + get basic IRQ stuff working Fix GDT (was missing a few entries) Fix TSS (IST1-4 interrupt stacks were missing) Implement interrupts for vectors #0 through #31 Update Panic for most interrupts to give way more details (example: pagefault in attached image) Replace Mutex with IrqMutex for text writing and holding CHAINED_PICS Current State I’ve noticed that my so far work has been kinda sloppy, so I’m currently working on improving everything (as u can see on the changelist above ive already started). I’m doing this because when I started with IRQs, stuff started breaking a lot. My goal is to get everything thread-safe and ready in a way that it’ll work in later stages of the kernel too. Next up? Next, I’ll be working on fixing logging (add a ringbuffer to prevent interrupts creating half-printed or partially drawn logs)
Refactoring into oblivion! Currently working on improving the kernel a lot to make it robuster for its later stages. What was so far improved Tooling Replace loose markdown docs with proper GitHub Pages hosted rustdoc Add docs.py , refactor build.py to work on linux too Kernel Serial Logging abstracted into trait + per-arch impls Improve panic handlers to give more info and dynamically use what is avail for outputting info (basic fb, serial) Refactor basic framebuffer and PSF2 font into cleaner struct+impl approach Remove text.rs and move its logic into kprint (which has also been refactored to be concurrency-safe) Other Stuff I also tested the kernel on real hardware !! To my suprise, grabbing the ISO from the build folder, flashing it onto a USB drive and plugging it into a laptop just… WORKED?!? well i guess my kernel is real hardware certified now :) I attached a video of the kernel booting on real hardware and IRQ0 (the BIOS-configured 18.2 Hz hardware timer) printing “a” repeatedly. :)
The refactoring hell continues… I’ve refactored PMM and VMM into SIMPLE_STATE , updated the build and docs scripts, enhanced kernel logging via log levels and more, added and fleshed out a Claude Code Skill for writing rustdoc and let Claude re-do a lot of the rust doc to reduce the over-documentation that was done by none other than… Claude. also look at this fancy new fb logging :D
funny stuff I’ve added a marker byte to the ring buffer that makes it possible to later properly parse log levels from the buffer. Also, I’ve finally figured out how buddy allocators work meaning I can start work on redoing the PMM :)
PMM Buddy Allocator done! The PMM now uses a buddy allocator which gives us many advantages, including: Tracking overhead doesn’t grow with RAM size Faster (bitmap is always O(n), buddy allocator is O(1) most times, worst case O(log n)) Order-based allocation (group a lot of free space together into one 4 MiB region for order 10 or 16 KiB for order 2 for example) Next is adding a few methods to the VMM, and then I’ll work on the heap allocator to finally be able to work on the kstate struct :)
Memory Management Updates! (oh and a logo too) VMM Changes I’ve added remap_page aswell as translate to the VMM (virtual memory manager). Also, I’ve added alignment-checking to ensure the VMM doesn’t corrupt memory when an incorrect virt/phys address is given to one of it’s methods. Additionally, I’ve cleaned up the code a little bit by making things like the raw pointer stuff more readable. LOGO!!! :) I’ve painstakingly (hope i spelled it right) put together a handmade logo for the kernel. It’s pretty simple but I still think it looks pretty nice.
VMM: Major Refactor aswell as Huge Pages! I’ve refactored the VMM, aswell as added support for huge pages (2 MiB & 1 GiB) to it. Also, I’ve added vmm-debug-logging . Also, I’ve updated the README to reflect the most up-to-date information and to look nicer.
Memory Stuff and General Fixes Currently working on the PMM and VMM a little more, also cleaned up the main for better readability. I also moved the PMM and VMM into the mm subsystem of the KState struct. Additionally, I switched from rust 2021 to rust 2024, which also needed a little fixing to get working again. Why? Well to use let chains in if statements of course cause its like a little bit nicer to write! :) Also, I partially upgraded scripts/x86_64/build.py to log to a log file and restore your terminal config on termination. Currently working on getting a bug in the PMM fixed, getting bootloader memory reclaim to work and then I’ll do the VMM’s VMAs.
Finally finished fixing the PMM free method! After needing to quickly relearn the logic of the free method because I somehow forgot it, it finally works again! Additionally, I’ve added a little safeguard to the bootloader memory reclamation. Next, I’ll be continuing on the VMM and refactoring it to support userland page tables too. Also, I’ve added a few core::fmt traits to PhysAddr and VirtAddr and also added a constructor which creates a new address type ( PhysAddr or VirtAddr ) with an address of zero. In addition, PhysAddr got a few new getters like as_ptr or as_mut_ptr .
AddressSpace and VMA data structs done! I’ve introduced a struct called AddressSpace which contains the virtual address of the root page table (the PML4 on x86_64) and a BTreeSet of VMAs (virtual memory areas). A BTreeSet is basically a type of set which organizes data in a tree-like structure (in a B-Tree structure to be exact) with a time complexity of O(log n) for inserting, removing and searching for elements. A VMA (virtual memory area) is an area in memory with specific flags, such as READ , WRITE , EXEC or USER . One of their main uses is determining what to do on a #PF (page fault): should we allocate a page here (lazy-mapping)? Or was this truly a faulty operation like trying to execute READ -only memory? Also, I briefly attempted to switch over to the linux boot protocol, but I decided to do that sometime later, because it is slightly complicated and definitely did not break everything ;)
Lots of stuff Changelog for the last two days Fixed the clean option in build.py Add kernel VMA setup method Update rustdoc in several places Add panic types to kernel_panic output Move SIMPLE_STATE out of main Improve per-arch code layout … and an INSANE amount of optimizations, including: Marking certain code paths as cold (which signals the compiler to expect that the code path is unlikely to occur and optimize accordingly) Remove a lot of overhead introduced by Once and similar by swapping them for UnsafeCell<MaybeUninit<T>> and others and taking advantage of things like niche optimization Using bit-packing instead of seperate booleans for state tracking Next up, I’ll be doing some more work on the VMM and finally add the full mapping methods, which map pages and VMAs together.
Virtual Memory Management is complete! (for now) I’ve added methods for mapping pages and VMA together. I’ve also improved the kernel VMA granularity and added a kernel paging remap step to mm_init() which applies the kernel VMA permissions to the kernel pages. This is only intermediate until I switch to the linux boot protocol. Next up, I’ll be adding a (very basic) ELF loader!
@everyone … WE HAVE USERSPACE!!! 🎉 :D This is something I’ve been waiting for sooo long… a binary is actually executing in USERSPACE now (ring 3 and stuff)!!! How it works Currently, the code is extremely hacked together, or “gepfuscht” as we like to call it in Austria ;). Basically, what should be cleanly integrated with KSTATE::procs , should use a proper userspace and kernel stack and so on, doesn’t. All of this of course won’t stay like that, but it currently is like that to just get something in userspace running. The one thing that is actually done cleanly is the ELF validation and definitions, which live under src/elf . There is a struct representing the ELF header, an ELF PHDR and also an enum for ELF validation errors. All other logic was just thrown into main for now, like creating the processes address space, mapping its memory, copying the binary into the mapped memor, setting up its stack and actually jumping to it. The current testing binary Written in good old assembly of course, the test binary currently has the job to cause some sort of CPU exception, as syscalls or any other way to interact with the kernel or output something doesn’t even exist yet. The current program manages that via just running the hlt instruction, which, when run from userspace (Ring 3) causes a general protection fault ( #GP ). This is because a userspace program shouldn’t just be able to halt the CPU or disable interrupts (via the cli instruction). That would defeat the whole purpose of rings and kernel/userspace. The userspace binary currently lies under src/user-binary.asm , next to src/kernel/ . It is build via running docker exec ferrite_os /ferrite_os/build-user-binary.sh . NASM was also added to the Dockerfile to make building the assembly file possible.
Refactoring hell is here again! Before I start on syscalls for the userspace binary, I’ve decided to first get most TODOs done, like updating the safety comments and rustdoc, refactoring the ELF loading stuff to properly utilize KSTATE::procs , proper error delegation and more. I’ve finally finished refactoring the TSS, GDT and IDT properly by removing the struct wrapper for GDT and IDT, getting rid of all the Once s and more. Additionally, I refactored KSTATE::cpu to be an arch-independent container of arch-specific structs. holy crap ive got so much left to refactor… and the new bodycam update just came out i wanna play but am stuck doing kernel stuff and some other stuff :’)
Big changes upcoming… I’ve decided to change what the kernel aims to be and how it will work. I’ve dropped full linux drop-in compatibility in favour of creating a clean, modern and optimized kernel which tries to fix design mistakes other kernels have already made. New Design The kernel will be POSIX-compatible, feature a namespaced VFS to seperate virtual files or devices like /dev/sda , /sys/class/hwmon , /proc/self/ and more from actual files. It will have namespaces like fs:/ for the actual file system, dev:/ for device files (like dev:/sda ), vdev:/ for virtual devices (like vdev:/urandom ) and more, including hw:/ for a modern hardware info interface, log:/ for a standardized logging interface and reg:/ for a global registry-like configuration system. Internally, the kernel will also feature KSTATE , a central static struct for storing everything the kernel uses. Also, ther kernel will move from strictly limine to a dynamic bootloader format, where each architecture has its own kernel_entry , which prepares the CPU and kernel into a common state (long mode, paging, …) and builds a standardized BootInfo struct, which then gets handed to kernel_main when it is called. New Source Tree I’m also currently working on massively reorganizing and cleaning up the kernel source, by for example standardizing where arch-specific code lands, categorizing and organizing the code better and more. Also, the kernel print system ( kprint ) is currently being rewritten to improve performance and integrate it into KSTATE .
Big changes are still happening oh boy this is taking way too long Rewritten kprint kprint , the kernel print macro has been fully rewritten. The public macros have not changed, but internally, the macros now: Construct a kernel writer Call core::fmt::write on the kernel writer The kernel writer collects the text of all calls made to it using core::fmt::write in an internal buffer, and on drop, it pushes a new entry into the kernel log ringbuffer in a format that is (currently only partially) syslog-compatible. Then, the kernel writer dispatches a log header and the full log text to all active log recievers (like UART or the basic framebuffer). New Types PackedU8 : A type which wraps all the bitwise logic for packing multiple pieces of data which are less than one byte into a byte UncheckedCell : A datatype with interior mutability, which can be uninitialized via MaybeUninit . When the cargo feature debug-checks is enabled, this type also checks for double initialization and uninitialized access NicheCell : A datatype with interior mutability, which can be uninitialized via Option . This datatype is designed to be used with values that can take advantage of niche optimization (where the datatype T has a value which is guaranteed to be invalid, like a NonZeroU64 being zero) and oh boy, the big refactor: redid kernel source tree, ... commit is gonna be one hell of a mess…
Finally… after all that work, we’re back!! After almost 30 hours of work over the last week, I’ve finally finished the big refactor . Changelog of the refactor Rewritten kprint backend Restructured KSTATE Hit my head on the wall A LOT Added a few new datatypes and macros Optimizations all over the place Added quite a few TODOs (more work 🥲) Restructured the source tree Changed the entrypoint (kernel now starts in an arch-specific kernel_entry , which builds a BootInfo and hands to kernel_main ) Moved the userspace/ELF stuff from main.rs into the sched subsystem Rewrote the UART driver … and so much more. and of course i organized it all into one slightly large 144 file commit :)
ELF Loading Refactor and improved build system After a week of basically no progress, I’ve finally finished moving the ELF init sequence from main.rs into the proper methods under the sched subsystem. I’ve also simplified and improved the build system ( build.py / docs.py ). They now use a shared lib.py to avoid code duplication. Also, build.py got a few changes: Removed build cache (cargo already does it) Swapped console restore for proper serial control char output filtering Bug Fixes The docker permissions have been fixed, as under Linux, root would own parts of the build files which is of course annoying as you can’t easily interact with those files then. The github-hosted rustdoc was failing to build as it could not find USER-BINARY , so I just faked user-binary via touch user-binary . time to go to bed yippeeeee :D
Syscalls! Changes so far: Added gs_info struct, which saves the user stack and holds a pointer to the kernel stack Added syscall::init which enables the syscall / sysret instructions and sets up the STAR , LSTAR and SFMASK registers Added syscall_entry which builds a UserFrame (a snapshot of the CPU registers on syscall entry) and calls syscall_dispatch Added syscall_dispatch , which currently logs the received syscall number and halts (the system call return path is not implemented YET) Added UserFrame which holds a CPU register snapshot which gets taken upon entering the kernel
Syscall Dispatching, Syscall returning and Exit Syscall Finally, after all this time: User programs can now interact with the kernel!! :) Changelog Added Syscall enum which represents all possible syscalls and their respective IDs Added SyscallArgs whichs contains syscall number and args in an architecture-independent format Added SyscallResult and SyscallError Added syscall dispatching logic Fixed kernel data leaking into userspace via initial_userspace_jump by clearing CPU registers before the jump the end of stardance is approaching… i wish all of you a lot of fun for the last 9 days.
DebugWrite syscall! The userspace process is now able to ask the kernel to write log messages! This means that finally, you can actually properly do something from the userspace program. Next, I’ll be implementing userspace page fault handling
whoops Turns out, when testing the new debug write syscall, it was slightly broken when passing wrong params, specifially when passing an unmapped or invalid address as a pointer to the string to write. Because of that, a page fault was triggered in the syscall, which blew up the stack, which was for some reason already filled more than halfway (around 9 KiB)?!? Side note: in a syscall handler, there are only 16 KiB of stack memory available, and therefore 9 KiB before any error handling is already kind of a lot (normally it should be less than half in this application) What was actually wrong Turns out, the error was the freaking kprint stuff. Yes, the funny kdebug!() and knotice!() logging macros.. The main issue is that they’re macros, which expand in place. Why that’s a problem In Syscall::dispatch , there’s two kdebug!() calls. That blows the method’s needed stack to almost 5 KiB for a simple dispatcher function with 10 lines, which is unacceptable. The problem is that the stack memory for all the kdebug!() writes gets reserved as soon as the method is called, and each kprint macro call has a 2 KiB buffer and more behind it that it needs. The Solution To fix this, I replaced the internal __kprint macro with a function. This means that now the needed stack for the logging action is only reserved during the logging, not during the whole parent function. By doing this, I managed to cut the debug write system call’s memory footprint from around 8 KiB to just 1.5 KiB, which is a huge improvement.
Memory Mapping and Paging Stuff With the stack no longer exploding, I finally got to refactoring the page fault handler. Before, it operated based on whether the fault happened in ring 3 or ring 0, when it should actually mainly distinguish between kernel and user addresses. Why distinguish between addresses instead of rings? Imagine this: we’re handling a syscall, attempting to read userspace-provided memory, and we page fault. The fault happens in ring 0 but we’re actually working with userspace memory. Now, if we handle it like a kernel page fault, where our mindset is “Either there’s a valid VMA and we can map this or there’s a kernel bug so we panic”, a malicious userspace process could just provide a wrong address and panic the whole system. Even worse: a valid pointer also brings down the system if it points into lazily allocated memory. Now, the page fault handler looks at the address (and for user address page faults also at the current ring) and decides based on that what to do. As proof that this works, the user binary now uses the stack for printing a text smiley, which works, even with the userspace stack now being lazily allocated. What’s still left to do Currently, bad user programs can still bring down the system because the page fault handler is among others still missing the so called “Exception Table” but more on that next devlog.
Exception Table done! Finally, bad userspace pointers don’t bring down the kernel anymore. What the Exception Table does Only a few instructions in a few places in kernel code can actually produce a pagefault which is the user’s fault, not the kernel’s. If we handle those normally, we’d panic. The exception table however gets built at compile time and contains every place in the kernel that could produce userspace-caused page faults. We simply check if the page fault happened at one of these instructions, and if it did, skip the instruction and return a BadAddress error instead.
Github Actions and try-it-out scripts I’m now finished preparing for shipping the kernel on stardance! :) Github Actions I’ve added a workflow which runs on tagged pushes in the v* format (anything starting with v ), so that a tag like v0.0.1 is recognized as release version 0.0.1. The workflow builds a debug and release ISO, and also ships scripts to easily try the kernel out. (and yes, the workflow and scripts were written with claude code, i really don’t wanna spend hours on that kinda stuff)
Ships:
Ferrite An OS kernel, build from the ground up to be modern, clean, optimized and do it right. Written in Rust. Ferrite aims to be a modern and optimized kernel as well as provide some advanced capabilities such as intent-aware scheduling, real-time scheduling/scheduling hints (“don’t preempt me for this long”), a namespaced VFS and more. Furthermore, Ferrite aims towards a clean design, fixing a lot of design mistakes in other kernels. How to try it out FerriteOS is a UEFI x86_64 kernel. The easiest way to try it is in QEMU. Linux Download ferrite_os-debug.iso (or ferrite_os-release.iso , but I recommend the debug one as it gives way more output) and ferrite_os-scripts.zip from the assets in the github release. Unzip the scripts and go to the Linux folder: unzip ferrite_os-scripts.zip -d ferrite_os-scripts cd ferrite_os-scripts/linux Install the dependencies. This installs QEMU system-wide (needs sudo) and downloads OVMF next to the script: ./install-deps.sh Start the ISO: ./run.sh Put the ISO next to run.sh . It picks up ferrite_os-release.iso first, then any other ferrite_os-*.iso . To choose one explicitly, pass the path, for example ./run.sh ferrite_os-debug.iso . Kernel output is printed to the terminal via the serial port. Windows Download ferrite_os-debug.iso (or ferrite_os-release.iso , but I recommend the debug one as it gives way more output) and ferrite_os-scripts.zip from the assets in the github release. Unzip the scripts and open the windows folder. Double-click install-deps.cmd . This installs QEMU via winget (needs admin approval) and downloads OVMF next to the script. Put the ISO next to run.cmd and double-click run.cmd . It picks up ferrite_os-release.iso first, then any other ferrite_os-*.iso . To choose one explicitly, run run.cmd ferrite_os-debug.iso from a terminal. Other platforms Scripts for other platforms are not available yet. You can run the ISO manually with qemu-system-x86_64 , -cdrom ferrite_os-release.iso , -m 4G and the OVMF code.fd and vars.fd files from ovmf-prebuilt attached as pflash drives. Thanks for trying out my project and have fun! :)...
|
1793512302.2154393
|
1793598702.2154393
|