From 5b19bf13e74b166fa94d4488a1647e9e4f6ba265 Mon Sep 17 00:00:00 2001 From: Aiken Harris Date: Sat, 14 Feb 2026 12:26:24 +0100 Subject: [PATCH] Development update --- content/news/20260214-development-update.md | 74 +++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 content/news/20260214-development-update.md diff --git a/content/news/20260214-development-update.md b/content/news/20260214-development-update.md new file mode 100644 index 0000000..e6b1f7b --- /dev/null +++ b/content/news/20260214-development-update.md @@ -0,0 +1,74 @@ ++++ +title = 'Development Update' +author = 'Aiken Harris' +date = '2026-02-14T11:17:38+01:00' ++++ +It has been several months since our last update. During this time, development of ExectOS has continued steadily, +although much of the work has taken place deep within the kernel and boot process. While the original goal of +delivering a fully functional memory manager has not yet been achieved, we believe this is a good moment to inform +the community about the progress made and the challenges addressed along the way. + + +## Focus Area: Memory Manager +Since the previous update, development has been focused exclusively on the kernel memory manager. This work required +substantial changes not only within the memory subsystem itself but also in the bootloader, where several critical +issues were discovered. + +## Bootloader Issues and Fixes +During integration work, it became clear that our custom bootloader still contained serious flaws that directly +impacted memory manager development: + * **XTLDR and XTOSKRNL incompatibility**: The bootloader and the kernel used different memory mapping strategies + across architectures. This mismatch resulted in inconsistent virtual memory layouts and prevented reliable + initialization of the memory manager. The mapping strategies have now been unified to ensure predictable behavior. + * **Incorrect identity mapping of UEFI-reserved memory**: The bootloader was identity-mapping memory regions reserved + by UEFI. This was a design error, as it unnecessarily consumed limited virtual address space. The mapping logic has + been corrected to avoid reserving virtual space for firmware-managed regions. + * **Critical mapping construction bug**: A severe issue in the bootloader's memory mapping builder caused overlapping + virtual memory regions. This made correct construction of the PFN (Page Frame Number) database impossible. The + mapping generation logic has been rewritten to guarantee non-overlapping, deterministic memory regions. + +### PAE Handling Fix (Bug #23) +We also resolved bug #23, which affected PAE support on the i686 platform. The root cause was improper handling of +memory descriptors above the 32-bit boundary. The fix ensures that all valid memory regions are correctly preserved and +passed to the kernel, allowing full utilization of available RAM under PAE. + +## Progress in the Memory Manager +Beyond bootloader fixes, significant progress has been made within the memory manager itself. + +### Dynamic Memory Layout Construction +The kernel now dynamically builds its memory layout based on the total amount of detected physical RAM. This ensures +that core memory regions, such as: PagedPool, NonPagedPool, or system structures and internal mappings are assigned +clearly defined and non-overlapping virtual address ranges. This dynamic approach improves scalability and makes the +layout adaptable to systems with varying memory configurations. + +### PFN Database Initialization +We are now successfully constructing the PFN database and mapping all memory regions required for the Memory Manager's +operation. This was a key milestone, as many higher-level memory operations depend on a correctly initialized PFN layer. + +### Stack Allocation and Deallocation +We have implemented internal routines for allocating and freeing memory intended for kernel stacks. These mechanisms +allow: + * Allocation of stacks with arbitrary sizes depending on requirements, + * Proper tracking and release of stack memory +This is a prerequisite for thread management. As a result, we can now begin work on kernel thread support. In the near +future, we plan to introduce functionality for dynamically growing existing stacks. + +## Current Work +At present, development is focused on implementing general-purpose allocation and deallocation routines for arbitrary +address pools, specifically: PagedPool and NonPagedPool. These allocators are essential for enabling further kernel +subsystems. +One of the next major items on our roadmap is SMT support. We intend to introduce it as early as possible to reduce +issues related to resource locking and race conditions. Bringing up symmetric multithreading early should help expose +and eliminate concurrency-related bugs before additional kernel complexity is introduced. + +## Summary +Although the fully functional memory manager is not yet complete, substantial foundational work has been finished: + * Critical bootloader memory mapping bugs resolved + * Proper PAE handling restored + * Dynamic kernel memory layout construction implemented + * PFN database successfully built + * Kernel stack allocation mechanisms introduced + +The project is now in a more stable and architecturally consistent state than it was before. The full and detailed +list of changes can be found in the **memmgr** branch. Further updates will follow as we continue stabilizing the +allocator infrastructure.