Compare commits
1 Commits
master
...
harraiken-
| Author | SHA1 | Date | |
|---|---|---|---|
5b29419c2d
|
@@ -4,7 +4,7 @@ title = "ExectOS Operating System"
|
||||
theme = "exectos"
|
||||
languageCode = "en-us"
|
||||
defaultContentLanguage = "en"
|
||||
ignoreErrors = ["error-disable-taxonomy"]
|
||||
paginate = 10
|
||||
|
||||
[menu]
|
||||
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
+++
|
||||
title = 'AI-Assisted Contributions'
|
||||
date = 2025-07-17T19:44:12+02:00
|
||||
+++
|
||||
This project includes content and code that has been partially created or assisted by artificial intelligence (AI) tools.
|
||||
|
||||
### Tools We Use
|
||||
We actively use the following AI tools to support development and documentation workflows:
|
||||
* **ChatGPT**: for generating and reviewing code, technical writing, and problem-solving assistance.
|
||||
* **Gemini Assistant**: integrated with VS Code, used for code suggestions and refactoring.
|
||||
* **GitHub Copilot**: provides real-time code suggestions and completions.
|
||||
* **Windsurf**: VS Code extension that helps streamline repetitive coding tasks.
|
||||
We are also experimenting with other AI-based tools to evaluate their usefulness in different areas of the project.
|
||||
|
||||
### Human Review
|
||||
All AI-assisted contributions are reviewed and validated by human developers before being committed to the project. We ensure
|
||||
that any code, documentation, or design produced with AI meets our quality and security standards.
|
||||
|
||||
### Why Use AI?
|
||||
Using AI tools allows us to:
|
||||
* Accelerate development and reduce boilerplate
|
||||
* Improve consistency in code and documentation
|
||||
* Explore alternative solutions more efficiently
|
||||
* Focus developer time on complex or creative tasks
|
||||
|
||||
### Transparency & Ethics
|
||||
We are committed to transparent and responsible use of AI. Our goal is to ensure that AI supports - but never replaces -
|
||||
thoughtful human contribution. If you have any questions or concerns about the role of AI in this project, please open an
|
||||
issue or contact the maintainers.
|
||||
@@ -14,5 +14,4 @@ date = 2024-06-16T22:31:24+02:00
|
||||
* [Developing ExectOS](/contributing/developing-exectos)
|
||||
* [ExectOS Coding Style](/contributing/coding-style)
|
||||
* [First steps with GIT](/contributing/working-with-git)
|
||||
* [Setting Up XTchain](/contributing/setting-up-xtchain)
|
||||
* [Submitting Patches](/contributing/submitting-patches)
|
||||
|
||||
@@ -2,56 +2,40 @@
|
||||
title = 'Building ExectOS'
|
||||
date = 2024-06-18T16:37:24+02:00
|
||||
+++
|
||||
ExectOS must be compiled with the XTChain toolchain. Make sure you have already [downloaded](/download) and installed
|
||||
XTchain and [set up the environment](/contributing/setting-up-xtchain) before proceeding. Once this is done and the
|
||||
source code is ready, open a terminal and launch the XTChain Build Environment.
|
||||
ExectOS must be compiled with the XTChain toolchain. Make sure you have already [downloaded](/download) and installed it.
|
||||
Once this is done and the source code is ready, open a terminal and launch the XTChain Build Environment.
|
||||
```
|
||||
xtchain
|
||||
```
|
||||
Next, set your target architecture and build type. You can choose between:
|
||||
* Architecture: i686 (32-bit) or amd64 (64-bit)
|
||||
* Build type: DEBUG (for development and debugging) or RELEASE (for production)
|
||||
|
||||
Use the following commands:
|
||||
Then, set your target build architecture and the build type and configure the project. Choose `i686` or `amd64` depending
|
||||
on the architecture you wish to target. The build type can be either `DEBUG` for debugging purposes or `RELEASE` for
|
||||
a production build.
|
||||
```
|
||||
charch [i686|amd64]
|
||||
chbuild [DEBUG|RELEASE]
|
||||
```
|
||||
Now, configure the project:
|
||||
* On Linux, run:
|
||||
```
|
||||
./configure.sh
|
||||
```
|
||||
* On Windows, run:
|
||||
```
|
||||
.\configure.ps1
|
||||
```
|
||||
Once configuration is complete, navigate to the build directory and compile the source:
|
||||
After the sources are configured, navigate to the build directory and compile the source code.
|
||||
```
|
||||
cd build
|
||||
xbuild
|
||||
```
|
||||
After the build completes, the compiled binaries will be located in the `build/output/binaries` directory. Additionally,
|
||||
depending on the selected build type, you may also find debug symbols in PDB format inside the `build/output/symbols`
|
||||
folder. Precompiled libraries intended for linking other software with XTDK are available in the `build/output/library`
|
||||
directory.
|
||||
Afterwards, you will find the binaries in `build/output/binaries` directory. Apart from that, depending on build type, you
|
||||
can also find a debug symbols in PDB format in `build/output/symbols` folder and precompiled libraries that can be used to
|
||||
link other software using XTDK in `build/output/library`
|
||||
|
||||
## Creating Qemu Disk Image
|
||||
If you want to build a bootable disk image for use with QEMU, you can do so with the following command:
|
||||
If you want to build a disk image that is ready to use with Qemu, you can use the following command:
|
||||
```
|
||||
xbuild diskimg
|
||||
```
|
||||
This command generates a virtual disk image that can be directly booted in a QEMU virtual machine. You can also convert
|
||||
this image into other formats compatible with alternative hypervisors such as VirtualBox or VMware.
|
||||
This will create a disk image that you can boot within a Qemu Virtual Machine, or convert to other format for use within
|
||||
other hypervisor, like VirtualBox.
|
||||
|
||||
## Testing with Qemu
|
||||
Once ExectOS has been successfully compiled, you can test it in a virtual environment. The recommended approach is to use
|
||||
QEMU, which supports both hardware virtualization (via KVM) and software emulation (via TCG).
|
||||
|
||||
After successfully compiling ExectOS, you can test it using QEMU with either hardware virtualization (KVM) or software
|
||||
emulation (TCG). To start QEMU with KVM, use the testkvm target, and for TCG emulation, use the testtcg target. These
|
||||
targets are provided for convenience and can be executed directly from the build directory. For example, run
|
||||
`xbuild testkvm` to launch QEMU with hardware virtualization, or `xbuild testtcg` to use software emulation:
|
||||
After successfully compiling ExectOS, you can test it using Qemu with either hardware virtualization (KVM) or software
|
||||
emulation (TCG). To start Qemu with KVM, use the `testkvm` target, and for TCG emulation, use the `testtcg` target. These
|
||||
targets are provided for convenience and you can run these commands from your build directory.
|
||||
```
|
||||
xbuild testkvm # For KVM hardware virtualization
|
||||
```
|
||||
@@ -59,20 +43,16 @@ or
|
||||
```
|
||||
xbuild testtcg # For TCG software emulation
|
||||
```
|
||||
These commands will automatically configure QEMU with the appropriate settings and start the virtual machine. Keep in mind
|
||||
that QEMU is not distributed with XTchain and must be installed manually.
|
||||
|
||||
In addition to QEMU, you can also test ExectOS using Bochs, an IA-32 (x86) PC emulator that offers a different emulation
|
||||
environment. To launch Bochs with the ExectOS disk image, run the following command:
|
||||
These commands will automatically configure Qemu with the appropriate settings for ExectOS and start the virtual machine.
|
||||
In addition to Qemu, you can also use Bochs for emulation. Bochs is an IA-32 (x86) PC emulator that provides a different
|
||||
emulation environment. To launch Bochs with the ExectOS image, run the following command:
|
||||
```
|
||||
xbuild bochsvm
|
||||
```
|
||||
This will start the Bochs emulator with ExectOS loaded, allowing you to test the system in a context that may offer
|
||||
different debugging capabilities compared to QEMU. Like QEMU, Bochs is not bundled with XTchain and needs to be installed
|
||||
separately.
|
||||
This will start the Bochs emulator with ExectOS loaded, allowing you to test the system in an environment that may offer
|
||||
different debugging capabilities compared to Qemu.
|
||||
|
||||
The performance and debugging experience will differ depending on which emulator you use. KVM requires a host system with
|
||||
hardware virtualization support and offers the best performance. TCG is a pure software-based emulator and works even on
|
||||
systems without virtualization capabilities, albeit at slower speeds. Bochs, while slower than both, provides a rich set of
|
||||
debugging features which can be especially helpful when troubleshooting complex or low-level issues. Alternatively, you may
|
||||
use other hypervisors like VMware or VirtualBox by manually configuring them to boot from the generated disk image.
|
||||
Remember that the performance and debugging features will vary depending on whether you use KVM, TCG, or Bochs. KVM
|
||||
requires that your host machine supports hardware virtualization, while TCG is a pure software emulator which works on
|
||||
systems without hardware virtualization. Bochs provides a rich set of debugging features, which can be useful when
|
||||
troubleshooting complex issues.
|
||||
|
||||
@@ -1,80 +0,0 @@
|
||||
+++
|
||||
title = 'Setting up XTchain'
|
||||
date = 2025-07-16T19:29:14+02:00
|
||||
+++
|
||||
XTchain is a script-based compilation environment designed to streamline complex build processes across multiple platforms.
|
||||
This guide explains how to set up and use XTchain efficiently on your system. You will learn how to launch the XTchain
|
||||
environment, configure your terminal for convenience, and ensure that your development setup is ready for building projects
|
||||
like ExectOS.
|
||||
|
||||
## Setting Up XTchain on Linux
|
||||
Setting up XTchain on Linux is designed to be as straightforward and hassle-free as possible. Once you have downloaded and
|
||||
extracted the XTchain archive, there is virtually no additional configuration required to get started. Simply navigate to the
|
||||
extracted directory and launch the main executable script:
|
||||
```
|
||||
/path/to/xtchain/xtchain
|
||||
```
|
||||
This single command initializes the entire XTchain environment, automatically setting up all necessary paths, environment variables,
|
||||
and dependencies behind the scenes. The script ensures that your terminal session is fully prepared for building ExectOS right away,
|
||||
so you can focus on development rather than setup.
|
||||
|
||||
Because Linux inherently supports scripting and shell environments, XTchain takes advantage of native shell features to provide
|
||||
a seamless experience. The environment is preconfigured to recognize common build tools and utilities, minimizing any need for
|
||||
manual intervention.
|
||||
|
||||
For convenience, you might consider either creating an alias in your shell configuration file (such as .bashrc or .zshrc):
|
||||
```
|
||||
alias xtchain="/path/to/xtchain/xtchain"
|
||||
```
|
||||
or adding the XTchain directory to your PATH environment variable:
|
||||
```
|
||||
export PATH="/path/to/xtchain:$PATH"
|
||||
```
|
||||
After making either change, reload your shell configuration with source ~/.bashrc (or source ~/.zshrc) to make the modification
|
||||
effective immediately. This way, you can start XTchain from anywhere simply by typing xtchain, without navigating to its folder
|
||||
each time.
|
||||
|
||||
## Setting Up XTchain on Windows
|
||||
On Windows, the simplest way to launch XTchain is by defining a dedicated profile in Windows Terminal that automatically
|
||||
opens PowerShell, loads the XTchain environment, and remains interactive for further commands.
|
||||
|
||||
### 1. Create a Custom Windows Terminal Profile
|
||||
A dedicated terminal profile ensures that every time you open XTchain, the environment is preconfigured and ready to use.
|
||||
|
||||
1. Open **Windows Terminal**.
|
||||
2. Click the down-arrow next to a tab and select **Settings**.
|
||||
3. Choose **Add a new profile**.
|
||||
4. In the **Command line** field, enter:
|
||||
```
|
||||
%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -NoExit -File "C:\Path\To\XTchain\xtchain.ps1"
|
||||
```
|
||||
Replace C:\Path\To\XTchain with the actual folder where xtchain.ps1 resides. Give the profile a name (for example, XTchain),
|
||||
assign an icon if desired, and save the settings. Now when you select the XTchain profile, Windows Terminal will launch
|
||||
PowerShell, bypass execution restrictions, run the initialization script, and keep the window open. The command works as follows:
|
||||
it starts the built-in PowerShell executable (%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe), temporarily disables
|
||||
the script execution policy with **-ExecutionPolicy Bypass**, prevents the terminal from closing after the script finishes using
|
||||
**-NoExit**, and runs the XTchain initialization script specified with **-File "C:\Path\To\XTchain\xtchain.ps1"**. This setup
|
||||
ensures that the XTchain environment is ready every time you open the terminal.
|
||||
|
||||
### 2. Handling Execution Policy Errors
|
||||
When running the profile, you might see the following error message: **File ... cannot be loaded because running scripts is
|
||||
disabled on this system**. This indicates that your system's execution policy is set to block all or unsigned scripts. To
|
||||
permanently fix it, open PowerShell as Administrator and enter:
|
||||
```
|
||||
Set-ExecutionPolicy RemoteSigned
|
||||
```
|
||||
This policy allows:
|
||||
- All locally created scripts (including xtchain.ps1) to run without signing.
|
||||
- Scripts downloaded from the internet to require a trusted digital signature.
|
||||
Warning: Only change execution policy if you trust the scripts you run.
|
||||
|
||||
### 3. Use Git Bash for a Unix-like Environment
|
||||
As an alternative, if you have installed Git for Windows along with the bundled MSYS2 tools, you can leverage its Bash shell and
|
||||
Unix-style scripts to initialize XTchain exactly as on Linux. During the installation process, you have the option to include a set
|
||||
of additional Unix utilities that, combined with the Bash shell, provide a powerful and familiar development environment. This setup
|
||||
allows you to run XTchain's shell scripts seamlessly on Windows, mirroring the Linux workflow and ensuring consistency across
|
||||
platforms without the need for platform-specific adjustments.
|
||||
|
||||
For added convenience, you can also configure a dedicated profile in Windows Terminal that automatically launches Git Bash with the
|
||||
XTchain environment pre-initialized. This makes it easy to jump straight into a ready-to-use shell session. The process for creating
|
||||
this terminal profile is analogous to the one described in section 1.
|
||||
@@ -3,7 +3,6 @@ title = 'Documentation'
|
||||
date = 2024-06-16T22:27:36+02:00
|
||||
+++
|
||||
### Getting Started
|
||||
* [AI-Assisted Contributions](/ai-assisted)
|
||||
* [Frequently Asked Questions](/faq)
|
||||
* [Get in Touch](/contact-us)
|
||||
* [Installation Guide](/documentation/install-guide)
|
||||
|
||||
@@ -3,23 +3,15 @@ title = 'Download'
|
||||
date = 2024-06-16T22:35:11+02:00
|
||||
+++
|
||||
### ExectOS
|
||||
ExectOS currently supports two architectures: i686 and AMD64. To get started, you can either download the appropriate
|
||||
binaries for your platform, or test-drive ExectOS using the QEMU emulator by downloading one of the available disk images.
|
||||
All available files are listed on [this page](https://artifacts.codingworkshop.eu.org/ExectOS/?C=M&O=D). Each filename
|
||||
includes the build date, commit hash, architecture, and build type (debug or release), as well as the artifact type.
|
||||
For example:
|
||||
|
||||
* ExectOS-20250730-61d5e36a4e-i686-release-bin.tar.gz - Contains binary files that can be copied to a USB disk and booted
|
||||
on bare metal hardware.
|
||||
* ExectOS-20250730-61d5e36a4e-i686-release-lib.tar.gz - Includes libraries useful for building 3rd party drivers.
|
||||
* ExectOS-20250730-61d5e36a4e-i686-release-sym.tar.gz - Contains debug symbols (PDB format) for release build.
|
||||
* ExectOS-20250730-61d5e36a4e-i686-release.img.gz - A compressed QEMU disk image for testing the release version of the
|
||||
system.
|
||||
* ExectOS-20250730-61d5e36a4e-i686-debug-bin.tar.gz - Contains debug version of system binaries, including additional debug
|
||||
output.
|
||||
* ExectOS-20250730-61d5e36a4e-i686-debug-lib.tar.gz - Libraries with debug symbols, suitable for development and testing.
|
||||
* ExectOS-20250730-61d5e36a4e-i686-debug-sym.tar.gz - Full debug symbols for the debug build, in PDB format.
|
||||
* ExectOS-20250730-61d5e36a4e-i686-debug.img.gz - A compressed QEMU disk image containing the debug build of ExectOS.
|
||||
ExectOS supports currently two architectures (i686 and AMD64). To get started today, either download a binaries for one of
|
||||
supported architectures or test drive ExectOS on the QEMU emulator by downloading a disk image. You can find a full list
|
||||
of files, available to download at [this page](https://artifacts.codingworkshop.eu.org/ExectOS/). Each file contains an
|
||||
information about build date, SHA1, architecture and a type of artifact, eg:
|
||||
* ExectOS-20240614-6b70074ec6-i686-bin.tar.gz - This archive contains a binary files that can be copied to USB disk and
|
||||
booted up on bare metal hardware.
|
||||
* ExectOS-20240614-6b70074ec6-i686-lib.tar.gz - Contains a libraries useful for building 3rd party drivers.
|
||||
* ExectOS-20240614-6b70074ec6-i686-sym.tar.gz - This archive includes a debug symbols in PDB format.
|
||||
* ExectOS-20240614-6b70074ec6-i686.img.gz - This is a compressed QEMU disk image, that can be used with this emulator.
|
||||
|
||||
### XTChain
|
||||
Get the latest prebuilt toolchain, needed to compile the ExectOS Operating System. This includes the Clang compiler, and
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
+++
|
||||
title = 'Streamlining for the Future: Inside the Latest XTchain Overhaul'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-07-10T15:48:42+02:00'
|
||||
+++
|
||||
In the dynamic world of software development, the ExectOS project has emerged as a captivating endeavor by a community
|
||||
of passionate programmers. This novel operating system, designed for enthusiasts and innovators alike, has recently
|
||||
undergone significant enhancements, particularly within its core toolchain, XTchain.
|
||||
|
||||
The XTchain toolchain, an integral component of the ExectOS build process, has been meticulously streamlined. The previous
|
||||
versions included an assortment of elements that, upon reflection, were deemed superfluous. Most notably, the entire
|
||||
mingw32 suite has been removed. The rationale behind this decision is founded on the existence of multiple established
|
||||
toolchains capable of building Windows applications using both GCC and LLVM. Given that ExectOS, despite employing PE
|
||||
executable files, is distinct from Windows, it was determined unnecessary for the dedicated toolchain to incorporate
|
||||
Windows-specific headers and libraries.
|
||||
|
||||
As the ExectOS project evolves, it may eventually integrate its own libraries and headers. However, at the present stage,
|
||||
the essential tools for system construction suffice. The leaner XTchain now comprises the LLVM compiler, additional Wine
|
||||
tools for resource file handling (such as wmc and wrc), CMake, and Ninja. Additionally, the toolchain's prior branding
|
||||
elements have been eliminated to pave the way for the use of system-native tools in building ExectOS—albeit not the
|
||||
recommended method—thereby facilitating contributions from unsupported systems like OS X.
|
||||
|
||||
A significant milestone for XTchain is the introduction of support for the Windows operating system. This advancement
|
||||
allows anyone to download a dedicated package containing the complete set of necessary tools to compile ExectOS under
|
||||
Windows, marking a pivotal step toward broadening developer accessibility.
|
||||
|
||||
These changes reflect ExectOS's ongoing commitment to accessibility, performance, and community involvement. We invite
|
||||
developers and enthusiasts to explore the updated toolchain and join us in shaping the future of this operating system.
|
||||
@@ -1,33 +0,0 @@
|
||||
+++
|
||||
title = 'XTchain 3.0.0 Released: Now Available for Windows'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-07-11T09:04:17+02:00'
|
||||
+++
|
||||
We are proud to announce the release of XTchain 3.0.0, marking a major milestone in the evolution of the XTOS
|
||||
build toolchain. This new version introduces a streamlined experience, improved platform support, and updated core
|
||||
components - further solidifying XTchain's role as a reliable foundation for building the ExectOS operating system.
|
||||
|
||||
The new release is available now. Download XTchain 3.0.0 from the [release page](https://github.com/xt-sys/xtchain/releases/tag/3.0.0).
|
||||
We invite developers and power users to explore the updated XTchain, try out the new Windows support or Lite editions, and
|
||||
contribute to the growing ExectOS ecosystem.
|
||||
|
||||
#### What's New in XTchain 3.0.0
|
||||
This release brings several key improvements and changes:
|
||||
* **Removal of mingw32:** XTchain no longer includes the mingw32 suite, significantly reducing complexity and size. Given
|
||||
that ExectOS uses PE executables but is not Windows, the inclusion of Windows-specific headers and libraries was deemed
|
||||
unnecessary.
|
||||
* **Toolchain simplification:** Redundant elements and legacy branding have been removed, making it easier for developers
|
||||
to integrate system-native tools if desired (not recommended for production use).
|
||||
* **Updated core tools:**
|
||||
* LLVM upgraded to 21.0.7
|
||||
* Wine tools upgraded to 10.11
|
||||
* CMake upgraded to 4.0.3
|
||||
* Ninja upgraded to 1.13.0
|
||||
* **Windows support:** For the first time, XTchain is officially available as a complete binary package for Windows,
|
||||
allowing users to build ExectOS natively on the platform.
|
||||
|
||||
#### Introducing the Lite Editions
|
||||
XTchain 3.0.0 also debuts **Lite versions** of the toolchain. These minimalist editions include only the essential build
|
||||
environment. The Lite editions are intended for advanced users who already have - or prefer to install - LLVM, CMake, and
|
||||
Ninja from official sources. These variants offer flexibility without compromising the core functionality required to build
|
||||
ExectOS.
|
||||
@@ -1,34 +0,0 @@
|
||||
+++
|
||||
title = 'XTchain 3.0.1 Released: Bug Fixes & Updates'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-07-18T10:54:37+02:00'
|
||||
+++
|
||||
Just days after the milestone release of XTchain 3.0.0, we have launched version 3.0.1, delivering targeted improvements
|
||||
and essential bug fixes to correct initial regressions and improve overall functionality across platforms.
|
||||
|
||||
XTchain 3.0.1 may appear as a minor version bump, but its release demonstrates the project's commitment to rapid iteration
|
||||
and user-focused refinement. The update includes the latest versions of LLVM, Ninja and Wine tools (such as widl, wmc and
|
||||
wrc) - three essential tools in the modern compilation ecosystem. These updates were integrated specifically in response
|
||||
to the recent upstream releases, ensuring XTchain remains compatible with the most up-to-date and performant build
|
||||
infrastructure.
|
||||
|
||||
While 3.0.0 marked a significant leap forward for the project - introducing a reworked toolchain architecture and broader
|
||||
platform support - version 3.0.1 builds on that foundation by focusing on polish, reliability, and cross-platform parity.
|
||||
|
||||
#### Key Highlights in XTchain 3.0.1
|
||||
* **Bug Fixes Across the Board**: We addressed several issues uncovered after the 3.0.0 release, including a lingering
|
||||
problem where XTchain attempted to display the version of the non-existent windres tool - an issue that stemmed from an
|
||||
incomplete cleanup of deprecated tool references.
|
||||
* **Improved Windows Environment Support**: One of the top user-requested features has landed: XTchain now works seamlessly
|
||||
within GIT Bash on Windows. This enhancement makes it easier for developers on Windows to interact with the toolchain
|
||||
using familiar Unix-like workflows. Additionally, a new script has been included to help initialize the build environment
|
||||
automatically.
|
||||
* **Enhanced XBUILD Behavior**: A particularly frustrating issue with the xbuild command - where it silently ignored
|
||||
user supplied parameters has been resolved. The tool now correctly passes all build arguments, ensuring predictable
|
||||
behavior for more complex project setups.
|
||||
* **Unified Cross-Platform Experience**: A key development goal in 3.0.1 was to harmonize behavior across Linux and Windows
|
||||
environments. From consistent output formatting to standardized toolchain handling, users can now expect a much more
|
||||
uniform experience regardless of their platform of choice.
|
||||
|
||||
Users are encouraged to update as soon as possible to benefit from the improved tooling, better compatibility, and refined
|
||||
user experience.
|
||||
@@ -1,60 +0,0 @@
|
||||
+++
|
||||
title = 'Major Bugfixing Milestone Unlocks New Development Path for ExectOS'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-08-20T11:13:37+02:00'
|
||||
+++
|
||||
Over the past month, since the release of XTbuild 3.0.1, development efforts on ExectOS have focused primarily on resolving
|
||||
critical bugs that were blocking further progress on the project. While feature development temporarily took a back seat,
|
||||
the work done during this period has been substantial - especially considering that it was carried out by a single developer.
|
||||
|
||||
These fixes were essential: as new features were being implemented, they frequently triggered unexpected Page Faults in
|
||||
regions of the system where memory behavior should have been stable. After extensive debugging, the root causes were traced
|
||||
to issues such as incorrect stack alignment and faulty memory mapping. Thankfully, many of these problems have now been
|
||||
resolved, paving the way for continued feature development.
|
||||
|
||||
## Key Fixes and Improvements
|
||||
Among the most important technical fixes completed in the past month:
|
||||
* **Removed compiler library dependencies**, particularly the reliance on memset(), ensuring full control over runtime behavior.
|
||||
* **Corrected stack initialization**, taking into account that the stack grows downward and should be initialized to its logical
|
||||
end.
|
||||
* **Proper stack alignment**, preventing subtle memory access issues.
|
||||
* **Enhanced COM port support**, improving serial communication stability.
|
||||
* Resolved a race condition in HlComPortReadLsr by making RingIndicator part of the CPPORT structure instead of using a
|
||||
shared static variable.
|
||||
* **Fixed TSS descriptor limit** to comply with the Intel architecture specification.
|
||||
* **Improved XTLDR behavior**:
|
||||
* Automatic booting of the default system after a timeout now works reliably.
|
||||
* Proper highlighting of the default boot menu entry.
|
||||
* Refined volume handling in XTLDR, allowing for correct detection of CD/DVD drives.
|
||||
* Fixed bootloader logic in XTOS_O and CHAINLDR modules to ensure the system is booted from the correct volume instead of
|
||||
always defaulting to the ESP partition.
|
||||
* **Improved build pipeline**:
|
||||
* Automated builds for both i686 and AMD64.
|
||||
* Separate outputs for debug and release builds.
|
||||
* **Explicit paging activation on AMD64**.
|
||||
* **Correct initialization of memory-related structures** through zero-filling before usage.
|
||||
* **Improved CPUID handling**.
|
||||
* **Replaced XTSTATUS with EFI_STATUS** in the bootloader, aligning with EFI standards.
|
||||
* **Memory mapping and PML handling fixes**:
|
||||
* Fixed a PDE conflict on i686 systems without PAE when mapping last 4MB
|
||||
|
||||
## Additional Enhancements
|
||||
In parallel with the bugfixes, several usability and maintainability improvements were introduced:
|
||||
* Added a build configuration script for compiling the project on Windows.
|
||||
* Replaced hardcoded values with constants and definitions for better maintainability.
|
||||
* Implemented menu scrolling support in the XTLDR bootloader.
|
||||
* Changed build output directories to allow separate debug and release folders.
|
||||
* Implemented kernel parameter support in the bootloader.
|
||||
* Full support for PML2 and PML3 in the bootloader, enabling system boot with or without PAE support.
|
||||
* Introduced the *NOXPA* parameter to disable PAE (i686) and LA57 (AMD64) as needed.
|
||||
* Automatic PML level detection:
|
||||
* The bootloader now queries CPUID for PAE or LA57 support depending on architecture.
|
||||
* If the user hasn't explicitly disabled XPA, the bootloader enables PAE when available.
|
||||
* LA57 remains unsupported for now, but groundwork has been laid.
|
||||
|
||||
## One Month, One Developer, One Giant Leap
|
||||
Given the size and complexity of the fixes, this is a remarkable milestone for just one month of development. More
|
||||
importantly, resolving these core issues has unblocked the project’s forward momentum. With stability greatly improved,
|
||||
the path is now clear to resume development of advanced features without being derailed by low-level system bugs.
|
||||
|
||||
Stay tuned - the next phase of ExectOS development promises to be an exciting one.
|
||||
@@ -1,79 +0,0 @@
|
||||
+++
|
||||
title = 'Dynamic Multi-Level Paging and Experimental LA57 Support'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-09-04T19:21:37+01:00'
|
||||
+++
|
||||
We have reached an important milestone in the development of ExectOS. While this is not a release announcement, we are
|
||||
excited to share significant progress on the virtual memory subsystem that lays the foundation for the future kernel
|
||||
memory manager.
|
||||
|
||||
Although the kernel does not yet have a fully functional memory manager, the latest work ensures that when we begin its
|
||||
implementation, it will seamlessly support all paging levels from the very start. With proper handling of PML2, PML3,
|
||||
PML4, and PML5 already integrated into the bootloader and partially into the kernel, we have created a highly flexible
|
||||
and future-proof paging framework.
|
||||
|
||||
## Dynamic Multi-Level Paging
|
||||
Unlike most operating systems, which require separate kernel builds or compile-time configuration to handle different
|
||||
paging models, ExectOS takes a unique approach. Instead of choosing a single paging mode at build time, the kernel
|
||||
selects the correct strategy **dynamically at runtime**.
|
||||
|
||||
To achieve this, we have introduced the concept of XPA — eXtended Physical Addressing. This is an internal term used to
|
||||
unify two different hardware features under a single abstraction:
|
||||
* On i686, XPA refers to PAE (Physical Address Extension), which enables PML3.
|
||||
* On AMD64, XPA refers to LA57 (5-level paging), which enables PML5.
|
||||
|
||||
The kernel automatically detects whether the CPU supports PAE or LA57 and chooses the appropriate paging strategy. If
|
||||
the user wants to disable extended paging modes, this can be done with a single boot parameter: **NOXPA**. Without this
|
||||
abstraction, we would have needed two separate options (**NOPAE** and **NOLA57**), complicating both code and
|
||||
configuration.
|
||||
|
||||
## The Strategy-Based Design
|
||||
The new paging subsystem uses the Strategy design pattern to provide both flexibility and maintainability. Instead of
|
||||
hardcoding assumptions about paging levels, we now have two cooperating structures:
|
||||
* **MMPAGEMAP_INFO**: Stores static information about the active paging mode, such as base virtual addresses, offsets
|
||||
for PTE/PDE/PPE/PXE/P5E levels, and flags describing whether XPA is enabled.
|
||||
* **MMPAGEMAP_ROUTINES**: Holds function pointers for all paging-related operations, such as validating PTEs,
|
||||
allocating new page tables, or changing caching attributes.
|
||||
|
||||
At runtime, ExectOS binds the appropriate function implementations to the MMPAGEMAP_ROUTINES structure based on the
|
||||
selected paging level, while the MMPAGEMAP_INFO structure provides the necessary constants and offsets for correct
|
||||
address translation.
|
||||
|
||||
This design allows us to maintain a single unified kernel image capable of running on CPUs with very different paging
|
||||
capabilities. More importantly, it sets the stage for a future memory manager that will "just work" with any supported
|
||||
paging level without requiring additional patches, recompilation, or feature flags.
|
||||
|
||||
## Performance Considerations
|
||||
You might expect that introducing function pointers and dynamic dispatch could hurt performance, but in practice, the
|
||||
overhead is negligible. The Strategy pattern affects only the rare moments when memory is being mapped or unmapped -
|
||||
operations that are infrequent compared to ordinary memory access.
|
||||
|
||||
Once a virtual address is mapped, the CPU uses the hardware-managed Translation Lookaside Buffer (TLB) to cache
|
||||
virtual-to-physical translations. Accessing mapped memory does not require any extra indirection through our structures.
|
||||
The cost of an additional function pointer lookup occurs only during the setup phase of a mapping, which involves much
|
||||
more expensive operations anyway - flushing TLB entries, updating PTEs, and invalidating caches. In other words, the
|
||||
small dispatch cost is completely overshadowed by the inherent complexity of memory mapping itself.
|
||||
|
||||
In summary, the Strategy-based design gives us dynamic flexibility without sacrificing performance, even on systems with
|
||||
tight CPU cycle budgets.
|
||||
|
||||
## Experimental LA57 Support
|
||||
For AMD64, we have added experimental support for 5-level paging (PML5), which expands the virtual address space from
|
||||
48-bits to 57-bits. To enable LA57, we implemented a small trampoline that temporarily switches the CPU into 32-bit
|
||||
compatibility mode, sets the CR4.LA57 bit, and then returns to long mode with the new PML5 hierarchy active.
|
||||
|
||||
Because we currently lack access to real hardware supporting LA57, this functionality has been tested exclusively in
|
||||
QEMU. While the implementation is complete and verified in a virtualized environment, we consider it experimental until
|
||||
we can validate it on physical systems.
|
||||
|
||||
During development, we also addressed an issue related to the calculation of virtual-to-physical translation offsets
|
||||
when working in 5-level paging mode. The constants defining the base virtual addresses for the page-table self-mapping
|
||||
region were previously chosen in a way that, under certain conditions, could lead to address calculation overflows when
|
||||
deriving the location of a Page Table Entry (PTE) from a given virtual address. We have corrected these base address
|
||||
definitions, ensuring that the entire PML5 hierarchy is now computed safely and consistently across all supported paging
|
||||
modes.
|
||||
|
||||
## Looking Ahead
|
||||
With the new foundation in place, the next major milestone will be implementing the kernel memory manager. Alongside
|
||||
this, we will continue refining the existing code through bug fixes, optimizations, and refactoring to ensure long-term
|
||||
stability and maintainability.
|
||||
@@ -1,111 +0,0 @@
|
||||
+++
|
||||
title = 'Deep Dive into Recent Bug Fixes'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-09-14T11:51:17+01:00'
|
||||
+++
|
||||
August has been an intense and productive month for the ExectOS project. When many thought development had slowed down,
|
||||
we managed to push forward with progress - a clear sign that ExectOS is still actively evolving. Alongside implementing
|
||||
Dynamic Multi-Level Paging and adding experimental LA57 support, we also spent a considerable amount of time addressing
|
||||
a wide range of bugs, improving stability and compatibility across the entire system.
|
||||
|
||||
In one of our recent updates, we published a full list of bug fixes included in the latest builds. This time we wanted to
|
||||
share some of the more interesting technical findings that came out during the development process. While many fixes were
|
||||
straightforward, others required a deep dive into low-level kernel internals, bootloader behavior, and hardware-specific
|
||||
quirks. In this article, we have selected several cases that stood out due to their complexity or the insights they
|
||||
provided. Our goal is to give you a closer look behind the scenes and highlight the complex engineering challenges we
|
||||
encounter.
|
||||
|
||||
|
||||
### Correcting String Parameter Types: From CONST PWCHAR to PCWSTR
|
||||
During a recent code review, we identified several inconsistencies in function definitions where parameters or return
|
||||
types used **CONST PWCHAR** instead of **PCWSTR**. At first glance, the two may seem equivalent, but the distinction is
|
||||
subtle and significant. The type **PCWSTR** is defined as **const WCHAR***, meaning it represents a pointer to immutable
|
||||
data - the function promises not to modify the contents of the string it receives. In contrast, **PWCHAR** is defined as
|
||||
**WCHAR* **, a pointer to mutable data, and when combined with the const qualifier as **CONST PWCHAR**, the compiler
|
||||
interprets it as **WCHAR * const**. This means the pointer itself is constant, but the data it points to can still be
|
||||
modified. In other words, the original declarations implied that the function would not change the pointer's address but
|
||||
made no guarantees about preserving the actual string content. Since our intention was to ensure that the functions never
|
||||
alter the provided strings, we corrected the signatures to use **PCWSTR**. This change improves type safety, prevents
|
||||
accidental data modifications, and better communicates the intended contract of these APIs.
|
||||
|
||||
|
||||
### Fixing a Stack Corruption Bug in BlEnumerateBlockDevices
|
||||
While investigating random crashes in the bootloader, we discovered a subtle but critical bug in the
|
||||
**BlEnumerateBlockDevices()** function, specifically within a **BlDebugPrint()** call responsible for logging detected
|
||||
hard disk partitions. The issue stemmed from using the wrong format specifier for printing the partition size. The code
|
||||
was using **%u** to display a value computed as **HDPath->PartitionSize * Media->BlockSize**, which is stored in a 64-bit
|
||||
**UINT64** variable. On 32-bit UEFI systems, **%u** expects a **32-bit unsigned int**, causing only the lower 32 bits of
|
||||
the value to be read while the upper 32 bits remained on the stack. As a result, all subsequent arguments were
|
||||
misaligned, leading **BlDebugPrint()** to interpret invalid data as a string pointer for the **%S** specifier. In
|
||||
practice, this caused corrupted output, random memory reads, and eventually a protection fault that froze the system. The
|
||||
fix was straightforward but essential: we replaced **%u** with **%llu** to correctly print the 64-bit value, preventing
|
||||
stack corruption and restoring XTLDR stability.
|
||||
|
||||
|
||||
### Eliminating Screen Flicker with Smarter Boot Menu Redrawing
|
||||
Previously, the boot menu suffered from a performance issue where every keypress - even a simple UP/DOWN arrow -
|
||||
triggered a complete redraw of the entire menu. This approach was inefficient and caused noticeable flickering. To fix
|
||||
this, we introduced a smarter redrawing mechanism controlled by two new flags: **RedrawBootMenu** and **RedrawEntries**.
|
||||
The **RedrawBootMenu** flag indicates when the entire menu, including the header and footer, must be redrawn - for
|
||||
example, after displaying the help screen or an error message. The **RedrawEntries** flag, on the other hand, signals
|
||||
when the list of visible entries needs refreshing, such as when the selection moves beyond the currently displayed area.
|
||||
For normal navigation, the optimized arrow key handling now tracks the previously highlighted item. When the new
|
||||
selection remains within the visible range, only two drawing operations are performed: the old item is redrawn without
|
||||
highlighting, and the new one is drawn with highlighting. The rest of the list remains untouched, eliminating unnecessary
|
||||
updates. This change completely removes flickering, significantly reduces CPU overhead, and results in smooth, menu
|
||||
navigation.
|
||||
|
||||
|
||||
### Restoring Build Compatibility by Replacing asm with \_\_asm\_\_
|
||||
After upgrading CMake from 4.0.3 to 4.1.0, the project became uncompilable when building with clang-cl under the C23
|
||||
standard. The root cause was the stricter enforcement of language compliance introduced in the new CMake release.
|
||||
Previously, CMake 4.0.3 did not explicitly pass a C standard flag, which allowed clang-cl to operate in a GNU-compatible
|
||||
mode where the shorthand asm keyword - provided as a GNU extension - was accepted. However, starting with CMake 4.1.0,
|
||||
the build system began explicitly adding the **-clang:-std=c23** flag, forcing the compiler into strict ISO C23 mode and
|
||||
disabling GNU-specific extensions. As a result, any occurrence of asm caused a compilation error:
|
||||
|
||||
```error: use of undeclared identifier 'asm'```
|
||||
|
||||
To resolve this, we replaced all instances of **asm** with the standard-compliant **\_\_asm\_\_** keyword. Unlike **asm**,
|
||||
which is a GCC-specific convenience, **\_\_asm\_\_** is a reserved identifier explicitly recognized by both GCC and Clang,
|
||||
even in strict standards mode. This change ensures consistent compatibility across compilers and aligns the codebase with
|
||||
the C23 standard.
|
||||
|
||||
|
||||
### Fixing a Race Condition in HlComPortReadLsr Caused by a Shared Static Variable
|
||||
We discovered a critical race condition and state management issue in the HlComPortReadLsr function, which handles
|
||||
reading from serial (COM) ports. The root cause was the use of a local static variable:
|
||||
|
||||
``STATIC UCHAR RingFlag;``
|
||||
|
||||
In C, declaring a local variable as static means it is allocated only once and retains its value across all future calls
|
||||
to the function. However, this also means there is only a single instance of RingFlag, shared by every call to
|
||||
**HlComPortReadLsr()** - regardless of which COM port (PCPPORT instance) is being accessed. This design flaw introduced
|
||||
two major problems - State Corruption. The function incorrectly allowed the state of one port to "leak" into another. If
|
||||
**HlComPortReadLsr()** was called for COM1 and set **RingFlag** to a specific value, a subsequent call for COM2 would
|
||||
reuse that value, leading to incorrect logic and inconsistent results. The solution was to remove the shared static
|
||||
variable and make **RingFlag** instance-specific, ensuring each port operates on its own state.
|
||||
|
||||
|
||||
### Fixing Framebuffer Rendering by Using Correct Pitch and Bytes-Per-Pixel Calculations
|
||||
We fixed a bug in the framebuffer driver that caused severe graphical glitches on many real devices. The issue stemmed
|
||||
from incorrect assumptions about how pixel data is laid out in video memory. The original implementation in
|
||||
**xtoskrnl/hl/fbdev.c** made two major mistakes when calculating pixel addresses:
|
||||
* **Ignoring the Pitch (Stride):** The code assumed that moving from one scanline to the next could be done by adding
|
||||
**screen_width * bytes_per_pixel**. It used **HlpFrameBufferData.PixelsPerScanLine** - the number of visible pixels
|
||||
per line - to perform the calculation. However, modern graphics hardware often aligns each scanline to specific memory
|
||||
boundaries (e.g., 16 bytes), which means the actual distance between the start of consecutive lines, known as the
|
||||
**Pitch (or stride)**, can be larger than **width * bytes_per_pixel**. By ignoring Pitch, the code miscalculated the
|
||||
start of each line, causing images to appear skewed, torn, or completely distorted.
|
||||
* **Hardcoding Pixel Size:** The original driver assumed that each pixel was always 4 bytes (32-bit color), using
|
||||
formulas like:
|
||||
``FrameBufferIndex = 4 * PositionY + 4 * PositionX;``
|
||||
This made the driver incompatible with display modes using 24-bit or 16-bit color depths.
|
||||
|
||||
We have addressed these issues by introducing a hardware-correct approach. We have added BytesPerPixel to the framebuffer
|
||||
data structure. It's value is calculated once during initialization. This eliminates all hardcoded assumptions about pixel
|
||||
size.
|
||||
|
||||
Additionally, all drawing functions were updated to use the proper formula for calculating pixel address. This fix ensures
|
||||
correct rendering across all supported color depths and hardware configurations, eliminates graphical artifacts, and makes
|
||||
the framebuffer driver fully compliant with modern graphics standards.
|
||||
@@ -1,64 +0,0 @@
|
||||
+++
|
||||
title = 'Migration of the Project to C++ with Maintained C Compatibility'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-09-28T17:12:45+01:00'
|
||||
+++
|
||||
The project’s kernel, bootloader, and all associated modules have been migrated to C++. This transition was a complex
|
||||
and time-consuming effort that we deliberately prioritized over adding new features. Because the codebase was still
|
||||
relatively compact, undertaking this migration at this stage was significantly easier than it would have been later.
|
||||
|
||||
In total, the migration involved modifications across **297 files**, with *16,671* lines of code added and *13,312*
|
||||
lines removed. The process culminated in the merge of **67 commits**. Despite these internal changes, the public
|
||||
interface exposed through XTDK remains fully compatible with C. As a result, developers can write drivers in either C
|
||||
or C++.
|
||||
|
||||
## Benefits of the Migration
|
||||
Although certain C++ features such as global constructors, destructors, runtime type information and exceptions are not
|
||||
suitable for use in the kernel, a large subset of the language can be used safely and effectively. Adopting C++
|
||||
internally allows us to benefit from:
|
||||
|
||||
- **Stronger static type checking** – templates, references and stricter type rules help eliminate entire classes of
|
||||
low-level bugs before they reach runtime.
|
||||
|
||||
- **Lightweight classes and structs** – we can define clear interfaces and encapsulate data without relying on heavy
|
||||
runtime mechanisms. Member functions and inline methods provide better organisation than scattered C functions.
|
||||
|
||||
- **Namespaces for isolation** – by grouping related functionality into namespaces we avoid symbol collisions, keep the
|
||||
global namespace clean and make it easier to navigate the codebase.
|
||||
|
||||
- **Inline and template-based utilities** – compile-time polymorphism and generic programming allow common patterns to
|
||||
be written once and reused without runtime overhead.
|
||||
|
||||
- **Clearer modular boundaries** – internal implementation details can be hidden behind class interfaces, improving
|
||||
encapsulation and making each subsystem easier to reason about and maintain.
|
||||
|
||||
This approach preserves the deterministic behaviour and performance required by kernel code while still providing
|
||||
a modern, well-structured environment for development. It enables a codebase that is easier to extend, review and
|
||||
maintain, without introducing mechanisms that are unsafe or impractical at the system level.
|
||||
|
||||
## Preserving C Compatibility
|
||||
Full C compatibility has been retained through the use of **C wrappers** around the underlying C++ components. This
|
||||
design choice means that the migration does not affect how external code interacts with the system.
|
||||
|
||||
Although the project is still in its initial phase and no drivers have been released yet, the architecture is prepared
|
||||
so that drivers can be written in **both C and C++** without restriction.
|
||||
|
||||
The SDK remains accessible to developers who prefer C, while at the same time opening the door for developers writing in
|
||||
C++ to integrate with the system using native language features. This approach ensures a stable and predictable API
|
||||
over time and provides flexibility for future contributors regardless of their chosen language.
|
||||
|
||||
## C Wrappers and Linker Optimisations
|
||||
The C wrappers are implemented as lightweight functions declared with `extern "C"` forwarding calls to the C++
|
||||
implementation. When building with **clang-cl**, linker flags help ensure these wrappers have negligible impact on
|
||||
performance and binary size:
|
||||
|
||||
- **`/OPT:REF`** (Remove Unreferenced Functions and Data) directs the linker to eliminate functions and data sections
|
||||
that are not referenced anywhere in the binary. This reduces the size of the resulting kernel and modules by
|
||||
discarding unused wrappers or internal routines.
|
||||
|
||||
- **`/OPT:ICF`** (Identical COMDAT Folding) merges byte-for-byte identical functions or sections into a single instance.
|
||||
In practice, thin C wrappers calling the same C++ routine can be folded together, removing redundancy and avoiding
|
||||
unnecessary call layers.
|
||||
|
||||
With these optimisations, the system delivers the advantages of a C++ implementation while maintaining a stable,
|
||||
efficient C interface for developers.
|
||||
@@ -1,24 +0,0 @@
|
||||
+++
|
||||
title = 'XTchain 3.0.2 Released: New Disk Image Tools'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-09-30T06:12:05+01:00'
|
||||
+++
|
||||
We are excited to announce the release of XTchain 3.0.2. While this update is relatively minor, it brings several
|
||||
important improvements aimed at enhancing Windows compatibility and streamlining workflows across platforms.
|
||||
<!--more-->
|
||||
|
||||
## Integrated Mtools for FAT Images
|
||||
XTchain 3.0.2 now comes with Mtools built-in. This integration allows developers to format disk or partition images with
|
||||
the FAT filesystem and copy data to and from those images directly - no additional setup required.
|
||||
|
||||
## New diskimg Utility
|
||||
Alongside Mtools, we have introduced our own diskimg application. This tool simplifies the creation of disk images,
|
||||
particularly on Windows systems where native support can vary depending on configuration. By bundling these tools
|
||||
directly into XTchain, we ensure that ExectOS builds can be created consistently across Linux and Windows environments.
|
||||
|
||||
## LLVM Updated to 21.1.2
|
||||
This release also includes an update to LLVM 21.1.2, bringing the latest compiler improvements and fixes into XTchain.
|
||||
|
||||
## Important Note for ExectOS Developers
|
||||
Version 3.0.2 is required to successfully build ExectOS. Older versions of the toolchain will not be able to generate
|
||||
disk images, which will cause the diskimg target to fail.
|
||||
@@ -1,38 +0,0 @@
|
||||
+++
|
||||
title = 'ExectOS Expands Beyond UEFI: Bringing BIOS Support'
|
||||
author = 'Aiken Harris'
|
||||
date = '2025-10-06T10:36:18+01:00'
|
||||
+++
|
||||
When we first started building ExectOS, we envisioned it as an operating system designed exclusively for UEFI-based
|
||||
systems. Over time, however, we realized that this requirement was an artificial limitation. Today, we are excited to
|
||||
announce that we have begun taking concrete steps toward supporting traditional PC/AT hardware with classic
|
||||
BIOS - broadening the reach of ExectOS to a much wider range of systems.
|
||||
<!--more-->
|
||||
|
||||
## First Steps Toward BIOS Compatibility
|
||||
We have already implemented an MBR boot sector and updated our CMake configuration to support BIOS-based emulation. As
|
||||
of now, ExectOS can be launched in several new testing configurations:
|
||||
* **bochsvm**: Bochs with BIOS
|
||||
* **testkvm/testtcg**: QEMU with classic BIOS (with KVM/WHPX acceleration or TCG without acceleration)
|
||||
* **testefikvm/testefitcg**: QEMU with modern UEFI (with or without acceleration), as before
|
||||
|
||||
This means developers can now experiment with ExectOS under both UEFI and BIOS environments using familiar tools like
|
||||
Bochs and QEMU.
|
||||
|
||||
## A Unified Boot Partition Model
|
||||
To simplify BIOS support, we are introducing a dedicated BOOT partition - similar to the recommended practice on
|
||||
Unix-like systems and analogous to the ESP (EFI System Partition) used by UEFI. Our bootloader will exclusively support
|
||||
FAT32 on this partition. It will house the bootloader itself, its configuration, and all modules. In other words, we are
|
||||
effectively bringing the concept of the ESP from the UEFI world into the BIOS realm.
|
||||
|
||||
## What's Next?
|
||||
Full BIOS support is still some way off. The current MBR implementation and CMake adjustments are just the first step. We
|
||||
still need to:
|
||||
* Implement a VBR (Volume Boot Record) capable of locating and loading files from disk
|
||||
* Develop a preloader to prepare the environment for the main bootloader (enabling paging, long mode, etc.)
|
||||
* Significantly refactor XTLDR, to eliminate assumptions about UEFI, such as memory allocation APIs
|
||||
* Ensure all modules are environment-agnostic, running seamlessly on both BIOS and UEFI systems
|
||||
* Move PE/COFF file handling from the module level into the main XTLDR code
|
||||
|
||||
While there is still a long road ahead, this change opens up new possibilities for ExectOS. By removing the UEFI-only
|
||||
limitation, we are making the system more flexible and accessible.
|
||||
@@ -1,74 +0,0 @@
|
||||
+++
|
||||
title = 'Development Update'
|
||||
author = 'Aiken Harris'
|
||||
date = '2026-02-14T11:17:38+01:00'
|
||||
+++
|
||||
It has been several months since our last update. During this time, development of ExectOS has continued steadily,
|
||||
although much of the work has taken place deep within the kernel and boot process. While the original goal of
|
||||
delivering a fully functional memory manager has not yet been achieved, we believe this is a good moment to inform
|
||||
the community about the progress made and the challenges addressed along the way.
|
||||
<!--more-->
|
||||
|
||||
## Focus Area: Memory Manager
|
||||
Since the previous update, development has been focused exclusively on the kernel memory manager. This work required
|
||||
substantial changes not only within the memory subsystem itself but also in the bootloader, where several critical
|
||||
issues were discovered.
|
||||
|
||||
## Bootloader Issues and Fixes
|
||||
During integration work, it became clear that our custom bootloader still contained serious flaws that directly
|
||||
impacted memory manager development:
|
||||
* **XTLDR and XTOSKRNL incompatibility**: The bootloader and the kernel used different memory mapping strategies
|
||||
across architectures. This mismatch resulted in inconsistent virtual memory layouts and prevented reliable
|
||||
initialization of the memory manager. The mapping strategies have now been unified to ensure predictable behavior.
|
||||
* **Incorrect identity mapping of UEFI-reserved memory**: The bootloader was identity-mapping memory regions reserved
|
||||
by UEFI. This was a design error, as it unnecessarily consumed limited virtual address space. The mapping logic has
|
||||
been corrected to avoid reserving virtual space for firmware-managed regions.
|
||||
* **Critical mapping construction bug**: A severe issue in the bootloader's memory mapping builder caused overlapping
|
||||
virtual memory regions. This made correct construction of the PFN (Page Frame Number) database impossible. The
|
||||
mapping generation logic has been rewritten to guarantee non-overlapping, deterministic memory regions.
|
||||
|
||||
### PAE Handling Fix (Bug #23)
|
||||
We also resolved bug #23, which affected PAE support on the i686 platform. The root cause was improper handling of
|
||||
memory descriptors above the 32-bit boundary. The fix ensures that all valid memory regions are correctly preserved and
|
||||
passed to the kernel, allowing full utilization of available RAM under PAE.
|
||||
|
||||
## Progress in the Memory Manager
|
||||
Beyond bootloader fixes, significant progress has been made within the memory manager itself.
|
||||
|
||||
### Dynamic Memory Layout Construction
|
||||
The kernel now dynamically builds its memory layout based on the total amount of detected physical RAM. This ensures
|
||||
that core memory regions, such as: PagedPool, NonPagedPool, or system structures and internal mappings are assigned
|
||||
clearly defined and non-overlapping virtual address ranges. This dynamic approach improves scalability and makes the
|
||||
layout adaptable to systems with varying memory configurations.
|
||||
|
||||
### PFN Database Initialization
|
||||
We are now successfully constructing the PFN database and mapping all memory regions required for the Memory Manager's
|
||||
operation. This was a key milestone, as many higher-level memory operations depend on a correctly initialized PFN layer.
|
||||
|
||||
### Stack Allocation and Deallocation
|
||||
We have implemented internal routines for allocating and freeing memory intended for kernel stacks. These mechanisms
|
||||
allow:
|
||||
* Allocation of stacks with arbitrary sizes depending on requirements,
|
||||
* Proper tracking and release of stack memory
|
||||
This is a prerequisite for thread management. As a result, we can now begin work on kernel thread support. In the near
|
||||
future, we plan to introduce functionality for dynamically growing existing stacks.
|
||||
|
||||
## Current Work
|
||||
At present, development is focused on implementing general-purpose allocation and deallocation routines for arbitrary
|
||||
address pools, specifically: PagedPool and NonPagedPool. These allocators are essential for enabling further kernel
|
||||
subsystems.
|
||||
One of the next major items on our roadmap is SMT support. We intend to introduce it as early as possible to reduce
|
||||
issues related to resource locking and race conditions. Bringing up symmetric multithreading early should help expose
|
||||
and eliminate concurrency-related bugs before additional kernel complexity is introduced.
|
||||
|
||||
## Summary
|
||||
Although the fully functional memory manager is not yet complete, substantial foundational work has been finished:
|
||||
* Critical bootloader memory mapping bugs resolved
|
||||
* Proper PAE handling restored
|
||||
* Dynamic kernel memory layout construction implemented
|
||||
* PFN database successfully built
|
||||
* Kernel stack allocation mechanisms introduced
|
||||
|
||||
The project is now in a more stable and architecturally consistent state than it was before. The full and detailed
|
||||
list of changes can be found in the **memmgr** branch. Further updates will follow as we continue stabilizing the
|
||||
allocator infrastructure.
|
||||
@@ -1,166 +0,0 @@
|
||||
+++
|
||||
title = 'Memory Manager Implementation'
|
||||
author = 'Aiken Harris'
|
||||
date = '2026-03-28T14:37:06+01:00'
|
||||
+++
|
||||
The successful integration of the Memory Manager through [PR #24](https://git.codingworkshop.eu.org/xt-sys/exectos/pulls/24)
|
||||
marks a pivotal moment in the development history of ExectOS, representing the culmination of several months of
|
||||
intensive low-level engineering and architectural refinement. As a project that aims to implement a brand-new XT
|
||||
architecture from scratch, the transition from a statically mapped environment to one with dynamic allocation
|
||||
capabilities is not merely a feature addition but a fundamental prerequisite for the maturation of all subsequent
|
||||
kernel services. The implementation of the Memory Manager provides the necessary infrastructure for the kernel to
|
||||
manage system resources with the fluidity required by modern general-purpose operating systems, moving beyond the
|
||||
limitations of pre-allocated buffers and fixed memory regions.
|
||||
<!--more-->
|
||||
|
||||
## The Architectural Foundation of the Memory Manager
|
||||
The design of the ExectOS Memory Manager is deeply rooted in the project's XT architecture, which derives its
|
||||
philosophical underpinnings from the Windows NT model while introducing modern enhancements and original implementation
|
||||
strategies. In the early stages of the kernel's lifecycle, the lack of a dynamic memory allocator served as a significant
|
||||
bottleneck, preventing the development of complex executive services such as the Object Manager or the Process and Thread
|
||||
Manager. The [PR #24](https://git.codingworkshop.eu.org/xt-sys/exectos/pulls/24) introduces a fully functional version of
|
||||
the Memory Manager that, while currently limited to the non-paged pool, establishes the core logic for physical frame
|
||||
management and virtual address space orchestration.
|
||||
|
||||
The non-paged pool is a critical component of any NT-style kernel. It consists of a range of virtual memory that is
|
||||
guaranteed to be resident in physical memory at all times. This guarantee is essential for kernel-mode components that
|
||||
must access memory at high Interrupt Request Levels (RunLevels), where the page fault handler cannot be invoked. By
|
||||
establishing this pool first, the ExectOS development team has enabled the implementation of other critical kernel
|
||||
mechanisms that require reliable, non-swappable memory.
|
||||
|
||||
### Cross-Architecture Paging Support and the Single-Binary Model
|
||||
One of the most technically demanding aspects of the ExectOS Memory Manager is its ability to support multiple paging
|
||||
structures within a single kernel binary. This design choice is a testament to the project's commitment to a unified
|
||||
Hardware Layer API, which replaces the traditional Hardware Abstraction Layer (HAL) with a more integrated,
|
||||
hardware-specific approach within the kernel itself.
|
||||
|
||||
The Memory Manager is engineered to adapt dynamically to the underlying hardware capabilities detected during the boot
|
||||
phase. This adaptation includes support for legacy 32-bit paging, expanded 36-bit physical addressing, and the latest
|
||||
57-bit linear addressing modes available on modern 64-bit processors.
|
||||
|
||||
|Architecture | Paging Level | Mechanism | Address Space Characteristics |
|
||||
|-------------|--------------|-----------------------|-------------------------------|
|
||||
| i686 | PML2 | Classic 32-bit Paging | Supports up to 4 GB of linear address space with 4 KB or 4 MB pages
|
||||
| i686 | PML3 | 3-Level Paging (PAE) | Extends physical addressing to 64 GB while maintaining a 4 GB virtual space per process
|
||||
| AMD64 | PML4 | 4-Level Paging | Provides a 48-bit virtual address space (256 TB) and 52-bit physical addressing
|
||||
| AMD64 | PML5 | 5-Level Paging (LA57) | Expands virtual address space to 57 bits (128 PB), supporting massive memory configurations
|
||||
|
||||
The support for PML5 (LA57) on the AMD64 architecture is particularly notable, as it places ExectOS among a selective
|
||||
group of kernels capable of leveraging the most advanced memory management features of contemporary CPUs.
|
||||
|
||||
## The Challenge of Early Development and the XTLDR Regression
|
||||
The road to a functional Memory Manager was fraught with significant engineering challenges, many of which originated
|
||||
outside the kernel itself. The development cycle was extended by several weeks due to unexpected bugs in the XTLDR
|
||||
(XT Boot Loader) and its interaction with UEFI firmware. These issues highlight the extreme sensitivity of early-stage
|
||||
kernel development, where the boundary between the loader and the kernel is a frequent source of catastrophic failures.
|
||||
|
||||
### Forensic Analysis of Issue 25: The CR3 Corruption Bug
|
||||
The most critical hurdle encountered was documented in [Issue #25](https://git.codingworkshop.eu.org/xt-sys/exectos/issues/25),
|
||||
which described a fundamental failure in the memory hand-off protocol between the XTLDR and the kernel. In a modern
|
||||
EFI-based boot process, the loader is responsible for preparing the initial execution environment for the kernel, which
|
||||
includes setting up a preliminary set of page tables and enabling paging. The physical address of the top-level Page Map
|
||||
Level (PML) table is stored in the CR3 register, and the CPU relies on this address for all subsequent memory translations.
|
||||
|
||||
The bug in the XTLDR occurred because the loader failed to explicitly reserve the physical frame containing the active
|
||||
PML table in the memory map passed to the kernel. Consequently, the physical address residing in the CR3 register was
|
||||
marked as LoaderFree (Type 2). In the XT memory map specification, Type 2 memory is intended for the kernel to reclaim
|
||||
and use as it sees fit once the loader has finished its task.
|
||||
|
||||
When the ExectOS Memory Manager initialized, it scanned the memory map and correctly identified the LoaderFree regions
|
||||
as available for allocation. As the kernel began to populate its non-paged pool, it eventually requested a new page
|
||||
for its own internal use. By sheer probability, the allocator selected the frame that was currently being used by the
|
||||
CPU as the top-level page table.
|
||||
|
||||
The catastrophic failure occurred during a call to ```RTL::Memory::ZeroMemory()```. As the kernel attempted to clear
|
||||
the newly allocated page for use, it effectively wiped the active page table entries. The immediate result was the
|
||||
destruction of the entire virtual address space. The CPU, suddenly unable to translate the address of the next
|
||||
instruction, the stack, or the Interrupt Descriptor Table (IDT), was forced into a series of faults.
|
||||
|
||||
The QEMU register dump provided in the bug report illustrates the state of the system at the moment of the crash:
|
||||
|
||||
| Register | Value | Description|
|
||||
|----------|--------------------|------------|
|
||||
| RIP | 0xfffff8080002c47a | Virtual address of the instruction causing the fault
|
||||
| RSP | 0xfffff80800051030 | Current stack pointer in the kernel's virtual space
|
||||
| CR0 | 0x80050011 | Paging enabled bit (WP, NE, ET, PE)
|
||||
| CR3 | 0x000000007ce0d000 | Physical address of the corrupt/zeroed PML table
|
||||
| EFER | 0x0000000000000d01 | LMA (Long Mode Active) and NXE (No-Execute) bits set
|
||||
|
||||
Forensic examination of the memory at physical address 0x7CE0D000 using the QEMU monitor confirmed the hypothesis:
|
||||
|
||||
```
|
||||
(qemu) xp /80x 0x7CE0D000
|
||||
000000007ce0d000: 0x00000000 0x00000000 0x00000000 0x00000000
|
||||
... (all zero entries)...
|
||||
```
|
||||
|
||||
Because the PML table contained nothing but zeros, the CPU could not handle the resulting Page Fault (PF) because it
|
||||
could not find the IDT or the code for the exception handler, leading instantly to a Double Fault (DF) and finally
|
||||
a Triple Fault, which triggers a hardware reset. This bug was particularly difficult to diagnose because it was
|
||||
a "delayed-action" failure; the system would appear to boot correctly until the Memory Manager reached a specific point
|
||||
in its allocation sequence, making the crash seem non-deterministic.
|
||||
|
||||
### UEFI Memory Map Volatility and Buffer Overflows
|
||||
Even as the CR3 reservation issue was being addressed, a second major challenge emerged related to the interaction
|
||||
between the bootloader and the UEFI GetMemoryMap service. The UEFI specification requires the loader to allocate a buffer
|
||||
large enough to hold the memory map, which consists of a series of descriptors representing every region of physical RAM.
|
||||
|
||||
The "hard buffer overflow" occurred because of a recursive side effect of the mapping process. In certain edge cases,
|
||||
the act of mapping memory within the bootloader required the allocation of additional memory to store metadata or new
|
||||
page tables. Under some UEFI implementations, these internal allocations would cause the UEFI firmware to update its own
|
||||
internal memory map, potentially splitting existing entries or adding new ones to reflect the change in status.
|
||||
|
||||
If the loader had already calculated the required buffer size based on a previous call to ```GetMemoryMap()```, and
|
||||
a subsequent allocation occurred before the final map was retrieved, the number of entries would increase beyond the
|
||||
capacity of the allocated buffer. This led to a situation where the firmware would write more descriptors than the loader
|
||||
had prepared for, resulting in a buffer overflow that corrupted adjacent memory structures in the loader's address space.
|
||||
|
||||
## Technical Innovations and Repository Evolution
|
||||
Beyond the implementation of the Memory Manager, the ExectOS repository has seen a series of significant updates that
|
||||
reflect a broader push toward code quality and architectural maturity.
|
||||
|
||||
### Enhancements to the Runtime Library (RTL)
|
||||
The development of the Memory Manager was accompanied by significant refinements to the RTL. Specifically,
|
||||
the standardization of the ```RTL::LinkedList::RemoveEntryList()``` routine signature and other related functions was
|
||||
necessary to support the complex data structures required by the memory allocator.
|
||||
|
||||
In a kernel without a Memory Manager, linked lists are often static or use pre-allocated pools. With the introduction
|
||||
of the non-paged pool, the kernel can now use dynamic linked lists to track free and used memory regions, manage object
|
||||
handles, and maintain list-based queues for the scheduler. The standardization of these routines ensures that the Memory
|
||||
Manager's internal bookkeeping is both reliable and performant.
|
||||
|
||||
### The Strategic Implementation of SHA-1 Hashing
|
||||
An interesting addition to the RTL is the implementation of SHA-1 hashing support. While SHA-1 is no longer considered
|
||||
secure for cryptographic signatures, its inclusion in the ExectOS likely serves a more practical, utility-driven purpose.
|
||||
Hashing is a fundamental tool for data deduplication, file system integrity verification, and the generation of unique
|
||||
identifiers for kernel objects.
|
||||
|
||||
## The Impact of the Memory Manager on the XT Architecture Roadmap
|
||||
The implementation of the non-paged pool, while only a "basic" version of the Memory Manager, is the catalyst for the
|
||||
next phase of ExectOS development. For several months, work on many critical kernel components had been stalled because
|
||||
they lacked a mechanism to request memory at runtime.
|
||||
|
||||
### Supporting the Process and Thread Manager
|
||||
The creation of a new process or thread requires the allocation of several structures, including the Process Environment
|
||||
Block (PEB), the Thread Environment Block (TEB), kernel stacks, and the process's own page tables. The Memory Manager's
|
||||
support for various paging levels (PML2 through PML5) is the foundation for this process creation logic. The kernel can
|
||||
now allocate a physical frame for a process's top-level page table and begin building a private virtual address space.
|
||||
|
||||
### Enabling the Object Manager and Handle Database
|
||||
In the XT architecture, almost all resources, like threads, events, and semaphores - are represented as objects. These
|
||||
objects must be allocated from kernel memory and tracked within an object handle database. Before the Memory Manager was
|
||||
integrated, the kernel could only support a fixed, hard-coded number of objects. With a functional non-paged pool now in
|
||||
place, the way is clear to implement the Object Manager, which will dynamically allocate memory for new objects as
|
||||
processes request them, allowing for a much more flexible and scalable system.
|
||||
|
||||
## Conclusion: A Milestone of Stability and Vision
|
||||
The successful merger of the Memory Manager represents more than just the addition of a few thousand lines of code; it
|
||||
is a validation of the XT architecture's design and the development team's ability to solve the most daunting problems
|
||||
in low-level systems programming. By overcoming the CR3 corruption bug and the UEFI memory map issues, ExectOS has
|
||||
demonstrated a level of robustness that is essential for any operating system aiming for general-purpose use.
|
||||
|
||||
As ExectOS moves forward, the presence of the Memory Manager will allow the focus to shift toward user-mode subsystems,
|
||||
driver compatibility layers, and the native application interface. The foundation is now set for ExectOS to transition
|
||||
from a kernel that simply boots to an operating system that can truly execute. The journey to this point was longer than
|
||||
expected, but the lessons learned in the process have resulted in a kernel that is more resilient and better prepared
|
||||
for the challenges of contemporary hardware.
|
||||
@@ -1,11 +0,0 @@
|
||||
+++
|
||||
title = 'The People behind ExectOS'
|
||||
date = 2025-08-17T17:44:12+02:00
|
||||
+++
|
||||
ExectOS is built by a tiny team of passionate developers. We are just two people (for now!) who love hacking on operating
|
||||
systems, learning new things, and sharing our work with the world. We keep things open and collaborative - and if you
|
||||
would like to join, you are more than welcome!
|
||||
|
||||
{{< authors >}}
|
||||
|
||||
Want to get involved? Check out our [GIT repository](https://git.codingworkshop.eu.org/xt-sys/exectos) and join us!
|
||||
@@ -1,8 +0,0 @@
|
||||
- nick: belliash
|
||||
name: Rafal Kupiec
|
||||
role: Core Developer & System Architect
|
||||
email: belliash@codingworkshop.eu.org
|
||||
- nick: harraiken
|
||||
name: Aiken Harris
|
||||
role: Core Developer
|
||||
email: harraiken91@gmail.com
|
||||
@@ -1,2 +1,2 @@
|
||||
{{ $paginator := .Paginate (where site.RegularPages.ByDate.Reverse "Type" "news") 5 }}
|
||||
{{ $paginator := .Paginate (where .Site.RegularPages "Type" "in" (slice "news")) }}
|
||||
{{ partial "list.html" (dict "context" . "paginator" $paginator) }}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
<div class="wrapper_footer">
|
||||
<div class="footer">
|
||||
<div class="footer-left">
|
||||
Copyright© 2017 - 2026, CodingWorkshop
|
||||
Copyright© 2017 - 2024, CodingWorkshop
|
||||
</div>
|
||||
<div class="footer-right">
|
||||
Content is available under <a href="https://www.gnu.org/licenses/fdl-1.3.html">GNU Free Documentation License 1.3 or later</a>.
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Nickname</th>
|
||||
<th>Name</th>
|
||||
<th>Role</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{{ range .Site.Data.authors }}
|
||||
<tr>
|
||||
<td class="nick-cell">
|
||||
<img src="https://www.gravatar.com/avatar/{{ .email | md5 }}?s=50&d=identicon" class="avatar" />
|
||||
<a href="https://git.codingworkshop.eu.org/xt-sys/exectos/commits/branch/master/search?q=author%3A{{ .nick }}">
|
||||
<span>{{ .nick }}</span>
|
||||
</a>
|
||||
</td>
|
||||
<td>{{ .name }}</td>
|
||||
<td>{{ .role }}</td>
|
||||
</tr>
|
||||
{{ end }}
|
||||
</tbody>
|
||||
</table>
|
||||
@@ -105,56 +105,25 @@ pre code {
|
||||
font-weight : normal;
|
||||
}
|
||||
|
||||
table {
|
||||
border-collapse : collapse;
|
||||
font-family : "trebuchet ms", verdana, sans-serif;
|
||||
font-size : 14px;
|
||||
margin : 1em auto;
|
||||
width : 90%;
|
||||
}
|
||||
|
||||
td, th {
|
||||
border-bottom : 1px solid #DDDDDD;
|
||||
padding : 6px 15px;
|
||||
text-align : left;
|
||||
}
|
||||
|
||||
td.nick-cell .avatar {
|
||||
border-radius : 50%;
|
||||
display : inline-block;
|
||||
flex-shrink : 0;
|
||||
height : 25px;
|
||||
margin : 0;
|
||||
margin-right : 10px;
|
||||
object-fit : cover;
|
||||
width : 25px;
|
||||
}
|
||||
|
||||
thead th {
|
||||
background-color : #BEE0FF;
|
||||
border-bottom : 2px solid #186196;
|
||||
color : #186196;
|
||||
font-size : 1.1em;
|
||||
}
|
||||
|
||||
tbody tr {
|
||||
background-color : #FFFFFF;
|
||||
}
|
||||
|
||||
tbody tr:nth-of-type(even) {
|
||||
background-color : #f2f2f2;
|
||||
}
|
||||
|
||||
tbody tr:hover {
|
||||
background-color : #e0e0e0;
|
||||
}
|
||||
|
||||
.content {
|
||||
box-sizing : border-box;
|
||||
font-size : 15px;
|
||||
margin : 1.5em auto 0 auto;
|
||||
max-width : 1060px;
|
||||
padding : 0 20px;
|
||||
margin-left : 30%;
|
||||
margin-right : 30%;
|
||||
margin-top : 1.5em;
|
||||
}
|
||||
|
||||
@media (max-width: 1280px) {
|
||||
.content {
|
||||
margin-left : 60px;
|
||||
margin-right : 50px;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 1024px) {
|
||||
.content {
|
||||
margin-left : 50px;
|
||||
margin-right : 40px;
|
||||
}
|
||||
}
|
||||
|
||||
.content img {
|
||||
@@ -276,11 +245,6 @@ tbody tr:hover {
|
||||
text-align : justify;
|
||||
}
|
||||
|
||||
.nick-cell {
|
||||
align-items : center;
|
||||
display : flex;
|
||||
}
|
||||
|
||||
.pagination {
|
||||
margin : 0px;
|
||||
padding : 0 0 0 0;
|
||||
|
||||
Reference in New Issue
Block a user