Implement initial memory manager infrastructure #24

Open

harraiken wants to merge 169 commits from memmgr into master

pull from: memmgr

merge into: xt-sys:master

xt-sys:master

Conversation 0 Commits 169 Files Changed 64 +7942 -610

harraiken commented 2026-02-24 22:16:08 +01:00

Owner

Copy Link

This merge request introduces the foundational infrastructure for the Memory Manager by implementing the initialization phase and critical bootloader fixes. While general-purpose heap allocation and freeing routines are not yet fully implemented, this changeset establishes the necessary groundwork for future memory management features.

Key features:

• Scans and processes memory descriptors provided by the bootloader.
• Dynamically initializes the memory layer based on the detected amount of available RAM.
• Introduces stack allocation and stack release routines.
• Lays structural groundwork for future physical and virtual memory management.
• Implements basic non-paged pool allocator
• Fixes multiple issues in the bootloader related to memory handling.
• Corrects PAE-related memory handling, including #23.

This merge request introduces the foundational infrastructure for the Memory Manager by implementing the initialization phase and critical bootloader fixes. While general-purpose heap allocation and freeing routines are not yet fully implemented, this changeset establishes the necessary groundwork for future memory management features. Key features: * Scans and processes memory descriptors provided by the bootloader. * Dynamically initializes the memory layer based on the detected amount of available RAM. * Introduces stack allocation and stack release routines. * Lays structural groundwork for future physical and virtual memory management. * Implements basic non-paged pool allocator * Fixes multiple issues in the bootloader related to memory handling. * Corrects PAE-related memory handling, including #23.

harraiken added 159 commits 2026-02-24 22:16:09 +01:00

Implement unified PTE accessors and management helpers ffa480d69a

Add missing EmptyPteList field to MMPAGEMAP_INFO f6dac12057

Add PTE management routines 1150b9ecdb

Add input qualifiers to paging interface definitions dfb0284427

Add input qualifiers to page map interface definitions c28c3f8344

Unify MMPML2_PTE field naming convention fd13091476

Unify PTE pointer types across MM subsystem cb2da54956

Add PTE attribute definitions dda8f88830

Replace writable flag with AttributesMask in PTE setup 071c840ca8

Make MM::Paging::GetExtendedPhysicalAddressingStatus public 10b8ab347a

Add GetPageFrameNumber() to PTE interfaces 66f27e4b9a

Fix class name 0cf178a648

Add page list and PTE pool type enums 7b11a8feb1

Prevent duplicate object generation by linking xtoskrnl with libxtos 35523a230a

Add kernel parameters section and fix minor formatting issues 20b0bfdfad

Correct LA57 paging base addresses, add self-map constants and extend PTE structures 0097cb88d7

Add PFN_COUNT typedef c099882866

Refactor PageMap to enable architecture-specific VA translation 931586eebd

Add MM::PageMap::GetXpaStatus() for querying PML level 4265ae92d0

Implement virtual address resolvers for all page map levels and add XPA status accessor d4287198b0

Split paging interface into arch-specific code 4a7494ad3f

Fix incorrect pointer types 86aa22e5f8

Fix incorrect pointer types 8054bb915a

Refactor XPA detection API c710ec4688

Add architecture-specific GetPteDistance 7694df7744

Add missing virtual GetPteDistance to pagemap interface 63c27a149a

Fix GetP5eAddress return type ee9514fd5c

Refactor memory manager initialization into MM::Manager 237f6a2974

Extract PFN management into separate module 783a9eea3a

Make memory type verification helpers accessible to PFN 17b5649362

Harden PFN initialization and expose page count eae48320f3

Tidy up memory type verification helpers b7bbf9ffa8

Compute PFN database size during MM initialization 5224dc315f

Introduce kernel virtual memory layout 070c508e42

Add bootstrap physical page allocator 5cf3dfa844

Implement early page table mapping routines 36c273ea13

Compute PTE count per page from entry size 7f0ca6a948

Split PTE implementation per architecture dc23f91110

Remove unsupported PML4/PML5 PTE interfaces 2265a4a522

Extend memory layout 020b7c7676

Architecture-specific system PTE limits 00d428d8de

Move memory layout initialization to architecture-specific code 728241f998

Add virtual address validation and system PTE helpers cb4d113e31

Call page table initialization f1a76bc01a

Update SelfMapAddress 049c9c6bbd

Implement initial virtual memory layout setup 687c58d923

Fix paging abstraction for PDE/PTE virtual address calculation b183d52806

Clarify page table entry offset semantics 570301bb35

Add early spin lock initialization 7b93c39348

Make PPE mapping architecture-specific 24dccf4bed

Respect architecture-specific PTE layouts and write PTEs via PML-aware helpers 755a167f2c

Extend PTE helpers with raw read and write support e237a944cc

Select correct self-map base for PAE and non-PAE paging 6aa148784b

Fix PTE free list sentinel handling 643fd0d1e8

Fix physical page count overflow by using 64-bit type 7575526f07

Bring up i686 page table initialization f0a06db7d2

Update memory manager type definitions and constants 560cd43b34

Update modified page list enum terminology 0fd2b8b729

Initialize memory manager during kernel startup b8e81e2223

Add storage for PFN database size c7cc536685

Introduce page coloring support to memory manager 288b2f8b24

Include mm/colors.cc in kernel build configuration b3b874d3ce

Initialize system page tables and configure kernel mappings 1e3917882c

Initialize system PTE pools and implement reservation routines 5012c8dc37

Implement tracking of available physical pages b7c004528a

Expose PFN database lookup via GetPfnEntry 4593a89a9b

Implement PFN database initialization and memory descriptor processing 0880a0f344

Refactor memory clearing calls and cleanup code style 04599161da

Update PTE support to use safe write accessors d891088b1a

Minor style fixes in MM includes 200e9132b1

Initialize paged pool sizing logic 7cb3d1764b

Ensure page map structures are self-mapped 28f49dd545

Reserve space for color tables to fix invalid memory access 49e97fb8b4

Merge branch 'master' into memmgr c1514557f6

Introduce page fault handling infrastructure 0d2d41dcda

Implement logic to link physical pages to PTEs 4a00179af2

Implement kernel stack allocation logic 4c7c914a1c

Simplify stack page count calculation f4c49e2f25

Adjust thread initialization to match new stack allocator signature d8fc223140

Improve formatting 53116b86a3

Prevent initialization of invisible memory ranges aced62e790

Initialize PTE template dynamically to resolve build warnings dbda6bbb29

Remove redundant check for invisible memory regions 00702bfb23

Introduce per-page-color modified page lists 5ff0cad094

Introduce helper functions for querying the software prototype and transition bits of PTE c080f74714

Implement generic PFN list linking function 7bcd78fdf3

Properly handle bad physical pages f653b9f79c

Add explicit default initialization for MM::Colors::ModifiedPages list 0159262ee0

Add support for transitioning PTE to invalid state cc76ea40ee

Abstract PTE list terminator into paging layer 329143b4f6

Initialize system PTEs with arch-specific list terminator 5e5b4a8392

Remove dead code from paging and PTE management 455349f2d7

Clean up paging code 9f359c10ed

Add mechanism to free system PTEs and merge adjacent clusters 0a43a93f41

Implement kernel stack deallocation and physical page freeing logic cb6efc648f

Add missing semicolon 46576398a2

Fix deadlock by reducing lock scope e66baa0da0

Replace ULONG with PFN_COUNT in system PTE variables 410286d012

Optimize system PTE deallocation by avoiding immediate and expensive TLB flush 3a087766cc

Allow PFN insertion at the beginning of standby list 154ca7be35

Add temporary fallback for BeginStandbyList insertion 4b50278ac9

Add logic to insert pages at the head of standby list bd1a3605d2

Make memory descriptor processing architecture-dependent 9027632c4f

Ensure every page in a bad memory region is marked as bad 36e53bfc8c

Correctly setup PFN database for ROM and in-use pages bee91d0c71

Prevent adding referenced pages to the free list b91c79e090

Temporary fix for PAE addressing limits and KSEG0 base mapping 9ed851ed1f

Relocate kernel and modules to KSEG0 memory space ec94e2341c

Remove manual virtual address tracking from boot sequence 5500192575

Update function documentation and remove debug prints 032cab7f2f

Clean up unused physical-to-virtual conversion routines 34c33a3b53

Extract MapDescriptor logic and simplify memory mapping API 15edd98242

Abstract base mapping address retrieval 11f7c25713

Drop obsolete KSEG0_KERNEL_BASE definition 0766eb4566

Annotate input parameters 587b85d0a4

Correctly initialize PFN entries for pre-mapped KSEG0 based memory 0f38d39705

Standardize ValidPte setup across architectures 7da6bcc75e

Add missing annotations c3607ea943

Add routine to retrieve installed memory size 46c24e653e

Fix broken PPE check on i686 d3edfef53b

Rollback bootloader memory management changes 0b1b76e9df

Correctly advance virtual address after mapping non-free regions b5f220a2ae

Ensure contiguous virtual memory mapping 31b0e4f441

Remove unused kernel base address definition 2c8eb6d692

Calculate total boot image size and pass it to kernel a4b9f495e5

Limit system PTE space mapping to calculated pool size 446ce920ec

Fix incorrect header guard comment fd29cf55ef

Add skeleton for memory pool allocator 7f6114f8e5

Calculate aligned boot image size from loader parameters a7820ff568

Add private helper declaration 72f34c8286

Refactor memory layout structure 58669d3074

Move PFN database size tracking to memory layout 5e3fb7a5a3

Fix PFN database size calculation call sites 54e75c9345

Enable non-paged pool setup 726fd84241

Overhaul kernel memory layout initialization and pool sizing 6a7bc64ac7

Remove redundant PFN database alignment 825de8b471

Handle non-paged pool overflow 19f5307be6

Centralize memory layout dumping b16dbb19f8

Fix PFN truncation on i686 with PAE enabled ba4ac6cec8

Fix physical address limit checks 7f0341bb83

Use 64-bit format specifiers for memory mapping logs 3d7f512377

Refactor PFN initialization to ensure proper page table setup eb0957dbd4

Fix missing assignment of PointerPte fae72f5326

Refactor PFN database initialization loop on i686 7b6e284d39

Map and zero entire PFN database upfront bc391d6e1e

Fix coding style 9b8417565b

Remove explicit identity mapping for internal page tables 0590ad3bcd

Introduce page directory initialization helper b51f21f55c

Ensure paging hierarchy exists before processing memory descriptors 0763a9522b

Fix stale comment a46f30045a

Deduplicate PFN descriptor processing logic across architectures 0ce2741e18

Minor style fixes 2e0e085acb

Fix boot image size alignment calculation 80ea0b49d0

Refactor EFI memory mapping to support distinct mapping strategies fa64507350

Map PDE and PTE ranges for i686 non-paged pool 9a5ef6fc00

Reorder initialization sequence and flush TLB 53f7945771

Fix PFN calculation truncation for memory above 4GB to prevent memory descriptor aliasing on PAE systems e2da6220f2

Add interface to retrieve page map level (PML) f7b7b61ea4

Merge branch 'master' into memmgr a136f21f4b

harraiken added 1 commit 2026-02-25 12:14:38 +01:00

Add definition for guarded PTE flag and remove hardcoded value 7a44901064

harraiken added 1 commit 2026-02-25 13:08:58 +01:00

Merge branch 'master' into memmgr 44fa2ca13a

harraiken added 1 commit 2026-02-25 19:28:08 +01:00

Refactor PFN linking logic bfdb7bc476

harraiken added 1 commit 2026-02-25 20:26:35 +01:00

Expose the number of available physical pages 0658e98436

harraiken added 1 commit 2026-02-26 10:57:31 +01:00

Correct comment phrasing 278def3081

harraiken added 1 commit 2026-02-26 13:43:31 +01:00

Implement page allocation interface 511dd15c0c

harraiken added 1 commit 2026-02-26 13:54:20 +01:00

Initialize paged pool alongside non-paged pool d7d125dd50

harraiken added 1 commit 2026-02-26 16:42:30 +01:00

Initialize paged pool after PFN database setup 5778a761b5

harraiken added 1 commit 2026-02-26 20:10:29 +01:00

Implement basic non-paged pool allocator 7bdd0dfe2c

harraiken added 1 commit 2026-02-26 20:12:38 +01:00

Add MIN and MAX helper macros a72bfd3902

harraiken referenced this pull request 2026-02-27 07:10:57 +01:00

Basic Memory Manager implementation #13

harraiken referenced this pull request 2026-02-27 07:11:16 +01:00

XTLDR truncates memory map entries above 4GB, preventing PAE utilization #23

All checks were successful

Hide all checks

Builds / ExectOS (i686, release) (push) Successful in 29s

Details

Builds / ExectOS (amd64, debug) (push) Successful in 40s

Details

Builds / ExectOS (amd64, release) (push) Successful in 32s

Details

Builds / ExectOS (i686, debug) (push) Successful in 38s

Details

This pull request doesn't have enough required approvals yet. 0 of 1 approvals granted from users or teams on the allowlist.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin memmgr:memmgr

git checkout memmgr

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

API CHANGE

Requires or requests API change

BUG

Something is not working

DUPLICATE

This issue or pull request already exists

ENHANCEMENT

New feature request or implementation

HELP WANTED

Needs some help

IDEA

Ideas and proposals

INVALID

Something is wrong

MODDING

Visual design modifications

QUESTION

More information is needed

REFACTORING

Code refactoring

SYNC

Synchronization with other project

TRANSLATION

Translation and localization issue or change

UPSTREAM

The patch needs to be sent upstream

WONTFIX

This won't be fixed

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

belliash harraiken

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: xt-sys/exectos#24