This commit is contained in:
@@ -0,0 +1,210 @@
|
||||
+++
|
||||
title = 'AI in ExectOS Development'
|
||||
author = 'Aiken Harris'
|
||||
date = '2026-10-30T06:31:11+01:00'
|
||||
+++
|
||||
Artificial intelligence has become difficult to avoid. People use it to write code, generate images, produce music,
|
||||
and create videos. People with little or no programming experience are building their first games. The Internet is
|
||||
filling with material whose origin is increasingly difficult to establish at a glance.
|
||||
|
||||
The discussion surrounding these changes often settles into two camps. One embraces AI for almost everything. The
|
||||
other rejects anything associated with it, sometimes dismissing a project as "AI slop" before looking at the work itself.
|
||||
|
||||
Where does ExectOS stand?
|
||||
<!--more-->
|
||||
|
||||
Somewhere in the middle. We see AI as an interesting and useful technology, and as another tool in our development
|
||||
environment, alongside the IDE and the debugger. Used carefully, it can save substantial time and help us find problems
|
||||
we might otherwise miss. In our experience, those benefits outweigh the difficulties, provided that we remain
|
||||
responsible for the result.
|
||||
|
||||
### Does ExectOS Contain AI-Generated Code?
|
||||
The answer needs more explanation than a simple yes or no.
|
||||
|
||||
Most ExectOS code is written by humans. We do not delegate the creation of complete, ready-to-use functions to a model
|
||||
and treat its output as finished implementation. AI does contribute proposed fixes and small changes to existing code.
|
||||
Those contributions undergo thorough review and are often rewritten before we accept them.
|
||||
|
||||
That distinction matters. An agent identifying an incorrect condition and proposing a minimal correction is useful.
|
||||
Handing over the design and implementation of a kernel subsystem, then committing whatever comes back, is a very
|
||||
different development process.
|
||||
|
||||
Current models are not reliable enough for us to commit their changes blindly. A proposed patch may compile and even
|
||||
appear to solve the immediate problem while introducing a workaround that we cannot accept. Suppressing an assertion,
|
||||
skipping a failing initialization step, or adding a special case can make a symptom disappear without correcting the
|
||||
underlying fault.
|
||||
|
||||
Operating system development makes this particularly visible. A change to object lifetime management can affect IPC.
|
||||
A lock can be correct in isolation and still create a deadlock through another call path. A memory-management change may
|
||||
work on AMD64 and fail on i686. Understanding the affected function is only part of understanding the change.
|
||||
|
||||
Our acceptance criteria remain the same regardless of who, or what, suggested a patch: it must address the cause, fit the
|
||||
architecture, preserve the relevant contracts, and come with evidence that supports the claimed result.
|
||||
|
||||
### Context Has a Cost
|
||||
A useful answer often requires a substantial amount of project context: implementation details, calling conventions,
|
||||
ownership rules, initialization order, architecture differences, and previous decisions. Providing that context
|
||||
increases processing time and, with hosted models, can increase cost. Leaving it out creates room for assumptions that
|
||||
do not match the project.
|
||||
|
||||
A large context window does not guarantee that a model will apply every instruction consistently. Over a long session,
|
||||
or after several rounds of corrections, we sometimes see it lose track of constraints and produce plausible
|
||||
explanations unsupported by the code. This is one form of what is commonly called hallucination.
|
||||
|
||||
The deterioration can also be less dramatic. An agent may start adding comments explaining why it reverted an earlier
|
||||
attempt. Those comments make sense within the conversation, but the final source tree contains no trace of that attempt.
|
||||
A future reader encounters an explanation of something the function does not do, with no reason to understand why it was
|
||||
mentioned.
|
||||
|
||||
Other changes gradually drift away from established naming, formatting, or implementation conventions. We can document
|
||||
project rules in AGENTS.md, but the presence of an instruction file does not ensure that every rule will be followed
|
||||
throughout a long task.
|
||||
|
||||
Specialized agents, reusable skills, and tool integrations through the Model Context Protocol (MCP) help us give tasks
|
||||
a clearer structure. They provide repeatable procedures and access to real development tools. They do not eliminate
|
||||
incorrect reasoning or missed instructions. Smaller tasks, explicit checkpoints, and human review remain necessary.
|
||||
|
||||
### Documentation Grounded in the Code
|
||||
One of the most practical applications is also one of the least spectacular: writing documentation.
|
||||
|
||||
AI can quickly turn an existing implementation into an initial set of comments or Doxygen documentation. It saves us
|
||||
time describing parameters, return values, and the sequence of operations. It can also identify places where a function's
|
||||
behavior is difficult to understand without an explanation of its purpose.
|
||||
|
||||
The source code provides a concrete reference, which makes this work easier to verify than an open-ended implementation
|
||||
request. We still check the result. A model can infer an ownership rule that does not exist, describe a failure path
|
||||
incorrectly, or turn an accidental implementation detail into a promised interface contract.
|
||||
|
||||
Good documentation explains actual behavior and the reasons behind decisions that would otherwise be unclear. It should
|
||||
not preserve the transcript of a debugging session or narrate every line of code. AI handles much of the drafting; we
|
||||
decide what belongs in the source tree.
|
||||
|
||||
### Giving an Agent a Real Debugging Environment
|
||||
Debugging is where we have invested the most effort in our AI workflow. We have refined the process to the point where
|
||||
an agent can carry out much of an investigation autonomously.
|
||||
|
||||
Our setup combines prepared agents and skills with a sandbox equipped with the tools needed to build and run ExectOS.
|
||||
An agent can compile the project, boot it in QEMU under different configurations, inspect logs, and use GDB or LLDB
|
||||
where appropriate. It can add temporary debug traces, follow execution paths, resolve addresses against symbols, form
|
||||
a hypothesis, and test a proposed correction.
|
||||
|
||||
This creates a useful investigation loop:
|
||||
1. Reproduce the failure and record the starting state.
|
||||
2. Add focused instrumentation or inspect execution with a debugger.
|
||||
3. Relate the observed behavior to the relevant source and invariants.
|
||||
4. Propose a small correction and repeat the failing scenario.
|
||||
5. Report the cause, the patch, the validation performed, and any remaining uncertainty.
|
||||
|
||||
While the agent works through that loop, we can work on something else. Instead of spending hours locating the first
|
||||
incorrect state transition, we can return to a focused patch and a detailed explanation of how the failure occurred.
|
||||
|
||||
The report is as important as the patch. A successful build proves that the code builds. A successful boot proves that
|
||||
a particular configuration reached a particular point. Neither establishes that a race has disappeared or that every
|
||||
affected architecture behaves correctly. We need to know which configurations were exercised, whether the original
|
||||
failure was reproduced, and what the investigation could not establish.
|
||||
|
||||
The agent's autonomy ends at proposing and validating a result. Accepting that result remains our responsibility.
|
||||
|
||||
### Another Pair of Eyes During Review
|
||||
AI also helps us review code beyond the path involved in a visible failure. It can follow callers, compare similar
|
||||
implementations, and look for inconsistencies in error handling or resource ownership.
|
||||
|
||||
For a kernel, that can mean examining whether a reference is released on every exit path, whether a lock remains held
|
||||
across a callback, or whether a size calculation can overflow before allocation. It can draw attention to failures that
|
||||
require a particular sequence of operations: repeated retries, partial initialization, a counter reaching a boundary,
|
||||
or a cleanup path entered only after an earlier operation fails.
|
||||
|
||||
These cases are easy to miss during ordinary testing because the usual execution path never reaches them. A model's
|
||||
ability to inspect several related paths quickly makes it a useful additional reviewer.
|
||||
|
||||
Its findings still need to be demonstrated. Models report false positives, overlook existing protections, and sometimes
|
||||
suggest a fix that breaks a different contract. We treat a review finding as a lead: establish the relevant state, trace
|
||||
the path, and determine whether the claimed failure is possible. A convincing explanation becomes useful engineering
|
||||
evidence when it survives that examination.
|
||||
|
||||
### Other Work Worth Delegating
|
||||
There are further opportunities to use AI without handing over the architectural decisions.
|
||||
|
||||
**Test design and failure reduction:** An agent can suggest boundary cases, derive a regression scenario from a bug
|
||||
report, and help reduce a large failure into a smaller reproducer. For ExectOS, useful dimensions include processor
|
||||
count, architecture, allocation failures, and repeated object creation and destruction. The test must exercise the
|
||||
intended contract; a test that merely repeats the implementation's assumptions offers little protection.
|
||||
|
||||
**Cross-architecture consistency checks:** Similar code paths can drift apart over time. AI can help compare i686 and
|
||||
AMD64 implementations, highlighting differences in pointer width, structure layout, alignment, or initialization.
|
||||
Those differences need architectural analysis before anyone decides that they should be removed.
|
||||
|
||||
**Development tooling:** Log parsers, debugger helpers, build scripts, and small diagnostic utilities are good
|
||||
candidates for bounded assistance. Improving these tools can shorten many later investigations. We can validate their
|
||||
output against known inputs before relying on them.
|
||||
|
||||
**Change analysis:** Before changing an interface, a model can help locate consumers, summarize dependencies, and
|
||||
identify documentation or tests that may need updating. Repository searches and compiler diagnostics provide the
|
||||
evidence; the model helps organize it into something easier to review.
|
||||
|
||||
**Technical writing:** Development notes, explanations of a resolved bug, and website articles benefit from editorial
|
||||
assistance, particularly when turning rough notes into clear English. The technical claims still need to come from work
|
||||
we understand and can substantiate.
|
||||
|
||||
Across these tasks, the most useful output is something we can inspect: a reproducible scenario, a list of concrete
|
||||
callers, a script with verifiable behavior, or a draft grounded in existing code.
|
||||
|
||||
### Reverse Engineering and Clean-Room Separation
|
||||
Understanding how another system behaves can be useful when implementing compatible interfaces or investigating
|
||||
an undocumented contract. AI can assist with that research, including the analysis of permitted reference material,
|
||||
disassembly, and observed behavior.
|
||||
|
||||
A traditional clean-room workflow separates two roles. One researcher investigates the reference implementation and
|
||||
prepares a behavioral specification. A separate developer implements the required behavior from that specification,
|
||||
without access to the reference implementation's source or disassembly. The specification describes observable behavior
|
||||
and interface requirements, rather than reproducing the original implementation.
|
||||
|
||||
An AI agent can successfully take on much of the research role. With a carefully defined task and suitable tools, it can
|
||||
trace paths, organize observations, and prepare documentation for a human implementer. For focused investigations, we
|
||||
have found this faster and more effective than doing every research step manually.
|
||||
|
||||
The documentation is not always satisfactory on the first attempt. It may omit an error case, blur the distinction
|
||||
between an observation and an inference, or leave an ownership rule ambiguous. We often need further research sessions
|
||||
to resolve those gaps before the specification is suitable for implementation.
|
||||
|
||||
A prompt alone does not establish clean-room separation. The research and implementation environments must remain
|
||||
separate, and the implementation side must receive only the reviewed behavioral specification. Reusing a research
|
||||
conversation that contains reference code would undermine that separation. AI makes parts of the process easier; it does
|
||||
not make the boundaries unnecessary or guarantee that generated material is free of copied expression.
|
||||
|
||||
### Choosing Models Without Choosing a Camp
|
||||
We take the same practical approach to model selection. AI is developing quickly enough that committing permanently to
|
||||
one provider would make little sense for us.
|
||||
|
||||
That does not mean trying every model released. We evaluate options that offer a plausible improvement for our work:
|
||||
better reasoning over unfamiliar code, more reliable tool use, useful local deployment, or comparable results at
|
||||
a substantially lower cost. A model that excels at drafting documentation may not be our first choice for investigating
|
||||
a kernel deadlock.
|
||||
|
||||
We do not limit ourselves to GPT models. If a new Gemini release handles a task better, or a competing model offers
|
||||
similar results at a much lower price, we are willing to change tools. The important comparison is how they behave in
|
||||
our actual workflow, including the time we spend checking and correcting their output.
|
||||
|
||||
Recently, we have spent considerable time testing GLM 5.3 Flash. We also actively use local inference, with a setup
|
||||
capable of running Qwen 3.8 Flash Next. Open-weight models give us the option to run the model on our own hardware and
|
||||
experiment with the inference environment. They still have hardware costs and capability limits, but they broaden our
|
||||
choices considerably.
|
||||
|
||||
And, of course, we are looking forward to Qwen 4. Whether it becomes part of our regular workflow will depend on what
|
||||
it can do for the project.
|
||||
|
||||
### Why This Fits ExectOS
|
||||
ExectOS is a hobby project. We work on it because operating system development interests us, because building it is
|
||||
enjoyable, and because it gives us opportunities to learn. We are not doing this to prove something to someone else or
|
||||
to turn it into a commercial product.
|
||||
|
||||
That makes our time valuable in a particular way. We want to spend it understanding the system, making design decisions,
|
||||
and building things we care about. If an agent can help locate a difficult bug while we work on another part of the
|
||||
kernel, that is a worthwhile use of the technology. Reviewing its explanation can also teach us something, provided that
|
||||
we verify the reasoning rather than simply accepting the conclusion.
|
||||
|
||||
We want ExectOS to have clear, maintainable, well-documented code. Using AI serves that goal only when the resulting work
|
||||
meets those standards. The tool can investigate, suggest, document, and challenge our assumptions. We remain responsible
|
||||
for understanding what enters the project.
|
||||
|
||||
P.S. This article was edited with AI assistance. We are glad you made it to the end anyway.
|
||||
Reference in New Issue
Block a user