From 058c7aaefe6eb05ec4880c04cc8d594c76c2907e Mon Sep 17 00:00:00 2001 From: Aiken Harris Date: Fri, 30 Oct 2026 08:34:05 +0100 Subject: [PATCH] AI in ExectOS Development --- .../20261030-ai-in-exectos-development.md | 210 ++++++++++++++++++ 1 file changed, 210 insertions(+) create mode 100644 content/news/20261030-ai-in-exectos-development.md diff --git a/content/news/20261030-ai-in-exectos-development.md b/content/news/20261030-ai-in-exectos-development.md new file mode 100644 index 0000000..567042b --- /dev/null +++ b/content/news/20261030-ai-in-exectos-development.md @@ -0,0 +1,210 @@ ++++ +title = 'AI in ExectOS Development' +author = 'Aiken Harris' +date = '2026-10-30T06:31:11+01:00' ++++ +Artificial intelligence has become difficult to avoid. People use it to write code, generate images, produce music, +and create videos. People with little or no programming experience are building their first games. The Internet is +filling with material whose origin is increasingly difficult to establish at a glance. + +The discussion surrounding these changes often settles into two camps. One embraces AI for almost everything. The +other rejects anything associated with it, sometimes dismissing a project as "AI slop" before looking at the work itself. + +Where does ExectOS stand? + + +Somewhere in the middle. We see AI as an interesting and useful technology, and as another tool in our development +environment, alongside the IDE and the debugger. Used carefully, it can save substantial time and help us find problems +we might otherwise miss. In our experience, those benefits outweigh the difficulties, provided that we remain +responsible for the result. + +### Does ExectOS Contain AI-Generated Code? +The answer needs more explanation than a simple yes or no. + +Most ExectOS code is written by humans. We do not delegate the creation of complete, ready-to-use functions to a model +and treat its output as finished implementation. AI does contribute proposed fixes and small changes to existing code. +Those contributions undergo thorough review and are often rewritten before we accept them. + +That distinction matters. An agent identifying an incorrect condition and proposing a minimal correction is useful. +Handing over the design and implementation of a kernel subsystem, then committing whatever comes back, is a very +different development process. + +Current models are not reliable enough for us to commit their changes blindly. A proposed patch may compile and even +appear to solve the immediate problem while introducing a workaround that we cannot accept. Suppressing an assertion, +skipping a failing initialization step, or adding a special case can make a symptom disappear without correcting the +underlying fault. + +Operating system development makes this particularly visible. A change to object lifetime management can affect IPC. +A lock can be correct in isolation and still create a deadlock through another call path. A memory-management change may +work on AMD64 and fail on i686. Understanding the affected function is only part of understanding the change. + +Our acceptance criteria remain the same regardless of who, or what, suggested a patch: it must address the cause, fit the +architecture, preserve the relevant contracts, and come with evidence that supports the claimed result. + +### Context Has a Cost +A useful answer often requires a substantial amount of project context: implementation details, calling conventions, +ownership rules, initialization order, architecture differences, and previous decisions. Providing that context +increases processing time and, with hosted models, can increase cost. Leaving it out creates room for assumptions that +do not match the project. + +A large context window does not guarantee that a model will apply every instruction consistently. Over a long session, +or after several rounds of corrections, we sometimes see it lose track of constraints and produce plausible +explanations unsupported by the code. This is one form of what is commonly called hallucination. + +The deterioration can also be less dramatic. An agent may start adding comments explaining why it reverted an earlier +attempt. Those comments make sense within the conversation, but the final source tree contains no trace of that attempt. +A future reader encounters an explanation of something the function does not do, with no reason to understand why it was +mentioned. + +Other changes gradually drift away from established naming, formatting, or implementation conventions. We can document +project rules in AGENTS.md, but the presence of an instruction file does not ensure that every rule will be followed +throughout a long task. + +Specialized agents, reusable skills, and tool integrations through the Model Context Protocol (MCP) help us give tasks +a clearer structure. They provide repeatable procedures and access to real development tools. They do not eliminate +incorrect reasoning or missed instructions. Smaller tasks, explicit checkpoints, and human review remain necessary. + +### Documentation Grounded in the Code +One of the most practical applications is also one of the least spectacular: writing documentation. + +AI can quickly turn an existing implementation into an initial set of comments or Doxygen documentation. It saves us +time describing parameters, return values, and the sequence of operations. It can also identify places where a function's +behavior is difficult to understand without an explanation of its purpose. + +The source code provides a concrete reference, which makes this work easier to verify than an open-ended implementation +request. We still check the result. A model can infer an ownership rule that does not exist, describe a failure path +incorrectly, or turn an accidental implementation detail into a promised interface contract. + +Good documentation explains actual behavior and the reasons behind decisions that would otherwise be unclear. It should +not preserve the transcript of a debugging session or narrate every line of code. AI handles much of the drafting; we +decide what belongs in the source tree. + +### Giving an Agent a Real Debugging Environment +Debugging is where we have invested the most effort in our AI workflow. We have refined the process to the point where +an agent can carry out much of an investigation autonomously. + +Our setup combines prepared agents and skills with a sandbox equipped with the tools needed to build and run ExectOS. +An agent can compile the project, boot it in QEMU under different configurations, inspect logs, and use GDB or LLDB +where appropriate. It can add temporary debug traces, follow execution paths, resolve addresses against symbols, form +a hypothesis, and test a proposed correction. + +This creates a useful investigation loop: + 1. Reproduce the failure and record the starting state. + 2. Add focused instrumentation or inspect execution with a debugger. + 3. Relate the observed behavior to the relevant source and invariants. + 4. Propose a small correction and repeat the failing scenario. + 5. Report the cause, the patch, the validation performed, and any remaining uncertainty. + +While the agent works through that loop, we can work on something else. Instead of spending hours locating the first +incorrect state transition, we can return to a focused patch and a detailed explanation of how the failure occurred. + +The report is as important as the patch. A successful build proves that the code builds. A successful boot proves that +a particular configuration reached a particular point. Neither establishes that a race has disappeared or that every +affected architecture behaves correctly. We need to know which configurations were exercised, whether the original +failure was reproduced, and what the investigation could not establish. + +The agent's autonomy ends at proposing and validating a result. Accepting that result remains our responsibility. + +### Another Pair of Eyes During Review +AI also helps us review code beyond the path involved in a visible failure. It can follow callers, compare similar +implementations, and look for inconsistencies in error handling or resource ownership. + +For a kernel, that can mean examining whether a reference is released on every exit path, whether a lock remains held +across a callback, or whether a size calculation can overflow before allocation. It can draw attention to failures that +require a particular sequence of operations: repeated retries, partial initialization, a counter reaching a boundary, +or a cleanup path entered only after an earlier operation fails. + +These cases are easy to miss during ordinary testing because the usual execution path never reaches them. A model's +ability to inspect several related paths quickly makes it a useful additional reviewer. + +Its findings still need to be demonstrated. Models report false positives, overlook existing protections, and sometimes +suggest a fix that breaks a different contract. We treat a review finding as a lead: establish the relevant state, trace +the path, and determine whether the claimed failure is possible. A convincing explanation becomes useful engineering +evidence when it survives that examination. + +### Other Work Worth Delegating +There are further opportunities to use AI without handing over the architectural decisions. + +**Test design and failure reduction:** An agent can suggest boundary cases, derive a regression scenario from a bug +report, and help reduce a large failure into a smaller reproducer. For ExectOS, useful dimensions include processor +count, architecture, allocation failures, and repeated object creation and destruction. The test must exercise the +intended contract; a test that merely repeats the implementation's assumptions offers little protection. + +**Cross-architecture consistency checks:** Similar code paths can drift apart over time. AI can help compare i686 and +AMD64 implementations, highlighting differences in pointer width, structure layout, alignment, or initialization. +Those differences need architectural analysis before anyone decides that they should be removed. + +**Development tooling:** Log parsers, debugger helpers, build scripts, and small diagnostic utilities are good +candidates for bounded assistance. Improving these tools can shorten many later investigations. We can validate their +output against known inputs before relying on them. + +**Change analysis:** Before changing an interface, a model can help locate consumers, summarize dependencies, and +identify documentation or tests that may need updating. Repository searches and compiler diagnostics provide the +evidence; the model helps organize it into something easier to review. + +**Technical writing:** Development notes, explanations of a resolved bug, and website articles benefit from editorial +assistance, particularly when turning rough notes into clear English. The technical claims still need to come from work +we understand and can substantiate. + +Across these tasks, the most useful output is something we can inspect: a reproducible scenario, a list of concrete +callers, a script with verifiable behavior, or a draft grounded in existing code. + +### Reverse Engineering and Clean-Room Separation +Understanding how another system behaves can be useful when implementing compatible interfaces or investigating +an undocumented contract. AI can assist with that research, including the analysis of permitted reference material, +disassembly, and observed behavior. + +A traditional clean-room workflow separates two roles. One researcher investigates the reference implementation and +prepares a behavioral specification. A separate developer implements the required behavior from that specification, +without access to the reference implementation's source or disassembly. The specification describes observable behavior +and interface requirements, rather than reproducing the original implementation. + +An AI agent can successfully take on much of the research role. With a carefully defined task and suitable tools, it can +trace paths, organize observations, and prepare documentation for a human implementer. For focused investigations, we +have found this faster and more effective than doing every research step manually. + +The documentation is not always satisfactory on the first attempt. It may omit an error case, blur the distinction +between an observation and an inference, or leave an ownership rule ambiguous. We often need further research sessions +to resolve those gaps before the specification is suitable for implementation. + +A prompt alone does not establish clean-room separation. The research and implementation environments must remain +separate, and the implementation side must receive only the reviewed behavioral specification. Reusing a research +conversation that contains reference code would undermine that separation. AI makes parts of the process easier; it does +not make the boundaries unnecessary or guarantee that generated material is free of copied expression. + +### Choosing Models Without Choosing a Camp +We take the same practical approach to model selection. AI is developing quickly enough that committing permanently to +one provider would make little sense for us. + +That does not mean trying every model released. We evaluate options that offer a plausible improvement for our work: +better reasoning over unfamiliar code, more reliable tool use, useful local deployment, or comparable results at +a substantially lower cost. A model that excels at drafting documentation may not be our first choice for investigating +a kernel deadlock. + +We do not limit ourselves to GPT models. If a new Gemini release handles a task better, or a competing model offers +similar results at a much lower price, we are willing to change tools. The important comparison is how they behave in +our actual workflow, including the time we spend checking and correcting their output. + +Recently, we have spent considerable time testing GLM 5.3 Flash. We also actively use local inference, with a setup +capable of running Qwen 3.8 Flash Next. Open-weight models give us the option to run the model on our own hardware and +experiment with the inference environment. They still have hardware costs and capability limits, but they broaden our +choices considerably. + +And, of course, we are looking forward to Qwen 4. Whether it becomes part of our regular workflow will depend on what +it can do for the project. + +### Why This Fits ExectOS +ExectOS is a hobby project. We work on it because operating system development interests us, because building it is +enjoyable, and because it gives us opportunities to learn. We are not doing this to prove something to someone else or +to turn it into a commercial product. + +That makes our time valuable in a particular way. We want to spend it understanding the system, making design decisions, +and building things we care about. If an agent can help locate a difficult bug while we work on another part of the +kernel, that is a worthwhile use of the technology. Reviewing its explanation can also teach us something, provided that +we verify the reasoning rather than simply accepting the conclusion. + +We want ExectOS to have clear, maintainable, well-documented code. Using AI serves that goal only when the resulting work +meets those standards. The tool can investigate, suggest, document, and challenge our assumptions. We remain responsible +for understanding what enters the project. + +P.S. This article was edited with AI assistance. We are glad you made it to the end anyway.