Memory Is Not Context in AI Systems

2026-03-11T00:00:00+00:00

Large language models rely on a context window</strong> to process information. Because the context can include prior conversation, code snippets, and instructions, it is often treated as a form of system memory.</p>

Architecturally, however, context and memory are fundamentally different concepts</strong>.</p>

Understanding this distinction becomes critical when building AI systems that go beyond simple interactions.</p>

Context vs Memory</h2>
Context</strong> is temporary input passed to the model during a request.</p>
Memory</strong> is persistent state that exists across requests, sessions, and system runs.</p>
Property</th> Context</th> Memory</th></tr></thead>

Lifetime</td> Single request</td> Persistent</td></tr>
Structure</td> Unstructured text</td> Structured data</td></tr>
Access</td> Sequential tokens</td> Queryable</td></tr>
Purpose</td> Provide immediate input</td> Store knowledge</td></tr> </tbody></table>
A context window exists only for the duration of a request. Memory persists across time and can be referenced by future operations.</p>
The Architectural Misconception</h2>
In many AI workflows, the system attempts to emulate memory by continually expanding the prompt:</p>

chat history</li>
code snippets</li>
system instructions</li>
past decisions</li> </ul>
This approach works surprisingly well at small scales.</p>
However, as system complexity grows, several problems appear.</p>
Failure Modes of Context-as-Memory</h2>
1. Context Growth</h3>
Even with large context windows, the total capacity is finite.</p>
Engineering systems cannot rely on prompts that grow indefinitely.</p>
2. Signal Dilution</h3>
When everything is included in the prompt, important information loses priority.</p>
Critical details become statistically less visible among tokens.</p>
3. Lack of Structure</h3>
Context is fundamentally a stream of text</strong>.</p>
Memory, on the other hand, requires:</p>

references</li>
relationships</li>
retrieval mechanisms</li>
query capabilities</li> </ul>
Without these, the system cannot efficiently locate relevant knowledge.</p>
At that point the system is not maintaining memory — it is simply constructing increasingly long prompts.</p>
The Role of Indexing</h2>
The solution is not expanding the context window.</p>
The solution is introducing indexed memory</strong>.</p>
In traditional software systems, data is not retrieved by scanning entire datasets on every request. Instead, systems rely on indexes to locate relevant information efficiently.</p>
AI systems benefit from the same principle.</p>
Instead of embedding all information into prompts, the system retrieves only what is relevant to the current request.</p>
Indexed Knowledge</h2>
Examples of indexed memory structures in AI-assisted development systems include:</p>

Indexed codebases</strong> Retrieve relevant modules instead of entire repositories.</p> </li>

Indexed architectural decisions</strong> Allow models to reference prior system design choices.</p> </li>

Indexed artifacts</strong> Enable reuse of generated outputs instead of regeneration.</p> </li>

Indexed conversations</strong> Preserve meaningful discussions across sessions.</p> </li> </ul>
With indexing in place, the workflow changes fundamentally.</p>
Instead of reading everything, the system performs targeted retrieval.</p>
Context as a Projection of Memory</h2>
In a well-structured system:</p>

Persistent knowledge is stored in memory.</li>
Retrieval mechanisms query that knowledge.</li>
Relevant fragments are projected into the model context.</li> </ol>
Context becomes a temporary view of memory</strong>, not the memory itself.</p>
Architectural Model</h2>
Example flow:</p>
Context Window System Memory</span></span> </span> +--------------------------+ +------------------------------+</span></span> | Prompt | | Persistent Knowledge |</span></span> |--------------------------| |------------------------------|</span></span> | instructions | | code index |</span></span> | chat history | | architectural decisions |</span></span> | code snippets | | documentation |</span></span> | logs | | artifacts |</span></span> | ... | | ... |</span></span> +-------------+------------+ +--------------+---------------+</span></span> | |</span></span> v v</span></span> +----------+ +---------------+</span></span> | MODEL | | RETRIEVAL |</span></span> +----------+ +-------+-------+</span></span> |</span></span> v</span></span> +---------+</span></span> | CONTEXT |</span></span> +---------+</span></span></code></pre>Implications for AI System Design</h2> Once systems introduce indexed memory, the design focus shifts.</p> Instead of asking:</p> How large is the context window?</p> </blockquote> Architects must ask:</p> How effectively can the system retrieve relevant knowledge?</p> </blockquote> This reframes the problem from prompt engineering</strong> to system architecture</strong>.</p> Conclusion</h2> Context windows are useful mechanisms for providing temporary input to language models.</p> They are not a substitute for persistent knowledge systems.</p> As AI becomes part of real engineering workflows, systems will require:</p> persistent memory</li> structured knowledge</li> indexing</li> retrieval mechanisms</li> </ul> In other words, knowledge infrastructure</strong>, not just longer prompts.</p> Originally published on GitHub — engineering-notes/memory-is-not-context-in-ai.md</a>.</em></p> Fuzz Testing in Modern Systems 2026-03-05T00:00:00+00:00 Context</h2> Modern software systems operate in increasingly unpredictable environments. They process untrusted inputs, integrate with external systems, and expose APIs that may receive malformed, adversarial, or unexpected data.</p> Traditional testing approaches — unit tests, integration tests, and even property-based testing — rely on explicitly defined test cases</strong>. While effective, they often fail to explore the vast input space that real systems encounter in production.</p> This is where fuzz testing</strong> becomes a powerful tool.</p> Fuzz testing (or fuzzing</em>) is a technique where a program is continuously executed with automatically generated, semi-random inputs</strong> in order to discover crashes, panics, memory safety violations, and logic errors.</p> Originally popular in security research, fuzzing is now widely used in modern software engineering, particularly in systems programming.</p> What Fuzz Testing Actually Tests</h2> Fuzzing focuses on discovering unexpected behavior under invalid or extreme inputs</strong>.</p> Typical issues discovered via fuzzing include:</p> Panics and crashes</li> Memory safety violations</li> Infinite loops</li> Unexpected error states</li> Parser bugs</li> Incorrect assumptions about input structure</li> </ul> Fuzzing is particularly effective for components such as:</p> Parsers</li> Protocol implementations</li> Serialization/deserialization logic</li> Binary formats</li> Compilers and interpreters</li> Networking layers</li> </ul> Many critical vulnerabilities in widely used software (OpenSSL, browsers, kernels) have been discovered through fuzzing.</p> How Modern Fuzzers Work</h2> Modern fuzzers are not purely random.</p> Most production-grade fuzzers use coverage-guided fuzzing</strong>.</p> The idea is simple:</p> The program is instrumented to track code coverage.</li> The fuzzer generates inputs.</li> Inputs that explore new execution paths are kept.</li> The fuzzer mutates those inputs to discover more paths.</li> </ol> Over time, the fuzzer builds a corpus of interesting inputs</strong> that maximizes code coverage.</p> Popular fuzzing engines include:</p> libFuzzer</li> AFL/AFL++</li> honggfuzz</li> cargo-fuzz (Rust wrapper around libFuzzer)</li> </ul> Coverage-guided fuzzing allows millions of test cases to be executed automatically.</p> Fuzz Testing in Rust</h2> Rust is already memory safe by design, but fuzzing is still extremely valuable.</p> It helps detect:</p> Panics caused by invalid assumptions</li> Logical inconsistencies</li> Incorrect parsing logic</li> Unexpected interactions between components</li> </ul> The most common approach in Rust is using cargo-fuzz</strong>, which integrates with libFuzzer</code>.</p> Example:</p> cargo</span> install</span> cargo-fuzz</span></span> cargo</span> fuzz</span> init</span></span> cargo</span> fuzz</span> run</span> fuzz_target</span></span></code></pre> A typical fuzz target might look like:</p> #</span>!</span>[</span>no_main</span>]</span></span> use</span> libfuzzer_sys</span>::</span>fuzz_target</span>;</span></span> </span> fuzz_target!</span>(</span>|</span>data</span>:</span> &</span>[</span>u8</span>]</span>|</span> {</span></span> let</span> _</span> =</span> my_parser</span>::</span>parse</span>(</span>data</span>)</span>;</span></span> }</span>)</span>;</span></span></code></pre> The fuzzer then continuously mutates data</code> to explore different execution paths.</p> The Role of the Corpus</h2> Fuzzers maintain a corpus</strong>, a collection of inputs that produce unique execution paths.</p> The corpus serves two important roles:</p> It improves the effectiveness of the fuzzer.</li> It becomes a growing library of edge cases discovered during testing.</li> </ol> A good practice is to seed the corpus with:</p> Valid examples</li> Known edge cases</li> Minimal inputs</li> Inputs derived from production logs</li> </ul> Over time, the corpus evolves into a powerful regression dataset.</p> Integrating Fuzzing into CI</h2> Fuzzing is computationally expensive and typically runs longer than standard test suites.</p> A practical strategy is:</p> Pull Requests</strong></p> Build fuzz targets</li> Ensure they compile</li> </ul> Nightly CI</strong></p> Run fuzzing for a fixed time window (e.g., 10–30 minutes)</li> </ul> Security/Reliability pipelines</strong></p> Run fuzzing continuously</li> </ul> When a crash is discovered, fuzzers automatically minimize the input and save it to the corpus for reproduction.</p> Where Fuzzing Delivers the Most Value</h2> Fuzz testing is most effective for:</p> Parsing logic</li> Protocol implementations</li> Low-level infrastructure</li> Serialization frameworks</li> Compilers and DSL interpreters</li> API gateways and routing layers</li> </ul> For example, in a web framework, fuzzing can target:</p> HTTP parsers</li> Routing logic</li> header parsing</li> query parsing</li> middleware pipelines</li> </ul> These are areas where malformed inputs are common in real-world traffic.</p> Fuzzing vs Property-Based Testing</h2> These two techniques are often confused but serve different roles.</p> Property-based testing verifies logical properties of a system</strong>.</p> Example:</p> For any valid input, serialization followed by deserialization should return the same value.</p> </blockquote> Fuzzing, on the other hand, is designed to discover unexpected failures</strong> caused by malformed inputs.</p> The two approaches complement each other and should often be used together.</p> Practical Guidelines</h2> Some practical lessons from applying fuzz testing in production systems:</p> 1. Fuzz small components first</strong></p> Fuzzing is most effective when targeting isolated functions such as parsers.</p> 2. Avoid complex setup</strong></p> Fuzz targets should be lightweight and deterministic.</p> 3. Fail loudly</strong></p> Panics and crashes should not be hidden — they are signals of potential bugs.</p> 4. Persist the corpus</strong></p> Corpus inputs are valuable regression tests.</p> 5. Run fuzzing continuously</strong></p> The longer fuzzing runs, the more paths it explores.</p> Final Thoughts</h2> Fuzz testing shifts part of the testing burden from human-designed test cases to automated exploration of input space.</p> As systems grow more complex and exposed to unpredictable environments, fuzzing becomes less of a security tool and more of a core engineering practice</strong>.</p> For systems that interact with external inputs — APIs, protocols, file formats — fuzzing often discovers bugs that traditional tests never reach.</p> In modern engineering teams, fuzz testing is increasingly becoming a default layer of reliability testing</strong>, particularly for infrastructure and systems-level software.</p> Originally published on GitHub — engineering-notes/fuzz-testing.md</a>.</em></p> std Channels vs Tokio Channels in Rust 2026-03-05T00:00:00+00:00 Context</h2> Rust offers multiple ways to communicate between concurrent tasks:</p> std::sync::mpsc</code></strong> (and third-party variants like crossbeam-channel</code>) for OS-thread-based concurrency.</li> tokio::sync</code> channels</strong> for async tasks scheduled by Tokio.</li> </ul> They look similar at the API level — send values, receive values — but they live in very different concurrency models. Mixing them casually can lead to performance problems, deadlocks, or accidental blocking inside an async runtime.</p> This note explains the differences, trade-offs, and how to choose.</p> Mental Model</h2> std channels (thread world)</h3> std::sync::mpsc</code> is designed for blocking</strong> communication between OS threads</strong>.</p> recv()</code> blocks the current thread.</li> send()</code> may block in bounded variants (std is unbounded, so mostly “doesn’t block”, but can still be expensive under contention).</li> Works with std::thread</code>, CPU-bound workers, and synchronous code.</li> </ul> tokio channels (async world)</h3> Tokio channels are designed for non-blocking</strong> coordination between async tasks</strong>.</p> recv().await</code> yields the task to the runtime instead of blocking a thread.</li> Backpressure is explicit via bounded channel capacity.</li> Designed to work with cancellation, task scheduling, and async shutdown patterns.</li> </ul> The Big Rule</h2> Never block a Tokio worker thread.</strong></p> If you call blocking std</code> primitives (including std::sync::mpsc::Receiver::recv</code>) directly inside an async task, you risk:</p> starving the runtime (other tasks stop progressing),</li> latency spikes,</li> deadlocks in worst cases.</li> </ul> If you must use blocking calls inside async code, move them into:</p> tokio::task::spawn_blocking(...)</code> (for CPU/blocking IO), or</li> a dedicated thread.</li> </ul> Channel Types Overview</h2> std::sync::mpsc</code></h3> Multi-producer, single-consumer</strong>.</li> Unbounded</strong> channel in stdlib.</li> Receiver is not cloneable.</li> Blocking receive (recv</code>, recv_timeout</code>).</li> Simple and stable, but limited and not async-friendly.</li> </ul> tokio::sync::mpsc</code></h3> Multi-producer, single-consumer</strong>.</li> Bounded channel with capacity.</li> Sender</code> is cloneable.</li> Receiver::recv().await</code> is async and cancellation-aware.</li> </ul> tokio::sync::broadcast</code></h3> Multi-producer, multi-consumer</strong> (each receiver gets every message).</li> Bounded ring-buffer semantics.</li> Receivers can lag and get Lagged</code> errors (important to handle explicitly).</li> Great for events, notifications, “pub-sub”.</li> </ul> tokio::sync::watch</code></h3> “Last value wins”.</li> Receivers always see the latest value, not every event.</li> Great for config/state propagation and shutdown flags.</li> </ul> tokio::sync::oneshot</code></h3> Single send, single receive.</li> Great for request/response, completion signals.</li> </ul> Backpressure and Memory Behavior</h2> std mpsc (unbounded)</h3> Because it’s unbounded, the risk is:</p> producers can outpace consumers,</li> memory usage grows without a natural limit,</li> latency grows because the queue grows.</li> </ul> This is often fine for small workloads, but it becomes dangerous for bursty or untrusted input.</p> Tokio mpsc (bounded)</h3> Tokio mpsc is typically bounded and will apply backpressure:</p> When the buffer is full, send().await</code> waits until there is capacity.</li> This helps keep memory stable and forces the system to self-regulate.</li> </ul> If you need “fire and forget”, Tokio also provides try_send()</code> patterns, but you must decide what to do on overflow.</p> Blocking vs Async Waiting: Why It Matters</h2> In a Tokio runtime, the runtime uses a limited number of worker threads to drive many tasks.</p> Blocking in a task blocks the whole worker thread.</li> Async waiting yields execution to other tasks.</li> </ul> So:</p> std channels are correct</strong> when your concurrency is “thread-per-worker”.</li> tokio channels are correct</strong> when your concurrency is “many async tasks on few threads”.</li> </ul> When std Channels Are Still a Great Choice</h2> Use std channels (or crossbeam-channel</code>) if:</p> You are building a synchronous</strong> library.</li> You are using std threads</strong> for CPU-bound parallelism.</li> You want “classic pipeline” style concurrency without async.</li> You are integrating with a non-async ecosystem.</li> </ul> In high-performance synchronous Rust, crossbeam-channel</code> is often preferred over std::sync::mpsc</code> due to better performance and features (bounded + select).</p> When Tokio Channels Are the Right Tool</h2> Use Tokio channels when:</p> Communication happens between Tokio tasks.</li> You need .await</code>-based receiving.</li> You want bounded queues + backpressure.</li> You care about async cancellation, graceful shutdown.</li> You want structured patterns like request/response, broadcast, watch.</li> </ul> If your code is already async, Tokio channels keep you in the same model.</p> Bridging Between std and Tokio Worlds</h2> Sometimes you must integrate sync and async components.</p> Pattern A: Dedicated thread → Tokio channel</h3> Sync component runs in a dedicated thread.</li> It sends messages into tokio::mpsc::Sender</code>.</li> </ul> This is common for:</p> reading from blocking sources,</li> stdin/stdout bridges,</li> legacy libraries.</li> </ul> Pattern B: Tokio task → std channel (rare)</h3> Usually avoid. If you must do it, don’t call blocking recv()</code> in async context. Instead:</p> use spawn_blocking</code> or a thread to receive, then forward into Tokio.</li> </ul> Pattern C: Use tokio::sync::mpsc</code> everywhere + spawn_blocking</code> at edges</h3> This is often the cleanest approach in async applications:</p> async core uses Tokio channels,</li> blocking edges are isolated.</li> </ul> Shutdown Semantics</h2> std channels</h3> Shutdown is “implicit”:</p> if all senders drop, receiver eventually gets Err(Disconnected)</code>.</li> </ul> Tokio channels</h3> Similar principle, but more structured:</p> recv().await</code> returns None</code> when all senders are dropped.</li> broadcast</code> has explicit lag handling.</li> watch</code> can signal change and also detect sender drop.</li> </ul> Engineering tip:</strong> Use explicit shutdown signals (watch</code> or broadcast</code>) rather than relying solely on “drop closes channel” when you need predictable shutdown ordering.</p> Performance Notes (Practical)</h2> Tokio channels integrate with the runtime scheduler and are tuned for async workloads.</li> std channels are simpler but can become bottlenecks under contention.</li> For high-throughput sync pipelines, consider crossbeam-channel</code>.</li> For high-throughput async pipelines, keep channels bounded and prefer batching when possible.</li> </ul> Avoid designing systems where:</p> a single receiver becomes a “hot spot” bottleneck,</li> messages are extremely small and extremely frequent (consider batching or different primitives).</li> </ul> Quick Decision Table</h2> Need</th> Best Tool</th></tr></thead> Pure sync + threads</td> std::sync::mpsc</code> or crossbeam-channel</code></td></tr> Async tasks + .await</code> receive</td> tokio::sync::mpsc</code></td></tr> 1-to-many event stream</td> tokio::sync::broadcast</code></td></tr> Share latest state/config</td> tokio::sync::watch</code></td></tr> Single response signal</td> tokio::sync::oneshot</code></td></tr> Bounded queue + backpressure</td> tokio::sync::mpsc</code> (bounded)</td></tr> </tbody></table> Final Takeaway</h2> The difference isn’t “which channel is better” — it’s which concurrency model you are in:</p> Threads</strong> → blocking primitives (std/crossbeam)</li> Async runtime</strong> → async primitives (Tokio)</li> </ul> The fastest way to introduce subtle production issues in async Rust is to accidentally use blocking channels in the hot path. Keep blocking at the edges, and use Tokio-native channels for the async core.</p> Originally published on GitHub — engineering-notes/rust-channels.md</a>.</em></p> ferrus 2026-03-01T00:00:00+00:00 ferrus is a deterministic orchestration system for real software execution.</p> Instead of treating agents as autonomous chat participants, ferrus models them as workers in a controlled lifecycle — with explicit state, enforced transitions, and restartable execution.</p> Projects are registered locally and maintain isolated runtime state, task execution, and recovery metadata through a dedicated orchestration database.</p> On the Evolution of Programming Languages in the Age of AI 2026-02-27T00:00:00+00:00 Context</h2> While building my own AI pipeline (model orchestration, structured outputs, constraint enforcement, tool chaining), I started questioning a deeper assumption about software engineering.</p> Programming languages were never designed for machines. They were designed for humans.</p> Even assembly is an abstraction layer. The CPU only understands machine code.</p> Yet today we ask LLMs to generate Rust, Go, C#, Java — languages optimized for human readability, maintainability, and collaboration.</p> If AI becomes a primary producer of code, are we still targeting the right abstraction layer?</p> Observation from Practice</h2> In AI-assisted workflows, the bottleneck is not syntax.</p> It is precision of intent</strong>.</p> The quality of AI-generated output scales with how well the following are formalized:</p> System guarantees</li> Invariants</li> Concurrency constraints</li> Failure semantics</li> Performance envelopes</li> Data ownership boundaries</li> Consistency models</li> </ul> When these properties are clearly expressed, output improves dramatically — even without changing the underlying language.</p> This suggests something deeper:</p> The limiting factor is not the programming language. It is the structure of system-level intent.</p> Hypothesis: Languages as Transitional Abstractions</h2> Traditional flow:</p> Human → Imperative code → Compiler → Machine code</span></span></code></pre> Potential AI-native flow:</p> Human → Structured intent (constraints DSL)</span></span> → AI-generated intermediate representation</span></span> → Verification layer (invariants, safety, properties)</span></span> → Optimizing compiler backend</span></span></code></pre> In this model:</p> Rust/Go/C#/Java become compilation targets</li> AI operates on structured intent</li> Verification becomes first-class</li> Properties replace instructions as the dominant abstraction</li> </ul> Why Not Just “Prompting”?</h2> Plain prompts are insufficient for serious engineering because they are:</p> Untyped</li> Non-verifiable</li> Hard to diff formally</li> Hard to compose safely</li> Weakly enforceable in enterprise environments</li> </ul> Enterprise systems require:</p> Versioned representations</li> Reviewable structures</li> Static validation</li> Explicit guarantees</li> Enforceable invariants</li> </ul> This strongly points toward constraint-driven DSLs rather than conversational programming.</p> Implications for Engineering</h2> If programming shifts from implementation to property declaration:</p> The most valuable skills become:</p> Architectural reasoning</li> Distributed systems thinking</li> Trade-off analysis</li> Invariant modeling</li> Failure domain design</li> Formal constraint specification</li> </ul> Syntax fluency becomes secondary.</p> Engineering evolves from “writing code” to “designing enforceable intent.”</p> Open Questions</h2> Will we see AI-native DSLs for distributed systems?</li> Will verification layers become mandatory in AI-generated systems?</li> Will mainstream languages evolve to embed property modeling more deeply?</li> Is the future of programming closer to formal methods than to autocomplete?</li> </ol> Working Thesis</h2> Programming languages may not disappear. But they may stop being the primary interface between human intent and executable systems.</p> They could become optimized backends.</p> And engineering might increasingly operate at the level of constraints, guarantees, and system properties.</p> If so, this is not the end of software engineering. It is a shift toward a more formal and system-oriented discipline.</p> Originally published on GitHub — engineering-notes/evolution-of-prog-lang.md</a>.</em></p> istok 2026-02-14T00:00:00+00:00 istok is an experimental HTTP/3 engine built around deterministic state machines, explicit protocol semantics, and test-first transport design.</p> The project focuses on making HTTP/3 behavior easier to reason about, validate, and evolve without relying on opaque runtime behavior or implicit protocol transitions.</p> Asymptotic Optimality vs Real Performance 2026-02-13T00:00:00+00:00 Context</h2> Sliding-window algorithm maintaining k smallest elements under a distance constraint.</p> Baseline implementations were already asymptotically optimal: O(n log k)</p> However, runtime performance varied dramatically depending on data representation and deletion strategy.</p> Observation</h2> The main bottlenecks were not algorithmic complexity but:</p> data layout</li> memory access patterns</li> indirection level</li> container design</li> </ul> Design Changes</h2> Transition from generic abstractions to specialised representations:</p> value-based structures → index-based state</li> hash maps → compact contiguous arrays</li> generic containers → specialised heaps</li> struct pairs → packed 64-bit keys</li> lazy deletion via counters → index flags</li> </ul> Results</h2> Same algorithmic complexity.</p> Different performance characteristics.</p> C# implementation:</p> ~550 ms</li> ~75 MB RSS (actual data much smaller, but CLR overhead is real)</li> </ul> Initial Rust implementation:</p> ~110 ms</li> ~6.8 MB RSS</li> </ul> Optimised Rust implementation:</p> ~60 ms</li> ~4.7 MB RSS</li> </ul> Key Insight</h2> Asymptotic optimality does not imply real-world efficiency.</p> Memory layout and indirection frequently dominate performance.</p> Implications</h2> Relevant beyond algorithmic problems:</p> low-latency services</li> streaming pipelines</li> infrastructure-level code</li> performance-sensitive middleware</li> </ul> Originally published on GitHub — engineering-notes/asymptotic-optimal-vs-real.md</a>.</em></p> When a Convenient API Becomes a Performance Constraint 2026-02-13T00:00:00+00:00 Context</h2> Convenience-focused APIs improve developer experience.</p> However, in performance-sensitive systems they may introduce hidden costs.</p> Patterns Observed</h2> Implicit Allocations</h3> Small per-request allocations accumulate in hot paths, increasing latency and memory pressure.</p> Hidden Control Flow</h3> Automatic conversions and implicit defaults obscure execution paths and complicate reasoning about performance.</p> Error Handling Abstractions</h3> Automatic transformation layers make it unclear where cost is paid.</p> Deep Middleware Chains</h3> Composable layers improve modularity but may introduce:</p> branching overhead</li> cache misses</li> reduced locality</li> </ul> Tradeoff Model</h2> Convenience is beneficial in:</p> business applications</li> productivity-oriented environments</li> </ul> Explicit APIs become preferable when:</p> latency is critical</li> throughput matters</li> predictable behaviour is required</li> </ul> Key Insight</h2> The best API for performance-critical systems often forces explicit ownership, cost visibility and execution flow.</p> Implications</h2> framework design</li> infrastructure libraries</li> async runtime abstractions</li> agent system architecture</li> </ul> Originally published on GitHub — engineering-notes/whan-a-convenient-api-becomes-a-perf-constraint.md</a>.</em></p> neva 2025-03-27T00:00:00+00:00 neva is a Rust SDK for building Model Context Protocol clients and servers with ergonomic APIs, async transports, and typed primitives for tools, resources, prompts, and protocol flows.</p> It is designed for developers who want Rust-level performance and reliability without turning simple MCP integrations into framework-heavy boilerplate.</p> volga 2024-10-02T00:00:00+00:00 Modern Rust web framework built around explicit HTTP abstractions and composable middleware.</p> volga avoids implicit behavior and hidden execution paths, enabling systems that remain understandable under load and at scale.</p>

Roman Emreis

Why this site exists

Organizations as Runtime Systems

Architect Playbook

0. Role Anchor</h2> I design systems, not execute tasks.</strong> I create clarity, ownership, and flow.</strong></p> </blockquote> Decision question:</strong></p>

1. Core Principles</h2>

1.1 Ownership > Effort</h3> No owner → no progress</li>

2. Decision Patterns</h2>

Status request</h3> → Redirect to owner</p>

Incident / failure</h3> → Engage only if you are the owner</p>

“Create ticket / ask DevOps”</h3> → Route to process</p>

Blocked work (review, dependency)</h3> → Highlight blocker + assign owner</p>

VP / stakeholder pressure</h3> → Acknowledge urgency → explain risk → propose sustainable path</p>

“They don’t know how”</h3> → Enable, don’t execute</p>

No-show / ignored meetings</h3> → Add impact → question priority</p>

3. Communication Patterns</h2>

5. Operating Model</h2>

Final Insight</h2> You are not the person who solves problems.</p> You are the person who:</p> builds a system where problems are solved without you</strong></p> </blockquote> Originally published on GitHub — engineering-notes/architect-playbook.md</a>.</em></p>

Memory Is Not Context in AI Systems

Failure Modes of Context-as-Memory</h2>

1. Context Growth</h3> Even with large context windows, the total capacity is finite.</p> Engineering systems cannot rely on prompts that grow indefinitely.</p>

2. Signal Dilution</h3> When everything is included in the prompt, important information loses priority.</p> Critical details become statistically less visible among tokens.</p>

Fuzz Testing in Modern Systems

How Modern Fuzzers Work</h2> Modern fuzzers are not purely random.</p> Most production-grade fuzzers use coverage-guided fuzzing</strong>.</p> The idea is simple:</p>

std Channels vs Tokio Channels in Rust

Mental Model</h2>

Channel Types Overview</h2>

Backpressure and Memory Behavior</h2>

Bridging Between std and Tokio Worlds</h2> Sometimes you must integrate sync and async components.</p>

Shutdown Semantics</h2>

ferrus

On the Evolution of Programming Languages in the Age of AI

Observation from Practice</h2> In AI-assisted workflows, the bottleneck is not syntax.</p> It is precision of intent</strong>.</p> The quality of AI-generated output scales with how well the following are formalized:</p>

Hypothesis: Languages as Transitional Abstractions</h2> Traditional flow:</p>

istok

Asymptotic Optimality vs Real Performance

Context</h2> Sliding-window algorithm maintaining k smallest elements under a distance constraint.</p> Baseline implementations were already asymptotically optimal: O(n log k)</p> However, runtime performance varied dramatically depending on data representation and deletion strategy.</p>

Observation</h2> The main bottlenecks were not algorithmic complexity but:</p>

Design Changes</h2> Transition from generic abstractions to specialised representations:</p>

Key Insight</h2> Asymptotic optimality does not imply real-world efficiency.</p> Memory layout and indirection frequently dominate performance.</p>

Implications</h2> Relevant beyond algorithmic problems:</p> low-latency services</li> streaming pipelines</li> infrastructure-level code</li> performance-sensitive middleware</li> </ul> Originally published on GitHub — engineering-notes/asymptotic-optimal-vs-real.md</a>.</em></p>

When a Convenient API Becomes a Performance Constraint

Context</h2> Convenience-focused APIs improve developer experience.</p> However, in performance-sensitive systems they may introduce hidden costs.</p>

Patterns Observed</h2>

Implicit Allocations</h3> Small per-request allocations accumulate in hot paths, increasing latency and memory pressure.</p>

Hidden Control Flow</h3> Automatic conversions and implicit defaults obscure execution paths and complicate reasoning about performance.</p>

Error Handling Abstractions</h3> Automatic transformation layers make it unclear where cost is paid.</p>

Deep Middleware Chains</h3> Composable layers improve modularity but may introduce:</p>

Tradeoff Model</h2> Convenience is beneficial in:</p>

Key Insight</h2> The best API for performance-critical systems often forces explicit ownership, cost visibility and execution flow.</p>

Implications</h2> framework design</li> infrastructure libraries</li> async runtime abstractions</li> agent system architecture</li> </ul> Originally published on GitHub — engineering-notes/whan-a-convenient-api-becomes-a-perf-constraint.md</a>.</em></p>

neva

volga

Status request</h3>
→ Redirect to owner</p>

Incident / failure</h3>
→ Engage only if you are the owner</p>

“Create ticket / ask DevOps”</h3>
→ Route to process</p>

Blocked work (review, dependency)</h3>
→ Highlight blocker + assign owner</p>

VP / stakeholder pressure</h3>
→ Acknowledge urgency → explain risk → propose sustainable path</p>

“They don’t know how”</h3>
→ Enable, don’t execute</p>

No-show / ignored meetings</h3>
→ Add impact → question priority</p>

Final Insight</h2>
You are not the person who solves problems.</p>
You are the person who:</p>

builds a system where problems are solved without you</strong></p> </blockquote>

Originally published on GitHub — engineering-notes/architect-playbook.md</a>.</em></p>

1. Context Growth</h3>
Even with large context windows, the total capacity is finite.</p>
Engineering systems cannot rely on prompts that grow indefinitely.</p>

2. Signal Dilution</h3>
When everything is included in the prompt, important information loses priority.</p>
Critical details become statistically less visible among tokens.</p>

Bridging Between std and Tokio Worlds</h2>
Sometimes you must integrate sync and async components.</p>

Context</h2>
Sliding-window algorithm maintaining k smallest elements under a distance constraint.</p>
Baseline implementations were already asymptotically optimal: O(n log k)</p>
However, runtime performance varied dramatically depending on data representation and deletion strategy.</p>

Key Insight</h2>
Asymptotic optimality does not imply real-world efficiency.</p>
Memory layout and indirection frequently dominate performance.</p>

Implications</h2>
Relevant beyond algorithmic problems:</p>

low-latency services</li>
streaming pipelines</li>
infrastructure-level code</li>
performance-sensitive middleware</li> </ul>

Originally published on GitHub — engineering-notes/asymptotic-optimal-vs-real.md</a>.</em></p>

Context</h2>
Convenience-focused APIs improve developer experience.</p>
However, in performance-sensitive systems they may introduce hidden costs.</p>

Implicit Allocations</h3>
Small per-request allocations accumulate in hot paths, increasing latency and memory pressure.</p>

Hidden Control Flow</h3>
Automatic conversions and implicit defaults obscure execution paths and complicate reasoning about performance.</p>

Error Handling Abstractions</h3>
Automatic transformation layers make it unclear where cost is paid.</p>

Key Insight</h2>
The best API for performance-critical systems often forces explicit ownership, cost visibility and execution flow.</p>

Implications</h2>

framework design</li>
infrastructure libraries</li>
async runtime abstractions</li>
agent system architecture</li> </ul>

Originally published on GitHub — engineering-notes/whan-a-convenient-api-becomes-a-perf-constraint.md</a>.</em></p>