The Bounded Auditor: Reclaiming “We the People” in an Age of Complexity

An Engineering Reality Check

In a recent discussion on Reddit, a machine learning engineer challenged our workflow by sharing a vital piece of recent research: Microsoft’s study, “LLMs Corrupt Your Documents When You Delegate”.  

The paper’s findings are a sobering and necessary reality check for anyone using AI: advanced models can silently alter or corrupt about 19% to 28% of core content when handed autonomous, multi-step editing tasks over massive documents. Worse, the smarter the model, the more smoothly it formats and hides its mistakes.

We completely agree with this research. If you blindly hand a multi-hundred-page omnibus bill to a chatbot and ask it to explain the whole thing in a vacuum, you are inviting hallucinated, decontextualized “slop.”

But that isn’t what we are trying to build here.

Turning Creators into Auditors

It is an undeniable reality that modern legislation is highly dense, complex, and lengthy. This natural structural complexity can leave the door open to information gaps, intense partisan spin, and conflicting media interpretations. When the law becomes too complex for a regular person to read, a vacuum opens up—leaving everyday citizens feeling more like spectators or targets of government policy rather than active participants.

Whether a politician is voting on a massive bill due to party alignment, institutional peer pressure, or other complex political realities, our goal as citizens remains the same: we want to infuse an element of objective, clear-eyed empiricism into the process. We are less interested in guessing the hidden motives behind a bill and more interested in verifying its actual text.

To bring that level of data-driven clarity to a system run by fallible humans, we don’t use LLMs as autonomous creatorsor blind summarizers. We use them as constrained semantic filters under strict human supervision.

The Methodology in Action: The SNAP/Tariff Claims

Look at the timeline of the One Big Beautiful Bill. During its drafting and debate phase—well before it eventually passed into law—prominent online claims circulated that specific provisions would immediately spike grocery prices and cut SNAP benefits.

We didn’t ask the AI for a broad, unconstrained prediction of the future. Instead, we isolated the specific agricultural and tariff provisions as they were written at that moment, locked them into the LLM’s prompt window alongside the outside claims, and forced the AI to act as an advanced, contextual Ctrl+F.

It stripped away the surrounding rhetoric and mapped out the actual legal mechanics of the text, allowing us to see exactly where the outside narratives deviated from the draft text.

A Retrospective Note: Obviously, looking back today, the actual relationship between those tariff policies and real-world food prices has become much clearer than it was during the initial debate. While some of those early warnings about rising grocery costs certainly materialized or evolved in ways the text alone couldn’t fully predict, our goal at the time wasn’t to act as economic prophets. Our goal was simply text verification.

By keeping the AI’s context restricted directly to the document, any technical hallucination or “silent rot” could be instantly caught and corrected by the human operator. It allowed us to separate what a bill actually said from the speculative noise surrounding it.

Learning and Err-ing in Public

AI is an evolving technology, and we are human researchers. We fully expect this journal to be messy. When a model falls victim to formatting bugs, struggles with precise data extraction, or misreads context, we will document those errors, misses, and course-corrections right alongside our insights.

True civic accountability means being willing to audit the process rigorously and taking valid data or technical criticism in stride, regardless of the source. We want to thank the community for pushing us to sharpen our parameters.

We don’t know exactly what happens if more everyday voters use these tools to step into the legislative process on a per-bill basis, but it offers a practical path toward greater individual ownership and a clearer sense of systemic accountability. In an era where many feel deeply divided and disconnected from the foundational idea of “We the People,” perhaps this form of direct, objective participation can help us feel genuinely connected to that phrase again.

It certainly can’t hurt to try. So let’s keep exploring, keep learning in public, and use the tools at our disposal to pull back the curtain together.