A bank verifies the numbers in an AI-generated filing summary by requiring every figure to trace back to a specific SEC filing, a specific accession, and a specific line or tagged fact, and then checking it there before the number enters a memo, a credit file or a model. My view is that this tracing is no longer a nice-to-have review step. Once AI tools are writing the first draft of filing analysis, verification against the source filing becomes the control itself. A figure you can trace is a figure you can defend. A figure you cannot trace is borrowed confidence.

Claim → Filing → Citation → Defense

Each AI-produced number is a claim until it is matched to the filing it came from. The match is recorded as a citation. That citation is what lets the firm stand behind the figure when a client, a committee, an auditor or a supervisor asks where it came from.

The thesis, stated plainly

Banks and asset managers are already asking AI tools to summarize 10-Ks, pull revenue and backlog figures, and draft first-pass memos. That speed is real, and it is now cheap. Accountability for the output has not become cheaper. When a large language model produces "contracted backlog of $X billion," that sentence looks just as authoritative whether it is right, slightly wrong, or invented. The National Institute of Standards and Technology has a name for this — confabulation, confidently stated but erroneous output — and recommends provenance tracking as one mitigation. The format of the output carries no signal about its accuracy.

In my conversation on Stack to Street with Dwight Gunning, a twenty-year financial technology veteran whose open-source EdgarTools library for SEC data has passed nine million downloads, he named this problem directly. Asked what firms will have to change as information gets easier to collect and analyze, he described the new workflow (pull the data, have the AI summarize it, get a report with numbers) and then asked the question that matters: "How do you know if it's correct?" His answer was that businesses will become far more interested in verification.

I agree, and I would go further. Verification is where the institutional value of AI analysis will be decided. The firms that build it in will be able to use agents aggressively. The firms that skip it will either slow down to compensate or carry risk they have not priced.

I hear several objections to this position from executives. Here they are, with my replies.

Objection one: "Our analysts already check the important numbers."

Some do, on some days. In practice, manual checking does not keep pace with AI-generated volume. A tool that produces ten summaries in the time an analyst used to read one does not come with ten times the reviewer hours. Under time pressure, review drifts from checking the source to rereading the summary, and the two are very different. Rereading the summary catches typos. It does not catch a figure pulled from the wrong period, the wrong segment, or the wrong company.

Ad hoc checking also leaves no record. If an analyst confirmed a number against a 10-K but did not record the accession and the line item, the firm cannot later show that the check happened. A control that leaves no evidence is, for practical purposes, a habit. That is why the Citation step in my sequence matters as much as the Filing step. Without it, the bank's ability to defend the figure depends on someone's memory.

Objection two: "Verifying AI output against filings is too slow to be worth it."

This was the strongest objection five years ago. It is the weakest one now, because the same tooling that makes AI summarization fast also makes verification fast.

The best illustration I know comes from Dwight's own work. He read an essay about the AI boom that cited very large contracted backlogs at the major cloud providers. Instead of taking the essay's figures at face value, he opened a Claude Code session, gave the agent the article, and, as he described it, told it to "go corroborate it", using EdgarTools to find the relevant information across the filings and pull it together. The result was roughly $2.36 trillion of contracted backlog across Oracle, Microsoft, Google and Amazon, assembled from the companies' own filings in what looked to me like about twenty lines of Python.

Two points matter here. First, the direction of the work. The agent was not asked to produce a number. It was asked to test someone else's number against primary sources. That is the posture I want banks to adopt, with AI as the checker as well as the writer. Second, the cost. A cross-company corroboration that would once have meant pulling four companies' filings by hand, finding the right disclosure in each and reconciling definitions now fits in a short script. When verification takes minutes, "too slow" stops being a reason. It becomes an excuse.

Objection three: "XBRL already standardized the data. Why not just trust the tagged numbers?"

XBRL was a major advance, and I would not want to go back to the period before it. Dwight was clear that it has "massively improved things," with almost all companies now reporting that way. The objection assumes, though, that a standard format means comparable data, and it does not.

As Dwight put it, among roughly 7,000 companies using the standard, "each company reports it slightly differently." His example was Intel versus an accounting company. They use the same taxonomy but have different business models, different choices about which elements to use, and different ways of carving up revenue and obligations. Standardizing that across industries, in his words, is a huge problem, and it is the work his contributors are still doing.

For verification, this means an AI tool can return a correctly tagged XBRL fact and still give the wrong answer to the question the analyst asked. The fact can be accurate for that filer and misleading when compared against a peer that tags the same economic item differently. Accounting researchers Sandip Dhole, Gerald Lobo, Sagarika Mishra, and Ananda Pal found that firms using more company-specific XBRL tags showed lower financial-statement comparability after the mandate. So the Filing step cannot stop at "the number exists in the data." It has to confirm that the number means what the summary says it means. Sometimes that requires reading the surrounding disclosure. A good verification process treats structured data as the fastest route to the source. It does not treat it as a substitute for the source.

Objection four: "This is a technology problem. Let the data team handle it."

The data team should build the tooling. The obligation itself belongs to the business. When a figure appears in a credit approval, a client report or an investment committee memo, the person who signs that document owns the number. If the firm cannot trace it to a filing, it is relying on an answer it does not fully own, however good the model was.

This is why I place Defense at the end of the sequence and do not treat it as an afterthought. The point of verification is to be able to answer, at any later date, the question "where did this come from?" with an accession number and a location, and not with "the tool said so." That is a governance design question. Which outputs require a citation? Who reviews exceptions? How long are verification records kept? Those are decisions for senior management, informed by the firm's general obligations around books, records and communications, and not left to whichever engineer wired up the API.

Objection five: "Agents will get good enough that verification won't matter."

I take this one seriously, because the trend behind it is real. Dwight predicted that within three years "you will have an AI agent investor", with a user setting a goal and the agent collecting information, reasoning over it, and investing toward that goal. He also expects these models' reasoning to surpass what humans can do.

Suppose that happens. It strengthens my thesis. An agent that acts on its own outputs compounds any error it makes. A human analyst who misreads a backlog figure writes a flawed memo, and someone may catch it. An agent that misreads the same figure may size a position on it before anyone looks. The more autonomy institutions give these systems, the more the citation trail becomes the only practical way to audit what the agent believed and why. Better models change who does the first pass. They do not remove the need to show the work. If anything, an agent investor without source-level traceability is precisely the kind of system a prudent firm should be unwilling to deploy.

What I would put in place now

  1. Require a citation for every AI-produced figure that leaves the analyst's desk: filing type, accession number, period, and the tagged element or page reference.
  2. Make corroboration a separate agent task. One process drafts and another checks the draft against the filings, so verification is not the same model grading its own work.
  3. Flag cross-company comparisons for definitional review, because consistent tagging does not guarantee comparable economics.
  4. Log verification results, including mismatches, so the firm can show that the check happened and learn where its tools fail.
  5. Assign ownership in the business, not only in technology. The signer of the document owns the figure.
  6. Revisit the controls before granting agents more autonomy, not after.

Speed will become a commodity. What a bank will be judged on is whether it can produce, for any figure an agent acted on, the filing that figure came from — and that is a decision to make now, before the agents are turned loose, not after.

About the author

Michael McLaughlin has more than 30 years in structured finance and private credit. At Macquarie Group from 2008 to 2023, he built the credit markets business from inception into a portfolio financing platform that averaged a $5 billion portfolio and generated more than $1 billion in cumulative net revenue. He has advised Hebbia on AI research software for financial services, and builds his own finance applications with AI coding tools and open-source software.

Through Path66, Michael works with financial institutions and investment firms on asset finance, liquidity, capital raising, data-driven due diligence and the practical application of emerging technology in finance. He hosts Stack to Street, where he interviews the developers building the software that will change financial institutions. Full bio.

Source

AI That Cites the SEC Filing, Stack to Street with Dwight Gunning, Creator of EdgarTools. Full transcript on the episode page.