Johann Dettweiler, CISO · stackArmor, a Quantum Sky company
Inside the 165-rule assessment of The Armory: 15 CR26 process families, visible gaps, and evidence behind the results.
BOTTOM LINE UP FRONT
On November 10, 2026, stackArmor’s automated compliance assessment engine assessed The Armory against a selected 165-rule baseline from FedRAMP’s Consolidated Rules for 2026 (CR26), spanning 15 process families. Every rule received an outcome. Not every rule passed. The achievement is a connected operating process: retained evidence, explicit human decisions, deterministic numeric results, and reporting that keeps the remaining gaps visible.
I do not judge compliance automation by how confidently it says “compliant.” I judge it by what happens when somebody asks a second question. Where did the evidence come from? What did the assessment actually examine? Who made the decision that could not be reduced to a check? And what is the organization doing about the work that remains?
Those questions are not obstacles to a good demonstration. They are the demonstration. A result that cannot support them may be useful as a signal, but it is not yet the operating record I want to put in front of a security leader, an engineer, or a reviewer.
For years, our argument at stackArmor has been that compliance should work like an engineering discipline. Requirements should connect to evidence. Recurring obligations should have repeatable procedures. Decisions should survive the meeting in which they were made. On November 10, we put that argument to work across The Armory’s selected CR26 baseline.
The November 10, 2026 run covers the selected Rev5 Agency Class D baseline. Coverage includes not-applicable outcomes; it does not mean all rules passed.
A dashboard is the view. The operating process is the achievement.
A dashboard has an important job: it helps people see where to look. But a status tile cannot, on its own, establish that the right environment was examined, that the source population was complete, or that somebody with the right authority reviewed an exception. Those responsibilities belong to the process behind the tile.
stackArmor’s automated compliance assessment engine evaluates retained workflow evidence and authoritative human decisions against the applicable rules in the selected baseline. The Armory provides the operating context around that assessment: the recurring work, its scope, the collection procedures, the review responsibilities, and the records that connect one stage to the next.
That connection is the distinction I care about. We are not asking a report to stand in for a security program. We are making the work of the program available to assessment. The resulting view is useful because there is something behind it that people can inspect, challenge, and improve.
A score points to where attention is needed. An operating record explains why—and what happens next.
We put the unfinished work on the screen.
The retained run reported 108 rules satisfied, 36 with managed gaps, one with an uncovered gap, and 20 not applicable. Together, those outcomes account for all 165 rules in the selected baseline. Complete coverage means that none was left unassessed; it does not mean that every rule passed.
I want that distinction to be impossible to miss. A system that makes the picture look healthier by blurring the difference between a satisfied requirement and a managed deficiency is making the next decision harder. The purpose of assessment is not to protect the appearance of the program. It is to tell the people operating it where attention is needed.
Rule outcomes for the selected November 10, 2026 baseline. Managed gaps remain gaps.
The gaps are not an awkward footnote to the milestone. They are part of what makes the result useful. An engineer needs to know what remains unfinished. A security leader needs to know whether that work is being handled. A reviewer needs to know what supports both conclusions. Those are related questions, but they are not the same question.
Retained report from November 10, 2026, shown unchanged. The 19 counts distinct linked native findings, not rule outcomes. The 100% indicates assessment coverage, not a pass rate. The report displays both 93.1 and the workflow-only mean of 74.5.
Managing a gap does not make the gap disappear.
Our report preserves that distinction through two measures. The workflow-only mean is 74.5. The management-adjusted score is 93.1, including bounded credit for gaps verified as actively managed. Both belong in the conversation. Neither should be presented as a substitute for the underlying rule outcomes.
Verified management matters: it tells a reader something about how unfinished work is being handled. It does not tell the reader that remediation is complete. The assessment therefore keeps the original gap determination visible rather than repainting it as a satisfied rule.
Reported measures for the retained November 10, 2026 run. Workflow-only performance and management credit remain separate views of the same assessment.
These are assessment measures used by Albatross, not a FedRAMP-issued rating or an authorization decision. They are also not simple percentages of rules passed. Without that context, a prominent number can imply more than the evidence supports. With it, the report can show both the state of the work and the discipline around addressing it.
An assessment should make the next decision easier—not make the current situation look better.
AI should explain the result, not negotiate it.
There is a useful role for AI in making a complex assessment readable. Our CR26 summary reporting uses it to produce an executive brief and family-by-family narratives from curated assessment facts. A leader should not have to reverse-engineer a collection of technical records simply to understand where attention is required.
But the numeric results stay outside the model’s response. They are deterministic fields, while the narrative operates under a defined contract and validation. The retained report also preserves the selected run and its finding snapshots, so the historical assessment remains connected to what was observed at that time.
That division of responsibility is deliberate. A fluent explanation is useful; it is not an authority to alter the result being explained. I want the model to help people understand the evidence, not become another place where the evidence’s meaning quietly changes.
The report is not where the work ends.
Consider Certification Data Sharing (CDS), one of the workflows operating inside The Armory. Collection and review preparation assemble the evidence, proposed package changes, and questions requiring an Information System Security Manager’s judgment. Reviewed publication is a separate stage. It checks the required decisions and merged source artifacts, publishes the approved materials, and verifies the delivered bytes.
That last step matters because a successful upload is not the same thing as a verified outcome. The artifact somebody receives should be the artifact the reviewer approved. Carrying that relationship through the workflow is less glamorous than a new dashboard animation, but it is considerably more useful.
The same principle explains stackArmor’s five-part operating model behind this work: Task Definitions, Playbooks, Runbooks, Profiles, and Skills. They give an agent a job, an organizational process, an execution sequence, a defined environment, and maintained procedures. The agent can then do useful work without being asked to invent the organization around it.
Do not take our adjective for it. Inspect the work.
“Automated” is not a sufficient evaluation criterion. The more meaningful test is whether a provider can show a recurring obligation moving through evidence collection, evaluation, review, and a controlled next action. Pick a result. Ask which baseline produced it. Ask what happens when collection is incomplete. Ask who owns the decision and how the completed work is retained.
That is the standard I want customers to apply to stackArmor. Not because our assessment shows a perfect environment—it explicitly does not—but because the relationship between the result and the work behind it is the capability we are putting forward.
The November 10 run is a concrete milestone, not a claim that all compliance work is now autonomous, that every gap is closed, or that an assessment engine replaces independent review. It demonstrates that these operating pieces can be connected against a real environment and a defined baseline. That is a much more substantial conversation than a promise about what might eventually be possible.
Compliance automation is not around the corner in this implementation. It is running. Bring us a workflow, choose a result, and ask the difficult questions. That is where the conversation should begin.
Questions worth asking
Did all 165 rules pass?
No. The retained run reports 108 satisfied, 36 managed gaps, one uncovered gap, and 20 not applicable. Assessment coverage is not a pass rate.
Does this run constitute a FedRAMP certification decision?
No. It is a retained November 10, 2026 assessment of the selected Rev5 Agency Class D baseline. No independent certification or authorization decision is implied.
Does AI calculate the score?
No. Numeric results remain deterministic fields outside the model’s response. AI produces bounded narrative explanations from curated assessment facts.
Assessment scope: historical November 10, 2026 run; selected Rev5 Agency Class D baseline; 165 rules across 15 process families; captured source versions retained. Assessment coverage is not a pass rate or an independent certification decision. Managed credit does not mean remediation is complete.
Sources and further reading
- FedRAMP — CR26 complete rulesets
- FedRAMP — Certification Data Sharing
- stackArmor — The Armory
- stackArmor — Dear Agencies: Let’s Make CR26 Work
Inside The Armory: A Three-Part Series
- Compliance Automation Isn’t Around the Corner. We’re Running It.
- Your AI Agent Needs More Than a Prompt. It Needs Accountability.
- Don’t Show Me a Compliance Score. Show Me the Evidence.
Choose a result. Ask us to defend it.
Walk through the evidence, the review decisions, and the next action with the people building the capability.