We at stackArmor have taken to heart the recent calls to “Blow up the Risk Management Framework (RMF)” and take the compliance drama head-on. ATOs are in the news almost daily, often associated with high costs and long approval cycles with questionable outcomes.
As we’re all about to light the RMF on fire and re-imagine it from first principles, we realize the real problem isn’t the RMF itself, it’s the fossilized way we’ve been playing the compliance game: binders packed with off-topic prose, screenshots that are outdated the moment they’re captured, and evidence packages that are obsolete the instant they are zipped.
Traditional Federal information system assessments have been an endless cycle of:
- Write 700 pages of implementation statements that are marginally on topic, and only sometimes accurate.
- Have your highly skilled/paid engineers copy/paste screenshots into Word docs like a freshly minted, unskilled intern.
- Ship the whole mess to auditors and pray nobody notices half the evidence is already outdated.
That’s not security, that’s cosplay with clipboards.
The General Services Administration (GSA) acknowledged this with the FedRAMP 20x pilot program, an initiative designed to bring compliance into the 21st century. The goals were refreshingly sane:
At stackArmor, we didn’t just nod politely at these goals and go about our merry way. We leaned in, hard. For our Armory20x system on Google Cloud Platform (GCP), we tossed the dusty playbook and submitted something different: real-time, code-driven compliance. Live, near-real-time proofs, not fossilized artifacts.
The Problem: Using Screenshots for Audits is like Firing Up a Tesla with Coal
To paint a picture, legacy compliance works like this: once or twice a year, an auditor shows up with a checklist that they’ve reused a hundred times for every system they assess, regardless of what the system actually does or how it was built. You, the system owner, scramble to grab artifacts: policy documents, console exports, and, of course, the coup de grace of bad evidence, the screenshot.
So, your highly paid, highly skilled engineer drops everything they are doing (including providing services to system customers) to log into a cloud console, set up the display to show only what they want the auditor to see, take a snap, paste it into Word, and label it Figure 12.1 – MFA Enabled. Now rinse and repeat 500 times, and that’s the prime example of an audit.
The problems are obvious:
Relying on screenshots to prove compliance in the cloud era is like checking your MySpace profile to gauge your social relevance. Sure, it exists, but your catchy profile page is just as lame as the day you wrote it, and no one’s impressed. Screenshots are for chumps.
A New Model: Trust the Code, Not the Clipboard
Our 20x pilot took a different approach: stop telling stories, start showing data. Instead of writing flowery, mostly off-topic, bloated paragraphs about how our disks are encrypted with customer-managed keys, we wrote queries that asked GCP directly: Hey, show me all disks and their encryption status. The system answered instantly and without bias.
We call this approach trust the code, not the clipboard. Or…in code, we trust.
Here’s our approach:
- Within ThreatAlert®, we implemented a lightweight Query Engine based on readily commercial and open technologies that normalizes cloud and Software-as-a-Service (SaaS) APIs into SQL tables. Want to know which Okta users lack MFA? Write code. Which GitLab repos lack branch protections? Query it with code. Which object storage buckets are publicly exposed? Code away.
- Additionally, we implemented a Validation Engine in ThreatAlert® that allows us to define FedRAMP controls as code, then structure them into benchmarks and run them en masse. Each control is a check; the suite of checks is the benchmark.
Put them together and suddenly FedRAMP’s Key Security Indicators (KSIs) aren’t prose, they’re executable code. Instead of a 700-page document, we delivered JSON. Instead of a binder of screenshots, we gave auditors a dashboard where they could drill into live results. Imagine being an assessor and actually getting useful, structured data instead of a PDF brick.
In this model, the auditor’s job shifts from scavenger hunt to referee. They’re not checking screenshots; they’re validating that the queries themselves are accurate and comprehensive. They’re doing real security work, and I’m willing to bet, are far more content than being elbow deep in off-topic screenshots and demonstrations.
How It Works: Compliance-as-Code in Action
Every control becomes a declarative statement plus a query:
This is compliance as code: versioned, peer-reviewed, and repeatable. When FedRAMP updates a requirement we tweak the query and re-run it. The running system state provides results, and the work is done.
In order to demonstrate KSI compliance, we needed to understand critical metrics around FIPS encryption, vulnerability scanning, etc. To that end, we wrote plugins for tools like Nessus and our own ThreatAlert® Security Workbench (TSW), but wherever possible we looked to use existing open-source plugins. In addition, we are actively contributing to the Turbot-managed Okta plugin and are developing an Okta STIG PowerPipe benchmark with queries that were written against Okta’s own STIG. If Okta says, “This is how you harden our product,” why not query the actual running instance in your environment and ensure that’s exactly what’s been deployed? Any service that has an API is fair game.
Run all the controls with one command. Get thousands of results in seconds. Export JSON, CSV, or light up a dashboard. That’s the whole FedRAMP assessment package, no screenshots required. We don’t just hand over results; we open the hood. Agencies and 3PAOs get full visibility into the actual queries, not just the outputs. That means the entire evidence pipeline is transparent; what we check, how we check it, and what the system answers back. That level of candor builds trust far better than any binder ever could.
The Payoff: Compliance Without the Theater
This isn’t just fun for engineers, it’s transformative for the whole ecosystem.
- Audit Fatigue? Gone. Instead of endless artifact chases, audits run on live queries. Auditors self-serve the evidence they need. It’s like an open-book test where the answers are always up to date.
- Faster ATOs. Old way: have expensive resources spend months prepping for assessment. New way: hit run benchmark and hand over results.
- Real Visibility. Dashboards show system owners, CISOs, and agencies the exact compliance posture right now. Not last quarter. Not last year. Now.
- Consistency & Confidence. Code doesn’t forget steps, doesn’t mislabel screenshots, and doesn’t get tired at 2 a.m. Results are repeatable and transparent.
The Bigger Picture: From Paperwork to Continuous Trust
FedRAMP 20x isn’t just a pilot, it’s a glimpse of the future. Compliance is moving from:
- Paperwork => Data
- Annual fire drills => Continuous evaluation
- Manual trust-but-verify => Automated trust-because-we-verify
Imagine an audit where the only human question is: Do the queries cover everything they should? not Please send me 400 screenshots by Friday. That’s where a better, more functional RMF is headed. stackArmor is here to light the way on our collective journey to a place where compliance contributes to actual system security, not just slows it down.
And this approach scales beyond FedRAMP: FISMA, DoD SRG, PCI, CMMC. Any framework that demands proof of implementation can benefit from compliance as code. Controls become queries, queries become continuous evidence, and auditors become validators of automation instead of artifact chasers.
But in our pursuit for true full system automation, we also understand compliance isn’t about appearances. It’s more than a dashboard with green checkmarks slapped on superficial data points because it looks nice and earns self-congratulatory pats on the back. Painting everything green doesn’t make systems more secure. Dashboards that show 100% compliance all the time are suspect. Real systems evolve, drift happens, and continuous compliance means catching it and addressing it, not purposefully obfuscating it while cracking unearned bottles of champagne.
A Word of Caution: Queries Cut to the Quick
Another tenet of the FedRAMP 20x program was encouraging CSPs to move towards a transparency mindset. To be clear, we’re not anti-transparency. In fact, we generally agree with the 20x stance that some compliance data should be made more publicly available. Not every detail about a system needs to be stamped sensitive, and too often agencies err on the side of secrecy for things that could safely build trust if shared. But here’s the catch: our approach goes deeper than surface-level checks. The queries we run don’t just scratch the UI for superficial attributes; they interrogate the guts of the system, real configuration state, and real security posture. That level of detail can expose more than what’s appropriate for public consumption. Transparency is good, but recklessly publishing system internals is the definition of a data leak. Want specific examples? Look at our pilot program submission: https://stackarmor.github.io/armory-20x-public/. We deliberately chose to share this data, as the Armory 20x tenant contains no sensitive information, and we thought it best to demonstrate the implications of this approach.
Compliance dashboards and the underlying data should be available to the people who matter: your authorizing officials, end-agency customers, and 3PAOs. Stakeholders that have skin in the game, as it were. Our vision for the future envisions curated dashboards backed by machine-readable data tied to identity services like Login.gov, where real stakeholders log in and see the truth that allows them to make informed risk-based decisions as opposed to competitors using compliance percentages as gotcha-based marketing ammo.
Conclusion: Delivering RMF ATOs the 20x Way!
FedRAMP 20x gave us permission to show what we’ve believed for years: compliance shouldn’t be theater; it should be engineering. Our code-driven model proves you can achieve continuous compliance faster, cheaper, and with more trust.
The takeaway is simple – gone are the days of clipboards, PDFs, and screenshots:
The future is code-driven, automated, and continuous.
Let’s stop romanticizing the binder era and start building compliance at cloud speed. stackArmor is ready, and frankly, we’ve already moved on from the old ways; we hope others do too. Come and see.
Copyright © 2025 stackArmor, Inc., a Tyto Athene Company. All rights reserved. All other trademarks not owned by stackArmor are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by stackArmor. This document does not provide you with any legal rights to any intellectual property in any stackArmor product or solution.
About the Author:
Johann Dettweiler is the Chief Information Security Officer at stackArmor, a Tyto Athene Company. He is a seasoned cybersecurity leader with deep expertise in FedRAMP, NIST Risk Management Framework (RMF), and Governance, Risk, and Compliance (GRC) practices. With a career spanning over 19 years in the security and compliance space, Johann has held key roles as a Lead 3PAO assessor, chief advisor, and operational leader, bringing a comprehensive understanding of both technical and organizational risk management.
As CISO, Johann leads stackArmor’s enterprise security strategy, driving risk-based decision-making and guiding the organization in developing secure and resilient solutions. He plays a critical role in informing executive leadership on risk posture, ensuring that cybersecurity is integrated into every aspect of strategic planning and service delivery. Johann is the internal and external face of security for stackArmor, providing trusted guidance to customers navigating complex compliance landscapes
