The FedRAMP PMO announced the Emerging Technology Prioritization Framework (ETPF) to fast-track AI solutions in code generation, image generation, and chatbots. Cloud service providers (CSP) with existing FedRAMP authorizations can now add OpenAI services to their current Cloud service offerings (CSO). This can be done by following FedRAMP’s prescribed change management process that is often referred to by its acronym – the SCR process.
In this blog we will walk you through how you can prepare for such a journey and add Gen AI capabilities to your existing FedRAMP accreditation boundary. stackArmor engineers have informed this blog with their work for existing customers and clients.
Select the right FedRAMP Accredited Cloud Service for GenAI
First, the journey begins with selecting the right FedRAMP accredited AI cloud service to add to your boundary based on the use case. Google, with Vertex AI, and Microsoft, with Azure OpenAI, both hold FedRAMP High accreditations currently.
Microsoft announced in August 2023, the JAB FedRAMP High P-ATO for their Azure OpenAI service in Azure Commercial. The announcement included support for Azure OpenAI’s foundational models, including GPT-3.5, GPT-4, and DALL-E. Since then, stackArmor has assisted a number of customers with their GenAI roadmap by helping them integrate Azure OpenAI’s capabilities with their existing FedRAMP authorized applications as a Significant Change Request (SCR) during their next annual assessment.
With the move towards multi-cloud adoption and the desire to incorporate and consume best-of-breed cloud services, the addition of a Azure-hosted service posed several challenging questions. Since many customers currently use AWS, this blog outlines our approach in responding to customer needs looking to add GenAI capabilities to their existing boundary.
Key issues and challenges that needed to be addressed include:
-
As a FedRAMP authorized surface, can one leverage Azure OpenAI service without extending the authorization boundary to include Azure?
- Did a virtual private network need to be established between AWS and Azure to protect traffic between the two?
- How will you monitor the security and compliance of the Azure-hosted resources to align with FedRAMP Continuous Monitoring control requirements?
- How will you manage the authentication of privileged users in Azure?
Extending the Authorization Boundary
For customers hosting their applications on AWS, a key question was whether they could use the Azure OpenAI service without extending their existing authorization boundary to include Azure. The Azure OpenAI and supporting services deploy within a customer’s Azure tenant. That tenant needs to be part of a FedRAMP authorization. While obtaining a separate authorization for the Azure tenant is possible, the cost and time required would be prohibitive. The faster and more economical way to incorporate Azure OpenAI capabilities into an existing authorization boundary that already has a FedRAMP ATO is by extending the existing boundary in AWS to include Azure and make it a multi-cloud solution.
Connecting AWS and Microsoft Azure
Once you decide to extend the authorization boundary to include both AWS and Azure, consider whether a virtual private network needs deployment between the two clouds to protect traffic. This decision depends on several factors and is not one-size-fits-all.
Will the services deployed in AWS and Azure require private networking to establish connectivity between them?
For Azure OpenAI, the publicly exposed service endpoints meet FedRAMP requirements for encryption. As a result, private networking may not be necessary for the AWS-hosted application to connect to Azure OpenAI. If using public endpoints, protect the service from unauthorized parties using a firewall, network security group, or IP filtering.
Are customers able to interact directly with Azure OpenAI, or is the connection to OpenAI brokered by the application?
If customers can interact directly with Azure OpenAI from their browsers or a desktop component, the Azure OpenAI endpoint needs to be publicly accessible. If the application brokers the connection, control the connections to Azure OpenAI through access control lists.
How much network data is expected to flow between the two environments?
The cost for sending large volumes of data between the two environments needs to be a consideration.
- If the application brokers the connection to Azure OpenAI, will the payloads going to or coming from Azure OpenAI be relatively small or could they be quite large?
- Does one need to replicate data between AWS and Azure to optimize the models for Azure OpenAI?
Continuous Monitoring Approach
One significant factor in the design of a multi-cloud solution is whether performing continuous monitoring should be entirely in a single cloud, or distributed across multiple clouds.
- Operationally, its typically more effective to centralize continuous monitoring activities into a single cloud. With this approach, one can monitor all environments through a single pane of glass. Security analysts won’t necessarily have to learn about the “other clouds” if their existing tooling in AWS provides all of the information they need.
- A centralized approach adds some complexity to ensuring evidence collection across the clouds works. This also makes the other cloud environments dependent on the primary cloud environment for monitoring. This approach also is likely to generate a lot of inter-cloud traffic and might require the use of a virtual private network. It will, at the very least, increase operational costs.
Authentication Considerations
Several engineering decisions need to be made regarding federating external identity providers for authentication and authorization. Entra ID is the cornerstone of authenticating users and services within Microsoft Azure. It relies on Microsoft Authenticator for multi-factor authentication. However, Authenticator uses FIPS-validated cryptography only on iOS devices with version 6.6.8 or higher. The Android version does not use FIPS-validated cryptography. Therefore, you will likely need a hard-token with FIPS-validated cryptography for multi-factor authentication unless all privileged users exclusively use iOS devices.
FedRAMP & AI!
Each deployment customizes application-specific requirements. We understand FedRAMP controls and Continuous Monitoring to determine the multi-cloud architecture. stackArmor helps customers develop cost-effective architectures. This includes maximizing Azure OpenAI features and performance and meeting FedRAMP requirements to ensure ATO renewal during the annual assessment.
Please contact us to schedule a free briefing to learn how we can add GenAI capabilities to your FedRAMP boundary.
