Reference Architecture & Hands-on Implementation
Building a Secure, Reusable Enterprise AI Platform on AWS
Amazon Bedrock | Amazon EKS | API Gateway | Cognito | RAG | Cloud Security
Functional POC completed. Production-hardening enhancements are in progress.
Project Background
Background and Objective
As part of my ongoing technical exploration into AI infrastructure and security, I initiated this reference architecture to extend my existing AWS cloud architecture expertise into designing and implementing secure, reusable infrastructure for enterprise generative AI workloads.
The objective was to move beyond standalone model invocation and explore how AI capabilities could be integrated into an enterprise application environment, with emphasis on private networking, workload identity, authentication, deployment automation and extensibility.
The lending domain was selected as an illustrative use case to demonstrate both Retrieval-Augmented Generation (RAG) and general-purpose foundation-model inference.
Project type: Reference Architecture | Technical POC
Architecture and Design
Authenticated requests pass from enterprise clients through Amazon API Gateway and private VPC connectivity to reusable AI workloads running on private EKS nodes.
01
Authenticate
Amazon Cognito issues OAuth 2.0 access tokens for client applications.
02
Invoke API
Clients call the REST API with a bearer token and use-case path.
03
Connect privately
API Gateway routes through VPC Link V2 to an internal NLB.
04
Run AI use case
FastAPI on private EKS routes to Bedrock and Knowledge Bases.
| Decision | Engineering rationale |
|---|---|
| Private EKS | Run containerized AI application workloads within controlled VPC networking. |
| API Gateway + Cognito | Separate API authentication from application execution. |
| VPC Link V2 + internal NLB | Provide private connectivity to the EKS backend. |
| EKS Pod Identity | Avoid embedding long-lived AWS credentials in containers. |
| Generic use-case registry | Support multiple AI workflows through a common API. |
| Bedrock Knowledge Bases | Implement evidence-grounded RAG with source attribution. |
| NAT-free lab environment | Explore private AWS connectivity while considering infrastructure costs. |
| GitHub Actions + OIDC | Automate delivery without maintaining static AWS credentials in CI/CD. |
Implementation and Validation
- Anonymous API request rejected with HTTP 401.
- Authenticated RAG invocation returned HTTP 200 with an S3 policy source reference.
- Authenticated application-summary invocation returned HTTP 200.
- Both use cases successfully executed through API Gateway, VPC Link V2, internal NLB and private EKS.
Engineering Challenges and Lessons Learned
| Challenge encountered | How it was addressed |
|---|---|
| Pulling public controller images from private EKS without NAT | Mirrored the AWS Load Balancer Controller image into private ECR. |
| Load Balancer Controller unable to reach required AWS APIs | Configured the necessary private connectivity through a VPC interface endpoint. |
| Optional Shield API dependency causing controller delays | Disabled the unused integration. |
| Authenticated API returned 404 | Isolated the problem through direct pod/service testing and identified an outdated deployed application image. |
Implementation Status
Validated POC
Implemented and functionally validated
- Private EKS application platform
- API Gateway REST API with Cognito authorization
- VPC Link V2 and internal NLB integration
- Bedrock inference and Knowledge Base RAG
- Two reusable AI use cases
- EKS Pod Identity and GitHub Actions CI/CD
Planned / In Progress
Production-hardening roadmap
- Per-use-case authorization
- NLB security-group restrictions
- Tenant-aware RAG access
- Observability and resilience
- Bedrock Guardrails and AI governance
- Evaluation framework and prompt/model lifecycle
For implementation details, including the source code and deployment guidance, see the AWS Enterprise GenAI Platform repository on GitHub.
Planning a secure AI platform?
Let's discuss your AWS architecture, AI infrastructure, security and production-readiness requirements.