How to Implement a Website AI Chat Agent

Overview

A website AI chat agent lets visitors ask questions without leaving your site. The proven pattern splits responsibilities: a thin frontend handles chat UI; your main site exposes a server-side proxy; a separate API owns sessions, prompts, and LLM calls.

• Chat UI: Floating widget and/or dedicated chat page — presentation only
• Website proxy: POST /api/chat — hides backend URL and API keys from the browser
• AI API: FastAPI (or similar) on Lambda, containers, or a VM — session I/O and LLM invoke
• LLM: Managed model API (Bedrock, OpenAI, Azure OpenAI, etc.)
• Memory: Per-session history in object storage or a database

Architecture

Visitor -> chat widget OR /chat page
  -> POST /api/chat { message, session_id? }
  -> website proxy (validate, rate limit, add secret header)
  -> AI API POST /chat
  -> load history -> build prompt -> LLM -> save session
  -> { response, session_id } -> render in UI

AWS Reference Architecture

On AWS, the AI API tier maps cleanly to a serverless stack. The main website (Amplify Hosting, CloudFront, or any host with a server-side route) proxies chat requests so Bedrock credentials never reach the browser.

AWS service	Role in the chat agent
Amplify Hosting / CloudFront	Serve the main website and optional standalone chat SPA
API Gateway (HTTP API)	Public HTTPS entry for POST /chat; throttling and CORS
Lambda + FastAPI	Session I/O, prompt assembly, Bedrock invoke (via Mangum or similar adapter)
Amazon Bedrock	Managed LLM (Nova, Claude, etc.) — no self-hosted inference
Amazon S3	Private bucket for per-session JSON; optional origin for static chat UI
IAM	Lambda execution role with least-privilege Bedrock and S3 permissions

# Path A - chat embedded in existing site (recommended)
Browser -> Next.js /api/chat proxy -> API Gateway -> Lambda -> Bedrock + S3

# Path B - standalone chat SPA
Browser -> CloudFront + S3 chat UI -> API Gateway -> Lambda -> Bedrock + S3
(CORS allowlist on API Gateway; no website proxy needed)

UI Entry Points

Pattern	Purpose
Floating widget	Compact panel on every page; low friction for quick questions
Full chat page	Larger viewport for long, markdown-formatted replies
Shared state	Both UIs read/write the same sessionStorage key so the conversation continues when switching views

Implementation Checklist

Layer	What to build
Chat component	Message list, input, send button, loading state; render assistant replies as markdown
Chat context	Hold messages, session_id, and isLoading; persist to sessionStorage per tab
Proxy route	GET returns { enabled }; POST forwards to backend with optional shared secret header
AI API	POST /chat accepts { message, session_id? }; returns { response, session_id }
Context injection	Static profile, FAQ, or product facts prepended to the prompt (no vector DB required for v1)
Session store	JSON per session_id in S3, Redis, or Postgres; trim to last N turns

API Contract

# Browser -> website proxy
POST /api/chat
{ "message": "What services do you offer?", "session_id": "optional-uuid" }

# Proxy -> AI backend (server-side only)
POST /chat
Headers: X-Assistant-Secret: {shared-secret}   # optional
{ "message": "...", "session_id": "..." }

# Response
{ "response": "markdown text", "session_id": "uuid" }

Keep the backend URL and any API keys in server environment variables — never expose them to the client.

Session and UX Behaviour

• First message: Omit session_id; backend creates one and returns it
• Follow-ups: Send the same session_id so the API loads conversation history
• Tab scope: Store messages in sessionStorage — closing the tab resets the UI; backend sessions can expire on a TTL
• Enter to send: Single-line input with Enter key; show a thinking indicator during LLM latency
• Graceful fallback: If the backend is unset, show a configure message instead of a broken widget

Security

• Server-side proxy: Browser never calls the LLM or AI API directly
• Rate limiting: Per-IP throttle on the proxy route (e.g. 30 requests/minute)
• Input validation: Reject empty messages; cap message length (e.g. 4 000 characters)
• Shared secret: Optional header between proxy and backend when both are on the public internet
• CORS: If you also host a standalone chat SPA, allowlist origins explicitly

Prompt and Content Tips

• Inject a system prompt with company name, tone, and scope boundaries
• Include static facts (services, pricing tiers, support email) in the prompt or a context file
• Direct out-of-scope or sales questions to a human contact address
• Start without RAG; add retrieval later when the knowledge base grows

Design Principles

Deploy the chat UI and AI API independently — the website is one client, not the backend. This mirrors enterprise portal patterns where multiple frontends (web, mobile, agent) call the same API. Keeping the UI thin makes it easy to swap models, add tool calls, or embed the agent elsewhere without rewriting the site.