Keeping a Bedrock chatbot on topic and on budget: defense in depth, not a single safeguard
A public LLM chatbot needs several independent safeguards: prompt hardening, a guardrail, rate limiting without WAF, a hard cost kill switch and EU-only routing. What I built, what I measured and what I learned.
The chatbot on chnet.solutions answers questions about my profile and services, using a small Claude model on Amazon Bedrock. The interesting part is the question every public LLM endpoint raises: how do you stop it becoming a free general-purpose AI on your AWS bill? A use case, not a showcase — the pattern fits any FAQ, support or product-advice bot.
Hypothesis
No single control keeps a public chatbot reliably on topic and within budget. Several independent layers, each bypassable on its own, should hold together: an attack that beats the system prompt hits the guardrail, a flood hits the rate limit, and if all else fails the cost kill switch pulls the plug.
Two questions decide it: does the bot answer legitimate questions and decline everything else with a defined sentence, and does spend stay capped under abuse?
Setup
The stack is deliberately small — static site, HTTP API (API Gateway v2), one Lambda function, two DynamoDB tables, Bedrock, all in AWS CDK. The safeguards, in the order a request meets them:
- Hardened system prompt with a refusal anchor. The prompt states what the bot covers and, just as explicitly, what it does not, with examples on both sides and one fixed refusal sentence (“This question is outside the scope I cover …”). That sentence is the real trick: it makes refusals machine-detectable and the bot testable.
- Bedrock Guardrail. Four denied topics (coding help, general knowledge, unrelated people and companies, role play), a prompt-attack filter, content filters and a short jailbreak word list — input and output, no code of my own.
- Pre-model filter. A heuristic catches ping floods, punctuation-only input and near-duplicates and returns a fixed notice without any model call.
- Conversation limits. At most 12 turns per conversation and 400 tokens per answer; history is cut at 8,000 tokens. The system prompt leads every turn, so long chats cannot dilute the role.
- Per-IP rate limit without WAF. A fixed quota per IP and time window, implemented as an atomic DynamoDB counter with TTL, plus a hard ceiling of 10 concurrent Lambda executions.
- Budget chain with auto-disable. An AWS Budget on Bedrock spend emails warnings at 50, 80 and 100 percent of forecast. At 100 percent of actual spend it triggers, via SNS, a tiny Lambda function that sets the chat handler’s concurrency to 0. It can do nothing else, and re-enabling the bot is deliberately manual.
- EU data residency. Bedrock is called through an EU cross-region inference profile that only routes between European regions. Conversation logs stay in the EU home region and expire automatically after 90 days.
Deliberate shortcuts: no separate classifier model, no fine-tuning, no streaming. The concurrency ceiling and the budget chain are the net underneath every other layer.
Measurement
The yardstick is an eval suite run against the deployed endpoint — a unit test with a real model call. It now holds 84 cases in 14 groups, from legitimate and disguised off-topic questions to prompt injection, multi-turn drift, spam and lead enquiries. Scoring relies on the refusal anchor: present when a refusal is expected, absent when an answer is.
The latest full run (27 October 2026, 84 cases) passed 73 of 84, or 86.9 percent against a 95 percent target; in May 2026 it was 61 of 73 (83.6 percent). All 11 failures are legitimate questions and lead flows: the bot declined simple skill questions, did not complete lead requests as intended, or repeated its refusal after a clarification. No off-topic, injection, context-drift or spam case failed. The gap is honest signal, not noise — which is the first lesson.
Lessons learned
- The test runner was the leak. An earlier version scored HTTP errors with an empty body as valid answers, so under load the bot “passed” cases it never answered. Now the runner fails hard on 4xx/5xx; the numbers are lower and trustworthy.
- Overly strict filters turn away real prospects. At its highest setting the prompt-attack filter flagged all 4 tested questions that simply addressed the bot directly (“can you do AWS?”) and returned an empty answer. One level lower it still catches jailbreaks. Denied topics behaved alike: a too-broad definition classed questions about my own skills as off-topic. Guardrail topics take no negative examples; the length-limited definition is the only lever.
- A scope that is too narrow loses leads. Requests like “I’d like to host my website on AWS” were refused at first. Lead enquiries are now explicitly in scope, and after a clarification the bot must not repeat its refusal.
- WAF does not fit every API. AWS WAF attaches to REST APIs and CloudFront, among others, but not to an API Gateway HTTP API v2 — hence the rate limit inside Lambda.
- Budget actions cannot throttle a Lambda function. They support IAM, SCP and SSM actions but no concurrency change, hence Budget → SNS → Lambda.
- Bedrock access is a Marketplace transaction. Anthropic models need an AWS Marketplace subscription, started from the Marketplace product page rather than the Bedrock console, plus a use-case form. It takes minutes to reach every EU region of the profile; meanwhile AccessDenied errors appear sporadically.
- IAM has to know every routing region. Permissions are checked against the model in the region a request is routed to. Omit one of the profile’s regions from the policy and requests fail at random.
Conclusion
The hypothesis holds, with a caveat. The eval stopped every off-topic and injection case. Spend is capped by design through the concurrency ceiling, rate limit and budget chain — reasoned, not measured; I have no documented case of the chain firing. The open issue is at the other end: false refusals. Safeguards add friction, and it hits the very visitors the bot was built for.
What I take into client work: an eval suite with an unambiguous refusal anchor matters as much as the safeguards, because only it shows whether tightening a control blocks attackers or customers. The kill switch belongs in the first deployment. And platform limits like WAF on HTTP APIs or the Marketplace subscription are best found during planning, not on deploy day.
Stack: Amazon Bedrock · Claude · Bedrock Guardrails · AWS Lambda · API Gateway HTTP API · DynamoDB · AWS Budgets · AWS CDK