🌐 Agent Web Search

⚡Executive Summary

Agent Web Search is a side-by-side comparison tool demonstrating the difference between LLM responses from training data alone vs. real-time grounded responses using AWS-managed Web Search via AgentCore Gateway. Built with Strands Agents on Amazon Bedrock, it features server-side governance with domain and date filters, agentic multi-turn retrieval where the model decides when and what to search, full execution traces, cost transparency ($7/1K searches), and zero third-party search SDK dependencies. The entire search path runs within AWS - no external search engine, no API keys, no third-party egress.

📋Situation

The Grounding Problem

Large language models answer from training data that has a knowledge cutoff. Ask about yesterday's stock price, last week's software release, or today's news, and the model either hedges ("as of my last training data...") or confidently states outdated facts. Enterprises building AI assistants needed a way to ground model responses in current information without introducing third-party search dependencies, managing API keys, or writing custom response-parsing code.

The Integration Challenges

• Knowledge Cutoff: Models cannot answer questions about events after their training data ends - a fundamental limitation for any time-sensitive use case
• Third-Party Dependencies: Traditional web search integration requires search engine API keys, SDK installations, response parsing, rate limit management, and billing relationships with external providers
• No Governance: When a model decides to search, operators have no server-side control over how many results are fetched, which domains are queried, or what date ranges are included
• Cost Opacity: Search costs are buried in third-party bills with no per-query attribution, making it impossible to project or control spend per run
• Security Concerns: Queries leave the AWS network boundary when routed through external search providers, creating data egress and compliance risks
• No Proof of Work: Users see a final answer but cannot inspect what the model searched for, what results it received, or whether it searched at all - a black box between question and answer

🎯Task

The Challenge: Proving Grounded Search with Zero Third-Party Dependencies

The goal was to build an interactive demo that makes the value of AgentCore Web Search immediately visible - run the same question through the same model twice, once without web search and once with it, and let the side-by-side comparison speak for itself.

• Side-by-Side Comparison: Two options running the same model on the same question - Option 1 from training data alone, Option 2 with AgentCore Gateway Web Search - rendered in parallel columns
• Zero Third-Party Search: No search SDK in requirements, no API key in config, no third-party egress. The query stays within AWS, served by the managed Web Search connector
• Model-Driven Search Decisions: The model - not the code - decides whether to search and what to search for. Stable-fact questions incur zero search costs; time-sensitive questions trigger retrieval automatically
• Server-Side Governance: Operator controls for maxResults caps, domain include/exclude filters, and published-date bounds - enforced on every search call regardless of what the model requests
• Agentic Multi-Turn Retrieval: The model can search multiple times in one run, refining queries based on initial results. maxResults is per-search, so multi-search runs accumulate more sources
• Full Execution Traces: Every tool call recorded with the model-generated query, the governance policy applied, the full response payload, and duration
• Cost Transparency: Real-time cost estimates per run showing Bedrock token costs plus $7/1K Web Search charges, with the search count visible in the trace
• Citation Integrity: Every grounded response includes source titles, URLs, publication dates, and snippets - rendered with markdown but with raw HTML disabled for security when consuming open-web content

⚙️Action

The Architecture

Fully serverless on AWS with a clear trust boundary - the search query crosses into an AWS service account, not out to the internet:

Browser ──► CloudFront ──┬──► S3 (private, OAC)                  static bundle
   (no sign-in)          │
                         └──► API Gateway ──► Lambda ──┬──► Bedrock (model)
                              (Lambda            │     └──► AgentCore Gateway
                               authorizer)       │            └──► web-search connector
                                                 └──► CloudWatch Logs

Trust Boundary:
  Query stays within AWS ──► AgentCore Gateway ──► AWS-managed Web Search
  No third-party egress  ──► No search SDK  ──► No API keys

Two Options - One Model

Technology Stack

Server-Side Governance (SearchPolicy)

A Strands BeforeToolCallEvent hook intercepts every WebSearch invocation before it leaves the agent. The operator's settings are authoritative - whatever the model requests is overridden:

• maxResults Pinning: Clamped to the operator-configured value (1-25 range). If the model omits it, the policy fills it in. If the model asks for 20, the policy pins it to the configured cap. The trace logs the override
• Domain Filters: Include and exclude lists enforced on every call. Up to 100 domains per list (connector 1.2.0+). Full URLs pasted by the user are normalized to bare domains (https://www.python.org/downloads/ becomes python.org)
• Published-Date Bounds: ISO-8601 date range filters restrict results to a time window, regardless of what the model's query text implies
• Policy Transparency: Every override is logged in the execution trace: "Policy: model omitted maxResults, pinned to 3; filters applied"

Agentic Multi-Turn Retrieval

• Model-Driven Search: The model writes its own search query based on the user's question. For compound questions ("how did it close, and why did it move?"), the model searches once per intent
• Iterative Refinement: On open-ended questions, the model reads the first set of results and may search again with a refined query. maxResults is per-search, so a run with three searches can return roughly three times as many unique sources
• Smart Cost Control: Ask something the model already knows ("explain what a Lambda function is") and it searches zero times, costing only Bedrock tokens. The model, not the code, decides whether searching adds value
• ToolCallLimiter Hook: A hard cap on tool calls per run prevents runaway loops. When the limit is reached, the model is forced to produce a final answer from what it has gathered

MCP Gateway Integration

• Discovery: The agent calls MCP tools/list on the Gateway and the WebSearch tool appears with its full schema - no hardcoded tool definitions
• Protocol: SigV4-signed Streamable HTTP MCP connection via mcp-proxy-for-aws. Credentials resolved from the Lambda execution role (deployed) or named profile (local)
• Gateway Target: CloudFormation provisions the Gateway, IAM service role, and web-search connector target. The tool ARN is AWS-owned (arn:aws:bedrock-agentcore:*:aws:tool/web-search.v1)
• No Third-Party Egress: The query crosses into an AWS service account, not out to the internet. No search SDK in requirements.txt, no API key in .env

Execution Tracing

A ToolCallRecorder hook (Strands AfterToolCallEvent) captures every tool invocation with full detail:

• Per-Call Recording: Tool name, arguments (including the model-generated query), result summary, full raw payload, duration, and any errors
• Citation Extraction: Parses Web Search response payloads to extract titles, URLs, publication dates, and snippets. De-duplicated across multiple searches in the same run
• Knowledge Graph Facts: Handles structured observations (null title/url with facts in text) by labeling them as "Knowledge graph fact" rather than dropping them
• Search Count Tracking: The trace header displays source count and number of searches, explaining why multi-search runs may return more sources than the per-search cap

Access Control & Security

No viewer authentication by design - anyone with the CloudFront URL can run the demo. Protections are about bounding cost and locking down the path to the agent:

• Reserved Concurrency: Lambda concurrency capped at 5 (configurable via API_CONCURRENCY_LIMIT). Beyond 5 in-flight runs, requests are throttled, not billed - this is the spend ceiling
• Gateway Authorizer: A REQUEST-type Lambda authorizer on API Gateway validates a secret that only CloudFront injects. Requests without it are rejected with 401 before the authorizer runs. Decisions cached 300 seconds
• Double Origin Check: The application Lambda independently verifies the CloudFront origin secret in constant time, so a request bypassing the authorizer still gets nothing
• Scoped Execution Role: The Lambda role is limited to InvokeGateway on the one Gateway ARN plus InvokeModel. The authorizer has a separate role with CloudWatch Logs only
• No Credentials to Browser: AWS credentials never reach the client. The browser sends settings; the server resolves credentials via its own execution role
• HTML Sanitization: Model output rendered as markdown with raw HTML deliberately disabled, since responses summarize pages fetched from the open web

Testing & Quality

• Offline Pipeline Tests (verify_pipeline.py): Drives the real Strands agent loop with a scripted fake model - Web Search response parsing, citation extraction, and $7/1K cost accounting exercised, not assumed. Also asserts no third-party search SDK is declared or imported anywhere
• API Contract Tests (verify_api.py): Filter payloads, domain normalization, error handling, and serialization verified with no AWS credentials needed
• Live End-to-End (live_check.py): One real run against a deployed Gateway costing ~$0.007 per search plus model tokens
• Type Safety: TypeScript strict mode with noUnusedLocals and noUnusedParameters in tsconfig.json - dead code fails the build
• Linting: Ruff for Python lint and format checks, configured in pyproject.toml

Key Design Decisions

📊Result

Key Capabilities

What the Demo Proves

• Grounding: Option 1 cannot answer about today and has no sources. Option 2 answers with titles, URLs, and publication dates from real-time web results
• Zero Integration Code: Adding web search means adding a connector to a Gateway - no SDK, no API key, no parsing glue. The model discovers the tool via MCP tools/list
• No Third-Party Egress: The query is served inside AWS. No external search engine, no data leaving the AWS boundary
• Model Intelligence: Ask something that needs no lookup and the model searches zero times, billing nothing. The model decides, not the code
• Governance Works: Set domain filters and date bounds in the UI, run again, and the trace shows "Policy: model omitted maxResults, pinned to 3; filters applied" - the operator's values are injected server-side
• Agentic Retrieval: Compound questions trigger multiple searches with refined queries. The model reads initial results and decides whether to search again
• Cost Transparency: Every run shows exact search count, Bedrock token costs, and Web Search charges. Switching from Sonnet to Haiku roughly halves the cost while maintaining grounded, cited answers

Suggested Demo Flow

• 1. Architecture Tab: Walk Option 1 (model alone), then Option 2 (model + Gateway). Land on the trust boundary - the query crosses into an AWS service account, not out to the internet
• 2. Run Tab: Pick a sample question, run both. Option 1 hedges and names its cutoff; Option 2 cites sources. Open the execution trace to show the model-generated query and full response payload
• 3. Search Controls: Restrict to one domain or set a date window, run again. The trace reads the governance override - proving server-side enforcement
• 4. No-Search Question: Ask something stable ("explain what a Lambda function is"). Option 2 searches zero times, costing only Bedrock tokens. The model decided
• 5. Side by Side Tab: Close on the cost row - $7 per 1,000 queries, only when the model searches

Enterprise Impact

• Compliance & Transparency: Enforced governance prevents parameter manipulation. All search decisions are audited in the execution trace with full payloads
• Cost Optimization: Model intelligence determines when searching occurs - stable-fact questions incur zero search cost. Switching to Haiku 4.5 halves token costs while maintaining grounding accuracy
• Operational Simplicity: No search SDK to maintain, no API key rotation, no response-parsing code to update when provider schemas change. The connector is managed by AWS
• Security Posture: No third-party data egress. Reserved concurrency caps spend. Double origin-secret validation in constant time. Raw HTML disabled in rendered output
• Teardown-Safe: Dedicated teardown scripts for both the hosting stack and the Gateway infrastructure. An open endpoint that can spend money should not outlive its purpose

🎥Demo Video

Watch Agent Web Search in action - see the side-by-side comparison between training-data-only responses and real-time grounded responses with AgentCore Gateway Web Search:

💡Key Learnings & Best Practices

Architecture Insights

• MCP Discovery Over Hardcoding: Having the agent discover the WebSearch tool via tools/list means the integration survives connector upgrades, schema changes, and target renames without code changes. The schema is always current because it comes from the Gateway, not from a local definition
• Governance as Hooks, Not Middleware: Implementing SearchPolicy as a Strands BeforeToolCallEvent hook means it applies to every search call including second and third searches in a multi-turn run. HTTP middleware would miss tool calls that happen inside the agent loop
• Side-by-Side Sells the Pattern: Showing the same question through both paths makes the value of grounding undeniable. Option 1 hedges; Option 2 cites. No slide deck achieves this
• Concurrency as a Cost Ceiling: Reserved Lambda concurrency is a more honest cost control than rate limiting. It bounds the maximum spend per unit time, not the maximum requests - which is what matters when each request costs money

Agent Behavior Insights

• Phrasing Matters: "What is AMZN's share price right now?" makes Haiku decline (it reads "right now" as a live-data request it cannot satisfy). "How did AMZN close most recently?" is a retrieval question, and it searches. Sample questions must be verified end-to-end on every model in the picker
• Single Intent, Predictable Results: Compound questions trigger multiple searches (one per intent), so the total source count exceeds the per-search cap. Single-intent questions give one search and exactly the requested count - which is less confusing for a demo
• Models That Fail on Tool Schemas: Nova Pro and Nova Lite cannot handle the nested filters object in the WebSearch input schema - they return modelStreamErrorException. Only models verified end-to-end on both options belong in the picker. This is why only Claude Sonnet 4.6 and Haiku 4.5 are offered
• Domain Normalization Matters: Users paste full URLs; the Web Search filter expects bare domains. Normalizing https://www.python.org/downloads/ to python.org prevents silent filter failures. Stripping www. is intentional - a root domain already matches subdomains

Security Insights

• Double Secret Validation: The Gateway authorizer rejects requests without the CloudFront origin secret, and the application Lambda checks it independently. Two layers, two failure modes, one secret
• Disable Raw HTML in Model Output: When the model summarizes pages fetched from the open web, its output may contain HTML from those pages. Rendering it as markdown with raw HTML disabled prevents XSS from untrusted web content
• Credentials Stay Server-Side: The browser sends settings (model ID, filters); the server resolves AWS credentials from its own execution role. No credential material ever reaches the client