An investigation engine that helps customers troubleshoot cost-related issues in AWS networking services. The system uses AWS DevOps Agent (Frontier agent) as the orchestrator and customer interface. The agent handles flow log analysis and CloudWatch metrics natively, guided by custom Skills. A CUR MCP server provides cost data from Athena that the agent cannot access directly (Athena queries are blocked as mutative operations in DevOps Agent).
AWS Networking Cost Investigation Engine using DevOps Agent + CUR MCP Server
Architecture
graph TD
subgraph Agent["DevOps Agent"]
A[Customer]
B[Orchestration]
M1[MCP: CUR Server]
S1[Skill: flowlogs-dns-correlation]
S2[Skill: cur-cloudwatch-correlation]
N1[Built-in: CloudWatch]
A --> B
B --> M1
B --> S1
B --> S2
B --> N1
end
APIGW[API Gateway + Auth]
subgraph Lambda["Lambda - eu-central-1"]
C[CUR MCP Lambda]
end
subgraph Sources["Data Sources"]
E[CUR 2.0 in S3 via Athena]
F["CloudWatch Logs<br/><small>FlowLogs, R53 Resolver Logs</small>"]
J[CloudWatch Metrics]
end
M1 --> APIGW
APIGW --> C
N1 --> J
N1 --> F
C --> E
Data Sources
Accessed by DevOps Agent natively (built-in CloudWatch integration)
Traffic patterns, bytes by destination, cross-AZ/cross-region volumes. Custom format with all v5 fields including pkt-src-aws-service, pkt-dst-aws-service, az-id, traffic-path, flow-direction
IP-to-FQDN mapping for enriching flow log destination IPs
CloudWatch Metrics
Standard NAT GW, TGW, ELB metrics
BytesProcessed, ErrorPortAllocation, PacketsDropCount, ActiveConnections, TGW data processed
Accessed via CUR MCP Server (agent cannot query Athena directly)
Data Source
Location
What it provides
CUR 2.0 (Data Exports)
S3 bucket + Athena table nci_cur.cur_data in eu-central-1
Cost breakdown by service, resource, region, AZ, UsageType, Operation, from/to locations. Includes effective cost (unblended + reservation + savings plan)
DevOps Agent Skills
Skill
Purpose
flowlogs-dns-correlation
Teaches the agent correct methodology for analyzing VPC Flow Logs with R53 DNS Resolver log correlation. Key: never use pkt-dst-aws-service for regional attribution, always correlate DNS-resolved IPs with flow log dstaddr
cur-cloudwatch-correlation
Teaches the agent how to correlate CUR cost data with CloudWatch metrics and flow logs for networking cost investigation
CUR MCP Server Tools
Tool
Purpose
getNetworkingCostBreakdown
Total networking cost by service, region, AZ with top spending resources
getResourceCostDetail
Full CUR breakdown for a specific resource — all dimensions including UsageType, Operation, from/to regions, effective cost
detectCostAnomalies
Compare current vs baseline period, flag resources with >20% cost increase
getTopNetworkingSpenders
Top N resources by networking cost
getCostTrend
Hourly, daily, or weekly cost trend for networking services
getCURDataRange
Exact time range covered by available CUR data
Investigation Process
CUR Analysis (via MCP server) → identify which networking services cost the most, detect anomalies, get resource-level detail with UsageType/Operation breakdown
Flow Log Deep Dive (agent native + flowlogs-dns-correlation skill) → for top cost resources, analyze traffic patterns with DNS enrichment to understand where traffic goes
CloudWatch Metrics (agent native) → verify with throughput/connection data (BytesProcessed trends, connection counts)
Correlate & Recommend (agent reasoning + cur-cloudwatch-correlation skill) → connect cost data with traffic patterns to produce actionable recommendations
Test Environment Architecture
graph TD
subgraph US["us-east-1"]
subgraph VPCA["VPC-A 10.0.0.0/16"]
subgraph AZ1["AZ1"]
NAT[NAT Gateway]
EC2A1["nci-instance-az1 m5.large"]
end
subgraph AZ2["AZ2"]
EC2A2["nci-instance-az2 m5.large"]
end
ALB[ALB - internal]
S3A["S3: nci-test-bucket-a"]
end
TGWA[Transit Gateway A]
CFA[CloudFront]
end
subgraph EU["eu-central-1"]
subgraph VPCB["VPC-B 10.1.0.0/16"]
NATB[NAT Gateway]
EC2B["nci-instance-b m5.large"]
S3B["S3: nci-test-bucket-b"]
end
TGWB[Transit Gateway B]
end
INTERNET[Internet]
EC2A1 --> NAT
EC2A2 --> NAT
NAT --> INTERNET
EC2B --> NATB
NATB --> INTERNET
ALB --> EC2A1
ALB --> EC2A2
CFA --> S3A
TGWA ---|TGW Peering| TGWB
EC2A1 --> TGWA
TGWB --> EC2B
Logging: VPC Flow Logs (all v5 fields) + R53 Resolver query logging on both VPCs. CUR 2.0 Data Export enabled. No S3 Gateway Endpoint in VPC-A (intentional — forces S3 traffic through NAT).
Project Structure
├── mcp-server/ # Self-contained CUR MCP Server
│ ├── template.yaml # SAM template (API GW + Lambda + Auth)
│ ├── src/
│ │ ├── cur_mcp/handler.py # MCP server Lambda (6 tools)
│ │ └── authorizer/handler.py # API Gateway Bearer token authorizer
│ └── layers/mcp-deps/
│ └── requirements.txt # awslabs.mcp_lambda_handler dependency
├── test-environment/ # CloudFormation for multi-region test lab
│ ├── region-a.yaml # us-east-1: VPC, NAT GW, TGW, ALB, CloudFront
│ ├── region-b.yaml # eu-central-1: VPC, NAT GW, TGW
│ ├── tgw-peering.yaml # Transit Gateway peering between regions
│ ├── athena/create_cur_table.sql # Athena table with all CUR 2.0 columns
│ └── scripts/ # Traffic generator scripts
├── skills/ # DevOps Agent Skills
│ ├── flowlogs-dns-correlation/ # Flow log + R53 DNS analysis methodology
│ └── cur-cloudwatch-correlation/ # CUR vs CloudWatch reconciliation
├── specs/ # Design docs, requirements, task list
│ ├── design.md
│ ├── requirements.md
│ └── tasks.md
└── docs/ # Architecture diagrams and learnings
├── POC-LEARNINGS.md
└── test-environment-architecture.drawio.png
Prerequisites
AWS account with admin access
AWS CLI configured
SAM CLI installed (brew install aws-sam-cli)
AWS DevOps Agent with MCP server capability enabled