Vector Ingest Pipeline

AuroraSOC uses Vector as the event normalization and routing layer between raw log sources and the detection/storage planes.

Architecture

┌──────────────┐     ┌──────────────┐     ┌──────────────────┐
│  Suricata    │────►│              │────►│  NATS JetStream   │
│  EVE JSON    │     │              │     │  (normalized)     │
├──────────────┤     │   Vector     │     ├──────────────────┤
│  Syslog      │────►│   Pipeline   │────►│  ClickHouse      │
│  (UDP/TCP)   │     │              │     │  (hot tier)       │
├──────────────┤     │              │     ├──────────────────┤
│  HTTP Push   │────►│              │────►│  Prometheus       │
│  (webhooks)  │     └──────────────┘     │  (metrics)        │
└──────────────┘                          └──────────────────┘

Sources

Source	Port/Path	Protocol
Suricata EVE	/var/log/suricata/eve.json	File tail
Syslog	UDP/TCP 1514	RFC 5424
HTTP Receiver	Port 8687	JSON over HTTP
NATS Raw	aurora.raw_events.>	NATS subscription

OCSF Normalization

Every event is normalized to OCSF 1.3 before routing:

• category_uid - Event category (1=System, 2=Detection, 4=Network, 6=Application)
• class_uid - Specific event class
• severity_id - 1 (Info) to 4 (Critical)
• src_endpoint / dst_endpoint - Network endpoints
• metadata.product - Source product identification
• time - Canonical event timestamp

Routing

Events are routed by category after normalization:

• Detection findings → aurora.events.normalized.detection
• Network activity → aurora.events.normalized.network
• System/Application → aurora.events.normalized.system

Deduplication

A content-hash based dedup filter runs with a 30-second window, keyed on:

• Source/destination IP
• Finding UID (for detections)
• Category UID

Deployment

Development

just stack-up-min  # Includes Vector with dev config

Production

docker compose -f docker-compose.yml -f docker-compose.vector.yml -f docker-compose.clickhouse.yml up -d

Monitoring

Vector exposes Prometheus metrics on port 9598. The /api/v1/graph endpoint shows the live pipeline topology.