Content-Aware PII Detection
Relevant source files
The following files were used as context for generating this wiki page:
Content-Aware PII (Personally Identifiable Information) Detection is a Phase 5 capability of the Honmoon project designed to provide deep packet inspection and Data Loss Prevention (DLP). While initial phases focus on egress domain filtering and protocol-level metadata (verbs, resources), PII detection inspects the actual payloads of outbound traffic to identify and potentially redact sensitive information before it leaves the protected boundary.
This system leverages wire-level parsers to extract facts and content from protocols like PostgreSQL and the Kubernetes API. The detection logic is intended to be transport-agnostic, residing within the honmoon-core crate to ensure it can be unit-tested without network overhead and embedded into various components of the Honmoon data plane.
Sources: ARCHITECTURE.md, wiki/deep-dive/roadmap-open-core.md:32-101, datasets/pii/README.md:1-5
Architecture and Components
The PII detection system is integrated into the honmoon-core crate, specifically within the pii.rs module. It works alongside other sensitive data detectors, such as secret detection (API keys/tokens), and a redaction engine that utilizes a reversible secret_tokenizer.
The diagram above shows the flow from raw wire bytes through protocol-aware parsing to the core detection and redaction modules.
Sources: ARCHITECTURE.md, crates/honmoon-core/src/lib.rs:25-119
Key Data Structures and Types
The detection system relies on the Facts structure to store metadata extracted during protocol parsing. For PII detection, the system specifically targets content fields that may be present in HttpFacts, SqlFacts, or K8sFacts.
| Component | Responsibility | Source |
|---|---|---|
pii.rs | Contains Tier-1 PII detection logic. | ARCHITECTURE.md |
secret_detect.rs | Identifies API keys and security tokens. | ARCHITECTURE.md |
redact.rs | Joins PII and secret detection into a unified redaction flow. | ARCHITECTURE.md |
secret_tokenizer | Provides reversible tokenization for identified sensitive strings. | ARCHITECTURE.md |
Protocol-Aware Facts Extraction
Before PII detection can occur, the system must parse the wire-level communication. The protocols.rs module provides specialized parsers for this purpose.
- PostgreSQL: The
parse_postgres_queryfunction identifies'Q'(Simple Query) packets and extracts the SQL string. - Kubernetes: The
parse_k8s_requestfunction decomposes HTTP methods and paths into verbs, resources, and namespaces.
Once content is extracted, the PII engine evaluates it against defined sensitive patterns.
Sources: crates/honmoon-core/src/protocols.rs:17-156
The sequence shows the proxy delegating parsing and subsequent PII detection to the core logic.
Sources: ARCHITECTURE.md, crates/honmoon-core/src/protocols.rs:17-35
Benchmark and Evaluation
To ensure accuracy and prevent regression, the project includes a specialized PII benchmark dataset. This dataset is used to calculate precision, recall, and F1 scores for the detection engine.
Dataset Layout
The evaluation tools are located in datasets/pii/ and include:
labels.yaml: A source of truth containing 33 canonical PII labels, mapping between various standards likeko-pii,Presidio, andKDPII.gen_synth.ts: A generator for positive records using format/checksum-valid but non-existent values.gen_negative.ts: A generator for "hard negatives" (PII-shaped non-PII) to test the engine's precision.
Sources: datasets/pii/README.md:8-25
Scoring Metrics
Detection performance is measured using an entity-level scorer (score.ts) that compares gold-standard JSONL records against prediction JSONL records.
Sources: datasets/pii/README.md:43-50
Implementation Status
Content-aware detection is currently categorized under the "Fragile / Incomplete" or "Planned" status within the project's roadmap and technical debt tracking.
- Tier-1 PII Detection: Basic logic exists in
pii.rs. - Live Relay (TD-006): While parsers exist and are tested, they are not yet fully wired to feed live socket traffic into the detection engine.
- TLS Termination: Required for inspecting encrypted HTTP bodies (e.g., Kubernetes API traffic); currently unbuilt.
Sources: ARCHITECTURE.md, wiki/deep-dive/roadmap-open-core.md:137-144
Summary
Content-Aware PII Detection represents the transition of Honmoon from a network-layer firewall to a deep-content security gateway. By combining wire-level protocol parsing with localized PII and secret detection, the project aims to prevent sensitive data exfiltration by AI agents. The current development focus involves validating the engine against synthetic and normalized datasets while preparing for live-traffic integration.