Data Masking & Redaction
Relevant source files
The following files were used as context for generating this wiki page:
Data Masking & Redaction in Hoop is a Layer 7 security feature designed to automatically detect and obscure sensitive information—such as PII (Personally Identifiable Information), PHI (Protected Health Information), and PCI (Payment Card Industry) data—in real-time. This processing occurs at the protocol layer before data leaves the gateway, ensuring that sensitive values never reach the user's terminal, IDE, or logs.
The system utilizes ML-powered classification rather than simple regex pattern matching, allowing it to understand the context of data (e.g., distinguishing a phone number from a build ID). It supports multiple wire protocols including PostgreSQL, MySQL, MSSQL, MongoDB, SSH, and HTTP.
Sources: README.md:15-18, README.md:144-149
System Architecture
The data masking system is integrated across the gateway backend, the database models, and the management web application.
Data Flow Overview
The gateway parses wire protocols natively. When a query result or API response is received, the gateway inspects the payload, identifies entities based on configured rules, and applies masking.
The diagram shows how the gateway sits between the client and infrastructure, intercepting raw data to apply masking before returning it to the user.
Sources: README.md:131-140, common/proto/spectypes/types.go:41-45
Data Models and Entities
Masking logic is governed by DataMaskingRule objects. These rules define what to mask, how confident the system must be, and which resources the rule applies to.
DataMaskingRule Structure
| Field | Type | Description |
|---|---|---|
ID | UUID | Unique identifier for the rule. |
Name | String | Unique name for identification. |
SupportedEntityTypes | List | Built-in ML groups (e.g., CONTACT_INFORMATION). |
CustomEntityTypes | List | User-defined regex or deny-lists. |
ScoreThreshold | Float | Confidence level (0.0 to 1.0) required for masking. |
ConnectionIDs | List | Connections where this rule is active. |
Attributes | List | Connection attributes identifying scoped resources. |
Sources: gateway/models/datamasking.go:15-27, gateway/api/mcpserver/tools_datamasking.go:31-39
Supported Entity Groups
The system provides several webapp presets for entity detection:
- CONTACT_INFORMATION: Includes
EMAIL_ADDRESS,PHONE_NUMBER,PERSON. - FINANCIAL_DATA: Includes
CREDIT_CARD,IBAN_CODE. - NETWORK_IDENTIFIERS: Includes
IP_ADDRESS,URL. - OTHERS:
LOCATION_DATA,US_DOCUMENTS,MEDICAL_DATA,CRYPTO_IDENTIFIERS.
Sources: gateway/api/mcpserver/tools_datamasking.go:34-36
Validation and License Gating
The implementation strictly enforces validation rules and differentiates between OSS and Enterprise licenses.
Core Validation Logic
The gateway validates that:
ScoreThresholdis between 0 and 1.- Entity type names are strictly uppercase.
- Custom entity types contain either a regex or a deny-list.
Sources: gateway/api/datamasking/datamasking.go:48-73
OSS Limitations
Organizations using the OSS license are subject to specific constraints:
- Limited to one data masking rule per organization.
- Cannot define both supported entity types and custom entity types in the same rule.
- Supported entity types are limited to one type group and one specific entity type.
Sources: gateway/api/datamasking/datamasking.go:78-95, gateway/api/mcpserver/tools_datamasking.go:137-143
API Endpoints
The system is managed via a REST API and integrated into the Model Context Protocol (MCP) server for AI agents.
REST API Reference
| Method | Endpoint | Description |
|---|---|---|
GET | /datamasking-rules | List all masking rules. |
POST | /datamasking-rules | Create a new masking rule. |
PUT | /datamasking-rules/{id} | Update an existing rule. |
DELETE | /datamasking-rules/{id} | Permanently remove a rule. |
Sources: gateway/api/datamasking/datamasking.go:278-360
MCP Server Integration
AI agents can manage masking rules through specific MCP tools:
datamasking_rules_list: List organization rules.datamasking_rules_create: Configure new detection patterns.datamasking_rules_update/delete: Manage rule lifecycles.
Sources: gateway/api/mcpserver/tools_datamasking.go:51-78
Protocol Layer Implementation
Masking information is carried within protocol packets using specific spec types.
type TransformationOverview struct {
TransformedBytes int64 `msgpack:"transformed_bytes"`
Summaries []TransformationSummary `msgpack:"transformation_summary"`
Err error `msgpack:"err"`
}
The TransformationOverview structure tracks the volume of data modified and provides a summary of which fields and info types were affected during a session.
Sources: common/proto/spectypes/types.go:41-45
User Interface
The management UI allows administrators to configure rules, set thresholds, and filter active rules by connection or attribute.
Management Workflow
- Rule Discovery: Users can search for rules applied to specific "Resource Roles" or "Attributes".
- Configuration: The create/edit form provides sections for rule information, resource role association, and specific output rules.
- Thresholding: Administrators can set an "Analyzer confidence threshold" (defaulting to 85%) to balance detection accuracy vs. false positives.
The sequence shows the standard data retrieval flow for the management interface.
Sources: webapp_v2/src/pages/Features/DataMasking/index.jsx:55-66, webapp_v2/src/pages/Features/DataMasking/Create/index.jsx:136-150
Conclusion
Data Masking & Redaction provides a critical security layer by ensuring PII and other sensitive data types are obscured at the gateway. By combining ML-based classification with flexible rule management and protocol-native parsing, the system allows engineers and AI agents to work with production data safely, minimizing the risk of data leakage without hindering operational visibility.