Email Ingestion

Relevant source files

The following files were used as context for generating this wiki page:

• src/app/(app)/settings/app/page.tsx/settings/app/page.tsx)
• src/components/AppSettingsForms.tsx
• src/lib/actions/app-settings.ts
• src/lib/actions/import.ts
• src/lib/appSettings.ts
• PRODUCT.md

Email Ingestion is an automated document capture system that periodically checks a configured IMAP mailbox for new messages. The system identifies attachments in incoming emails, extracts their content, and places them into the household "inbox" for review. This feature serves as a frictionless entry point for financial and personal paperwork, allowing users to forward digital documents directly to their Hearth instance.

Sources: PRODUCT.md, src/app/(app)/settings/app/page.tsx:143-151

System Architecture

The Email Ingestion module operates as a scheduled background process on the Hearth server. It connects to an external mail server using the IMAP protocol to retrieve documents.

High-Level Data Flow

The following diagram illustrates how a document moves from an email sender to the Hearth inbox:

The system periodically checks the mailbox, guesses the document type, and drops it into the Inbox for manual confirmation.
Sources: PRODUCT.md, src/app/(app)/settings/app/page.tsx:143-146

Configuration

Admins configure Email Ingestion through the System Settings. The configuration includes IMAP connection details and credentials, which are stored encrypted in the database.

Configuration Parameters

Parameter	Type	Description
host	String	The IMAP server hostname (e.g., imap.gmail.com).
port	Number	Connection port (typically 993 for SSL/TLS).
secure	Boolean	Whether to use implicit TLS.
user	String	The email address or username for the mailbox.
password	String	Encrypted password for mailbox access.
mailbox	String	The specific folder to check (default is "INBOX").

Sources: src/lib/appSettings.ts, src/components/AppSettingsForms.tsx:84-114

Connection Testing

Administrators can verify settings using the testEmailIngestConnection action. This action attempts to establish a connection with the IMAP server using the current form values before they are committed to permanent storage.
Sources: src/lib/actions/app-settings.ts, src/components/AppSettingsForms.tsx:121-125

Document Processing Logic

When the system retrieves an email, it processes attachments through a multi-stage pipeline:

1. Extraction: The system extracts raw bytes from email attachments.
2. Storage: Files are saved to local storage with a generated unique name to prevent collisions.
3. Searchable Text: The system runs extractSearchableText to process PDFs (via pdftotext) or images (via OCR).
4. Intake Computation: The computeInboxIntake logic analyzes the extracted text and file hash.
5. Record Creation: A record is created in the inboxDocument table with a status of NEEDS_REVIEW or POSSIBLE_DUPLICATE.

Sources: src/lib/actions/import.ts:109-130, CLAUDE.md

Processing Sequence

Security and Privacy

Hearth prioritizes privacy by ensuring the sender of an email is never implicitly trusted. Documents ingested via email are never filed automatically to a contract or warranty record. Instead, they must be reviewed by a household member who confirms the extracted details.

• Encryption at Rest: Mailbox passwords and AI API keys (if used for further extraction) are encrypted using the ENCRYPTION_KEY via AES-256-GCM.
• Manual Review: All ingested documents land in src/app/(app)/documents/inbox where they await classification.

Sources: PRODUCT.md, src/app/(app)/settings/app/page.tsx:143-151, CLAUDE.md

Standalone Mode Limitations

In Standalone (Mobile) mode, automated Email Ingestion is unavailable. Standalone instances rely on local file selection for imports rather than server-side mailbox polling. Standalone users must manually upload documents to the inbox.

Sources: ios-shell/www/app.js:655-659

Conclusion

Email Ingestion provides a specialized background service for document collection. By integrating IMAP polling with Hearth's extraction pipeline, it automates the first step of document management while maintaining strict security through a mandatory manual review process in the household inbox.