Architecture Overview
Relevant source files
The following files were used as context for generating this wiki page:
LogUtil is a high-performance, real-time Nginx log analyzer built in Rust. It utilizes an asynchronous architecture to monitor log files, parse entries via regex, and provide an interactive Terminal User Interface (TUI) for data visualization and security analysis.
The system is designed around a central in-memory database (MemoryDB) that indexes log entries for rapid retrieval, supporting real-time updates through a background file monitoring task. The interface is modular, employing a tab-based management system to separate different analytical concerns such as security, performance, and general traffic overview.
Sources: README.md:12-25, src/main.rs:1-15
Core System Components
The architecture consists of four primary layers: the CLI/Entry Point, the Configuration/Interactive Selector, the Data Management Layer, and the TUI Presentation Layer.
This diagram illustrates the high-level flow from application startup to the real-time monitoring and display loop.
Sources: src/main.rs:114-165, src/app.rs:36-65
Entry Point and Execution Modes
The application supports two primary execution modes:
- Direct Mode: When a file path is provided via CLI arguments, the application immediately initializes analysis.
- Interactive Mode: Launched when no file is specified. It provides a TUI-based file browser and settings editor.
Sources: src/main.rs:125-155, src/file_settings.rs:88-140
Data Management Layer
The MemoryDB serves as the central synchronized storage for all parsed log entries. It is designed for high-concurrency and fast indexing using DashMap.
MemoryDB Structure
The database indexes records by IP, URL, and Status Code to facilitate instant lookups for different TUI tabs.
| Component | Technology/Logic | Description |
|---|---|---|
| Primary Storage | DashMap<u64, LogRecord> | Thread-safe concurrent hash map for log entries. |
| Indexing | DashMap<String, Vec<u64>> | Multi-indexes for IPs, URLs, and Status Codes. |
| Stats Tracking | Arc<RwLock<DBStats>> | Real-time counters for unique visitors and total traffic. |
| Eviction Policy | FIFO / Count-based | Removes entries when exceeding max_records (default 10M). |
Sources: src/memory_db.rs:32-55, src/memory_db.rs:104-125
Data Flow Sequence
Sequence of events from a new log line detection to UI update.
Sources: src/main.rs:325-345, src/memory_db.rs:83-100
TUI Architecture
The TUI is built using the ratatui library and managed by a central App struct that coordinates input handling and rendering across multiple tabs.
Tab Management
The application implements a modular tab system where each tab (e.g., Overview, Security, Performance) implements a common Tab trait.
| Tab Name | File Reference | Functionality |
|---|---|---|
| Overview | overview.rs | Displays Top IPs and URLs with real-time request counts. |
| Requests | requests.rs | Searchable list of all log entries with pagination. |
| Detailed | detailed.rs | Drill-down view of requests associated with a specific IP. |
| Security | security.rs | Detects SQLi, XSS, and Brute Force patterns. |
| Heatmap | heatmap.rs | Visualizes hourly and daily request distribution. |
Sources: src/app.rs:50-80, src/tabs/mod.rs:1-12
Layout Management
The App::draw function defines a standardized layout for all screens, consisting of a Header, Summary Panel, and a Content Area.
Sources: src/app.rs:190-215, src/tui_manager.rs:24-35
Configuration and Settings
The FileSettings module handles the interactive configuration before analysis begins. It allows users to browse directories and toggle analysis features.
| Setting | Type | Default | Description |
|---|---|---|---|
Count | Number | 0 (Tail) | Number of lines to read from the end. |
Regex | Text | Nginx Default | Pattern used to parse log lines. |
Security | Boolean | false | Enables/Disables threat detection. |
Top N | Number | 10 | Limit for "Top" lists in various tabs. |
Sources: src/file_settings.rs:114-165, src/main.rs:50-100
Performance and Concurrency
LogUtil employs several strategies to maintain high performance:
- Asynchronous I/O: The file monitor runs in a separate
tokiothread to prevent UI blocking. Sources: src/main.rs:325-345 - Atomic Stats: Uses
AtomicUsizefor record IDs to avoid lock contention. Sources: src/memory_db.rs:34 - Caching: The
Appstruct caches summary statistics (Total Requests, Unique IPs) for 1 second to reduceMemoryDBread locks during high-frequency UI refreshes. Sources: src/app.rs:260-285