Relevant source files

The following files were used as context for generating this wiki page: - [tools/omfile.c](https://github.com/rsyslog/rsyslog/blob/main/tools/omfile.c) - [doc/source/configuration/modules/omfile.rst](https://github.com/rsyslog/rsyslog/blob/main/doc/source/configuration/modules/omfile.rst) - [tests/omfile-outchannel.sh](https://github.com/rsyslog/rsyslog/blob/main/tests/omfile-outchannel.sh) - [DEVELOPING.md](https://github.com/rsyslog/rsyslog/blob/main/DEVELOPING.md) - [doc/source/development/dev_oplugins.rst](https://github.com/rsyslog/rsyslog/blob/main/doc/source/development/dev_oplugins.rst) - [AGENTS.md](https://github.com/rsyslog/rsyslog/blob/main/AGENTS.md) - [doc/source/development/architecture.rst](https://github.com/rsyslog/rsyslog/blob/main/doc/source/development/architecture.rst)

# Output Module: Writing to Files (omfile) The `omfile` module is a core rsyslog output plugin responsible for writing log messages to files. As one of the most fundamental and widely used output modules, it provides a reliable and flexible mechanism for local log storage. It supports both static and dynamic filenames, allowing for complex log rotation and organization schemes based on message properties. `omfile` is considered a legacy module, residing in the `tools/` directory rather than the modern `plugins/` directory. Architecturally, it serves as a canonical example of an "inherently serial" output plugin, using a mutex to ensure that file write operations are properly synchronized, even in a multi-threaded worker environment. Sources: [tools/omfile.c](), [doc/source/configuration/modules/omfile.rst](), [DEVELOPING.md](), [AGENTS.md]() ## Architecture and Concurrency The `omfile` module is designed for safe, sequential access to a single file handle. This is critical because file I/O operations on a single descriptor are not inherently thread-safe. Sources: [DEVELOPING.md]() ### Concurrency Model `omfile` implements a simple but robust concurrency model by using a mutex to serialize access to the file. All message processing within the `doAction` entry point is guarded by this lock. * **Shared State (`pData`)**: The per-action `instanceData` structure holds the file descriptor (`fd`), filename, and the `mutWrite` mutex. This data is shared among all worker threads processing messages for a single action. * **Worker State (`WID`)**: The per-worker `wrkrInstanceData_t` is minimal for `omfile`, as the critical state (the file handle) is shared and must be protected. * **Serialization**: Before any file operation (write, flush, or close), a worker thread must acquire the `mutWrite` mutex. This ensures that only one thread can write to the file at a time, preventing data corruption. The diagram below illustrates the locking strategy for concurrent workers. ```mermaid graph TD subgraph "omfile Action Instance" direction LR pData["instanceData (pData)
fd, f_fname, mutWrite"] end Worker1["Worker Thread 1"] --> Lock1{Acquire mutWrite} Worker2["Worker Thread 2"] --> Lock2{Acquire mutWrite} Worker3["Worker Thread 3"] --> Lock3{Acquire mutWrite} Lock1 -->|Granted| Write1["doAction()
write()"] Write1 --> Unlock1{Release mutWrite} Lock2 -->|Blocks| Write2["doAction()
write()"] Lock3 -->|Blocks| Write3["doAction()
write()"] style pData fill:#f9f,stroke:#333,stroke-width:2px ``` This diagram shows multiple worker threads competing for the `mutWrite` mutex stored in the shared `pData`. Only one worker can acquire the lock and proceed to write to the file, while others block. Sources: [DEVELOPING.md](), [tools/omfile.c:109-131]() ### Data Structures The module's state is managed by two primary data structures defined via C macros. | Structure | Scope | Key Members | Description - | `instanceData` | Per-Action | `pthread_mutex_t mutWrite;`
`int fd;`
`uchar *f_fname;`
`int iUncompressedBytesInBuf;` | Holds state shared across all worker threads for a given file output action. The `mutWrite` mutex is the most critical component, serializing file access. | | `wrkrInstanceData` | Per-Worker | `instanceData *pData;` | Holds a pointer to the shared `instanceData`. It is primarily a lightweight container for thread-local access to the shared action state. | Sources: [tools/omfile.c:109-131, 134-138]() ## Execution Flow The `omfile` module follows the standard rsyslog output plugin interface. The core of its logic resides in the `doAction` function, which is called by a worker thread for each message to be written. ### Message Processing in `doAction` The following diagram shows the sequence of operations inside the `doAction` function. ```mermaid sequenceDiagram participant Worker as Worker Thread participant Core as Rsyslog Core participant OMFile as omfile Module Core->>Worker: Dequeue message Worker->>OMFile: doAction(pMsg) activate OMFile OMFile->>OMFile: Lock(pData->mutWrite) Note right of OMFile: Serialize file access OMFile->>OMFile: Check if file is open (pData->fd) alt File not open OMFile->>OMFile: openFile(pData) Note right of OMFile: Opens/creates file, sets pData->fd end OMFile->>OMFile: write(pData->fd, pMsg) Note right of OMFile: Writes message to file buffer alt Sync enabled OMFile->>OMFile: fsync(pData->fd) Note right of OMFile: Flush data to disk end OMFile->>OMFile: Unlock(pData->mutWrite) OMFile-->>Worker: Return RS_RET_OK deactivate OMFile ``` The key steps are: 1. **Locking**: The `mutWrite` mutex is acquired to ensure exclusive access. 2. **File Check**: The module checks if the file descriptor is already open. If not (e.g., on the first message or after an error), `openFile()` is called. 3. **Writing**: The formatted message string is written to the file using the `write()` system call. The module may buffer writes for efficiency. 4. **Syncing**: If the `sync` parameter is enabled, `fsync()` is called to force the operating system to write the data to the physical disk, ensuring durability at the cost of performance. 5. **Unlocking**: The mutex is released. Sources: [tools/omfile.c:452-498](), [doc/source/development/dev_oplugins.rst]() ### Transaction Handling `omfile` uses the simplest form of transaction handling. It does not implement the batching interface (`beginTransaction`, `endTransaction`). Instead, each call to `doAction` is treated as a complete, atomic transaction. A successful return (`RS_RET_OK`) implies the message has been committed. If an error occurs (e.g., a write failure), the module can return `RS_RET_SUSPENDED`, signaling the rsyslog core to retry the message later. Sources: [doc/source/development/dev_oplugins.rst](), [tools/omfile.c:496]() ## Configuration Parameters `omfile` is highly configurable through various parameters set in `rsyslog.conf`. | Parameter | Type | Default Value | Description - | `file` | String | (none) | Static file path to write logs to. Either `file` or `dynaFile` is required. - | `dynaFile` | String | (none) | A template that resolves to a dynamic file path for each message. - | `template` | String | "RSYSLOG_FileFormat" | Specifies the format of the message written to the file. - | `sync` | Boolean | `off` | If `on`, forces a sync (`fsync()`) after every write. - | `ioBufferSize` | String | "4k" | The size of the I/O buffer. Larger values can improve performance for high-volume logging. - | `dirCreateMode` | Octal | `0700` | The file mode (permissions) to use when creating directories for dynamic file paths. - | `fileCreateMode` | Octal | `0644` | The file mode (permissions) for newly created log files. - | `outchannel` | String | (none) | Defines an "output channel" which is a set of files. This can be used to write a single message to multiple files simultaneously. The test `omfile-outchannel.sh` demonstrates this feature. | Sources: [doc/source/configuration/modules/omfile.rst](), [tests/omfile-outchannel.sh]() ## Conclusion The `omfile` module is a cornerstone of rsyslog's functionality, providing a simple yet powerful way to persist logs to the filesystem. Its design as a serial, mutex-protected module ensures data integrity in concurrent environments and serves as a clear architectural pattern within the rsyslog codebase. With support for dynamic files, I/O buffering, and explicit data syncing, it offers the flexibility and performance required for a wide range of logging scenarios.