Installation via Meltano

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• cron_ext/extension.py
• cron_ext/main.py
• cron_ext/stores/crontab.py
• cron_ext/stores/stdout.py
• tests/meltano.yml

Installation of the cron-ext utility allows a Meltano project to manage job scheduling through the system's cron daemon. The extension functions by transforming Meltano's internal schedules into executable shell scripts and corresponding cron entries, which are then injected into the user's crontab or directed to standard output.

This process involves adding the utility to the meltano.yml configuration, installing the package via pip, and then invoking the install command to synchronize the Meltano project's schedules with the host's cron system. Sources: README.md:1-26, cron_ext/extension.py:126-149

Plugin Configuration

To enable the extension, it must be defined as a utility plugin within the Meltano project environment. This requires specifying the pip_url (pointing to the repository) and defining the executable command. Sources: README.md:28-40, tests/meltano.yml:11-18

Meltano.yml Definition

The plugin is typically added under the utilities section of the plugins block:

plugins:
  utilities:
  - name: cron
    namespace: cron
    pip_url: git+https://github.com/meltano/cron-ext.git
    executable: cron
    commands:
      describe:
        args: describe
        executable: cron

Sources: README.md:30-40

Installation Commands

After updating the configuration file, the utility is installed into the project's virtual environment using the standard Meltano CLI.

Step	Command	Description
1	meltano add utility cron	Registers and downloads the extension.
2	meltano invoke cron install	Installs project schedules into the crontab.

Sources: README.md:42-62

Installation Architecture and Logic

When the install command is invoked, the extension performs a sequence of operations to bridge Meltano schedules with the system crontab.

Data Flow for Schedule Installation

The following diagram illustrates how Meltano schedule definitions are converted into active cron jobs.

The extension retrieves schedule data via a subprocess call to meltano schedule list --format=json. Sources: cron_ext/extension.py:70-78, cron_ext/stores/crontab.py:34-60

Component Responsibilities

Component	Responsibility	Relevant Files
Cron Class	Orchestrates the installation, generates script files, and handles logic for selective installation.	cron_ext/extension.py
CrontabEntryStore	Manages reading/writing to the system crontab -l and crontab - buffers.	cron_ext/stores/crontab.py
StdoutEntryStore	Simple store that redirects generated cron lines to standard output.	cron_ext/stores/stdout.py
EntryPattern	Regex used to identify and parse existing cron-ext entries.	cron_ext/entry.py

Sources: cron_ext/extension.py:33-51, cron_ext/stores/crontab.py:16-24, cron_ext/stores/stdout.py:12-18

Detailed Installation Process

1. Script Generation

For every schedule being installed, the extension creates a wrapper shell script located at .meltano/run/cron-ext/<schedule_name>.sh. These scripts are responsible for:

• Navigating to the MELTANO_PROJECT_ROOT.
• Setting required environment variables (including PATH and VIRTUAL_ENV).
• Executing the appropriate meltano run or meltano elt command.
• Piping output to /usr/bin/logger -t meltano.

Sources: README.md:20-24, cron_ext/extension.py:61-68, cron_ext/extension.py:90-111

2. Crontab Manipulation

The CrontabEntryStore manages a specific section in the user's crontab identified by the MELTANO_PROJECT_ID. This prevents the extension from interfering with non-managed cron entries.

3. Installation Targets

Users can choose between two installation targets via the --target flag:

• crontab (Default): Persists entries directly to the system scheduler. Sources: cron_ext/main.py:112-116
• stdout: Outputs the cron-formatted strings to the console, allowing for manual verification or redirection to other scheduling tools. Sources: cron_ext/stores/stdout.py:32-33

Selective and Error Handling

The installation command supports installing a subset of schedules by providing specific IDs:
meltano invoke cron install schedule_a schedule_b.

If a provided ID does not exist within the Meltano project, the extension logs an error and exits with code 1 to prevent partial or incorrect installations. Sources: cron_ext/extension.py:136-149, cron_ext/main.py:112-116

Installation via Meltano provides a robust way to automate data pipelines using native Linux scheduling while maintaining the environment context (virtual environments and project-specific variables) required for Meltano to operate correctly. Sources: README.md:1-26