The Update Process (`update_providers.py`)

Relevant source files

The following files were used as context for generating this wiki page:

• update_providers.py
• update_sing_box.py
• Clash/Providers/ProxyGFWlist.yaml
• Clash/Providers/Ruleset/Wikipedia.yaml
• Clash/GeneralClashConfig.yml
• README.md

The update_providers.py script is a core automation tool within the Self-Rule repository. Its primary function is to generate Clash provider .yaml files from a set of master .list files. This process ensures that the rule configurations for the Clash proxy client are kept consistent and are easily maintainable from a single source of truth. The repository is based on the ACL4SSR project and is intended for self-use rule management.

This script works in tandem with other tools in the repository, such as update_sing_box.py, which performs a similar function for the sing-box client. Together, these scripts form a system for managing and distributing proxy rules for multiple platforms from a common .list file format.

Sources: update_providers.py, README.md, update_sing_box.py

Core Architecture

The update process is driven by two main Python scripts: update_providers.py for Clash and update_sing_box.py for sing-box. The central architectural concept is that .list files stored throughout the Clash/ directory (but not in Clash/Providers/) act as the definitive source for all rule sets.

Directory Structure and Roles

Directory/File Type	Role
Clash/**/*.list	Source of truth for rule definitions. These are plain text files containing rules.
Clash/Providers/**/*.yaml	Generated Clash provider files. These are created and updated by update_providers.py.
sing-box/**/*.json	Intermediate JSON files generated by update_sing_box.py from .list files.
sing-box/**/*.srs	Compiled, binary rule-set files for sing-box, generated from the intermediate .json files.

Sources: update_providers.py:10-16, update_sing_box.py:70-73

update_providers.py Workflow

The script executes two main tasks: generation of provider files and cleanup of stale files.

1. Provider File Generation

The script first traverses the Clash/ directory, looking for any file ending with the .list extension. It explicitly ignores the Clash/config and Clash/Providers subdirectories to avoid processing generated files or main configuration files.

For each .list file found, it performs a conversion to a Clash-compatible .yaml provider format.

This diagram illustrates the generation flow.

Line Conversion Rules

The script applies specific formatting rules when converting lines from a .list file to a .yaml file.

Original Line Type	Condition	YAML Output
Standard Rule	Any line not matching other conditions.	- {line_content}
Comment	Line starts with #.	# {comment_content}
Unsupported Rule	Line starts with USER-AGENT, URL-REGEX, or PROCESS-NAME.	# {line_content}
Empty Line	Line is a newline character \n.	Ignored (passes).

The following snippet from update_providers.py shows this logic:

# File: update_providers.py:29-42
if line == "\n":
    pass
elif line.lstrip().startswith("#"):
    if not line.lstrip().startswith("# "):
        f_out.write(f"  # {line.strip()[1:]}")
    else:
        f_out.write(f"  {line.strip()}")
elif line.lstrip().startswith("USER-AGENT"):
    f_out.write(f"  # {line.strip()}")
elif line.lstrip().startswith("URL-REGEX"):
    f_out.write(f"  # {line.strip()}")
elif line.lstrip().startswith("PROCESS-NAME"):
    f_out.write(f"  # {line.strip()}")
else:
    f_out.write(f"  - {line.strip()}")

Sources: update_providers.py:29-42

An example of a generated file is Clash/Providers/Ruleset/Wikipedia.yaml, which follows this format.

# File: Clash/Providers/Ruleset/Wikipedia.yaml
payload:
  # 内容：Wikipedia 维基相关域名
  # 数量：12条
  - DOMAIN-SUFFIX,mediawiki.org
  - DOMAIN-SUFFIX,wikibooks.org
  - DOMAIN-SUFFIX,wikidata.org
  - DOMAIN-SUFFIX,wikileaks.org
  - DOMAIN-SUFFIX,wikimedia.org
  - DOMAIN-SUFFIX,wikinews.org
  - DOMAIN-SUFFIX,wikipedia.org
  - DOMAIN-SUFFIX,wikiquote.org
  - DOMAIN-SUFFIX,wikisource.org
  - DOMAIN-SUFFIX,wikiversity.org
  - DOMAIN-SUFFIX,wikivoyage.org
  - DOMAIN-SUFFIX,wiktionary.org

Sources: Clash/Providers/Ruleset/Wikipedia.yaml

2. Stale Provider Cleanup

After the generation step, the script performs a cleanup to ensure that no orphan .yaml provider files exist. It iterates through all .yaml files within the Clash/ directory. For each one, it determines the path of the original .list file it would have been generated from. If this source .list file does not exist, the script deletes the .yaml file.

This diagram shows the cleanup logic.

The logic for deriving the source path is as follows: Clash/Providers/Ruleset/Example.yaml -> Clash/Ruleset/Example.list.

# File: update_providers.py:50-52
# eg: `Clash/Providers/Ruleset/58.yaml` -> `Clash/Ruleset/58.list`
read_path = os.path.join(*path_l[:1], *path_l[2:])
read_path = os.path.splitext(read_path)[0] + ".list"

Sources: update_providers.py:50-52

Related Script: update_sing_box.py

A parallel script, update_sing_box.py, exists to convert the same source .list files into a format compatible with the sing-box proxy client. This reinforces the design pattern of using .list files as a universal source format.

update_sing_box.py Workflow

1. Find .list files: Similar to update_providers.py, it walks the Clash/ directory.
2. Parse and Convert: It parses each line of a .list file and maps the rule type (e.g., DOMAIN-SUFFIX, IP-CIDR) to a corresponding field in a HeadlessRule data structure.
3. Generate JSON: The populated HeadlessRule objects are wrapped in a Rule object and serialized into a .json file in the sing-box/ directory.
4. Compile to .srs: The script then calls the sing-box rule-set compile command-line tool to convert the .json file into a binary .srs file.
5. Cleanup: Finally, the intermediate .json file is deleted.

This diagram shows the process for updating sing-box rules.

Data Structures

The script uses dataclasses to represent the sing-box rule structure before serializing to JSON.

This diagram shows the relationship between the Rule and HeadlessRule classes.

Conclusion

The update_providers.py script is a key component of the repository's rule management strategy. It automates the conversion of human-readable .list files into machine-optimized .yaml provider files for Clash. This, combined with its cleanup functionality and the parallel update_sing_box.py script, creates a robust and centralized system for maintaining proxy rules across different client applications, ensuring consistency and simplifying updates.

Sources: update_providers.py, update_sing_box.py