Ad Blocking Rules

Relevant source files

The following files were used as context for generating this wiki page:

• Clash/Providers/BanAD.yaml
• Clash/Providers/BanEasyListChina.yaml
• Clash/Providers/BanProgramAD.yaml
• Clash/Providers/UnBan.yaml
• update_providers.py
• update_sing_box.py
• Tool/New.md

The Self-Rule repository provides a set of ad-blocking rules designed for use with Clash and other proxy tools. These rules are based on the ACL4SSR project and are structured to block a wide range of advertisements, from general ad networks to specific in-app ads. The system uses a layered approach, combining multiple rule lists to achieve comprehensive blocking while also providing an "unban" list to prevent breakage of essential services.

The rules are maintained in .list files and are automatically converted into .yaml provider files for Clash and .json files for sing-box using Python scripts. This automated process ensures consistency and simplifies maintenance. The history of rule updates shows a continuous effort to refine the lists by adding new targets, removing obsolete entries, and optimizing performance.

Rule Architecture and Processing

The ad-blocking rules are managed through a semi-automated pipeline that converts source .list files into consumable formats for different proxy applications.

Rule Generation Flow

The primary flow involves Python scripts that read .list files and generate .yaml files for Clash providers and .json/.srs files for sing-box rule-sets.

This diagram illustrates the conversion process for Clash and sing-box rule files.

Clash Provider Generation (update_providers.py)

The update_providers.py script walks through the Clash/ directory, finds all .list files, and converts them into .yaml provider files located in Clash/Providers/.

The script performs the following logic:

1. Reads each line from the source .list file.
2. Creates a corresponding .yaml file in the Clash/Providers/ directory.
3. Writes a payload: header to the YAML file.
4. Processes each line:
   • Lines starting with # are treated as comments.
   • Lines for USER-AGENT, URL-REGEX, and PROCESS-NAME are commented out in the final YAML, as they are not standard Clash payload rules.
   • All other lines are prefixed with - to format them as YAML list items.

Sources: update_providers.py:11-33

sing-box Rule-Set Generation (update_sing_box.py)

A similar script, update_sing_box.py, handles the conversion for the sing-box format. It parses .list files and creates structured .json files.

1. It defines a HeadlessRule data class to hold different rule types (domain, domain_suffix, ip_cidr, etc.).
2. It iterates through each line of a .list file, parsing the rule type (e.g., DOMAIN-SUFFIX, IP-CIDR) and value.
3. The parsed rules are added to a HeadlessRule object.
4. This object is then serialized into a .json file within the sing-box/ directory.
5. Finally, it uses the sing-box rule-set compile command to create a binary .srs file and deletes the intermediate .json file.

Sources: update_sing_box.py:48-97

Rule Categories and Files

The ad-blocking functionality is split across several provider files, each targeting different types of ad content.

File Name	Purpose	Rule Count (Approx.)
BanAD.yaml	Blocks common ad keywords and domains of major domestic and international ad networks.	450+
BanProgramAD.yaml	Targets ads within specific applications and services like Baidu, QQ, iqiyi, and Youku.	700+
BanEasyListChina.yaml	A comprehensive list derived from the EasyListChina project, containing thousands of ad-related domains.	4200+
UnBan.yaml	A whitelist to exempt specific domains from being blocked, preventing interference with legitimate services.	20+

Sources: Clash/Providers/BanAD.yaml, Clash/Providers/BanProgramAD.yaml, Clash/Providers/BanEasyListChina.yaml, Clash/Providers/UnBan.yaml

General Ad Blocking (BanAD.yaml)

This file forms the base layer of ad blocking. It contains rules targeting common ad-related keywords and the domains of well-known ad networks.

Key Components:

• Ad Keywords: A list of DOMAIN-KEYWORD rules that block any domain containing terms like admaster, guanggao, adsage, etc.
• Ad Networks: An extensive list of DOMAIN-SUFFIX rules for both domestic (e.g., allyes.com, tanx.com) and international (e.g., adcolony.com, criteo.com) ad-serving platforms.
• Junk Sites & Operator Ads: Includes rules to block known malicious sites and ads injected by mobile carriers.

Example rules from BanAD.yaml:

payload:
  # 本碎片只包含常见广告关键字、广告联盟。无副作用，放心使用

  # 广告关键词
  - DOMAIN-KEYWORD,admarvel
  - DOMAIN-KEYWORD,admaster
  - DOMAIN-KEYWORD,adsage
  
  # 广告联盟-国内****************
  - DOMAIN-SUFFIX,09mk.cn
  - DOMAIN-SUFFIX,100peng.com
  - DOMAIN-SUFFIX,allyes.com
  - DOMAIN-SUFFIX,tanx.com

  # 广告联盟-国外****************
  - DOMAIN-SUFFIX,adcolony.com
  - DOMAIN-SUFFIX,criteo.com

Sources: Clash/Providers/BanAD.yaml:4-500

Application-Specific Ads (BanProgramAD.yaml)

This provider focuses on blocking ads embedded within popular applications and software. It is more targeted and may have minor side effects, though it aims to avoid breaking core functionality.

Covered Applications:

• Chinese Tech Giants: Baidu, Alibaba, Tencent (QQ), 163, Sina.
• Video Apps: iqiyi, Youku, LeTV, PPTV, Sohu.
• Mobile ROMs: Xiaomi (MIUI) and Meizu (Flyme).
• Software: WPS Office, Xunlei, Adobe.
• News and Social Media: Toutiao, Zhihu, Twitter.

This file uses a mix of DOMAIN-SUFFIX, DOMAIN-KEYWORD, DOMAIN, and IP-CIDR rules to block ad servers, tracking domains, and ad-related IP addresses. For example, it blocks iqiyi and Youku ad IPs directly.

payload:
  # iqiyi PPS 爱奇艺
  - DOMAIN-SUFFIX,ad.m.iqiyi.com
  - DOMAIN-SUFFIX,afp.iqiyi.com
  - IP-CIDR,101.227.200.0/24,no-resolve
  
  # Youku & Tudou
  - DOMAIN-SUFFIX,ad.api.3g.youku.com
  - DOMAIN-SUFFIX,stat.youku.com
  - IP-CIDR,117.177.248.17/32,no-resolve

Sources: Clash/Providers/BanProgramAD.yaml:900-910, 1005-1015

EasyList China (BanEasyListChina.yaml)

This is the largest rule set, sourced directly from the EasyListChina project. It provides broad-spectrum blocking of ad-related domains relevant to the Chinese internet landscape. The file contains over 4,200 DOMAIN-SUFFIX and IP-CIDR rules.

Sources: Clash/Providers/BanEasyListChina.yaml

Whitelist (UnBan.yaml)

To counteract the aggressive blocking of the other rule sets, UnBan.yaml serves as a whitelist. It contains domains that should not be blocked, even if they match a rule in one of the "Ban" lists. This is crucial for ensuring that essential services, like Google's dl.google.com or Epic Games' ol.epicgames.com, remain functional.

Example rules from UnBan.yaml:

payload:
  # Unbreak 白名单 不应该被广告链接的网站

  # Epicgames
  - DOMAIN-SUFFIX,ol.epicgames.com

  # Google
  - DOMAIN,dl.google.com
  - DOMAIN-SUFFIX,googletraveladservices.com
  
  # Miui 小米
  - DOMAIN,tracking.miui.com

Sources: Clash/Providers/UnBan.yaml:4-22

Historical Development

The Tool/New.md file provides a changelog for the rule sets, dating back to 2017. It highlights a consistent effort to improve the ad-blocking rules with the following goals:

• Performance Optimization: Merging IP ranges, simplifying regular expressions, and removing redundant rules to speed up matching.
• Coverage Expansion: Adding rules for new apps (e.g., TikTok/Douyin, book apps), ad networks, and tracking services.
• Maintenance: Regularly de-duplicating, sorting, and splitting rules into more manageable files.
• Fixing Issues: Addressing problems where ad-blocking rules interfered with normal app functionality, such as Google Play updates or map services.

Sources: Tool/New.md:5-100