Expert Atlas Generator
Relevant source files
The following files were used as context for generating this wiki page:
The Expert Atlas Generator characterizes the functional specialization of the 19,456 experts within the GLM-5.2 744B Mixture-of-Experts (MoE) model. It utilizes a probe harness to execute topic-tagged prompts, captures expert-routing histograms, and calculates topic-affinity vectors for each expert. This system enables developers to visualize the "mind" of the model as a structured hierarchy rather than an opaque set of parameters.
Sources: c/tools/expert_atlas/README.md:1-5, site/index.html:150-155
System Architecture and Data Flow
The generator operates through a three-stage pipeline: data collection via a sweep script, statistical analysis for affinity mapping, and cross-prompt validation.
The diagram shows the flow from probe definitions through the inference engine to the final analytical outputs used by the web interface.
Sources: c/tools/expert_atlas/README.md:7-12, c/tools/expert_atlas/analyze.py:1-10
Measurement Control and Traps
Accurate atlas generation requires strict control over engine settings to avoid data corruption. The system specifically addresses four primary traps:
| Trap | Effect | Control Mechanism |
|---|---|---|
| Top-P Sampling | Hides ~38% of distinct experts by pruning low-probability candidates. | Set TOPP=0 during sweeps. |
| Speculative Drafts | Counts experts routed for rejected tokens that were never emitted. | Set MTP=0 and DRAFT=0. |
| Accumulated Usage | Naive dumps include all prior history from .coli_usage. | Remove .coli_usage before each run. |
| Autocorrelation | Single-prompt flukes look like perfect specialists due to context correlation. | Require replication across ≥2 independent prompts. |
Sources: c/tools/expert_atlas/README.md:17-30
Analytical Methodology
The analyze.py script processes raw routing counts to determine which topics an expert "owns."
Statistical Processing Steps
- Normalization: Calculates the selection frequency of expert e while running category c, normalized by the total category size to prevent verbose categories from skewing results.
- Base-Rate Correction: Renormalizes the distribution into a topic distribution per expert ($p(c|e)$) to identify true specialists rather than just globally popular experts.
- Specialization Metric: Calculates entropy-based specialization ($spec(e) = 1 − H(p(c|e)) / log C$). A value of 0 indicates a generalist, while 1 indicates an expert that fires for exactly one topic.
- Replication Gate: Filters out experts that do not fire in at least two independent prompts for a specific category.
Sources: c/tools/expert_atlas/README.md:32-45, c/tools/expert_atlas/analyze.py:55-85
Probing Suite
The probes.json file contains the dataset used to stimulate the model. It includes 10 categories with 3 varied prompts each to ensure results reflect the topic rather than specific wording.
- Categories:
code_python,code_sql,math_proof,chinese,german,poetry,law,medicine,json_format, andcasual_chat. - Design Requirement: Prompt lengths are kept in a narrow band because prefill routing counts toward the total, and long prompts would artificially inflate counts for their category.
Sources: c/tools/expert_atlas/probes.json:1-10
Validation Framework
The validate.py tool uses a "leave-one-prompt-out" protocol to ensure the atlas generalizes to new input.
This diagram illustrates the cross-validation logic used to prove that specialization is a property of the topic.
Sources: c/tools/expert_atlas/validate.py:1-15
Web Integration and Visualization
The generated experts.json is consumed by the web dashboard to provide live insights during model inference.
Atlas Galaxy and Brain皮层
- Atlas Galaxy: A 3-D visualization (implemented in
site/index.html) where expert positions represent measured routing affinity. Specialists cluster by topic (e.g., poetry, law, code). - Brain Tooltips: The
Brain.tsxcomponent uses the atlas data to provide hover information for every expert in the model's 75 layers. - Data Fields:
affinity: Record of topic percentages.entropy: Measurement of routing predictability.top: The primary topic the expert specializes in.label: Classified as "specialist" (if $spec \geq 0.5$) or "generalist".
Sources: web/src/Brain.tsx:75-95, site/index.html:230-260, c/tools/expert_atlas/analyze.py:125-135
Summary of Findings
Analysis of GLM-5.2 744B reveals that specialisation is a property of the topic, with a 96.7% accuracy in leave-one-prompt-out tests. Only 7.9% of experts are considered "strong specialists" ($spec \geq 0.5$), refuting the idea that one expert corresponds to exactly one topic for the majority of the model. Furthermore, specialisation increases with layer depth, moving from general syntactic processing in early layers (layer 3 $\approx$ 0.07) to high-level semantic reasoning in later layers (layers 18–58 $\approx$ 0.19–0.27).
Sources: c/tools/expert_atlas/README.md:49-60, c/tools/expert_atlas/analyze.py:100-115