Performance Case Studies

Relevant source files

The following files were used as context for generating this wiki page:

• posa/00_introduction.html.md
• posa/03_ninja.html.md
• posa/08_talos.html.md
• posa/07_infinispan.html.md
• posa/02_from-socialcalc-to-ethercalc.html.md

Performance Case Studies within this project explore the methodologies used by open-source experts to find and fix performance bottlenecks in real-world applications. These studies move beyond the theoretical "premature optimization" warning to address practical needs for speed in mobile devices, big data analysis, and distributed systems. They cover a range of performance goals, from reducing response times and data structure sizes to optimizing large-scale build frontends and automation frameworks.

Sources: posa/00_introduction.html.md

Build System Optimization: Ninja

Ninja was developed to minimize the "front-of-build" latency—the time between starting a build and the execution of the first compilation. Its design philosophy centers on "doing less work" by shifting complex logic (like conditionals or file extension rules) to a separate generator program.

Architecture and Data Flow

Ninja functions like an assembler for builds. It loads a .ninja file containing a graph of dependencies where nodes (input files) point to edges (build commands) which point to output nodes.

Key Performance Components

Component	Optimization Technique	Result
Parsing	Hand-written lexer replaced with re2c lookup tables.	Reduced parsing time of large builds (e.g., Chrome).
Node Mapping	Path pointers and unique Node objects for pointer equality.	Avoided costly string comparisons for dependency checks.
Build Log	Command hashing instead of full command recording.	Reduced log size from 200MB to 2MB and improved load speed 20x.
Depfiles	Eager background processing of Makefile dependencies.	Shifted parsing work away from the critical startup path.

Sources: posa/03_ninja.html.md:104-300

Performance Testing Frameworks: Talos and Datazilla

Talos is Mozilla's performance testing harness designed to measure attributes like startup time and page load speeds. The system underwent a transition to reduce "noise"—statistical variance in results that obscured actual performance regressions.

Regression Detection Flow

To identify regressions, the system transitioned from simple averages to raw data collection and complex statistical filtering via a new backend called Datazilla.

Noise Reduction Requirements

1. Raw Data Storage: Database must store all collected data points, not just averages.
2. Sample Size: Each test must collect at least 20 useful data points.
3. Warm-up Filtering: The first 5 data points are recorded but ignored to bypass initial caching/warm-up noise (covering 95% of noisy results).
4. Independent Calculation: No more averaging values across different pages; each page is treated independently to prevent masking regressions.

Sources: posa/08_talos.html.md:28-150

Distributed Data Grids: Infinispan

Infinispan is a distributed in-memory key-value store optimized for low-latency data access. Its architecture is peer-to-peer to avoid single points of failure and performance bottlenecks.

Core Performance Subsystems

Subsystem	performance strategy
Networking	Use of JGroups with UDP multicast and message bundling.
Serialization	Magic numbers and "Externalizers" to avoid standard Java serialization overhead.
Concurrency	Software Transactional Memory (STM) and lock-free data structures.
Persistence	Append-only structures for durable stores and B-tree structures for paging.

Resource Contention and Multi-core Scaling

Infinispan utilizes non-blocking algorithms to maximize parallelism on multi-core systems. It implements lock amortization and adaptive eviction policies to manage memory without excessive synchronization overhead.

Sources: posa/07_infinispan.html.md:40-230

Web Spreadsheet Scaling: EtherCalc

EtherCalc evolved from SocialCalc, transitioning from a Perl prototype to a Node.js-based architecture to support simultaneous collaborative editing and reduce client-side CPU usage.

Multi-threaded Web Worker Model

To handle high-concurrency REST API calls on multi-core servers, EtherCalc uses background threads to distribute the workload of command execution.

Evolution of Architecture

• Prototype: A simple WebSocket chat-room style server relaying commands.
• Node.js Integration: Used jsdom to run the in-browser spreadsheet engine on the server to maintain state snapshots.
• Worker Scaling: Replaced a single-threaded bottleneck with webworker-threads to allow one background thread per spreadsheet.

Sources: posa/02_from-socialcalc-to-ethercalc.html.md:20-250

Summary of Optimization Principles

The case studies demonstrate several recurring performance principles:

1. Do Less Work: Shifting work out of critical paths (Ninja).
2. Shift to Client: Offloading computation and rendering to browsers where possible (EtherCalc).
3. Statistical Integrity: Collecting raw data instead of losing information through premature averaging (Talos).
4. Lock-Free Design: Utilizing non-blocking structures to scale across multi-core hardware (Infinispan).

Sources: posa/00_introduction.html.md, posa/03_ninja.html.md, posa/08_talos.html.md, posa/07_infinispan.html.md, posa/02_from-socialcalc-to-ethercalc.html.md