TopoJSON & FIPS Data Management
Relevant source files
The following files were used as context for generating this wiki page:
The Actionmap project utilizes a sophisticated pipeline for managing geographical data, combining Federal Information Processing Series (FIPS) codes with TopoJSON files to render interactive maps. This system allows the application to represent political boundaries at both the national (state boundaries) and state (county boundaries) levels, facilitating the search for representatives and local news.
The management system consists of backend Ruby task helpers that automate the retrieval and processing of United States Census Bureau shapefiles, and a frontend JavaScript architecture powered by D3.js and TopoJSON tools to project and render these maps dynamically.
Geographical Data Pipeline
The project automates the generation of map assets through specialized task helpers. This process involves fetching raw geospatial data from the U.S. Census Bureau and transforming it into lightweight TopoJSON formats optimized for web delivery.
Data Acquisition and Transformation
The transformation process is managed by TopojsonTaskHelper, which acts as a wrapper for the mapshaper Node.js utility. The pipeline follows a strictly defined sequence: fetching, unzipping, simplifying, and dissolving polygons.
The diagram above illustrates the automated flow of converting government shapefiles into application-ready TopoJSON assets.
Sources: lib/task_helpers/topojson_task_helper.rb:38-55, lib/task_helpers/national_shapefile_request.rb:24-29, lib/task_helpers/state_shapefile_request.rb:30-34
Processing Specifications
The system applies specific transformations depending on the scope of the request:
| Feature | National View | State/County View |
|---|---|---|
| Source URL | cb_2018_us_state_500k.zip | cb_2019_FIPS_cousub_500k.zip |
| Simplification | 10% retention | 10% retention (keeps shapes) |
| Key Operations | Keep-shapes | Dissolve COUNTYFP |
| Output Path | app/assets/topojson/national/states | app/assets/topojson/states/ |
Sources: lib/task_helpers/national_shapefile_request.rb:16-29, lib/task_helpers/state_shapefile_request.rb:18-34
FIPS Code Management
FIPS codes serve as the primary unique identifiers linking geographical polygons to database records. The StateFipsTaskHelper manages the ingestion and correction of these codes.
Database Schema
Geographical entities are stored with their corresponding FIPS identifiers to allow for efficient queries and frontend lookups.
The relationship between states and counties is bridged by FIPS codes, which are stored as integers with specific bit-limits in the schema.
Sources: db/schema.rb:13-21, db/schema.rb:78-90
FIPS Data Cleaning
The project includes logic to handle historical changes in FIPS assignments, specifically updating records for Alaska, South Dakota, and Virginia to maintain data integrity.
- Alaska: Replaces Wade Hampton Census Area (FIPS 270) with Kusilvak Census Area (FIPS 158).
- South Dakota: Replaces Shannon County (FIPS 113) with Oglala Lakota County (FIPS 102).
- Virginia: Deletes Bedford City (FIPS 515) as it is no longer an independent city.
Sources: lib/task_helpers/state_fips_task_helper.rb:45-85
Map Rendering Architecture
The frontend uses D3.js to render TopoJSON assets. Because TopoJSON is a topology-preserving format, it must be converted back to GeoJSON features within the client's browser for rendering.
Projection and Utilities
The state_map_utils.js file provides the mathematical logic for map projections. It utilizes geoAlbersUsa for the national view and geoMercator for state-level views, with specific overrides for non-contiguous regions like Alaska.
This sequence demonstrates how geographical assets are transformed from compressed storage formats to interactive SVG elements.
Sources: app/javascript/packs/state_map_utils.js:38-52, app/javascript/packs/national_states_map.js:23-35
Interactive Features
The rendering logic attaches data attributes directly to SVG paths to enable interactivity:
data-state-fips-code: Used for identifying states in the national view.data-county-fips-code: Used for identifying counties within a state.data-state-symbol: Facilitates navigation to state-specific routes (e.g.,/state/CA).
Sources: app/javascript/packs/national_states_map.js:43-45, app/javascript/packs/state_map.js:16-17
Conclusion
Actionmap's geographical system provides a robust framework for handling complex spatial data. By automating the conversion of raw Census Bureau data and utilizing standardized FIPS codes, the application ensures a consistent link between visual map elements and the underlying political database. This architecture supports high-performance rendering by utilizing TopoJSON's efficiency while maintaining the flexibility of D3.js for user interaction.