Data Seeding & Maintenance

Relevant source files

The following files were used as context for generating this wiki page:

• db/seeds.rb
• lib/task_helpers/state_fips_task_helper.rb
• lib/task_helpers/topojson_task_helper.rb
• lib/task_helpers/state_shapefile_request.rb
• lib/task_helpers/task_helper.rb
• db/schema.rb

The Data Seeding & Maintenance system in Actionmap is responsible for initializing the application's database with core political and geographical information. This includes populating tables for states, counties, representatives, and events, as well as managing complex geographical data (FIPS codes and TopoJSON) required for the application's mapping features.

Sources: db/seeds.rb, lib/task_helpers/topojson_task_helper.rb

Database Seeding Logic

The primary entry point for seeding the database is db/seeds.rb. The process follows a strict order to maintain referential integrity: it destroys existing records for states, counties, and representatives before recreating them from defined data sources.

Seeding Workflow

The seeding process iterates through SeedData to populate different models:

1. States and Counties: States are created first. For each state, the system reads a corresponding JSON file located in lib/assets/counties_fips_data/ to populate the counties association.
2. Representatives and News Items: Representatives are created, and their associated news items (title, description, link) are nested within the creation loop.
3. Events: Events are linked to both a state and a specific county using fips_code.

Geographical Data Maintenance

Actionmap uses specialized task helpers to maintain FIPS (Federal Information Processing Standards) data and geographical shapefiles. This ensures the application reflects accurate administrative boundaries.

State FIPS Helper

The StateFipsTaskHelper manages the retrieval and correction of county-level FIPS data from external sources like census.gov. It includes specific logic to handle historical or administrative changes in counties.

Method	Purpose
fips_csv_to_hash	Converts raw CSV data from the Census into a structured Ruby hash.
fetch_fips_data_for_state	Downloads state-specific FIPS text files from census.gov.
alaska_update	Corrects data for Kusilvak Census Area (formerly Wade Hampton).
virginia_update	Removes Bedford City, which is no longer an independent city.

Sources: lib/task_helpers/state_fips_task_helper.rb:6-76

TopoJSON and Shapefile Processing

The system processes raw shapefiles into lightweight TopoJSON for web rendering. This is managed by TopojsonTaskHelper and StateShapefileRequest.

Data Schema Summary

The database schema defines the structure for seeded data, ensuring constraints like null: false are maintained for critical identifiers like fips_code and state_id.

Table	Key Fields	Constraints
states	name, symbol, fips_code	fips_code limit 1, all non-null
counties	name, state_id, fips_code	fips_code limit 2, indexed on state_id
representatives	name, ocdid, title, party
events	name, county_id, start_time	Linked to counties table

Sources: db/schema.rb:14-106

Utility Helpers

The TaskHelper module provides low-level file system operations used during data maintenance tasks, such as reinitializing directories and extracting ZIP archives.

• reinit_dir(dir_name): Forcefully removes and recreates a directory to ensure a clean state for new data downloads.
• extract_zip(zip_file, dest_folder): Extracts shapefile components for processing.
• write_json(filename, content): Saves processed FIPS or geographical data in a pretty-printed JSON format with UTF-8 encoding.

Sources: lib/task_helpers/task_helper.rb:7-35

Conclusion

The Data Seeding & Maintenance system provides a robust pipeline for initializing political data and processing complex geographical assets. By combining static seeds with dynamic shapefile conversion and FIPS data correction, the system ensures that the Actionmap visualizations and representative data remain accurate and performant.