cloud-server/migrations

Database Migrations

This directory contains SQL migration files for database schema changes.

Automated Migration System

The project uses an automated migration tracking system that:

• ✅ Automatically detects and runs unapplied migrations
• ✅ Tracks migration history in the database
• ✅ Executes migrations in numerical order
• ✅ Safe to run multiple times (idempotent)
• ✅ Records execution time and errors

Quick Start

# Check migration status
npm run db:migrate:status

# Run all pending migrations (local)
npm run db:migrate

# Run all pending migrations (remote)
npm run db:migrate:remote

Migration Files

All migration files are named with a numeric prefix for ordering:

• 000_migration_history.sql - Creates migration tracking table (always runs first)
• 002_add_composite_indexes.sql - Query performance optimization
• 003_add_retail_pricing.sql - Add retail pricing columns
• 004_anvil_tables.sql - Anvil-branded product tables

Migration Details

000_migration_history.sql

Date: 2026-01-25 Purpose: Create migration tracking system

• Creates migration_history table to track applied migrations
• Records execution time, success/failure status, and error messages

002_add_composite_indexes.sql

Date: 2026-01-21 Purpose: Add composite indexes to optimize query performance

Indexes Added:

1. idx_instance_types_provider_family_specs - Optimizes instance filtering by provider, family, and specs
2. idx_pricing_instance_region_price - Optimizes pricing queries with JOIN operations and sorting
3. idx_regions_provider_code - Optimizes region lookups by provider and region code

Performance Impact:

• Reduces query execution time for filtered instance searches
• Improves JOIN performance between instance_types, pricing, and regions tables
• Enables efficient ORDER BY on hourly_price without additional sort operations

003_add_retail_pricing.sql

Date: 2026-01-23 Purpose: Add retail pricing fields to all pricing tables

• Adds hourly_price_retail and monthly_price_retail columns
• Backfills existing data with 1.21x markup

004_anvil_tables.sql

Date: 2026-01-25 Purpose: Create Anvil-branded product tables

• anvil_regions - Anvil regional datacenters
• anvil_instances - Anvil instance specifications
• anvil_pricing - Anvil retail pricing with cost tracking
• anvil_transfer_pricing - Data transfer pricing

Running Migrations

Automated (Recommended)

# Check current status
npm run db:migrate:status              # Local database
npm run db:migrate:status:remote       # Remote database

# Run all pending migrations
npm run db:migrate                     # Local database
npm run db:migrate:remote              # Remote database

Manual (Backward Compatibility)

Individual migration scripts are still available:

npm run db:migrate:002                 # Local
npm run db:migrate:002:remote          # Remote
npm run db:migrate:003                 # Local
npm run db:migrate:003:remote          # Remote
npm run db:migrate:004                 # Local
npm run db:migrate:004:remote          # Remote

Migration Best Practices

1. Idempotent Operations: All migrations use IF NOT EXISTS to ensure safe re-execution
2. Backwards Compatible: New indexes don't break existing queries
3. Performance Testing: Test migration impact on query performance before deploying
4. Rollback Plan: Each migration includes rollback instructions in comments

Query Optimization Details

Main Query Pattern (query.ts)

SELECT ...
FROM instance_types it
JOIN providers p ON it.provider_id = p.id
JOIN pricing pr ON pr.instance_type_id = it.id
JOIN regions r ON pr.region_id = r.id
WHERE p.name = ?
  AND r.region_code = ?
  AND it.instance_family = ?
  AND it.vcpu >= ?
  AND it.memory_mb >= ?
ORDER BY pr.hourly_price

Optimized By:

• idx_instance_types_provider_family_specs - Covers WHERE conditions on instance_types
• idx_pricing_instance_region_price - Covers JOIN and ORDER BY on pricing
• idx_regions_provider_code - Covers JOIN conditions on regions

Expected Performance Improvement

• Before: Full table scans on instance_types and pricing tables
• After: Index seeks with reduced disk I/O
• Estimated Speedup: 3-10x for filtered queries with sorting

Verifying Index Usage

You can verify that indexes are being used with SQLite EXPLAIN QUERY PLAN:

# Check query execution plan
npm run db:query "EXPLAIN QUERY PLAN SELECT ... FROM instance_types it JOIN ..."

Look for "USING INDEX" in the output to confirm index usage.

Notes

• SQLite automatically chooses the most efficient index for each query
• Composite indexes follow the "leftmost prefix" rule
• Indexes add minimal storage overhead but significantly improve read performance
• Write operations (INSERT/UPDATE) are slightly slower with more indexes, but read performance gains outweigh this for read-heavy workloads