cloud-server/CACHE_KV_IMPLEMENTATION.md

KV-Based Cache Index Implementation

Overview

Implemented KV-based cache index for CacheService to enable pattern-based cache invalidation and enumeration. This resolves the TODO at line 274 in src/services/cache.ts.

Changes Summary

1. Core Implementation (src/services/cache.ts)

Constructor Update:

• Added optional kvNamespace: KVNamespace | null parameter
• Defaults to null for backward compatibility
• Logs KV index availability on initialization

New Features:

• invalidatePattern(pattern: string): Promise<number> - Pattern-based cache invalidation with wildcard support
• clearAll(prefix?: string): Promise<number> - Actually clears and counts entries when KV is available
• getStats(): Promise<{ supported: boolean; indexed_keys?: number }> - Returns actual key count when KV is available

Private Helper Methods:

• _registerCacheKey(key: string, ttlSeconds: number): Promise<void> - Registers cache keys in KV index
• _unregisterCacheKey(key: string): Promise<void> - Unregisters cache keys from KV index
• _listCacheKeys(prefix?: string): Promise<string[]> - Lists all cache keys from KV index

Key Design Decisions:

• KV Index Prefix: cache_index: to separate from rate limiting data
• Auto-Expiration: KV entries use same TTL as cache entries (automatic cleanup)
• Graceful Degradation: All KV operations are non-blocking, failures are logged but don't break cache operations
• Backward Compatibility: Works with or without KV namespace, existing code unchanged

2. Route Updates

src/routes/instances.ts:

• Updated getCacheService() to accept env: Env parameter
• Passes env.RATE_LIMIT_KV to CacheService constructor
• Enables KV index for all instance cache operations

src/routes/recommend.ts:

• Updated getCacheService() to accept env: Env parameter
• Passes env.RATE_LIMIT_KV to CacheService constructor
• Enables KV index for all recommendation cache operations

3. Test Coverage (src/services/cache-kv.test.ts)

24 comprehensive tests:

• Cache key registration/unregistration
• Pattern invalidation with wildcards
• clearAll() with and without KV
• Error handling and graceful degradation
• Backward compatibility (no KV)
• Large-scale operations (150 keys)

Test Results:

✓ 24 tests passed (24)
Duration: 26ms

API Changes

Pattern Invalidation

Before:

async invalidatePattern(pattern: string): Promise<void> {
  logger.warn(`Pattern invalidation not supported: ${pattern}`);
  // TODO: Implement with KV-based cache index if needed
}

After:

async invalidatePattern(pattern: string): Promise<number> {
  // Returns count of invalidated entries
  // Supports wildcards: *, case-insensitive
  // Examples:
  //   '*instances*' - All instance caches
  //   '*provider=linode*' - All linode caches
  //   '*pricing*' - All pricing caches
}

clearAll() Enhancement

Before:

async clearAll(prefix?: string): Promise<number> {
  logger.info('Cache clearAll requested', { prefix });
  return 0; // Always returns 0
}

After:

async clearAll(prefix?: string): Promise<number> {
  // Returns actual count of cleared entries when KV is available
  // Returns 0 when KV is not available (backward compatible)
}

getStats() Enhancement

Before:

async getStats(): Promise<{ supported: boolean }> {
  return { supported: false };
}

After:

async getStats(): Promise<{ supported: boolean; indexed_keys?: number }> {
  // Returns indexed_keys count when KV is available
  // Returns { supported: false } when KV is not available
}

Usage Examples

With KV Index (Production)

import { Env } from './types';
import { CacheService } from './services/cache';
import { CACHE_TTL } from './constants';

// Initialize with KV namespace for full functionality
const cache = new CacheService(CACHE_TTL.INSTANCES, env.RATE_LIMIT_KV);

// Set cache entries
await cache.set('https://cache.internal/instances?provider=linode', instancesData);
await cache.set('https://cache.internal/instances?provider=vultr', vultrData);
await cache.set('https://cache.internal/pricing?provider=linode', pricingData);

// Pattern invalidation (wildcard support)
const count = await cache.invalidatePattern('*instances*');
console.log(`Invalidated ${count} instance cache entries`);

// Clear all caches
const cleared = await cache.clearAll();
console.log(`Cleared ${cleared} total cache entries`);

// Clear with prefix filter
const clearedInstances = await cache.clearAll('https://cache.internal/instances');
console.log(`Cleared ${clearedInstances} instance cache entries`);

// Get statistics
const stats = await cache.getStats();
console.log(`Cache has ${stats.indexed_keys} indexed keys`);

Without KV Index (Development/Backward Compatible)

// Initialize without KV namespace (backward compatible)
const cache = new CacheService(CACHE_TTL.INSTANCES);

// Basic cache operations work normally
await cache.set('key', data);
const result = await cache.get<MyType>('key');

// Pattern invalidation returns 0 (logs warning)
const invalidated = await cache.invalidatePattern('*pattern*');
// → Returns 0, logs warning

// clearAll returns 0 (logs info)
const cleared = await cache.clearAll();
// → Returns 0, logs info

// Stats indicate not supported
const stats = await cache.getStats();
// → { supported: false }

KV Index Structure

Index Key Format:

cache_index:{original_cache_key}

Example:

cache_index:https://cache.internal/instances?provider=linode

KV Entry Metadata:

{
  cached_at: "2026-01-25T15:00:00.000Z",
  ttl: 300
}

Auto-Cleanup:

• KV entries use same TTL as cache entries
• KV automatically deletes expired entries
• No manual cleanup required

Pattern Matching

Wildcard Support:

• * matches any characters
• Case-insensitive matching
• Regex special characters are properly escaped

Examples:

// Match all instance caches
invalidatePattern('*instances*');

// Match specific provider
invalidatePattern('*provider=linode*');

// Match specific endpoint
invalidatePattern('https://cache.internal/pricing*');

// Match exact pattern
invalidatePattern('*instances?provider=vultr*');

Performance Characteristics

With KV Index:

• set(): +1 KV write (async, non-blocking)
• delete(): +1 KV delete (async, non-blocking)
• clearAll(): 1 KV list + N cache deletes + N KV deletes
• invalidatePattern(): 1 KV list + N cache deletes + N KV deletes
• getStats(): 1 KV list operation

Without KV Index:

• All operations identical to original implementation
• No performance overhead

Graceful Degradation:

• KV failures don't break cache operations
• Errors are logged but not thrown
• Cache operations continue normally

Migration Guide

For Existing Code (No Changes Required)

Existing code continues to work without changes:

// This still works (no KV index)
const cache = new CacheService(CACHE_TTL.INSTANCES);

For New Features (Enable KV Index)

To enable pattern invalidation and enumeration:

// Pass KV namespace as second parameter
const cache = new CacheService(CACHE_TTL.INSTANCES, env.RATE_LIMIT_KV);

Production Deployment

Option 1: Reuse RATE_LIMIT_KV (Current Implementation)

• No configuration changes required
• Cache index data stored in same KV as rate limiting
• Prefix separation prevents conflicts (cache_index: vs ratelimit:)

Option 2: Dedicated CACHE_INDEX_KV (Future Enhancement)

• Add new KV namespace in wrangler.toml:

  [[kv_namespaces]]
  binding = "CACHE_INDEX_KV"
  id = "your-kv-id-here"
  

• Update routes to use env.CACHE_INDEX_KV instead of env.RATE_LIMIT_KV

Testing

Run all cache tests:

npm run test -- cache-kv.test.ts

Coverage:

• 24 unit tests
• All edge cases covered
• Error handling validated
• Backward compatibility verified

Security Considerations

• KV index only stores cache keys (not data)
• Same security as rate limiting KV
• No sensitive data exposure
• Automatic cleanup via TTL expiration

Benefits

1. Pattern Invalidation: Invalidate multiple cache entries at once
2. Accurate Counts: clearAll() now returns actual count
3. Cache Statistics: Monitor cache usage and size
4. Backward Compatible: Works with or without KV
5. Automatic Cleanup: KV TTL keeps index clean
6. Graceful Degradation: KV failures don't break cache operations

Known Limitations

• KV eventual consistency: Recent writes may not appear immediately in list operations
• Pattern matching is done in-memory after KV list (not server-side)
• Large cache sizes (>1000 keys) may require pagination handling
• KV list operations have ~100ms latency