IceFireDB Storage Drivers #

Overview #

IceFireDB supports multiple storage backends through a unified driver interface. This allows you to choose the best storage engine for your specific use case, whether you need high-performance local storage or decentralized distributed storage.

Note: Currently, IceFireDB NoSQL uses command-line flags for configuration rather than YAML configuration files. All storage driver configuration is done through runtime flags.

Driver Architecture #

All storage drivers implement the same interface:

type IDB interface {
    Put(key, value []byte) error
    Get(key []byte) ([]byte, error)
    Delete(key []byte) error
    Close() error
    // ... additional methods
}

This consistent interface enables seamless switching between different storage engines.

Configuration Approach #

Currently, IceFireDB NoSQL uses command-line flags for all configuration. YAML configuration files are not yet supported for the NoSQL component. All storage driver settings are specified as command-line arguments when starting the server.

Basic Usage #

# Select storage backend and data directory
./icefiredb-nosql -storage-backend leveldb -j ./data

# With additional options
./icefiredb-nosql -storage-backend badger -j ./badger_data --cache-size 2048

Available Drivers #

LevelDB Driver #

Best for: General purpose, high-performance local storage

Description: Google LevelDB is a fast key-value storage library that provides ordered mapping from string keys to string values.

Features:

• High performance for read-intensive workloads
• Snappy compression support
• Atomic batch operations
• Crash consistency

Configuration: Configured through command-line flags:

./icefiredb-nosql \
  -storage-backend leveldb \
  -j ./leveldb_data \
  --compression \
  --block-size 4096 \
  --write-buffer-size 4194304

Performance:

• Read: ~2M ops/sec
• Write: ~250K ops/sec
• Latency: <1ms

BadgerDB Driver #

Best for: Write-intensive workloads, SSD optimization

Description: Dgraph BadgerDB is a fast key-value store written purely in Go, optimized for SSDs.

Features:

• Faster writes than LevelDB
• SSD-optimized design
• Memory-mapped I/O
• Encryption support

Configuration: Configured through command-line flags:

Configuration:

Performance:

• Read: ~1.8M ops/sec
• Write: ~300K ops/sec
• Latency: <1ms

IPFS Driver #

Best for: Decentralized storage, content-addressable data

Description: InterPlanetary File System driver stores data on IPFS network with local caching.

Features:

• Content-addressable storage
• Distributed and decentralized
• Data deduplication
• Local hot cache for performance

Configuration: Configured through command-line flags:

Configuration:

Performance:

• Read: ~50K ops/sec (depends on network)
• Write: ~20K ops/sec (depends on network)
• Latency: 10-100ms

CRDT Driver #

Best for: Conflict-free replicated data, eventual consistency

Description: Conflict-free Replicated Data Types driver for automatic conflict resolution in distributed systems.

Features:

• Automatic conflict resolution
• Eventual consistency
• Multi-master replication
• Type-specific merge semantics

Configuration: Configured through command-line flags:

Configuration:

Performance:

• Read: ~800K ops/sec
• Write: ~150K ops/sec
• Latency: 2-5ms

IPFS-LOG Driver #

Best for: Append-only logs, audit trails, immutable data

Description: IPFS-based append-only log storage for immutable data records.

Features:

• Immutable append-only storage
• Cryptographic verification
• Distributed audit trails
• Historical data preservation

Configuration: Configured through command-line flags:

Configuration:

Performance:

• Append: ~15K ops/sec
• Read: ~100K ops/sec
• Latency: 5-20ms

OSS Driver #

Best for: Cloud object storage integration, large datasets

Description: Object Storage Service driver for cloud storage integration (S3-compatible).

Features:

• Cloud storage integration
• Cost-effective for large datasets
• Durability and availability
• Multi-region support

Configuration: Configured through command-line flags:

Configuration:

Performance:

• Read: ~5K ops/sec (depends on cloud)
• Write: ~2K ops/sec (depends on cloud)
• Latency: 50-200ms

HybridDB Driver #

Best for: Tiered storage, cost optimization

Description: Hybrid storage driver that automatically moves data between hot and cold storage tiers.

Features:

• Automatic data tiering
• Cost optimization
• Performance tuning
• Custom migration policies

Configuration: Configured through command-line flags:

Command Line:

./icefiredb-nosql \
  -storage-backend hybriddb \
  --hot-storage badger \
  --cold-storage oss \
  --hot-data-dir "./data/hot" \
  --migration-threshold 604800 \
  --hot-cache-size 2048 \
  --cold-bucket "my-cold-storage"

Performance:

• Hot storage: Same as underlying driver
• Cold storage: Same as underlying driver
• Migration: Background process

Driver Selection Guide #

Performance Considerations #

Consistency Models #

Storage Requirements #

Configuration Examples #

Basic LevelDB Configuration #

./icefiredb-nosql \
  -storage-backend leveldb \
  -j "/var/lib/icefiredb" \
  --compression \
  --block-size 4096 \
  --write-buffer-size 4194304 \
  --max-open-files 1000

Production IPFS Configuration #

./icefiredb-nosql \
  -storage-backend ipfs \
  -j "/var/lib/ipfs" \
  --hot-cache-size 2048 \
  --p2p-listen "/ip4/0.0.0.0/tcp/4001" \
  --bootstrap-peers "/ip4/192.168.1.100/tcp/4001/p2p/QmPeer1,/dns4/bootstrap.libp2p.io/tcp/443/p2p/QmNnoo..."

Hybrid Storage Configuration #

./icefiredb-nosql \
  -storage-backend hybriddb \
  --hot-storage badger \
  --cold-storage oss \
  --hot-data-dir "/var/lib/icefiredb/hot" \
  --migration-threshold 2592000 \
  --hot-cache-size 4096 \
  --oss-endpoint "https://oss-cn-hangzhou.aliyuncs.com" \
  --oss-bucket "icefiredb-cold-storage" \
  --oss-access-key "your-access-key" \
  --oss-secret-key "your-secret-key" \
  --oss-region "cn-hangzhou"

Driver Selection #

Selecting Storage Drivers #

Storage drivers are selected at startup using command-line flags. Driver selection cannot be changed at runtime - you must restart IceFireDB with the new storage backend.

# Start with LevelDB backend
./icefiredb-nosql -storage-backend leveldb -j ./leveldb_data

# Start with BadgerDB backend  
./icefiredb-nosql -storage-backend badger -j ./badger_data

# Start with IPFS backend
./icefiredb-nosql -storage-backend ipfs -j ./ipfs_data --p2p-enable

Checking Current Driver #

You can check the current storage driver using Redis commands:

# Get driver information
127.0.0.1:11001> DRIVER.INFO

Driver-specific Commands #

Each driver may support additional commands:

# CRDT synchronization
127.0.0.1:11001> CRDT.SYNC

# IPFS peer management
127.0.0.1:11001> P2P.PEERS

# HybridDB migration status
127.0.0.1:11001> HYBRID.STATUS

Performance Tuning #

LevelDB/BadgerDB Tuning #

./icefiredb-nosql \
  --cache-size 2048 \
  --max-open-files 5000 \
  --compression \
  --write-buffer-size 67108864

IPFS Performance Optimization #

./icefiredb-nosql \
  -storage-backend ipfs \
  --hot-cache-size 4096 \
  --chunk-size 262144 \
  --replication-factor 3 \
  --low-water 100 \
  --high-water 200

Monitoring Storage Performance #

# Get storage statistics
127.0.0.1:11001> INFO storage

# Monitor driver performance
127.0.0.1:11001> DRIVER.STATS

Migration Between Drivers #

Data Migration Process #

1. Backup current data
2. Install new driver dependencies
3. Update configuration
4. Restart IceFireDB
5. Verify data integrity

Automated Migration Tools #

IceFireDB provides tools for migrating between storage drivers while maintaining data availability.

Troubleshooting #

Common Issues #

• Driver not found: Install required dependencies
• Permission denied: Check filesystem permissions
• Out of memory: Adjust cache sizes
• Network issues: Check connectivity for cloud/ipfs drivers

Debug Commands #

# Get detailed driver information
127.0.0.1:11001> DRIVER.DEBUG

# Check storage health
127.0.0.1:11001> STORAGE.HEALTH

# Reset driver configuration
127.0.0.1:11001> DRIVER.RESET

Best Practices #

1. Choose the right driver for your workload
2. Monitor performance and adjust configuration
3. Regularly backup important data
4. Test migrations in staging environment
5. Keep drivers updated with latest versions
6. Use hybrid approaches for cost optimization
7. Monitor storage health proactively

Configuration Status #

Current Implementation: IceFireDB NoSQL currently uses command-line flags for all configuration. YAML configuration file support has not yet been implemented for the NoSQL component.

Future Plans: YAML configuration support may be added in future releases to provide a more structured configuration approach alongside the existing CLI flags.

See Also #

• Configuration Guide - Command-line configuration options
• Performance Benchmarking - Performance testing guide
• API Reference - Driver management commands