Model Sync

Sync ML model files between servers.

Quick Reference

Basic Sync

rsync -avz --progress \
  ./models/ user@server:/models/

With Bandwidth Limit

rsync -avz --bwlimit=10000 \
  ./models/ user@server:/models/

Checksum Verification

rsync -avzc \  # -c for checksum
  ./models/ user@server:/models/

Common Patterns

Sync Specific Model

rsync -avz ./models/llama-7b/ server:/models/llama-7b/

Exclude Checkpoints

rsync -avz --exclude='*.ckpt' --exclude='*.tmp' \
  ./models/ server:/models/

Dry Run (Preview)

rsync -avzn ./models/ server:/models/

Multi-Server Sync

# Sync to multiple servers
for server in gpu1 gpu2 gpu3; do
  rsync -avz ./models/ $server:/models/ &
done
wait

Verification

# Generate checksum
sha256sum models/model.bin > model.sha256

# Verify on remote
ssh server "cd /models && sha256sum -c model.sha256"

Best Practices

• Use --checksum for critical models
• Limit bandwidth on production networks
• Always verify after sync
• Use --delete carefully (removes extra files)