Large Repository Performance Optimization

What you will learn

• Understand the core purpose of Large Repository Performance Optimization
• Master the basic usage and common options of Large Repository Performance Optimization
• Strategies for optimizing Git performance in large repositories, including partial clone, sparse checkout, shallow clone, git gc, and Git LFS.
• Understand key concepts: Diagnose Repository Performance
• Know when to use this feature and when to avoid it

Start with a problem

Your Git repository keeps growing, clones are getting slower, and everyday operations are starting to feel sluggish. You want to know what optimization techniques are available and which ones fit your project.

One-Sentence Understanding

Git can slow down in very large repositories, but strategies like partial clone, sparse checkout, shallow clone, and regular gc can dramatically improve performance.

Diagnose Repository Performance

Before optimizing, check your repo's current state:

# Repository size
git count-objects -vH

# Largest files
git rev-list --objects --all | git cat-file --batch-check='%(objecttype) %(objectsize) %(rest)' | awk '/^blob/ {print}' | sort -k2 -n -r | head -10

# Command timing
time git status --porcelain | wc -l
time git log --oneline -1

Partial Clone

Only download objects you need, fetching the rest on-demand:

# Clone without blob objects
git clone --filter=blob:none <url>

# Clone without tree or blob objects
git clone --filter=tree:0 <url>

# On-demand fetch
git checkout main  # triggers missing file downloads

Filter Comparison

Sparse Checkout

Only check out specific paths:

# Enable during clone
git clone --sparse <url>

# Enable in existing repo
git sparse-checkout init --cone

# Set directories to check out
git sparse-checkout set src/api src/lib

# Add more directories
git sparse-checkout add docs

# List active paths
git sparse-checkout list

Shallow Clone

Clone only recent commits:

# Last 5 commits
git clone --depth 5 <url>

# Since a date
git clone --shallow-since=2025-01-01 <url>

# Exclude tags
git clone --shallow-exclude=v1.0.0 <url>

# Convert to full clone
git fetch --unshallow

Repository Maintenance

# Regular GC
git gc

# Aggressive GC
git gc --aggressive

# Auto GC (runs automatically)
git gc --auto

# Verify integrity
git fsck

# Repack
git repack -a -d --depth=250 --window=250

Git LFS

# Install LFS
git lfs install

# Track large file types
git lfs track "*.psd"
git lfs track "*.zip"

# Migrate existing binaries
git lfs migrate import --include="*.psd" --everything

Strategy Guide

Try it yourself

1. Practice the large-repo-optimization command in a test repository and observe state changes before and after
2. Experiment with different options and compare the output differences
3. Simulate a real scenario where you would need to use this, and walk through the full process

Continue Learning

1. concepts/worktree — Multiple worktrees for parallel dev
2. internals/packfiles-and-storage — Packfile & storage
3. commands/git-sparse-checkout — Sparse checkout reference

Further reading

Keep going on the same topic:

• Git Maintenance Deep Dive
• Partial Clone: On-Demand Git Object Fetching
• Scalar Git Deep Dive