Secret Scanning & Leak Prevention in Git

What you will learn

• Understand the core purpose of Secret Scanning & Leak Prevention in Git
• Master the basic usage and common options of Secret Scanning & Leak Prevention in Git
• How secrets end up in Git, automated scanning with git-secrets, truffleHog, Gitleaks, platform-level detection, BFG cleanup, and pre-commit prevention.
• Understand key concepts: How Secrets End Up in Git
• Know when to use this feature and when to avoid it

Start with a problem

You're concerned about Git repository security — maybe key management isn't ideal, or commits aren't signed. You're not sure if your security setup is sufficient or where to start hardening it.

One-Sentence Understanding

Secrets (API keys, passwords, tokens) accidentally committed to Git persist in history forever — automated scanning, pre-commit hooks, and history-cleanup tools are the defense layers.

How Secrets End Up in Git

# Common leak scenarios:
git add .                      # Adds .env accidentally
git commit -m "fix config"     # config with hardcoded keys
git push origin main           # Now secret is public

Even if removed in a later commit, the secret lives in the object database:

git log --all --diff-filter=D --summary  # Find deleted files
git show COMMIT_HASH:config.js           # Secret still visible

git-secrets

Scans for regex patterns before commits and across history.

# Install
brew install git-secrets                    # macOS
sudo apt install git-secrets                # Linux

# Register AWS patterns
git secrets --register-aws

# Install pre-commit hook
git secrets --install

# Scan entire history
git secrets --scan-history

# Add custom patterns
git secrets --add 'api[_-]?key\s*[:=]\s*.+'
git secrets --add '(--|password|passwd)\s*[:=]\s*.+'

truffleHog

Entropy-based secret detection — finds high-entropy strings without predefined patterns.

# Scan a repo
trufflehog git https://github.com/user/repo.git

# Scan with specific branch
trufflehog git https://github.com/user/repo.git --branch main

# Scan local directory
trufflehog filesystem /path/to/repo

Gitleaks

Rule-driven scanning with extensive built-in detectors.

# Install
brew install gitleaks

# Scan a repo
gitleaks detect --source . -v

# Scan with a custom config
gitleaks detect --source . --config .gitleaks.toml

# Scan history
gitleaks detect --source . --log-opts="--all"

# Generate config
gitleaks detect --source . --report-format json --report report.json

Example .gitleaks.toml:

title = "Gitleaks custom config"
[[rules]]
  id = "custom-api-token"
  description = "Custom API token pattern"
  regex = '''(?i)myapp_token[ =]+[0-9a-zA-Z_-]{20,}'''
  tags = ["custom", "token"]

[[rules]]
  id = "private-key"
  description = "Private key block"
  regex = '''-----BEGIN\s?RSA\s?PRIVATE\s?KEY-----'''
  tags = ["key", "crypto"]

Platform-Level Detection

GitHub Secret Scanning

Automatically scans all public repos for known partner patterns (AWS, Google, Azure, etc.) and alerts both the user and the provider.

# Enable in private repos:
# Settings → Code security → Secret scanning → Enable

GitLab Secret Detection

Built into GitLab CI via a dedicated job:

secret_detection:
  stage: test
  script:
    - gitlab-org/ci-cd/secret-detection
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

.gitignore Best Practices for Secrets

# Environment files
.env
.env.*
.env.local
.env.production

# Secrets and credentials
**/secrets/
credentials.*
*.cred
**/config/credentials/

# Key files
*.pem
*.key
*.p12
*.jks
id_rsa
id_ed25519

# Build artifacts with secrets
*.log
npm-debug.log*

Global .gitignore

git config --global core.excludesFile ~/.gitignore_global

Pre-Commit Hooks for Secret Detection

Using pre-commit framework:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: detect-private-key
  - repo: https://github.com/awslabs/git-secrets
    rev: master
    hooks:
      - id: git-secrets

Manual pre-commit hook (.git/hooks/pre-commit):

#!/bin/sh
git secrets --pre-commit
gitleaks protect --source .

Removing Secrets from History

BFG Repo-Cleaner

Fast, simple tool for removing large files or text patterns.

# Clone mirror
git clone --mirror https://github.com/user/repo.git

# Replace sensitive text
java -jar bfg.jar --replace-text passwords.txt repo.git

# Remove specific files
java -jar bfg.jar --delete-files .env repo.git

# Expire and gc
cd repo.git
git reflog expire --expire=now --all
git gc --prune=now --aggressive
git push --force

git filter-repo

More powerful, Python-based alternative.

pip install git-filter-repo

# Remove specific file from all history
git filter-repo --path .env --invert-paths

# Replace text
git filter-repo --replace-text <(echo "AKIAIOSFODNN7EXAMPLE==>REPLACED")

Preventing Accidental Commits

Use a credential manager for authentication instead of hardcoded tokens:

git config --global credential.helper osxkeychain

Set up pre-push hooks:

#!/bin/sh
# .git/hooks/pre-push
git secrets --scan || exit 1
gitleaks detect --source . -v || exit 1

Key Takeaways

1. Secrets in history are permanent unless actively purged
2. Platform scanning helps but is reactive, not preventive
3. Pre-commit hooks are the most effective prevention
4. Treat any leaked secret as compromised — rotate immediately
5. Combine tools: git-secrets + Gitleaks + platform scanning for defense in depth

Try it yourself

1. Practice the secret-scanning command in a test repository and observe state changes before and after
2. Experiment with different options and compare the output differences
3. Simulate a real scenario where you would need to use this, and walk through the full process

Continue Learning

1. best-practices/security-with-git — Broader Git security practices
2. security/credential-helper — Managing credentials without hardcoding
3. security/gpg-signing — Verifying commit authenticity
4. commands/git-fsck — Repository integrity checking