Gitleaks

┌─○───┐
│ │╲  │
│ │ ○ │
│ ○ ░ │
└─░───┘

[!警告]
Gitleaks 功能已完整。我不会将新功能合并到 Gitleaks 中。未来的版本将仅包括安全补丁。我将把注意力转向 Betterleaks

[][构建] [][Gitleaks-操作] [][Go-文档] [][Go-报告卡] [][许可证]

Gitleaks 是一个用于 检测 git 仓库、文件以及你想通过 stdin 投入的任何其他内容中的密码、API 密钥和令牌等秘密的工具。如果你想了解更多关于检测引擎如何工作的内容，请查看这篇博客：正则表达式几乎就是你所需要的一切。

➜  ~/code(master) gitleaks git -v

    ○
    │╲
    │ ○
    ○ ░
    ░    gitleaks

Finding:     "export BUNDLE_ENTERPRISE__CONTRIBSYS__COM=cafebabe:deadbeef",
Secret:      cafebabe:deadbeef
RuleID:      sidekiq-secret
Entropy:     2.609850
File:        cmd/generate/config/rules/sidekiq.go
Line:        23
Commit:      cd5226711335c68be1e720b318b7bc3135a30eb2
Author:      John
Email:       john@users.noreply.github.com
Date:        2022-08-03T12:31:40Z
Fingerprint: cd5226711335c68be1e720b318b7bc3135a30eb2:cmd/generate/config/rules/sidekiq.go:sidekiq-secret:23

入门

Gitleaks 可以通过 Homebrew、Docker 或 Go 安装。Gitleaks 也可以以二进制形式提供，适用于许多流行的平台和操作系统类型，可在 releases page 获取。此外，Gitleaks 还可以直接作为预提交钩子在你的仓库中实施，或作为 GitHub 操作使用 Gitleaks-Action。

安装

# MacOS
brew install gitleaks

# Docker (DockerHub)
docker pull zricethezav/gitleaks:latest
docker run -v ${path_to_host_folder_to_scan}:/path zricethezav/gitleaks:latest [COMMAND] [OPTIONS] [SOURCE_PATH]

# Docker (ghcr.io)
docker pull ghcr.io/gitleaks/gitleaks:latest
docker run -v ${path_to_host_folder_to_scan}:/path ghcr.io/gitleaks/gitleaks:latest [COMMAND] [OPTIONS] [SOURCE_PATH]

# From Source (make sure `go` is installed)
git clone https://github.com/gitleaks/gitleaks.git
cd gitleaks
make build

预提交

1. 从 https://pre-commit.com/#install 安装预提交

2. 在你的仓库根目录创建一个 .pre-commit-config.yaml 文件，内容如下：

   repos:
    - repo: https://github.com/gitleaks/gitleaks
      rev: v8.24.2
      hooks:
        - id: gitleaks

   用于 gitleaks 的本地执行 或使用 gitleaks-docker 预提交 ID 来使用 官方 Docker 镜像 执行 gitleaks

3. 通过执行 pre-commit autoupdate 自动将配置更新到最新的仓库版本

4. 使用 pre-commit install 安装

5. 现在你一切都准备好了！

➜ git commit -m "this commit contains a secret"
Detect hardcoded secrets.................................................Failed

注意：要禁用 gitleaks pre-commit 钩子，您可以在提交命令前加上 SKIP=gitleaks，它将跳过运行 gitleaks

➜ SKIP=gitleaks git commit -m "skip gitleaks check"
Detect hardcoded secrets................................................Skipped

使用

Gitleaks scans code, past or present, for secrets

Usage:
  gitleaks [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  dir         scan directories or files for secrets
  git         scan git repositories for secrets
  help        Help about any command
  stdin       detect secrets from stdin
  version     display gitleaks version

Flags:
  -b, --baseline-path string          path to baseline with issues that can be ignored
  -c, --config string                 config file path
                                      order of precedence:
                                      1. --config/-c
                                      2. env var GITLEAKS_CONFIG
                                      3. env var GITLEAKS_CONFIG_TOML with the file content
                                      4. (target path)/.gitleaks.toml
                                      If none of the four options are used, then gitleaks will use the default config
      --diagnostics string            enable diagnostics (http OR comma-separated list: cpu,mem,trace). cpu=CPU prof, mem=memory prof, trace=exec tracing, http=serve via net/http/pprof
      --diagnostics-dir string        directory to store diagnostics output files when not using http mode (defaults to current directory)
      --enable-rule strings           only enable specific rules by id
      --exit-code int                 exit code when leaks have been encountered (default 1)
  -i, --gitleaks-ignore-path string   path to .gitleaksignore file or folder containing one (default ".")
  -h, --help                          help for gitleaks
      --ignore-gitleaks-allow         ignore gitleaks:allow comments
  -l, --log-level string              log level (trace, debug, info, warn, error, fatal) (default "info")
      --max-archive-depth int         allow scanning into nested archives up to this depth (default "0", no archive traversal is done)
      --max-decode-depth int          allow recursive decoding up to this depth (default "0", no decoding is done)
      --max-target-megabytes int      files larger than this will be skipped
      --no-banner                     suppress banner
      --no-color                      turn off color for verbose output
      --redact uint[=100]             redact secrets from logs and stdout. To redact only parts of the secret just apply a percent value from 0..100. For example --redact=20 (default 100%)
  -f, --report-format string          output format (json, csv, junit, sarif, template)
  -r, --report-path string            report file
      --report-template string        template file used to generate the report (implies --report-format=template)
      --timeout int                   set a timeout for gitleaks commands in seconds (default "0", no timeout is set)
  -v, --verbose                       show verbose output from scan
      --version                       version for gitleaks

Use "gitleaks [command] --help" for more information about a command.

命令

⚠️ v8.19.0 引入了一个变更，使 detect 和 protect 不再推荐使用。这些命令仍然可用，但在 --help 菜单中被隐藏。可以查看这个 gist 来获得简单的命令对照。如果你发现 v8.19.0 破坏了现有命令（detect/protect），请提交一个问题。

有三种扫描模式：git、dir 和 stdin。

Git

git 命令允许你扫描本地 git 仓库。在底层，gitleaks 使用 git log -p 命令来扫描补丁。你可以使用 log-opts 选项配置 git log -p 的行为。例如，如果你想在一系列提交上运行 gitleaks，你可以使用以下命令：gitleaks git -v --log-opts="--all commitA..commitB" path_to_repo。有关更多信息，请参见 git log 文档。如果未指定作为位置参数的目标，则 gitleaks 将尝试扫描当前工作目录作为 git 仓库。

目录

dir（别名包括 files、directory）命令允许您扫描目录和文件。例如：gitleaks dir -v path_to_directory_or_file。如果没有将目标作为位置参数指定，则 gitleaks 将扫描当前工作目录。

标准输入

您还可以使用 stdin 命令向 gitleaks 流式传输数据。例如：cat some_file | gitleaks -v stdin

创建基线

在扫描大型仓库或历史悠久的仓库时，使用基线会很方便。使用基线时，gitleaks 将忽略基线中存在的任何旧发现。基线可以是任何 gitleaks 报告。要创建 gitleaks 报告，请使用 --report-path 参数运行 gitleaks。

gitleaks git --report-path gitleaks-report.json # This will save the report in a file called gitleaks-report.json

一旦创建了基线，当再次运行检测命令时可以应用它：

gitleaks git --baseline-path gitleaks-report.json --report-path findings.json

在使用 --baseline-path 参数运行 detect 命令后，报告输出（findings.json）将只包含新的问题。

预提交钩子

您可以通过将示例 pre-commit.py 脚本复制到您的 .git/hooks/ 目录中来将 Gitleaks 作为预提交钩子运行。

加载配置

优先顺序为：

1. --config/-c 选项：

     gitleaks git --config /home/dev/customgitleaks.toml .

2. 环境变量 GITLEAKS_CONFIG 的文件路径：

     export GITLEAKS_CONFIG="/home/dev/customgitleaks.toml"
     gitleaks git .

3. 环境变量 GITLEAKS_CONFIG_TOML 的文件内容：

     export GITLEAKS_CONFIG_TOML=`cat customgitleaks.toml`
     gitleaks git .

4. 目标路径中的 .gitleaks.toml 文件：

     gitleaks git .

如果四个选项都未使用，那么 gitleaks 将使用默认配置。

配置

Gitleaks 提供了一种配置格式，您可以按照该格式编写自己的秘密检测规则：

# Title for the gitleaks configuration file.
title = "Custom Gitleaks configuration"

# You have basically two options for your custom configuration:
#
# 1. define your own configuration, default rules do not apply
#
#    use e.g., the default configuration as starting point:
#    https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml
#
# 2. extend a configuration, the rules are overwritten or extended
#
#    When you extend a configuration the extended rules take precedence over the
#    default rules. I.e., if there are duplicate rules in both the extended
#    configuration and the default configuration the extended rules or
#    attributes of them will override the default rules.
#    Another thing to know with extending configurations is you can chain
#    together multiple configuration files to a depth of 2. Allowlist arrays are
#    appended and can contain duplicates.

# useDefault and path can NOT be used at the same time. Choose one.
[extend]
# useDefault will extend the default gitleaks config built in to the binary
# the latest version is located at:
# https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml
useDefault = true
# or you can provide a path to a configuration to extend from.
# The path is relative to where gitleaks was invoked,
# not the location of the base config.
# path = "common_config.toml"
# If there are any rules you don't want to inherit, they can be specified here.
disabledRules = [ "generic-api-key"]

# An array of tables that contain information that define instructions
# on how to detect secrets
[[rules]]
# Unique identifier for this rule
id = "awesome-rule-1"

# Short human-readable description of the rule.
description = "awesome rule 1"

# Golang regular expression used to detect secrets. Note Golang's regex engine
# does not support lookaheads.
regex = '''one-go-style-regex-for-this-rule'''

# Int used to extract secret from regex match and used as the group that will have
# its entropy checked if `entropy` is set.
secretGroup = 3

# Float representing the minimum shannon entropy a regex group must have to be considered a secret.
entropy = 3.5

# Golang regular expression used to match paths. This can be used as a standalone rule or it can be used
# in conjunction with a valid `regex` entry.
path = '''a-file-path-regex'''

# Keywords are used for pre-regex check filtering. Rules that contain
# keywords will perform a quick string compare check to make sure the
# keyword(s) are in the content being scanned. Ideally these values should
# either be part of the identiifer or unique strings specific to the rule's regex
# (introduced in v8.6.0)
keywords = [
  "auth",
  "password",
  "token",
]

# Array of strings used for metadata and reporting purposes.
tags = ["tag","another tag"]

    # ⚠️ In v8.21.0 `[rules.allowlist]` was replaced with `[[rules.allowlists]]`.
    # This change was backwards-compatible: instances of `[rules.allowlist]` still  work.
    #
    # You can define multiple allowlists for a rule to reduce false positives.
    # A finding will be ignored if _ANY_ `[[rules.allowlists]]` matches.
    [[rules.allowlists]]
    description = "ignore commit A"
    # When multiple criteria are defined the default condition is "OR".
    # e.g., this can match on |commits| OR |paths| OR |stopwords|.
    condition = "OR"
    commits = [ "commit-A", "commit-B"]
    paths = [
      '''go\.mod''',
      '''go\.sum'''
    ]
    # note: stopwords targets the extracted secret, not the entire regex match
    # like 'regexes' does. (stopwords introduced in 8.8.0)
    stopwords = [
      '''client''',
      '''endpoint''',
    ]

    [[rules.allowlists]]
    # The "AND" condition can be used to make sure all criteria match.
    # e.g., this matches if |regexes| AND |paths| are satisfied.
    condition = "AND"
    # note: |regexes| defaults to check the _Secret_ in the finding.
    # Acceptable values for |regexTarget| are "secret" (default), "match", and "line".
    regexTarget = "match"
    regexes = [ '''(?i)parseur[il]''' ]
    paths = [ '''package-lock\.json''' ]

# You can extend a particular rule from the default config. e.g., gitlab-pat
# if you have defined a custom token prefix on your GitLab instance
[[rules]]
id = "gitlab-pat"
# all the other attributes from the default rule are inherited

    [[rules.allowlists]]
    regexTarget = "line"
    regexes = [ '''MY-glpat-''' ]

# ⚠️ In v8.25.0 `[allowlist]` was replaced with `[[allowlists]]`.
#
# Global allowlists have a higher order of precedence than rule-specific allowlists.
# If a commit listed in the `commits` field below is encountered then that commit will be skipped and no
# secrets will be detected for said commit. The same logic applies for regexes and paths.
[[allowlists]]
description = "global allow list"
commits = [ "commit-A", "commit-B", "commit-C"]
paths = [
  '''gitleaks\.toml''',
  '''(.*?)(jpg|gif|doc)'''
]
# note: (global) regexTarget defaults to check the _Secret_ in the finding.
# Acceptable values for regexTarget are "match" and "line"
regexTarget = "match"
regexes = [
  '''219-09-9999''',
  '''078-05-1120''',
  '''(9[0-9]{2}|666)-\d{2}-\d{4}''',
]
# note: stopwords targets the extracted secret, not the entire regex match
# like 'regexes' does. (stopwords introduced in 8.8.0)
stopwords = [
  '''client''',
  '''endpoint''',
]

# ⚠️ In v8.25.0, `[[allowlists]]` have a new field called |targetRules|.
#
# Common allowlists can be defined once and assigned to multiple rules using |targetRules|.
# This will only run on the specified rules, not globally.
[[allowlists]]
targetRules = ["awesome-rule-1", "awesome-rule-2"]
description = "Our test assets trigger false-positives in a couple rules."
paths = ['''tests/expected/._\.json$''']

请参考默认的 gitleaks 配置 作为示例，或者如果您想为默认配置做贡献，请遵循 贡献指南。此外，您还可以查看 这篇 gitleaks 博客文章，其中介绍了高级配置设置。

附加配置

复合规则（多部分或 required 规则）

在 v8.28.0 版本中，Gitleaks 引入了复合规则，这些规则由一个“主”规则和一个或多个辅助或 required 规则组成。要创建复合规则，请在主规则中添加一个 [[rules.required]] 表格，指定一个 id，并可选地设置 withinLines 和/或 withinColumns 接近约束。片段是 Gitleaks 一次处理的一块内容（通常是一个文件、文件的一部分或 git diff），接近匹配指示主规则仅在辅助 required 规则也在片段的指定区域中发现匹配项时才报告发现。

接近匹配： 使用 withinLines 和 withinColumns 字段指示主规则仅在辅助 required 规则也在指定接近范围内找到匹配时才报告发现。您可以设置：

• withinLines: N - 必须在 N 行以内（垂直）
• withinColumns: N - 必须在 N 个字符以内（水平）
• 两者 - 创建一个矩形搜索区域（必须满足两个约束）
• 都不是 - 片段级匹配（所需发现可以出现在同一片段的任何位置）

以下是说明每种接近行为的图示：

p = primary captured secret
a = auxiliary (required) captured secret
fragment = section of data gitleaks is looking at

    *Fragment-level proximity*
    Any required finding in the fragment
          ┌────────┐
   ┌──────┤fragment├─────┐
   │      └──────┬─┤     │ ┌───────┐
   │             │a│◀────┼─│✓ MATCH│
   │          ┌─┐└─┘     │ └───────┘
   │┌─┐       │p│        │
   ││a│    ┌─┐└─┘        │ ┌───────┐
   │└─┘    │a│◀──────────┼─│✓ MATCH│
   └─▲─────┴─┴───────────┘ └───────┘
     │    ┌───────┐
     └────│✓ MATCH│
          └───────┘

   *Column bounded proximity*
   `withinColumns = 3`
          ┌────────┐
   ┌────┬─┤fragment├─┬───┐
   │      └──────┬─┤     │ ┌───────────┐
   │    │        │a│◀┼───┼─│+1C ✓ MATCH│
   │          ┌─┐└─┘     │ └───────────┘
   │┌─┐ │     │p│    │   │
┌──▶│a│  ┌─┐  └─┘        │ ┌───────────┐
│  │└─┘ ││a│◀────────┼───┼─│-2C ✓ MATCH│
│  │       ┘             │ └───────────┘
│  └── -3C ───0C─── +3C ─┘
│  ┌─────────┐
│  │ -4C ✗ NO│
└──│  MATCH  │
   └─────────┘

   *Line bounded proximity*
   `withinLines = 4`
         ┌────────┐
   ┌─────┤fragment├─────┐
  +4L─ ─ ┴────────┘─ ─ ─│
   │                    │
   │              ┌─┐   │ ┌────────────┐
   │         ┌─┐  │a│◀──┼─│+1L ✓ MATCH │
   0L  ┌─┐   │p│  └─┘   │ ├────────────┤
   │   │a│◀──┴─┴────────┼─│-1L ✓ MATCH │
   │   └─┘              │ └────────────┘
   │                    │ ┌─────────┐
  -4L─ ─ ─ ─ ─ ─ ─ ─┌─┐─│ │-5L ✗ NO │
   │                │a│◀┼─│  MATCH  │
   └────────────────┴─┴─┘ └─────────┘

   *Line and column bounded proximity*
   `withinLines = 4`
   `withinColumns = 3`
         ┌────────┐
   ┌─────┤fragment├─────┐
  +4L   ┌└────────┴ ┐   │
   │            ┌─┐     │ ┌───────────────┐
   │    │       │a│◀┼───┼─│+2L/+1C ✓ MATCH│
   │         ┌─┐└─┘     │ └───────────────┘
   0L   │    │p│    │   │
   │         └─┘        │
   │    │           │   │ ┌────────────┐
  -4L    ─ ─ ─ ─ ─ ─┌─┐ │ │-5L/+3C ✗ NO│
   │                │a│◀┼─│   MATCH    │
   └───-3C────0L───+3C┴─┘ └────────────┘

gitleaks:允许

如果你明知在提交一个 gitleaks 会捕捉到的测试密钥，你可以在该行添加一个 gitleaks:allow 注释，这将指示 gitleaks 忽略该密钥。例如:

class CustomClass:
    discord_client_secret = '8dyfuiRyq=vVc3RRr_edRk-fK__JItpZ'  #gitleaks:allow

.gitleaksignore

你可以通过在仓库根目录创建一个 .gitleaksignore 文件来忽略特定的发现。在 v8.10.0 版本中，Gitleaks 在报告中增加了一个 Fingerprint 值。每个泄露或发现都有一个指纹（Fingerprint）来唯一标识一个秘密。将此指纹添加到 .gitleaksignore 文件中即可忽略特定的秘密。参见 Gitleaks 的 .gitleaksignore 示例。注意：此功能为实验性，未来可能会有所变动。

解码

有时秘密会以某种方式编码，这可能使它们仅靠正则表达式难以被发现。现在你可以告诉 gitleaks 自动查找并解码已编码的文本。标志 --max-decode-depth 启用此功能（默认值 "0" 表示该功能默认禁用）。

由于解码后的文本也可能包含编码文本，因此支持递归解码。标志 --max-decode-depth 设置递归深度限制。递归会在没有新的编码文本段可解码时停止，所以设置很高的最大深度并不意味着会执行那么多次，它只会执行解码文本所需要的次数。总体来说，解码仅会轻微增加扫描时间。

编码文本的发现与普通发现有以下不同：

• 位置指示编码文本的边界
  • 如果规则在编码文本之外匹配，边界会被调整为
    也包括那个
• 比赛和秘密包含解码后的值
• 添加了两个标签 decoded:<encoding> 和 decode-depth:<depth>

当前支持的编码方式：

• percent - 任意可打印的 ASCII 百分比编码值
• hex - 任意可打印的 ASCII 十六进制编码值，长度 ≥ 32 个字符
• base64 - 任意可打印的 ASCII base64 编码值，长度 ≥ 16 个字符

归档扫描

有时秘密会打包在像 zip 文件或 tar 包这样的归档文件中，很难被发现。现在你可以让 gitleaks 自动提取并扫描归档内容。标志 --max-archive-depth 启用此功能，适用于 dir 和 git 扫描类型。默认值“0”表示此功能默认被禁用。

支持递归扫描，因为归档文件中也可能包含其他归档。--max-archive-depth 标志设置递归限制。当没有新的归档可提取时递归停止，因此设置非常高的最大深度只是设置可能达到的深度。它只会深入到实际需要的深度。

在归档中定位到的秘密的发现结果将包含归档内部文件的路径。内部路径用 ! 分隔。

示例发现（为简短起见已缩短）：

Finding:     DB_PASSWORD=8ae31cacf141669ddfb5da
...
File:        testdata/archives/nested.tar.gz!archives/files.tar!files/.env.prod
Line:        4
Commit:      6e6ee6596d337bb656496425fb98644eb62b4a82
...
Fingerprint: 6e6ee6596d337bb656496425fb98644eb62b4a82:testdata/archives/nested.tar.gz!archives/files.tar!files/.env.prod:generic-api-key:4
Link:        https://github.com/leaktk/gitleaks/blob/6e6ee6596d337bb656496425fb98644eb62b4a82/testdata/archives/nested.tar.gz

这意味着在 testdata/archives/nested.tar.gz 中的 archives/files.tar 中的 files/.env.prod. 的第 4 行检测到一个秘密。

当前支持的格式：

The compression and archive formats supported by mholt's archives package are supported.

报告

Gitleaks 内置支持多种报告格式：json、csv、junit 和 sarif。

如果这些格式都不符合您的需求，您可以使用 Go text/template .tmpl 文件 和 --report-template 标志创建您自己的报告格式。该模板可以使用 Masterminds/sprig 模板库 的扩展功能。

例如，以下模板提供了自定义的 JSON 输出：

# jsonextra.tmpl
[{{ $lastFinding := (sub (len . ) 1) }}
{{- range $i, $finding := . }}{{with $finding}}
    {
        "Description": {{ quote .Description }},
        "StartLine": {{ .StartLine }},
        "EndLine": {{ .EndLine }},
        "StartColumn": {{ .StartColumn }},
        "EndColumn": {{ .EndColumn }},
        "Line": {{ quote .Line }},
        "Match": {{ quote .Match }},
        "Secret": {{ quote .Secret }},
        "File": "{{ .File }}",
        "SymlinkFile": {{ quote .SymlinkFile }},
        "Commit": {{ quote .Commit }},
        "Entropy": {{ .Entropy }},
        "Author": {{ quote .Author }},
        "Email": {{ quote .Email }},
        "Date": {{ quote .Date }},
        "Message": {{ quote .Message }},
        "Tags": [{{ $lastTag := (sub (len .Tags ) 1) }}{{ range $j, $tag := .Tags }}{{ quote . }}{{ if ne $j $lastTag }},{{ end }}{{ end }}],
        "RuleID": {{ quote .RuleID }},
        "Fingerprint": {{ quote .Fingerprint }}
    }{{ if ne $i $lastFinding }},{{ end }}
{{- end}}{{ end }}
]

用法：

$ gitleaks dir ~/leaky-repo/ --report-path "report.json" --report-format template --report-template testdata/report/jsonextra.tmpl

退出代码

当发现泄漏时，您可以始终使用 --exit-code 标志设置退出代码。下面是默认的退出代码：

0 - no leaks present
1 - leaks or error encountered
126 - unknown flag

Join Discord！