# GitHub AI Scan found two shell risks, but the security check passed

> Canonical: https://vxi.me/en/posts/github-ai-scan-pr-security-checks/
> Locale: en-us
> Published: 2026-10-16
> Updated: 2026-10-16
> Markdown: https://vxi.me/en/posts/github-ai-scan-pr-security-checks.md

# GitHub AI Scan found two shell risks, but the security check passed

Today I opened a public test [PR #1](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1) containing two deliberately unsafe Bash patterns. Roughly 81 seconds later, `github-advanced-security[bot]` had left two line-level comments, identifying both risks.

Yet the [GitHub Actions run #38835686338](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/actions/runs/38835686338) marked its `github-advanced-security` job `Success`. Both things happened: the warnings were real, and the check was green.

GitHub had announced the [general availability of AI Scan for pull requests](https://github.blog/changelog/2026-10-15-ai-scan-for-pull-requests-generally-available/) the day before this test. The useful question is not whether an AI scanner can flag suspicious code, but **why a PR can contain flagged vulnerabilities while its security check succeeds**.

The distinction matters: **a successful AI Scan job does not certify that the code is safe. Its findings are advisory, not merge-blocking gates.**

## Two risky Bash patterns, and what it actually caught

I put the fixture under `ai-scan-test/` in a temporary PR. It did not modify production application code or use real credentials or servers. The complete file is preserved at [commit `29e24722d3`](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/blob/29e24722d3f8868322da3aa28ec01c9648f90726/ai-scan-test/vulnerable-deploy.sh). These lines are intentionally dangerous; do not use them in production:

```bash
eval "systemctl restart ${service_name}"

cmd="curl -fsSL http://${host}/${path} -o /tmp/svc.conf"
$cmd
```

The first issue is in the service-restart function. A caller-controlled `service_name` is interpolated into `eval`, which tells the shell to parse the resulting text as a command again. A malicious service name can therefore become more than a name. The [bot's comment on line 17](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1#discussion_r4289992720) classified it as CWE-78 and recommended removing `eval`, quoting the argument and restricting which services are permitted.

The second pattern is different. The script builds a full `curl` command in `cmd`, then executes `$cmd` unquoted. Word splitting lets an attacker influence which arguments and options `curl` receives via `host` or `path`. The [line 27 comment](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1#discussion_r4289992738) also called out the tampering risk of fetching a configuration over plain `http://`.

| Location | What AI Scan reported | Original comment |
| --- | --- | --- |
| Line 17, `eval` | Input may be interpreted again as shell syntax (CWE-78) | [Read comment](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1#discussion_r4289992720) |
| Line 27, `$cmd` | Argument/option injection, plus insecure HTTP transport | [Read comment](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1#discussion_r4289992738) |

Do not collapse these two into an identical vulnerability. The first passes input back through the shell grammar. For the second, the confirmed mechanism is principally **argument/option injection**; unquoted expansion alone does not prove arbitrary, separate shell-command execution.

There is another important caveat. This was a **clearly labeled positive fixture**. Its [README](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/blob/29e24722d3f8868322da3aa28ec01c9648f90726/ai-scan-test/README.md) and script comments explicitly say `INTENTIONAL VULNERABILITY (CWE-78)` and mention an expected `String injection` category. The bot's exploitation examples were not copied from the script, but these hints could have affected detection. So 2/2 in this small test is not a detection rate for unmarked real-world vulnerabilities.

You can also verify the timing. The [PR](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1) was created at **2026-10-16 01:32:31 UTC**; both comments appeared at **01:33:52 UTC** — an **81-second** gap. That is one observed PR-to-comment interval, not an averaged scanner-runtime benchmark.

Both planted issues received comments. There was no separate clean-code PR as a negative control, so this test cannot estimate a false-positive rate either.

## Two vulnerability comments, one green Actions run

In the [dynamic scan run #38835686338](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/actions/runs/38835686338), the workflow was named `Code scanning AI findings on PR #1`. GitHub recorded the `github-advanced-security` job as successful at **01:33:34 UTC**. The comments arrived at **01:33:52 UTC**, **18 seconds later**.

First came a green check, then the vulnerability comments. This sequence alone shows why `Success` cannot be read as “no security findings.”

![Illustration of GitHub AI Scan PR #1: two labeled Bash risks receive advisory comments; the job succeeds at 01:33:34 UTC, comments arrive at 01:33:52 UTC, and merge rules are separate](/images/posts/github-ai-scan-pr/ai-scan-advisory.en-us.svg)

*Diagram derived from public [PR #1](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1) and its [Actions run](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/actions/runs/38835686338), **not a screenshot of GitHub**. The 81 seconds measure PR creation to comments, not scanner execution time. Each language version uses its own localized diagram.*

This is not evidence that the scanner malfunctioned. GitHub's [AI Scan documentation](https://docs.github.com/en/code-security/concepts/code-scanning/ai-powered-security-detections) describes its findings as **advisory**: they do not fail the code scanning check and cannot themselves enforce a Ruleset-based block on merging. AI Scan helps a reviewer find code worth examining; it does not decide whether merging is allowed.

| GitHub signal | What it actually means |
| --- | --- |
| AI Scan inline comments | Potential risks for the author/reviewer; not a direct merge block |
| Dynamic AI Scan job: `Success` | The scan completed, possibly with advisory findings |
| CodeQL or other supported scanning plus Rulesets | May enforce configured merge conditions; verify the repository's actual rules |
| Required checks and human review | Merge permissions depend on the configured check/review policy |

GitHub documents a separate [code scanning merge protection](https://docs.github.com/en/code-security/how-tos/find-and-fix-code-vulnerabilities/manage-your-configuration/set-merge-protection) mechanism. If the requirement is “do not merge while high-severity risks remain,” you need an enforceable configuration, not a green AI Scan advisory run.

I did **not** run an actual Ruleset blocking experiment on this unsafe PR. The observation here is narrower: **the AI Scan job was green at the same time that its PR carried two vulnerability advisories**. It does not prove GitHub would permit every unsafe merge.

## Before enabling it: CodeQL, eligibility and credits

The experiment used a **public repository under a personal account**. Under GitHub's [October 2026 GA documentation](https://docs.github.com/en/code-security/concepts/code-scanning/ai-powered-security-detections), personal public repositories can enable AI Scan while personal private repositories cannot. Public organization repositories are eligible; private/internal organization repositories require the applicable GitHub Code Security or GitHub Advanced Security licensing. An extra Copilot license is not required simply to use AI Scan.

CodeQL default setup is **not** a prerequisite. During the test, I recorded its status as `not-configured`, while AI Scan still left two comments. The [official docs](https://docs.github.com/en/code-security/concepts/code-scanning/ai-powered-security-detections) likewise say AI Scan does not depend on CodeQL default setup or a successful CodeQL analysis.

It is not a historical, full-repository scan. It focuses on eligible PR changes, including some Shell/Bash, Terraform HCL and Dockerfile cases not covered by CodeQL. Fork-originated and Dependabot PRs are outside the supported scope described for this release; teams relying on those workflows need other scanning and review paths.

To enable it, look in the repository's Code scanning settings. In an organization, check the organization policy and security-feature permissions as well.

Billing deserves attention before a high-volume rollout. At GA, usage is measured through **Code Security AI Credits**. Who pays, which allowances apply and how spending limits work depend on [GitHub's billing policy](https://docs.github.com/en/code-security/concepts/code-scanning/ai-powered-security-detections). I did not verify a final charge for this test, so I am not claiming a cost of `$0`. Check Billing limits and hard budgets before turning it on for a busy repository.

## What actually prevents an unsafe merge

The bot's [two comments](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1) point toward ordinary defensive fixes: don't send parameters back through `eval`, and don't assemble a shell command as one string for expansion. The call sites might instead look like this:

```bash
# 调用形式示意：service_name 和 url 仍需单独校验
systemctl restart "$service_name"
curl -fsSL "$url" -o /tmp/svc.conf
```

This is a **direction for a fix, not a security patch that I retested**. Restrict `service_name` to an allowlist. The download URL needs HTTPS enforcement, allowed destinations, redirect handling, and — depending on the configuration's sensitivity — integrity verification.

At the repository level, attach CodeQL or another genuinely enforceable scanner to appropriate Rulesets/required checks, then verify that missing or failing checks are actually rejected on protected branches. Keep AI Scan's comments for reviewers to assess. A green dynamic scan job is not a reason to ignore those comments.

After the experiment, I [closed PR #1](https://github.com/kuangyongjie/cliproxyapi-codex-deploy/pull/1) without merging. The test branch was deleted and public `main` stayed at `87624c26a5e6`. An owner REST receipt recorded AI Scan as `disabled` again. Closing a PR does not erase its historical comments or pinned source, so the public links remain auditable.

This one labeled-positive sample shows AI Scan can provide useful, line-level risk warnings, but **a completed scan and an enforceable merge gate are separate things**. One finds problems; the other is configured and verified by the repository owner.

For a related risk at a different enforcement point, see the [OpenQodex local pre-push hook test](/posts/openqodex-prepush-review-fail-open/), which looks at whether AI review can fail open before Git push.

*Test date: October 16, 2026 (UTC).*