AI-Infra-Guard Explained: AI, MCP and Agent Skill Scans

AI-Infra-Guard brings several AI security checks into one platform, but choosing the right scanner matters more than running the largest possible scan. A skill repository, a running inference service, and a model endpoint are different targets. The project documentation describes separate capabilities for each—and warns that the platform itself lacks built-in authentication and should not be exposed publicly.

Match the scanner to the thing you operate

The skill scanner inspects agent skills against the SkillTrustBench risk taxonomy. The MCP and agent scanners address tool and workflow behavior, while the infrastructure scanner fingerprints running services for known vulnerabilities. Jailbreak evaluation exercises a model endpoint using attack datasets. These are complementary forms of evidence, not interchangeable certificates of safety. Source

For a team deciding how to introduce scanning, the useful first step is to identify a concrete boundary: code being admitted into an agent environment, tools an agent can call, or a service reachable on the network. That choice determines both the scan input and what a negative result can reasonably mean.

Your question	Relevant documented capability	What the result does not establish
Does this skill contain risky instructions or code?	Skill scanning	That every possible execution is safe
Does this running AI service match known vulnerabilities?	Infrastructure fingerprinting	That all application-level behavior is secure
Can this model be induced to violate a policy?	Jailbreak evaluation	That unseen attacks will fail

A service address is not a repository URL

The infrastructure scanner connects to the network address of a running service. The README explicitly distinguishes this from supplying a GitHub URL or source path. By contrast, the documented MCP/skill workflow accepts source inputs. Selecting the wrong target type can produce a scan that never examines the system you intended to assess. Source

Dynamic verification adds another boundary. The release notes describe tool whitelisting to reduce remote-code-execution risk during dynamic scans. This is a specific control, not proof that arbitrary third-party tools can be executed without isolation.

Read the benchmark as a detection tradeoff

The project reports SkillTrustBench F1, precision, recall, and false-positive rates for several model backends. For example, the table reports recall of 0.9974 and FPR of 0.0663 for Claude Opus 4.6, compared with recall of 0.9641 and FPR of 0.0120 for Gemini 3.5 Flash. These are project-reported results on that benchmark. Source

The useful interpretation is a tradeoff: fewer false alarms can accompany more missed positives. A single F1 ranking hides that choice. The table does not establish detection rates on your repository, nor does it supply a universal best backend for every security policy.

Plan the scanner's own deployment boundary

The documented Docker setup requires Docker 20.10 or later, at least 4 GB of RAM, and at least 10 GB of disk. The skill CLI also requires model access for the selected backend. Follow the linked quick start for exact setup rather than treating the resource minimums as sizing guidance for every workload. Source

For adoption, the most consequential issue is who can reach the scanning service and what credentials or target access it receives. A broader scanning platform is useful when you have several AI-specific surfaces to inspect. If your immediate problem is one skill repository, start with that narrower workflow and make its findings actionable before expanding coverage.

A rollout worksheet for an internal skill repository

Consider a hypothetical team admitting third-party skills into a coding environment. Its first decision is whether a finding should block admission, request investigation, or be recorded for later. That is a policy decision distinct from the scanner's model score. The documented separation between skill scanning, infrastructure scanning and jailbreak evaluation makes a mixed target list a poor starting point. Scanner scope

Trial input	Evidence to retain	Admission decision
A known prohibited skill behavior	The passage or code location that supports the finding	Block the skill when the prohibited behavior is established
A benign skill that resembles an attack	The exact reason the scanner flagged it	Investigate false alarms before expanding the block rule
A scan that cannot finish	Connection errors and whether the intended target was examined	Treat the result as incomplete, rather than a clean bill of health

Build a small labeled set from your own admission policy and keep the skill revision and scanner configuration with each result. Review missed positives separately from false positives: an attractive aggregate F1 score does not decide which error is acceptable. Compare backends on that same set before changing the admission rule.

The rollout can expand after findings consistently lead to a reproducible action. Keep the scanning service on its documented internal deployment boundary, and use isolated copies of untrusted targets for any dynamic investigation. This example is a proposed evaluation procedure; it is not a scan result or an assurance that a passing skill is safe.

Evidence and scope

Primary repository documentation was refreshed on 14 October 2026. The worked scenario is a planning exercise derived from documented behavior, not a product test. Reported measurements remain attributed to their source and workload.

AI-Infra-Guard README