Introduction

This guide covers Model Guardian, a Pragatix solution for scanning third-party AI models with a focus on security and compliance aspects.
The model guardian allows clients to set guardrail policies to the use of external AI models in general and Large Language Models in specific.

List of Available Scans

Provenance & Identity

• Verified Org Check: Confirms the model's publisher is officially verified by the hosting platform (e.g., Hugging Face).

• Hash Match & GPG Check: Cryptographically ensures the model hasn't been altered or tampered with since publication.

• Maturity & Popularity: Assesses community trust and active maintenance by reviewing downloads, likes, and recent update history.

Static Analysis

• Unsafe Files Detection: Identifies any files flagged as malicious or risky by the hosting platform.

• File Formats Filtering: Blocks unapproved or risky file types, ensuring the model only includes approved formats.

• Static Analysis (Roadmap): will offer running static analysis on the model’s weight files to detect hidden arbitrary code executions.

Red Teaming

• Jailbreak: Tests the model's resistance to attacks designed to bypass its safety guardrails (e.g., "DAN" attacks).

• Hallucination: Measures on what level is the model grounded to reality to ensure it doesn't fabricate facts.

• Prompt Injection: Evaluates defenses against malicious instructions hidden inside standard user prompts.

• Sensitive Info Disclosure: Tests if the model can be tricked into revealing secrets, such as API keys or PII.

• System Prompt Leakage: Checks the model can be forced to reveal its core internal instructions.

• Custom Dataset Inference (Roadmap): Will enable running inference on the model against a custom dataset uploaded by the client, evaluating the model’s responses against expected keywords or semantic meaning.

Compliance & Legal

License Validation: Confirms the model's legal license aligns with the organization's approved usage policies.

Vendor Compliance (Deep Research): Evaluates the model's creator against the organization's internal standards to ensure full corporate and security compliance.

How to Use the Model Guardian

Step 1 - Rule Creation​

Rules for the Model Guardian are listed under AI Security Suite > Model Guardian > Model Rules.

To add a new rule:

1. Click Add.

2. A window for creating a new rule will open.

3. Fill in the relevant fields:

   1. Name - the name for your new rule (e.g. “models with unsafe files”).

   2. Description - a short description for the content of your new rule.

   3. Rule category - categories are split into 4 sections, covering different aspects of the scan

Step 2 - New Policy Creation​

• Under AI Security Suite > Model Guardian > Model Policies, click Add.
  A window for creating a new Policy will open. Fill in the relevant fields, then click Save.

Step 3 - Scan Models

• Go to AI Security Suite > Model Guardian > Model Auditing, Fill in the fields as shown in the following images, then click Scan Model to start the scan and receive the relevant results.