Understanding what the AI Firewall is

The firewall allows the company to monitor and manage the risks of using Generative AI services, focusing on public AI services.

Pragatix firewall allows companies to enforce AI policies by analyzing and understanding the use case, users' objectives for using AI, and the sensitivity of the relevant data.

The firewall safeguards against misuse of AI that could pose a business risk or violate company policies. It also enforces AI guardrails to ensure the AI response is aligned with the company ethics and responsible use of AI.

Policies can be tailored to suit the specific needs and characteristics of individual users or groups, optimizing the advantages of AI utilization while mitigating associated risks.

Pragatix gives the company complete visibility and analytics of what the users are doing with GenAI, whether ChatGPT, Copilot, Gemini, or a custom AI model.

Pragatix also offers complete end-to-end Private/ On-prem AI solutions for highly regulated companies, ensuring zero data exposure. You can read more about this solution here .

Accessing the Pragatix dashboard

To access the dashboard, follow this link - Pragatix Demo Dashboard

You can sign-in with your Miscrosoft or Google account.

How do you get your prompts into the system?

The system relies on analyzing the interactions with the public AI service. For this reason, you need to configure your demo environment to get the interactions (prompts and responses) into the system.

You can demo the firewall using the Pragatix chatbot or your ChatGPT / Copilot in the browser. The easiest way is to use the browser extension.

There are several interfaces available for the system.

1. Use the system's built-in Chatbot. The chatbot can interact with ChatGPT directly, or you can ask questions about the content you upload or sync to the system.

2. Installing a browser extension (Chrome / Edge).
   For Chrome go to the Chrome web store here: ​https://chromewebstore.google.com/detail/businessgpt-ai-firewall/nmclpkidicoigfkjbphinajeekefeoel?hl=en-US&utm_source=ext_sidebar
   For Edge go to the Microsoft Edge add on page here: Pragatix AI Firewall - Microsoft Edge Addons

   Notes:
   a. Currently, the extension supports Chrome, Edge and ChatGPT.
   b. The next versions will support Copilot and Gemini

3. Configure Network proxy - contact support for this

Who can access the account Firewall settings?

The first user who signs into the Dashboard from a specific domain is automatically the account admin. Any other users with the same domain who sign in will be members of that account. You can change this by contacting our support.

The firewall components

All components are visible to the company admin under the Account Firewall section.

Dashboard - The product's web UI.

Data classification rules- Rules classifying data, included in the prompt/response and relevant content (documents). Each rule has a sensitivity level.

Usage classification rules- Rules classifying the intention of using the GenAI services.

Firewall Policies- Policies that define each scenario's risk and action (block/ allow/ monitor)

based on groups/users/data classification rules/usage classification rules.

Firewall auditing - Report of all prompts and responses with the policy applied and the matching classification rules

Firewall Dashboard- A management graphic summary of all Firewall results

Classification rules- overview

There are two types of classification rules in the system:

1. Data classification rules - These rules cover data written in the prompt or response and files uploaded to the prompt as part of the interaction.

2. Usage classification rules- These rules apply only to the prompt to analyze the user’s objectives/use case.

For each classification rule (data and usage), there are two types of rules - Regular or AI

1. Regular expression: The engine will search for text defined in the value. You can also enter more advanced patterns using regular expressions. If you want text, just type it in the field without other characters.

2. AI - Define the rule in your own language (NLP)

Configure Data classification rules

The data classification rules classify content, including the prompt, response, and data involved.

The data can be a file added to the prompt or any data type synced into the system.

Each rule has a sensitivity level set to it.

Any new account includes default rules, allowing you to add more manually.

Default rules are marked with a green background but can not be deleted. If you don’t want to use the default rules, disable them from the edit rule page.

Try it yourself:
1. Go to the Account AI Firewall > Data Classification page

2. 

3. Click the Add button to define a new rule as follows.

2. AI data classification rules - these types of rules allow you to define in natural language the terms of classification

Try it yourself

Go to new rules and define the following rule.

You can copy the text into the value: Legal content, such as contracts, statutes, regulations, case law, and legal literature.

2. Goto data sources > Documents and upload a legal document like an NDA; for example, you can download this file and upload it into the system:

3. Wait a short time (typically 1-2 min) until uploaded and classified

4. Once ready, you should see a classification level set, and if you expand the line, you should see the classification rule set :
   

   

   
   

Data classification language

Each rule has a language set. By default, it is any, which means the rule is checked regardless of the document language.

Writing the rules in the local language is recommended to achieve better results in non-English documents. When done, set the rule language to the relevant value.

When an account contains rules with non-English rules, it will check the language of each document and only send rules that are either set to “Any” or to the document's matching language.

Configure your usage classification rules

Usage classification rules identify the objectives and intentions of using the AI service to manage the risks.

These rules only apply to the prompts and not the response or the attached files.

Usage classification rules are typically defined using Natural Language (AI) but can also be defined using regular expressions to find words.

Firewall Auditing

Get help from auditing.

Note that clicking here on the auditing page takes you to this section in the wiki

Firewall auditing is available at the user or account level.

Auditing details

Each prompt creates a record in the auditing. When a chat contains several prompts - it will be presented as several lines with the same chat name.

The auditing records have the following parts as can be seen in the image below:

At the very top, there is the policy that was applied and an icon indicating its risk.

In the second section, there are details of the prompt, response, and data sources. Data sources are either files attached to the prompt when using public AI services or the data the AI has based it's response on when using the Pragatix chatbot with company data.

The prompt contains the following parts:
Data classification: The data classification of the prompt text
Usage classification: The usage classification rules that matched the prompt
Specific usage - a summary of the usage regardless of any classifications

Below, is the response data classification, and at then comes the data sources classifications

In the last section, there are technical details, including the Firewall type:
Pragatix API is reflected to control the Pragatix chatbot

Prompt ID is used for troubleshooting.

Firewall Policy actions

The action can have the following values:

Allowed - The activity was allowed after validating the policies

Inspecting - The activity was allowed, but the policies have not yet been validated. This action will change shortly and be set to Allowed or Flagged if it matches a policy set to Flag.

Flagged- The activity was allowed but found to match a policy set to “Flag.” This means that this activity should be looked at, but it is not necessarily high risk and does not require blocking.

Blocked - The activity was blocked in real-time.

Mask - when set, the system will mask all Data Sensitivity classification rules that are part of the policy matched. This will apply to both LLM and auditing in real time.

Other data sensitivity rules classified at rest will be masked during auditing by the BG service. This is done regardless of policies, but needs to set “Mark auditing” to true.

• Note that you can’t use “mask” as a policy action in practice unless it is set to be inspected in real-time via the “Real-time firewall inspection action” setting. If the user chooses mark and does not set it to real-time, they will see a warning message that the mask will not work.

Firewall risks

The risks are the results of the matching policies. A full circle represents them

Empty: When no risk is displayed, the policy validation has not yet been done.
None: The Policy validation was completed, but no matching policies were found.
High: The activity matched a high-risk policy
Medium: The activity matched a medium-risk policy
(Yellow) Low: The activity matched a low-risk policy

Firewall policy Engine

Can get the following values:

Firewall Service - means that the BG Service analyzed the interaction at rest after completing the interaction

Real-time Firewall- means that interaction was analyzed in real-time

Firewall Type

The type of component that proxies the data into the dashboard.

It can get the following values:

• ChromeExtention

• Proxy

• PragatixApi (Dashboard)

• FileLoader

• CopilotManualUpload

• CopilotAPI

• FirewallAPI (For external integration)

Chat AI service

• ChatGPT

• Gemini

• PragatixDashboard

• MicrosoftCopilot

• MicrosoftWordCopilot

• MicrosoftTeamsCopilot

• MicrosoftBrowserCopilot

• MicrosoftExcelCopilot

• MicrosoftPowerPointCopilot

• MicrosoftOutlookCopilot

• Cohere

• Claude

• SlackBot

Algorithm

• Retrieve - When the entire file fits within the model's context window the system fully sent it to the AI

• Stuff - When the content scope exceeds the model's context window (e.g., All company data), the system identifies and retrieves the most relevant chunks (portions of the content) based on similarity and keyword search, then sends them to the AI for processing.

• Refine - When the chat scope is a single large file that exceeds the model's context window, the system processes it in segments and iteratively builds and refines the response.

• Selected Documents from search- When the system searches for relevant chunks within a large content scope, it identifies complete documents from those chunks that can be fully included within the effective context limit

• Dominant Document from Search - If the system is unable to send all documents as specified in the Selected Documents from Search, it will attempt to find a dominant document to send. A dominant document is considered as one that 60% of its content was returned in the search results

Chat scope

• General chat (Public Knowledge)- a question from general knowledge chat using public knowledge

• General chat (Company Knowledge) - a question from general knowledge chat using company knowledge.

• Uploaded file - a file/ files uplaoded to the general chat

• Knowledge chat [chat name & ID] - a question from a knoweldge chat

• Data analysis agent [Database name] - question on a DB

• Documet [Document name]- User asked on a specific document from the dashboard.

• AI Agent - Qeustion from the Agent chat

• Smart search- Searched in the smart search

Chat Type

KnowledgeBot

ChatContext

The context of the chat. From which application it was used. For example the name of the Slack channel of the name of the word document that the AI was used from.

Sensitivity Levels

Sensitivity levels are a result of data classification rules.

If data classification is not completed, the system displays a message

A light blue means no matching data classification rules were found. Under the rules, the system writes:

“The content did not match any data classification rules.”

In other classifications, it shows the risk in yellow, orange, and red rings.

Configure Firewall policies

The firewall policies define the risk and the firewall activity (Block/Allow) to be applied for every use of AI.
The policy has two parts:
1. Details
2. Conditions

On the policy page, you establish the criteria for policy matching, leveraging data, and usage classification rules. You can select from existing conditions or seamlessly incorporate new ones directly within the policy page interface. Additionally, defining the specific users or groups the policy applies to, is integral to its configuration.

Furthermore, the policy encompasses risk assessment for each scenario and the firewall's response: whether to block or allow access.

Firewall Details

In this part, you can set the policy's name and description and whether it is enabled or not.
In addition, you set the risk of this policy as you see it in your company.
Risk
Note that the risk can change based on the user performing the activity. For example, a member of the legal group poses less risk than any other user when asking for legal advice.

Action:
Block- will block the prompt or response

Allow- will allow the interaction

Flag- will allow the interaction but will flag it in the auditing for further inspection.When set to real-time , the ser will get a friendly message why his content was flagged

Customizing the messages when a prompt is flagged or blocked

There are two types of message that are used- email and popup in the context of the AI service ( a message shown in the UI of Copilot, ChatGPT..)

Email message:

By default, this is the message shown to the end user:

{
"title": "Prompt [PromptID] by [User] was blocked due to policy [PolicyName]",
"bodyText": "Your prompt was blocked according to company policy [PolicyName].<br>The prompt: '[PromptValue]'<br>File Name: [FileName]<br>The content matched the following classification rules: [ClassificationRules].<br>For more information, you can contact support with the following PromptID: [PromptID]"
}

Popup message

{
"title": "AI Firewall notificatie",
"bodyText": "Your prompt was <strong>Blocked</strong> by the policy <strong>[PolicyName]</strong>.<br>The content matched the following rules:<br> <strong>[ClassificationRules].</strong><br>You will receive a detailed email.<br><em>Please refresh the page to continue.</em>",
"buttonText": "Refresh"
}

Notes

• The email is sent to the “Firewall admin notification email” and to the user who submitted the prompt.

• You can use HTML tags to format and edit the message.

• Under the Firewall settings in the Account Settings page, you can customize the message displayed to the end user.

• There is a separate configuration for the Flag message and the Block message.

• The Flag popup message can be disabled via the setting: Show Popup On Flag Event.

Firewall Conditions:

Start by configuring the identity to whom the policy applies. It can be a group or a user.
Groups can be created in the system and/or synced from your Azure AD.
Contact support to learn how to sync your Azure AD groups and users.

Below this, you choose or create the data and usage classification rule. When you create a rule from this page, it will be added to your classification rules list.

Firewall dashboard

See here: ​https://agatsoftware.atlassian.net/wiki/spaces/VA/pages/3341287429

Data classification report

See here: ​https://agatsoftware.atlassian.net/wiki/spaces/VA/pages/3341090858

Usage management

See here ​https://agatsoftware.atlassian.net/wiki/spaces/VA/pages/3341615137