Components overview
Dashboard
.NET website for the end user and admin UI
The Dashboard manages the synced content items, permissions, building collections, chat interfaces, and settings at the user and site levels.
LLM
Customers can choose between using OpenAI’s LLM models such as Chat GPT 4o and Meta's Llama3.1. Llama 3.1 is royalty free for commercial use for for applications with less than 700 million monthly users.
Ingestor service
The ingestor service is a service with connectors to various data sources. It pulls the content and the permissions and sends them to the Gateway embedding queue.
Content loader:
The loader extracts the text from the content items and cleans unnecessary parts from the content, such as email signatures and disclaimers in emails
Embedding service-
Gets data from the Loader, splits it into chunks, and transforms the chunked content into vectors.
Each file type requires a different loader to extract the text content. We will use the Unstructured library for most file types.
We will endeavor to keep many file types, paragraphs, and sections in a single chunk, with some context before and after for context.
The embedding vectors will be stored in a vector DB - we’ve elected to use Chroma DB.
Chroma DB
The Chroma stores embedding as vector content and the chunked text with metadata.
Embedded vector binary content is also stored here.
Insights Engine
This business Logic processes questions, searches for relevant organizational content, and interacts with the AI models.
The API gets the query and decides the best algorithm to generate the answer.
Token classification model
An AI model used for specific data classification, such as PII
Bastion Proxy
Network proxy supporting HTTP and Web sockets needed for analysing the AI services traffic
Firewall Service
This is a BL that is run by the BG SErvice for Classifies and analyses data at rest against policies
Firewall API
This is a website exposing API to be used in real-time for inspection and classification