AI model servers
The main cost of the BusinessGPT deployment is the AI servers responsible for answering questions. These are utilizing graphic card GPUs (Graphic Process Units).
These servers will also be used to create embedding for processing and preparing the data for AI.
Graphics Cards
Below are two card options that are available for purchase for standard servers. The difference between the cards is the number of questions per minute that can be processed.
The system supports multiple servers with a load balancer to boost performance.
Card | Answer Speed | Time taken to answer 10 questions asked simultaneously | Purchase cost* (One time cost) |
|---|---|---|---|
H100 80GB SXM5 | 1.9Sec | 15 | $33,000 |
Nvidia RTX 4090 24 GB VRAM 4 vCPU | 2.1sec ( 29 Questions /Min) | 17 Sec | $2289 |
Nvidia RTX 4070 Ti 12GB VRAM 8vCPU | 4.2 sec (14 Questions /Min) | 25 Sec |
*Costs were taken from Amzaon.com however these graphic cards can also be purchased elsewhere.
Dashboard website
Windows Server 2016 Enterprise (or Higher), 4CPU, 8GB RAM, 50G disk space, IIS installed. HDD/SSD with a R/W speed of at least 50MB/s.
Ingestor service
Windows Server 2016 Enterprise (or Higher), 4CPU, 8GB RAM, 50G disk space, IIS installed. HDD/SSD with a R/W speed of at least 50MB/s.
Dashboard SQL DB
Configured with SQL file storage
Windows Server 2016 Enterprise (or Higher), 4CPU, 8GB RAM, IIS installed. HDD/SSD with a R/W speed of at least 50MB/s.
In general, the integrated content is not saved in this DB . Instead, the product keeps a link to the source file
In case files are uploaded manually to the Dashboard, they are saved in the Dashboard db as is (same size)
Gateway server
Linux server Ubuntu 4CPU, 8GB RAM, IIS installed. HDD/SSD with a R/W speed of at least 100MB/s.
The disk size should be 30% larger than the original content file size.