AI model servers

The main hardware cost of the BusinessGPT deployment is are the AI servers responsible for answering questions. These are utilizing graphic card GPUs (Graphic Process Units)utilize GPUs.

These servers will also be used to create embedding embeddings for processing and preparing the data for AI.

Graphics Cards

Below are two card GPU options that are available for purchase for standard servers. The difference between the cards is the number of questions per minute that can be processed.

...

*Costs were taken from Amzaon.com however these graphic cards can also be purchased elsewhere.

** Approximate times when using Wizard-Vicuna-13B model.

Dashboard website

Windows Server 2016 Enterprise (or Higher), 4CPU, 8GB RAM, 50G disk space, IIS installed. HDD/SSD with a R/W speed of at least 50MB/s.

Ingestor service

May be co-located with Dashboard server (above)

Windows Server 2016 Enterprise (or Higher), 4CPU, 8GB RAM, 50G disk space, IIS installed. HDD/SSD with a R/W speed of at least 50MB/s.

...

Linux server Ubuntu 4CPU, 8GB RAM, IIS installed. . HDD/SSD with a R/W speed of at least 100MB/s.
The disk size should be 30% larger than the original content file size.

AI Server - Docker container

Linux server Ubuntu 2CPU, 4GB RAM HDD/SSD with a R/W speed of at least 100MB/s.

GPU: CUDA 11.8+, Min 12GB RAM.
The disk size should be 30% larger than the original content base file size.

Multiple docker containers may be used for higher performance / High availability with a load balancer.