Versions Compared

Old Version 24

changes.mady.by.user Meir Davis

Saved on

compared with

New Version 25

changes.mady.by.user Meir Davis

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents

Containers

Container Purpose

Option 1 (GPU)

Option 2 (CPU)

Gateway (Linux)

16 GB

4 vCPUs

32 16 GB

8 4 vCPUs

Embedder (Linux)

16 GB

16GB GPU

None. Embedding happens on Gateway.16 GB GPU

2 vCPUs

16 GB

4 vCPUs

LLM (Linux)

16 GB

24 GB GPU

2 vCPUs

To be determined.

Dashboardt3a.large/Ingestor (Windows Server- Not a container)

8 GB

4 vCPUs

t3a.large

Info

  1. Example CPU processors: Intel Xeon Platinum 8000.

  2. Example GPU for embedding: NVIDIA T4 Tensor Core.

  3. Example GPU for LLM: NVIDIA L4 Tensor Core.

  4. Database can be installed on the Windows server or any other convenient location, including an existing instance of Microsoft SQL Server.

  5. The Linux containers may be deployed on one server or spread over multiple servers.

AI model servers

The main hardware cost of the BusinessGPT deployment are the AI servers responsible for answering questions. These utilize GPUs.

...