Embedding Estimates Nov 2024

Embedding model: gte-Qwen

GPU: Nvidia Tesla T4 (12 GB vRAM)

This is a list of sample documents and the time taken to process them. It includes extracting the text from the document, splitting it up into appropriately sized chunks, embedding the text into vector format and storing the extracted text and vector representations in the vector database.

Example

100,000 documents would take approximately 8 days with 1 embedding GPU

or 3 days using the embedding GPU + LLM GPU temporarily (on prem)

or 2 days using 4 GPUs (cloud).

12m documents with 10 RTX 4090 GPUs would take approximately 8 weeks.

Depends on source of documents (e.g. SharePoint) - would need extra time to download each file