Embedding model: gte-Qwen
GPU: Nvidia Tesla T4 (12 GB vRAM)
This is a list of sample documents and the time taken to process them. It includes extracting the text from the document, splitting it up into appropriately sized chunks, embedding the text into vector format and storing the extracted text and vector representations in the vector database.
Document | Description | Chunks | Time Taken (seconds) |
|---|---|---|---|
10 pages, 2155 words | 20 | 7 | |
Example
To be safe, we assume, on average, 10 pages per document based on ChatGPT, but in our experience at AGAT, the ratio is four pages per document.
100,000 documents would take approximately 8 days with 1 embedding GPU
or 3 days using the embedding GPU + LLM GPU temporarily (on prem)
or 2 days using 4 GPUs (cloud).
12m documents with 10 RTX 4090 GPUs would take approximately 8 weeks.
Depends on source of documents (e.g. SharePoint) - would need extra time to download each file