...
72GB disk space used by model
Tested on Runpod
--model meta-llama/Meta-Llama-3.1-70B-Instruct --gpu-memory-utilization 0.95 --tensor-parallel-size 4--max-model-len 8000
Page Comparison
General
Content
Integrations
...
72GB disk space used by model
Tested on Runpod
--model meta-llama/Meta-Llama-3.1-70B-Instruct --gpu-memory-utilization 0.95 --tensor-parallel-size 4--max-model-len 8000