...
8x 4090 runs full context window at 90% vram. 20k window at 87%. (64GB RAM)
4x 4090 20k window: works (32GB RAM)
2x 4090 20k window: doesn’t work
...
...
8x 4090 runs full context window at 90% vram. 20k window at 87%. (64GB RAM)
4x 4090 20k window: works (32GB RAM)
2x 4090 20k window: doesn’t work
...