Layer loading leaves most GPU memory unused
Large-model inference runs slowly because it loads one layer at a time and leaves most of the GPU or Mac memory unused.
Tier II · Emerging Problems that are still building up.
3 public reports · first reported 30 November 2023 · last reported 2 February 2024
What's going wrong
Owners of consumer GPUs with 11 to 24 GB of memory, and some Mac users, find that the tool fills only a small part of available memory, roughly one layer at a time.
AI-assisted: written with the help of AI from the reports. The idea report has the full summary.
In the idea report
- The full summary.
- The products the 3 reports are about, and how many came each month.
- Related problems.
No business has been found with this problem yet, so the report has no contacts. All sales are final — no refunds. Pricing.