Imagination details GPU strategies for LLM acceleration

Imagination Technologies previewed its work on making large language models (LLMs) like ChatGPT run faster on everyday devices using its PowerVR GPUs. The company explained two core speed metrics—how long until the first word shows up, and how fast each next one appears. They’re focusing on speeding up LLM inference with smarter caching and tailored GPU operations, aiming to bring more practical AI capabilities to phones and edge devices where power and speed matter most.

• Imagination working to speed up LLMs on PowerVR GPUs
• Highlights importance of 'first word' and per-word delays
• LLM inference benefits from GPU-parallel operations
• KV caching skips repetitive calculations for faster AI
• Aims to enable fast, local AI even on modest devices