Caterpillar showcases in-cab AI assistant for excavators at CES
The heavy equipment group used a live demonstration in Las Vegas to show how voice-driven artificial intelligence could assist operators directly from the cab, while outlining wider plans for AI-enabled factories and workforce training.
Caterpillar demonstrated a Cat 306 CR mini excavator fitted with an in-cab AI assistant at CES, highlighting how artificial intelligence could be embedded directly into industrial machinery to interpret operator requests and respond in natural speech.
The company said the assistant runs on NVIDIA’s Jetson Thor platform, enabling real-time inference at the edge without relying on cloud connectivity. Speech recognition and synthesis are handled by NVIDIA Riva using Nemotron speech models, while a locally hosted Qwen3 4B model, served via vLLM, parses operator intent and generates responses. Caterpillar’s Helios system supplies what the company described as trusted machine context, grounding the assistant’s answers in live operational data.
During the live demonstration, a video feed from inside the cab showed the assistant responding to a spoken instruction, replying with synthesised speech and then moving the excavator’s arm accordingly. Caterpillar said the demo was designed to show how AI could reduce cognitive load for operators by allowing them to interact with machines conversationally rather than through traditional controls.
Chief executive Joe Creed said the company is focused on building what he described as the “invisible layer” of modern industrial technology, where software and AI augment physical equipment.
Beyond the cab, Caterpillar said it is piloting factory digital twins at multiple US sites using NVIDIA Omniverse libraries and OpenUSD. These virtual environments are intended to simulate production line changes, test scheduling scenarios and optimise material flows before any physical retooling takes place.
Caterpillar also announced a $100 million, five-year commitment to workforce training and education, including a $25 million Global Workforce Innovation Challenge aimed at identifying and scaling programmes that prepare workers for AI-enabled industrial systems.
The Recap
- Caterpillar demoed in‑cab AI and voice interaction on excavator.
- Cat AI Assistant ran on NVIDIA Jetson Thor platform.
- Company pledged $100 million over five years for training.
