Google DeepMind demos AI-aware mouse pointer that understands what you're pointing at
Google DeepMind released experimental demos of a Gemini-powered pointer that identifies content under the cursor and acts on short spoken commands.
Google DeepMind published experimental demos showing a reimagined mouse pointer powered by Gemini. Unlike a traditional pointer that only tracks location, the AI-enabled version identifies the specific content under the cursor — a word, image, code block, or video frame — and lets users act on it using brief spoken commands or gestures rather than detailed typed prompts. Examples from the thread on X include turning a photo of a handwritten note into an interactive to-do list, converting a paused video frame into a restaurant booking link, and doubling recipe quantities by hovering over an ingredient list and saying "double these."
The demos are available to try in Google AI Studio. For developers and product teams building AI-native tools, the experiments signal a design direction where context-awareness at the cursor level reduces the need for explicit, structured prompts — shifting interaction toward the pointing-and-speaking shorthand people already use in face-to-face settings. Google DeepMind describes the capability as guiding its thinking on next-generation interfaces.