Google is working on a AI tool that could take over your computer with a simple command. According to The Information, the tool, dubbed internally as Project Jarvis, can browse the internet and perform tasks like purchasing products and booking flights on its Chrome browser.
The tool reportedly works by taking screenshots of a computer screen and “interpreting the shots before taking actions like clicking on a button or typing into a text field,” much like Microsoft’s controversial Recall feature, although that is used for storing and retrieving a user’s computer behavior.
A better comparison of Jarvis’s capabilities is the analysis feature Anthropic launched last week for its Claude large language model (LLM), which can write and run JavaScript code. However, unlike Claude, which is tailored to programmers for operating software applications, Jarvis is reportedly browser-based, which one could imagine being marketed towards a more mainstream audience for shopping, booking flights, and various productivity tasks.
Enhancing productivity and automating certain mundane tasks is the direction Google has taken many of its AI products in search of killer use cases. Google has AI features for Workspace apps like summarizing and writing text in Gmail and Docs, but the competition has been fierce. Google has lagged behind OpenAI, which recently launched a purported reasoning model called o1 that might soon evolve to have more autonomous web-browsing capabilities.
According to the report, Project Jarvis might launch in December with the release of the latest version of its Gemini LLM.
