Windows 11 opens the door to Copilot: the assistant that “moves its hands” inside the computer | Festina Lente - Your leading source of AI news | Turtles AI

Microsoft is experimenting with a mode called Copilot Actions for Windows 11, which will allow the AI ​​assistant to act on local files (sort photos, extract data from documents, etc.). The feature will be initially tested by Insiders, with rigorous controls and usage limits.

Key points:

• Copilot Actions extends AI beyond the browser: it will be able to operate on files stored on the PC.
• Access is reserved for Windows Insider and Copilot Labs users and is disabled by default.
• Actions are executed in an isolated environment, and the user can intervene at any time.
• The update also includes "Hey Copilot" voice activation, taskbar integration, and connection to external services (Gmail, OneDrive, etc.).

Microsoft has announced that it will introduce an experimental feature called Copilot Actions in Windows 11, designed to allow the AI ​​assistant to perform autonomous tasks directly on local files, not just online ones, such as sorting images, extracting information from PDFs, generating content from personal documents, or applying changes to items stored on the PC. The idea is for the user to tell the assistant "I want you to do this" (in natural language), and Copilot will attempt to carry out the request by interacting with desktop or web applications, while the user can continue working on other tasks or intervene when necessary.

The trial will initially be reserved for members of the Windows Insider program and the Copilot Labs group, where Microsoft tests real-world scenarios under strict supervision. To ensure security and data protection, this feature is disabled by default: the user must voluntarily enable it, explicitly granting access to specific folders (Desktop, Documents, Downloads, Pictures), and can suspend its activity at any time. Its operations will be performed in a separate "agent environment," with its own desktop, so that the activity does not interfere with the user’s work and it is possible to monitor what happens step by step.

One of the first practical examples involves a contextual item in the File Explorer context menu: by selecting a file, the user can click "Create website with Manus," invoking a Manus agent (a Singaporean startup) that converts local content into web pages. Other uses include grouping and analyzing recent images, automatic reorganization, and data extraction from various document formats.

In parallel with the expansion of Copilot Actions, Microsoft is introducing new ways to interact with the assistant: the "Hey Copilot" voice activation allows you to invoke the assistant by speaking, similar to smart speakers. The Vision functionality (analysis of the content displayed on the screen) becomes more ubiquitous and can be queried via text as well as voice. Furthermore, a dedicated "Ask Copilot" or "Copilot" button is expected to be added to the taskbar to quickly start conversations, activate Vision, or activate Actions with a single click. Microsoft explained that these new features use existing interfaces (search APIs, file system, settings) and do not automatically grant access to personal files unless expressly authorized.

This launch comes just as Microsoft has ended free support for Windows 10, pushing users to migrate to Windows 11. In this context, these AI innovations reinforce the central role that Copilot plays in the Windows 11 user experience, attempting to transform the system into an "intelligent" environment where the assistant is not just an advisor but can actually intervene in tasks. Some argue that Microsoft is "rewriting" Windows 11 around AI, merging voice input, vision, and agents capable of acting within the operating system.

Uncertainties remain regarding efficiency on complex interfaces, error handling, privacy implications, and user acceptance.