Microsoft Copilot Now Enhances Edge with Screen Viewing Capability

Microsoft's AI assistant, Copilot Vision, which can analyze your screen and assist with various tasks, is now accessible for free within the Edge browser. This exciting update was shared by Microsoft AI's CEO, Mustafa Suleyman, via Bluesky today. Copilot Vision offers a unique "talk-based experience," where you simply speak your commands and wait for Copilot to respond.

According to Suleyman, if you choose to enable Copilot Vision, it can "literally see what you see on screen." Imagine having it guide you through a recipe as you cook, or even decoding job descriptions to help you prepare for interviews or brainstorm cover letter ideas. However, it's worth noting that using AI for your resume might not always be the best approach. As per Microsoft's support page, Copilot Vision can highlight parts of your screen to pinpoint relevant information, but it won't click links or take actions on your behalf.

While the broader, system-wide features of Copilot Vision remain exclusive to Copilot Pro subscribers, the free version within Edge is still quite powerful. With a subscription, Vision can extend beyond Edge to assist with tasks in applications like Photoshop or video editing software, or even guide you through games like *Minecraft*, as demonstrated by *The Verge*'s Tom Warren earlier this month.

To give Copilot Vision a try, simply open this link to Microsoft's website in the Edge browser. You should be prompted to opt into the feature. Once you've granted permission, you can activate the Copilot sidebar on any website, click the microphone icon, and start your Vision session, which will be marked by a chime and a change in your browser's hue.

However, my experience wasn't as smooth as expected. It took a few attempts before Edge asked me to opt in, and even after opting in and starting a Vision session, the controls didn't appear. Instead, I was left with a persistent message saying "One moment…" at the bottom of my browser. Keep in mind, I'm using an older, less powerful laptop, so your experience might be different.

Microsoft assures users that while they log Copilot's responses, they do not collect your inputs, images, or page content during a Copilot Vision session. To stop sharing your screen, you can end the session or simply close the browser window.