On Thursday, Microsoft began rolling out a limited preview of Copilot Vision, a tool that can understand and respond to questions about sites you’re visiting using Microsoft Edge. This feature is part of the company’s experimental AI capabilities, which are available through Copilot Labs.
What is Copilot Vision?
Copilot Vision uses artificial intelligence (AI) to analyze text and images on web pages, allowing it to answer queries like "What’s the recipe for this lasagna?" The tool can also summarize and translate text, as well as handle tasks like spotlighting discounted products in a store catalog.
How Does Copilot Vision Work?
When you choose to enable Copilot Vision, it sees the page you’re on, reads along with you, and allows you to talk through the problem you’re facing. This means that you can have a conversation with AI as you navigate the web, making it easier to get help when you need it.
Limitations of Copilot Vision
While Copilot Vision is an exciting feature, it’s not without its limitations. The tool deletes data following every session, and processed audio, images, or text aren’t stored or used to train models. This means that your personal data will be kept safe from any potential misuse.
However, the current preview release of Copilot Vision has some limitations in terms of the types of websites it can view. Microsoft is blocking the feature from working on paywalled and "sensitive" content, limiting Vision to a pre-approved list of popular sites.
What Constitutes Sensitive Content?
So what constitutes sensitive content? Microsoft wouldn’t say, but it appears that they’re being cautious in their approach to this new technology. They’ve stated that the list of allowed websites is determined by category and on a case-by-case basis.
The List of Allowed Websites
Microsoft has started with a small list of sites that they’ve thoroughly tested, and will continue to add more sites over time. However, it’s worth noting that major publishers have opted to block AI tools from trawling their websites due to concerns about data being used without permission.
Collaboration with Publishers
Microsoft is committed to taking feedback from publishers to allay their concerns. They’ve stated that they’re collaborating with third-party publishers to understand how Vision could be used to help people better engage and make decisions on their pages.
What Does This Mean for You?
The rollout of Copilot Vision marks a significant step forward in the development of AI technology. While there are still some limitations to this feature, it’s clear that Microsoft is committed to making it more accessible and useful to users.
As you navigate the web with Copilot Vision enabled, you’ll be able to have a conversation with AI as you need help or guidance. This could revolutionize the way we interact with technology, making it easier for people to access information and solve problems.
The Future of AI
While there are still many questions about the future of AI, one thing is clear: Microsoft is committed to pushing the boundaries of what’s possible with this technology. With Copilot Vision, they’re taking a significant step forward in making AI more accessible and useful to users.
Benefits of Copilot Vision
- Analyzes text and images on web pages to answer queries
- Summarizes and translates text
- Handles tasks like spotlighting discounted products
- Deletes data following every session
- Committed to taking feedback from publishers
Limitations of Copilot Vision
- Blocked from working on paywalled and sensitive content
- Limited to a pre-approved list of popular sites
- Major publishers have opted to block AI tools from trawling their websites
Conclusion
The rollout of Copilot Vision marks an exciting new chapter in the development of AI technology. While there are still some limitations to this feature, it’s clear that Microsoft is committed to making it more accessible and useful to users.
As you navigate the web with Copilot Vision enabled, you’ll be able to have a conversation with AI as you need help or guidance. This could revolutionize the way we interact with technology, making it easier for people to access information and solve problems.
What’s Next?
The future of AI is exciting and uncertain. With Copilot Vision, Microsoft is taking a significant step forward in making AI more accessible and useful to users. As this technology continues to evolve, we can expect to see even more innovative applications of AI in the years to come.
