With Windows 10 support sunsetted, Microsoft is enhancing Windows 11 at breakneck speed. Following multiple AI upgrades last week, Microsoft continues to expand the way you can interact with your computer by adding Bing Visual Search for anything you see on your screen. In the same way you would draw a box around the portion of the screen you want to copy, you can now select an item you would like to learn more about and send the image to Bing.
You’ll find this new feature in the newly updated Snipping Tool app. Once your computer has been updated to the latest version of Windows 11 (and version 11.2508.29.0 of the Snipping Tool), you’ll receive a walkthrough of Visual Search when you next open the Snipping Tool.
Having played with the feature, laughed at some results, and cursed at the interface, my first impression is that it's a potentially powerful tool. However, there are a couple of significant design flaws.
When I first started using Visual Search, I found the experience puzzling. Bing immediately recognized Taylor Swift and Mark Zuckerberg from standard press photos. The results were a bit confusing, though, with a shot of Sabrina Carpenter's Saturday Night Live performance. At first, Bing appeared to be confused by what it was seeing, returning images of dresses and other blond stars. Then I noticed a tiny thumbnail image at the top of the results (just above a giant ad) with the words "looks like sabrina carpenter." Clicking there returned solid photos and web page results. So much for celebrities.
How about merchandise? When I highlighted an image of Apple AirPods Pro 2 from a recent Techlicious article, Bing got the general idea, pulling up reviews and ecommerce pages for various AirPods models, some closer than others. Aside from also being white and having a similar name, the aged AirPods 2 that Bing placed near the top of search results (with a link to an Amazon page) were rather off the mark, as was the pair of Lexie hearing aids.
Then I saw the little "Sponsored" tag on both results, and I finally caught on. Bing Visual Search is actually pretty good at recognizing people, products, and other items, but sponsored results mangle the experience. Ads have been a part of web search for decades. But Bing's prominent, irrelevant results are particularly confusing for a new service meant to showcase its visual prowess. At least I now know why images of dresses obscured the Sabrina Carpenter results. (They were also ads.) Perhaps Microsoft will find a better way to designate these paid placements, as Google did years ago with text ads, so that people won't miss the useful results of Bing Visual Search.
In any case, Bing Visual Search is worth trying out. Here's how to do it.
- Make sure your Windows 11 is updated. Press Windows icon > Settings > System > Windows Update. Or just type that last item into the Windows search bar.
- Whenever you see something you are curious about, click Windows key + Shift + S to open the Snipping Tool.
- Drag your pointer over the area you are interested in and let go. You'll see a pop-up window from the Snipping Tool with a preview of your screen capture.
- Click the "Markup and share" button to bring up the app.
- Click the three dots in the upper right corner of the Snipping Tool window and select "Visual Search with Bing."
The search results will appear in a new tab in your default browser.
[Image credit: Screenshots via Sean Captain/Techlicious]
