Google made a subtle announcement today on Twitter: In the process of rolling out a new AI features for its Lens platform that will let you search your Google Photos library for text that appears within photos and screenshots. Then, you can then easily copy and paste that text into a note, document, or form.
Both of the new features make use of the technique known as optical character recognition (OCR), with the copy / paste option building on Lens' existing ability to understand and pull out the text found within the photos, be it a screenshot or a photo of a physical sign or document. According to 9to5Google, that feature is available now on some Android devices, though it doesn't appear to be active quite yet on iOS. You may already be able to search your photos using Google Photos on the web.
Google is positioning the Lens feature as a way to easily snag a confounding Wi-Fi password, but it can also be used to grab any portion of text from any photo or screenshot in your Google Photos library on a case-by-case basis. The more impressive feat, though, is the ability to search for any snippet of text without having to pick a photo first. That suggests Google is performing OCR on your entire photo collection automatically.
Wow, @googlephotos has OCR to turn screenshots into copy / paste text!
A. Open google photos and select screenshot
B. Pick the “Lens” feature [image 1]C. Highlight text [image 2]D. Pick copy / paste [image 2,3]
Nicely done Google team! pic.twitter.com/Um49ika2yT
– ☕️ (@hunterwalk) August 21, 2019
"Starting this month, we're rolling out the ability to search your photos by text," Google wrote in response to prominent venture capitalist Hunter Walk, who noticed earlier this week that the Lens feature has been turned on for his account. “Once you find the photo you are looking for, click the Lens button to easily copy and paste text. Take that, impossible wifi password. "
You spotted it! Starting this month, we're rolling out the ability to search your photos by texting them.
Once you find the photo you are looking for, click the Lens button to easily copy and paste text. Take that, impossible wifi passwords
– Google Photos (@googlephotos) August 22, 2019
The features are related to one announced back during Google's 2017 I / O developer conference. At the time, CEO Sundar Pichai celebrated a new, in-the-works feature that would let the AI-powered Lens camera understand the context of a photo. That way, you could point your smartphone camera at the router's password sticker and get automatically connected to the Wi-Fi by way of software reading the password and understanding that you like to connect to the network using it.
Since then, Google Lens has added the ability to search the web for photos of menu items, add contact information from a business card to your address book, and all manner of other OCR-related tricks. But these new photos feature like a large-scale deployment of the AI technique done so on a user's entire library to make it searchable and to give you the ability to pull that text out of the photos.