Google’s Gemini seems to be improving so quickly that Bloomberg’s Mark Gurman said Apple Intelligence was a disappointment compared to Gemini, and Google is determined to put its AI into every app it can, with Gemini now recognizing and working with YouTube.
There are actually two really cool Gemini features rolling out simultaneously in Gemini and Google app beta: “Ask about this screen” and “Ask about this video.” The former was shown off at Google I/O this year, where the company also showed off Project Astra. “Ask about this video” is essentially the same feature, but it works with YouTube videos.
There are use cases for both, so let’s go into a bit more detail.
Ask a Gemini about screens
Image credit — Google
Gemini is smart enough to understand what’s going on on your screen, and Apple Intelligence is said to be able to do that too. You don’t have to stick to that phrase, either. That means you don’t have to specifically say, “Gemini, what’s on my screen?” Instead, there’s a handy button that tells the AI to receive queries about what’s currently being displayed. Like all modern LLM (Large Language Model) AIs, you can converse with Gemini in a very natural way. Are you on a webpage with a really long article? Just call up Gemini and tell it to show you a summary. Have an image open and you can’t figure out what’s what? Our AI friend can probably help.
The idea is that Gemini is always at hand, ready to assist whenever you are in trouble.
Ask Gemini to summarize your YouTube videos
Image credit — PhoneArena
The same feature has a slight change when used in the YouTube app: the button that says “Ask a question about this screen” changes to one that says “Ask a question about this video.” You’ll also see a little YouTube icon to let you know that you’re about to ask Gemini a question related to the video. What’s interesting is that Gemini uses the YouTube video’s captions to generate responses, so you can have Gemini answer your question about the video without having to play the video or watch a previous ad.
I doubt this exploit will last long — Gemini will likely tell you to finish watching an ad or “kindly” offer you a YouTube Premium subscription — but enjoy it while it lasts.
Always listening… | Image credit — Google
As I said above, Google is determined to put Gemini into everything. I don’t disagree, but it’s nice to know that users can turn off most of the AI tools in case they don’t want them. AI is in:
These are also Google services. Many other platforms are also integrating their own AI into their apps. Soon there will be no device without AI. Naturally, Google doesn’t want to be left behind in this AI race.
While AI is being deployed in places where it might not seem all that useful (just look at the countless emails about AI products), there’s no denying that many services would benefit from it, and with on-device AI as well as AI built into nearly every service, Google seems pretty prepared for the AI era.
It’s no wonder the Pixel is one of the best smartphones you can buy right now.