One of the leaders of the private messaging app Signal calls the feature 'dangerous' because it could potentially spy on users.
Google’s AI model will potentially listen in on all your phone calls — or at least ones it suspects are coming from a fraudster.
To protect the user’s privacy, the company says Gemini Nano operates locally, without connecting to the internet. “This protection all happens on-device, so your conversation stays private to you. We’ll share more about this opt-in feature later this year,” the company says.
“This is incredibly dangerous,” says Meredith Whittaker, the president of a foundation for the end-to-end encrypted messaging app Signal.
Whittaker —a former Google employee— argues that the entire premise of the anti-scam call feature poses a potential threat. That’s because Google could potentially program the same technology to scan for other keywords, like asking for access to abortion services.
“It lays the path for centralized, device-level client-side scanning,” she said in a post on Twitter/X. “From detecting ‘scams’ it’s a short step to ‘detecting patterns commonly associated w/ seeking reproductive care’ or ‘commonly associated w/ providing LGBTQ resources’ or ‘commonly associated with tech worker whistleblowing.’”
One of the things they glided around was whether a lot of this on-device stuff needs a special processor chip with AI+security to work?
The Pixel phones (especially newer ones) made by Google have them, but the vast majority of Android phones don’t.
So either these features only work on latest Google phones (which will piss off licensees and partners), or they’re using plain old CPU/GPUs to do this sort of detection, in which case it will be sniffable by malicious third-parties.
And let’s not forget that if the phone can listen to your conversation to detect malicious intent, any country can legally compel Google to provide them with the data by claiming it is part of a law-enforcement investigation.
It’s not just google who have AI stuff built into their phones. All recent SoCs I have seen have had NPUs going back the last couple generations. A lot of older or cheap phones won’t have one, but the new devices will.
I don’t see the problem with using the phones normal GPU. This shouldn’t be more insecure than making a call currently is. I am pretty sure android phones don’t have a secure enclave just for making calls as you can give different apps access to calling features, and most calls I make are through third party apps anyway, not via POTS. That being said android is pretty secure anyway provided you don’t give permissions to the wrong app. It’s more secure than your average Linux system, as each app has its own user and is only allowed to access things it has explicit permissions to access. Secure enclaves aren’t all that in my opinion.
And let’s not forget that if the phone can listen to your conversation to detect malicious intent, any country can legally compel Google to provide them with the data by claiming it is part of a law-enforcement investigation.
The point of doing it locally is the audio never gets sent to google directly. That being said they could definitely do some dodgy things by training the ML model to search for words like abortion, drugs, transgender, etc depending on what the laws are in the country the phone is being used in.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !privacyguides@lemmy.one
In the digital age, protecting your personal information might seem like an impossible task. We’re here to help.
This is a community for sharing news about privacy, posting information about cool privacy tools and services, and getting advice about your privacy journey.
You can subscribe to this community from any Kbin or Lemmy instance:
Check out our website at privacyguides.org before asking your questions here. We’ve tried answering the common questions and recommendations there!
Want to get involved? The website is open-source on GitHub, and your help would be appreciated!
This community is the “official” Privacy Guides community on Lemmy, which can be verified here. Other “Privacy Guides” communities on other Lemmy servers are not moderated by this team or associated with the website.
Moderation Rules:
We prefer posting about open-source software whenever possible.
No soliciting engagement: Don’t ask for upvotes, follows, etc.
Surveys, Fundraising, and Petitions must be pre-approved by the mod team.
Be civil, no violence, hate speech. Assume people here are posting in good faith.
Don’t repost topics which have already been covered here.
News posts must be related to privacy and security, and your post title must match the article headline exactly. Do not editorialize titles, you can post your opinions in the post body or a comment.
Memes/images/video posts that could be summarized as text explanations should not be posted. Infographics and conference talks from reputable sources are acceptable.
No help vampires: This is not a tech support subreddit, don’t abuse our community’s willingness to help. Questions related to privacy, security or privacy/security related software and their configurations are acceptable.
No misinformation: Extraordinary claims must be matched with evidence.
Do not post about VPNs or cryptocurrencies which are not listed on privacyguides.org. See Rule 2 for info on adding new recommendations to the website.
General guides or software lists are not permitted. Original sources and research about specific topics are allowed as long as they are high quality and factual. We are not providing a platform for poorly-vetted, out-of-date or conflicting recommendations.
One of the things they glided around was whether a lot of this on-device stuff needs a special processor chip with AI+security to work?
The Pixel phones (especially newer ones) made by Google have them, but the vast majority of Android phones don’t.
So either these features only work on latest Google phones (which will piss off licensees and partners), or they’re using plain old CPU/GPUs to do this sort of detection, in which case it will be sniffable by malicious third-parties.
And let’s not forget that if the phone can listen to your conversation to detect malicious intent, any country can legally compel Google to provide them with the data by claiming it is part of a law-enforcement investigation.
Things are going to get spicy in Android-land.
It’s not just google who have AI stuff built into their phones. All recent SoCs I have seen have had NPUs going back the last couple generations. A lot of older or cheap phones won’t have one, but the new devices will.
I don’t see the problem with using the phones normal GPU. This shouldn’t be more insecure than making a call currently is. I am pretty sure android phones don’t have a secure enclave just for making calls as you can give different apps access to calling features, and most calls I make are through third party apps anyway, not via POTS. That being said android is pretty secure anyway provided you don’t give permissions to the wrong app. It’s more secure than your average Linux system, as each app has its own user and is only allowed to access things it has explicit permissions to access. Secure enclaves aren’t all that in my opinion.
The point of doing it locally is the audio never gets sent to google directly. That being said they could definitely do some dodgy things by training the ML model to search for words like abortion, drugs, transgender, etc depending on what the laws are in the country the phone is being used in.