Android and Google Play comprise a vibrant ecosystem with billions of users around the globe and millions of helpful apps. Keeping this ecosystem safe for users and developers remains our top priority. However, like any flourishing ecosystem, it also attracts its share of bad actors. That’s why every year, we continue to invest in more ways to protect our community and fight bad actors, so users can trust the apps they download from Google Play and developers can build thriving businesses.
Last year, those investments included AI-powered threat detection, stronger privacy policies, supercharged developer tools, new industry-wide alliances, and more. As a result, we prevented 2.36 million policy-violating apps from being published on Google Play and banned more than 158,000 bad developer accounts that attempted to publish harmful apps.
But that was just the start. For more, take a look at our recent highlights from 2024:
That’s enabled us to stop more bad apps than ever from reaching users through the Play Store, protecting users from harmful or malicious apps before they can cause any damage.
To protect user privacy, we’re working with developers to reduce unnecessary access to sensitive data. In 2024, we prevented 1.3 million apps from getting excessive or unnecessary access to sensitive user data. We also required apps to be more transparent about how they handle user information by launching new developer requirements and a new “Data deletion” option for apps that support user accounts and data collection. This helps users manage their app data and understand the app’s deletion practices, making it easier for Play users to delete data collected from third-party apps.
We also worked to ensure that apps use the strongest and most up-to-date privacy and security capabilities Android has to offer. Every new version of Android introduces new security and privacy features, and we encourage developers to embrace these advancements as soon as possible. As a result of partnering closely with developers, over 91% of app installs on the Google Play Store now use the latest protections of Android 13 or newer. Safeguarding apps from scams and fraud is an ongoing battle for developers. The Play Integrity API allows developers to check if their apps have been tampered with or are running in potentially compromised environments, helping them to prevent abuse like fraud, bots, cheating, and data theft. Play Integrity API and Play’s automatic protection helps developers ensure that users are using the official Play version of their app with the latest security updates. Apps using Play integrity features are seeing 80% lower usage from unverified and untrusted sources on average.
We’re also constantly working to improve the safety of apps on Play at scale, such as with the Google Play SDK Index. This tool offers insights and data to help developers make more informed decisions about the safety of an SDK. Last year, in addition to adding 80 SDKs to the index, we also worked closely with SDK and app developers to address potential SDK security and privacy issues, helping to build safer and more secure apps for Google Play.
Google Play Protect automatically scans every app on Android devices with Google Play Services, no matter the download source. This built-in protection, enabled by default, provides crucial security against malware and unwanted software. Google Play Protect scans more than 200 billion apps daily and performs real-time scanning at the code-level on novel apps to combat emerging and hidden threats, like polymorphic malware. In 2024, Google Play Protect’s real-time scanning identified more than 13 million new malicious apps from outside Google Play1.
Google Play Protect is always evolving to combat new threats and protect users from harmful apps that can lead to scams and fraud. Here are some of the new improvements that are now available globally on Android devices with Google Play Services:
Google Play Protect’s enhanced fraud protection pilot analyzes and automatically blocks the installation of apps that may use sensitive permissions frequently abused for financial fraud when the user attempts to install the app from an Internet-sideloading source (web browsers, messaging apps, or file managers). Building on the success of our initial pilot in partnership with the Cyber Security Agency of Singapore (CSA), additional enhanced fraud protection pilots are now active in nine regions – Brazil, Hong Kong, India, Kenya, Nigeria, Philippines, South Africa, Thailand, and Vietnam.
In 2024, Google Play Protect’s enhanced fraud protection pilots have shielded 10 million devices from over 36 million risky installation attempts, encompassing over 200,000 unique apps. By piloting these new protections, we can proactively combat emerging threats and refine our solutions to thwart scammers and their increasingly sophisticated fraud attempts. We look forward to continuing to partner with governments, ecosystem partners, and other stakeholders to improve user protections.
In 2024, we introduced a new badge for government developers to help users around the world identify official government apps. Government apps are often targets of impersonation due to the highly sensitive nature of the data users provide, giving bad actors the ability to steal identities and commit financial fraud. Badging verified government apps is an important step in helping connect people with safe, high-quality, useful, and relevant experiences. We partner closely with global governments and are already exploring ways to build on this work.
We also recently introduced a new badge to help Google Play users discover VPN apps that take extra steps to demonstrate their strong commitment to security. We allow developers who adhere to Play safety and security guidelines and have passed an additional independent Mobile Application Security Assessment (MASA) to display a dedicated badge in the Play Store to highlight their increased commitment to safety.
In addition to our partnerships with governments, developers, and other stakeholders, we also worked with our industry peers to protect the entire app ecosystem for everyone. The App Defense Alliance, in partnership with fellow steering committee members Microsoft and Meta, recently launched the ADA Application Security Assessment (ASA) v1.0, a new standard to help developers build more secure mobile, web, and cloud applications. This standard provides clear guidance on protecting sensitive data, defending against cyberattacks, and ultimately, strengthening user trust. This marks a significant step forward in establishing industry-wide security best practices for application development.
All developers are encouraged to review and comply with the new mobile security standard. You’ll see this standard in action for all carrier apps pre-installed on future Pixel phone models.
This year, we’ll continue to protect the Android and Google Play ecosystem, building on these tools and resources in response to user and developer feedback and the changing landscape. As always, we’ll keep empowering developers to build safer apps more easily, streamline their policy experience, and protect their businesses and users from bad actors.
1 Based on Google Play Protect 2024 internal data.
Modern AI systems, like Gemini, are more capable than ever, helping retrieve data and perform actions on behalf of users. However, data from external sources present new security challenges if untrusted sources are available to execute instructions on AI systems. Attackers can take advantage of this by hiding malicious instructions in data that are likely to be retrieved by the AI system, to manipulate its behavior. This type of attack is commonly referred to as an "indirect prompt injection," a term first coined by Kai Greshake and the NVIDIA team.
To mitigate the risk posed by this class of attacks, we are actively deploying defenses within our AI systems along with measurement and monitoring tools. One of these tools is a robust evaluation framework we have developed to automatically red-team an AI system’s vulnerability to indirect prompt injection attacks. We will take you through our threat model, before describing three attack techniques we have implemented in our evaluation framework.
Threat model and evaluation framework
Our threat model concentrates on an attacker using indirect prompt injection to exfiltrate sensitive information, as illustrated above. The evaluation framework tests this by creating a hypothetical scenario, in which an AI agent can send and retrieve emails on behalf of the user. The agent is presented with a fictitious conversation history in which the user references private information such as their passport or social security number. Each conversation ends with a request by the user to summarize their last email, and the retrieved email in context.
The contents of this email are controlled by the attacker, who tries to manipulate the agent into sending the sensitive information in the conversation history to an attacker-controlled email address. The attack is successful if the agent executes the malicious prompt contained in the email, resulting in the unauthorized disclosure of sensitive information. The attack fails if the agent only follows user instructions and provides a simple summary of the email.
Automated red-teaming
Crafting successful indirect prompt injections requires an iterative process of refinement based on observed responses. To automate this process, we have developed a red-team framework consisting of several optimization-based attacks that generate prompt injections (in the example above this would be different versions of the malicious email). These optimization-based attacks are designed to be as strong as possible; weak attacks do little to inform us of the susceptibility of an AI system to indirect prompt injections.
Once these prompt injections have been constructed, we measure the resulting attack success rate on a diverse set of conversation histories. Because the attacker has no prior knowledge of the conversation history, to achieve a high attack success rate the prompt injection must be capable of extracting sensitive user information contained in any potential conversation contained in the prompt, making this a harder task than eliciting generic unaligned responses from the AI system. The attacks in our framework include:
Actor Critic: This attack uses an attacker-controlled model to generate suggestions for prompt injections. These are passed to the AI system under attack, which returns a probability score of a successful attack. Based on this probability, the attack model refines the prompt injection. This process repeats until the attack model converges to a successful prompt injection.
Beam Search: This attack starts with a naive prompt injection directly requesting that the AI system send an email to the attacker containing the sensitive user information. If the AI system recognizes the request as suspicious and does not comply, the attack adds random tokens to the end of the prompt injection and measures the new probability of the attack succeeding. If the probability increases, these random tokens are kept, otherwise they are removed, and this process repeats until the combination of the prompt injection and random appended tokens result in a successful attack.Tree of Attacks w/ Pruning (TAP): Mehrotra et al. (2024) [3] designed an attack to generate prompts that cause an AI system to violate safety policies (such as generating hate speech). We adapt this attack, making several adjustments to target security violations. Like Actor Critic, this attack searches in the natural language space; however, we assume the attacker cannot access probability scores from the AI system under attack, only the text samples that are generated.
We are actively leveraging insights gleaned from these attacks within our automated red-team framework to protect current and future versions of AI systems we develop against indirect prompt injection, providing a measurable way to track security improvements. A single silver bullet defense is not expected to solve this problem entirely. We believe the most promising path to defend against these attacks involves a combination of robust evaluation frameworks leveraging automated red-teaming methods, alongside monitoring, heuristic defenses, and standard security engineering solutions.
We would like to thank Vijay Bolina, Sravanti Addepalli, Lihao Liang, and Alex Kaskasoli for their prior contributions to this work.
Posted on behalf of the entire Google DeepMind Agentic AI Security team (listed in alphabetical order):
Aneesh Pappu, Andreas Terzis, Chongyang Shi, Gena Gibson, Ilia Shumailov, Itay Yona, Jamie Hayes, John "Four" Flynn, Juliette Pluto, Sharon Lin, Shuang Song
Today, people around the world rely on their mobile devices to help them stay connected with friends and family, manage finances, keep track of healthcare information and more – all from their fingertips. But a stolen device in the wrong hands can expose sensitive data, leaving you vulnerable to identity theft, financial fraud and privacy breaches.
This is why we recently launched Android theft protection, a comprehensive suite of features designed to protect you and your data at every stage – before, during, and after device theft. As part of our commitment to help you stay safe on Android, we’re expanding and enhancing these features to deliver even more robust protection to more users around the world.
Identity Check rolling out to Pixel and Samsung One UI 7 devices
We’re officially launching Identity Check, first on Pixel and Samsung Galaxy devices eligible for One UI 71, to provide better protection for your critical account and device settings. When you turn on Identity Check, your device will require explicit biometric authentication to access certain sensitive resources when you’re outside of trusted locations. Identity Check also enables enhanced protection for Google Accounts on all supported devices and additional security for Samsung Accounts on One UI 7 eligible Galaxy devices, making it much more difficult for an unauthorized attacker to take over accounts signed in on the device.
As part of enabling Identity Check, you can designate one or more trusted locations. When you’re outside of these trusted places, biometric authentication will be required to access critical account and device settings, like changing your device PIN or biometrics, disabling theft protection, or accessing Passkeys.
Identity Check is rolling out now to Pixel devices with Android 15 and will be available on One UI 7 eligible Galaxy devices in the coming weeks. It will roll out to supported Android devices from other manufacturers later this year.
Theft Detection Lock: expanding AI-powered protection to more users
One of the top theft protection features introduced last year was Theft Detection Lock, which uses an on-device AI-powered algorithm to help detect when your phone may be forcibly taken from you. If the machine learning algorithm detects a potential theft attempt on your unlocked device, it locks your screen to keep thieves out.
Theft Detection Lock is now fully rolled out to Android 10+ phones2 around the world.
Protecting your Android device from theft
We're collaborating with the GSMA and industry experts to combat mobile device theft by sharing information, tools and prevention techniques. Stay tuned for an upcoming GSMA white paper, developed in partnership with the mobile industry, with more information on protecting yourself and your organization from device theft.
With the addition of Identity Check and the ongoing enhancements to our existing features, Android offers a robust and comprehensive set of tools to protect your devices and your data from theft. We’re dedicated to providing you with peace of mind, knowing your personal information is safe and secure.
You can turn on the new Android theft features by clicking here on a supported Android device. Learn more about our theft protection features by visiting our help center.
Timing, availability and feature names may vary in One UI 7. ↩
With the exclusion for Android Go smartphones ↩
In December 2022, we announced OSV-Scanner, a tool to enable developers to easily scan for vulnerabilities in their open source dependencies. Together with the open source community, we’ve continued to build this tool, adding remediation features, as well as expanding ecosystem support to 11 programming languages and 20 package manager formats.
Today, we’re excited to release OSV-SCALIBR (Software Composition Analysis LIBRary), an extensible library for SCA and file system scanning. OSV-SCALIBR combines Google’s internal vulnerability management expertise into one scanning library with significant new capabilities such as:
SCA for installed packages, standalone binaries, as well as source code
OSes package scanning on Linux (COS, Debian, Ubuntu, RHEL, and much more), Windows, and Mac
Artifact and lockfile scanning in major language ecosystems (Go, Java, Javascript, Python, Ruby, and much more)
Vulnerability scanning tools such as weak credential detectors for Linux, Windows, and Mac
SBOM generation in SPDX and CycloneDX, the two most popular document formats
Optimization for on-host scanning of resource constrained environments where performance and low resource consumption is critical
OSV-SCALIBR is now the primary SCA engine used within Google for live hosts, code repos, and containers. It’s been used and tested extensively across many different products and internal tools to help generate SBOMs, find vulnerabilities, and help protect our users’ data at Google scale.
We offer OSV-SCALIBR primarily as an open source Go library today, and we're working on adding its new capabilities into OSV-Scanner as the primary CLI interface.
All of OSV-SCALIBR's capabilities are modularized into plugins for software extraction and vulnerability detection which are very simple to expand.You can use OSV-SCALIBR as a library to:
1.Generate SBOMs from the build artifacts and code repos on your live host:
import (
"context"
"github.com/google/osv-scalibr"
"github.com/google/osv-scalibr/converter"
"github.com/google/osv-scalibr/extractor/filesystem/list"
"github.com/google/osv-scalibr/fs"
"github.com/google/osv-scalibr/plugin"
spdx "github.com/spdx/tools-golang/spdx/v2/v2_3"
)
func GenSBOM(ctx context.Context) *spdx.Document {
capab := &plugin.Capabilities{OS: plugin.OSLinux}
cfg := &scalibr.ScanConfig{
ScanRoots: fs.RealFSScanRoots("/"),
FilesystemExtractors: list.FromCapabilities(capab),
Capabilities: capab,
}
result := scalibr.New().Scan(ctx, cfg)
return converter.ToSPDX23(result, converter.SPDXConfig{})
2. Scan a git repo for SBOMs:
Simply replace "/" with the path to your git repo. Also take a look at the various language extractors to enable for code scanning.
3. Scan a remote container for SBOMs:
Replace the scan config from the above code snippet with
...
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/osv-scalibr/artifact/image"
filesys, _ := image.NewFromRemoteName(
"alpine:latest",
remote.WithAuthFromKeychain(authn.DefaultKeychain),
ScanRoots: []*fs.ScanRoot{{FS: filesys}},
4. Find vulnerabilities on your filesystem or a remote container:
Extract the PURLs from the SCALIBR inventory results from the previous steps:
for _, i := range result.Inventories {
fmt.Println(converter.ToPURL(i))
And send them to osv.dev, e.g.
$ curl -d '{"package": {"purl": "pkg:npm/dojo@1.2.3"}}' "https://api.osv.dev/v1/query"
See the usage docs for more details.
Users looking for an out-of-the-box vulnerability scanning CLI tool should check out OSV-Scanner, which already provides comprehensive language package scanning capabilities using much of the same extraction as OSV-SCALIBR.
Some of OSV-SCALIBR’s capabilities are not yet available in OSV-Scanner, but we’re currently working on integrating OSV-SCALIBR more deeply into OSV-Scanner. This will make more and more of OSV-SCALIBR’s capabilities available in OSV-Scanner in the next few months, including installed package extraction, weak credentials scanning, SBOM generation, and more.
Look out soon for an announcement of OSV-Scanner V2 with many of these new features available. OSV-Scanner will become the primary frontend to the OSV-SCALIBR library for users who require a CLI interface. Existing users of OSV-Scanner can continue to use the tool the same way, with backwards compatibility maintained for all existing use cases.
For installation and usage instructions, have a look at OSV-Scanner’s documentation here.
In addition to making all of OSV-SCALIBR’s features available in OSV-Scanner, we're also working on additional new capabilities. Here's some of the things you can expect:
Support for more OS and language ecosystems, both for regular extraction and for Guided Remediation
Layer attribution and base image identification for container scanning
Reachability analysis to reduce false positive vulnerability matches
More vulnerability and misconfiguration detectors for Windows
More weak credentials detectors
We hope that this library helps developers and organizations to secure their software and encourages the open source community to contribute back by sharing new plugins on top of OSV-SCALIBR.
Get our FREE eBook "10 Programming Tips That Changed Everything" when you subscribe!
No spam. Unsubscribe anytime.
Follow