Android users can choose from more than 2.7 million apps in the Google Play Store — a daunting number for a privacy researcher who wants to investigate if those apps comply with privacy laws.

But fear not, privacy researchers. There's a new tool in town, and it's revealed some eye-opening data about the state of privacy for Android apps.

A team of researchers from Carnegie Mellon University and Fordham University recently created the Mobile App Privacy System (MAPS), a tool that uses natural language processing, machine learning and code analysis to identify potential privacy compliance issues by inspecting apps' privacy policies and code. The researchers tested MAPS on more than a million Android apps, and presented their findings at last month's Privacy Enhancing Technologies Symposium in Stockholm, Sweden.

"The sheer number of apps in app stores, combined with their complexity and all of the different third-party interfaces they may use, make it impossible for regulators to systematically look for privacy compliance issues," said Norman Sadeh, a professor in the Institute for Software Research and principal investigator on the study. "This tool provides a system for systematically identifying potential privacy issues at scale, and can be customized to help app store operators or regulators focus on issues relevant to specific privacy regulations."

The tool also allows users to filter privacy results to focus on, such as apps with more than a certain number of downloads, specific categories of apps or particular types of potential compliance issues.

To analyze the state of privacy for Android apps, the team used MAPS to analyze 1,039,003 apps downloaded from the Google Play Store. The analysis took the tool about three weeks, working at an average rate of 2,023 apps per hour. The researchers found that nearly half of the apps did not have privacy policy links, despite most of them (89%) suggesting that their code engages in at least one practice that would require disclosure under a particular jurisdiction.

"When policies do exist, many seem to inaccurately portray the practices performed by the app," said Sadeh, who is also affiliated with Carnegie Mellon's CyLab Security and Privacy Institute. "For example, 12% of apps' policies did not seem to accurately describe how the app is handling your location data."

Sadeh cautioned that these results require further manual vetting, because not all potential compliance issues are necessarily actual violations. For instance, code that may appear to share sensitive information with third parties may not actually be executed.

"For practical reasons, we were only able to fully vet a tiny fraction of our results, but many of those results that were checked proved to correspond to actual compliance issues," Sadeh said. "In particular, the tool was used as part of a project with a large European electronics manufacturer to check several of their mobile apps for compliance with the European General Data Protection Regulation (GDPR)."

On average, the researchers found about three potential privacy compliance issues per app. They also found that while newer apps were more likely to have privacy policies, they also had more potential issues than older apps.

"Overall, we found that Google's efforts to push developers to post privacy policies may not be enough," Sadeh said. "Developers may not be able or willing to adequately describe their apps' behaviors without proper tools and incentives."

Sadeh further noted that this particular study was conducted just before the GDPR took effect. Under GDPR, companies are subject to more stringent disclosure requirements and face steeper penalties for not complying.

"One can hope that with GDPR, the number of compliance issues will diminish over time," Sadeh said. "At the same time, our research as well as that of others suggests that many app developers simply lack the sophistication and the resources necessary to be fully compliant. This is an area where, in my view, App Store operators should be more proactive and provide additional support to app developers."

Other researchers on the study included former CMU computer science postdoctoral associate Sebastian Zimmeck; graduate students Peter Story, Daniel Smullen, Abhilasha Ravichander and Ziqi Wang; and Fordham University law faculty members Joel Reidenberg and N. Cameron Russell.