UPDATE: Apple has released official details about the scanning system, and says it’ll only target images about to be stored in iCloud Photos.
The system is closer to existing child abuse image scanning that already takes place on social media and cloud storage platforms. Nevertheless, Apple’s scanning will take place on the iPhone itself. An on-device process will check if an image uploaded to an iCloud Photos account matches any images already flagged in a database indexing known child sexual abuse material.
Apple can only decrypt and see the uploaded images if a “threshold” on how much suspected child sexual abuse material in iCloud account is crossed.
“The threshold is set to provide an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account,” the company said. “Apple then manually reviews each report to confirm there is a match, disables the user’s account, and sends a report to NCMEC (the National Center for Missing and Exploited Children.)”
Apple plans on rolling out the scanning feature in iOS 15 and iPadOS 15, which arrive this fall.
“Apple only learns about users’ photos if they have a collection of known CSAM (child sexual abuse material) in their iCloud Photos account,” the company added. “Even in these cases, Apple only learns about images that match known CSAM.”
That said, the company is implementing a second scanning system on iMessage to detect sexual explicit images, but it’ll be geared toward iPhones belonging to children. The system will also tap on-device machine-learning algorithms to detect sexual explicit images sent to and from a child’s account on the messages app.
“When receiving this type of content, the photo will be blurred and the child will be warned, presented with helpful resources, and reassured it is okay if they do not want to view this photo,” Apple explained. “As an additional precaution, the child can also be told that, to make sure they are safe, their parents will get a message if they do view it.”
However, the Electronic Frontier Foundation is no fan of the system. The privacy group is worried the same approach pokes a hole into the end-to-end encryption on iMessage.
“All it would take to widen the narrow backdoor that Apple is building is an expansion of the machine learning parameters to look for additional types of content, or a tweak of the configuration flags to scan, not just children’s, but anyone’s accounts,” the EFF wrote. “That’s not a slippery slope; that’s a fully built system just waiting for external pressure to make the slightest change.”
Original story:Apple may be working on a system to scan iPhones to detect if they’re carrying child sexual abuse imagery, according to The Financial Times.
The intent is to stop crime, but it could come at the expense of an iPhone owner’s privacy. The main concern is that Apple is creating a beachhead to monitor the activity on a consumer’s personal hardware, a step beyond simply surveilling the data on a social network, video site, or email service.
According to the Financial Times, Apple has been briefing US scholars on the upcoming system, which has been dubbed “neural match.” The technology works by warning a human reviewer if a suspected child pornographic image has been detected on an iPhone; the reviewer can then contact law enforcement.
On Wednesday, cryptography expert Matthew Green also said Apple is preparing to release a tool for child sexual abuse imagery scanning, citing his own sources. “This is a really bad idea,” he wrote in a tweet.
“This sort of tool can be a boon for finding child pornography in people’s phones. But imagine what it could do in the hands of an authoritarian government?” he added in a follow-up tweet.
Apple did not immediately respond to a request for comment. But according to Green, the company’s plan is to first use the system to scan for photos stored inside Apple’s cloud storage service. However, Green fears Apple will eventually expand the system to scan images inside iMessage, the company’s end-to-end encrypted chat system.
The technology will work by scanning for hashes, or digital signatures tied to known child sexual abuse images circulated on the internet. Specifically, the hashes are compared to photos in a database containing already flagged images of child sexual abuse. However, if the scanning isn’t smart enough, it could accidentally flag harmless images files as problematic, Green notes.
The system, if real, is already alarming security researchers over its potential for widescale surveillance. “Apple are walking back privacy to enable 1984,” tweeted Alex Muffett, a security expert who’s worked for Facebook and Sun Microsystems.
However, the system likely has one big supporter: law enforcement. The Justice Department and the FBI have been urging tech companies to roll back plans to encrypt personal devices, citing the need to obtain evidence in criminal investigations. In 2019, former US Attorney General William Barr specifically pointed to stopping child sexual predators as a reason for Facebook to stop expanding its end-to-end encrypted messaging.
