IBM scraps facial recognition citing racial profiling concerns

In the wake of Black Lives Matter protests, the computing firm is canning work with the technology.
9 June 2020

A display shows a facial recognition system for law enforcement during the NVIDIA GPU Technology Conference. Source: AFP

  • In an open letter to Congress, IBM’s CEO has announced the firm is ceasing its development of facial recognition
  • The key announcement comes amid ongoing Black Lives Matter protests around the world
  • Sources have questioned the motivation behind the decision, with IBM’s facial recognition technology business yet to take off

IBM has announced it is pulling out of the facial recognition market, and is now urging a “national dialogue” on how the technology is used by law enforcement. 

The announcement comes as technology companies face growing scrutiny over their contracts with authorities, who in the US have been criticized for their use of force in disbanding peaceful protests following the death of George Floyd. 

Facial recognition technology – while already bringing potential benefits for private businesses and their customers – has been criticized for racial and gender bias, use in racial profiling, and its invasion of citizen privacy when used for mass surveillance. 

In 2018, a study by MIT found that while determining gender using three different facial recognition programs, the error rate for light-skinned men was 0.8%, while darker-skinned women were misgendered 20% to 34% of the time.

And while many of us use facial recognition every day to unlock our iPhones, in China, the government is claimed to have used the technology to track and aid in the detention of Uighurs, a largely Muslim minority.

Announcing that the firm “no longer offers general purpose IBM facial recognition or analysis software,” Arvind Krishna, the computing giant’s CEO explained the company’s decision to end its facial recognition initiatives in a public letter to Congress

“IBM firmly opposes and will not condone uses of any technology, including facial recognition technology offered by other vendors, for mass surveillance, racial profiling, violations of basic human rights and freedoms, or any purpose which is not consistent with our values and Principles of Trust and Transparency.

“We believe now is the time to begin a national dialogue on whether and how facial recognition technology should be employed by domestic law enforcement agencies,” Krishna wrote. 

The IBM CEO continued to call artificial intelligence a “powerful tool” that can help law enforcement keep citizens safe, but urged “shared responsibility” in ensuring AI is tested for bias, especially when used by law enforcement, and that bias testing is “audited and reported.” 

Krishna also said national policy should “encourage and advance uses of technology that bring greater transparency and accountability to policing, such as body cameras and modern data analytics techniques.”

IBM last year opposed an outright ban on facial recognition’s use by law enforcement, calling instead for “precision regulation” to protect privacy and civil liberties. It said policymakers should understand that “not all technology lumped under the umbrella of ‘facial recognition’ is the same,” and that “casting such a wide regulatory net runs the very real risk of cutting us off from the many — and potentially life-saving — benefits these technologies offer.” 

That followed a direct address to Congress by Microsoft calling for regulation of facial recognition over its “societal ramifications and potential for abuse” and potential to contribute to a “dystopian” future.

In light of recent Black Lives Matter protests, with Predator drones circling the skies of Mineappolis, individuals and organizations alike are facing pressure to take a yet more definitive stance.

Amazon, for one, continues to work with law enforcement. Customers of its facial recognition software Rekognition – which has been criticized for a dangerous lack of accuracy – include US government agencies like Immigration and Customs Enforcement (ICE). Amazon’s Ring smart doorbell platform has also partnered with hundreds of law enforcement agencies, offering departments access to its platform in exchange for outreach to residents. 

Amazon’s founder Jeff Bezos has been vocal in his support of the Black Lives Matter movement, and shared a “sickening response” to that stance on his Instagram account, but has also refused to answer questions on the use of its facial recognition technology in policing protests – a dichotomy leading to headlines like this one

Multiple sources, however, have questioned the motive behind IBM’s move to halt its work with facial recognition, arguing that the technology has not been making the company much money and that it has so far not really been a contender behind even the sub-par products of Amazon. IBM’s caveat of “general purpose” facial recognition, meanwhile, doesn’t count it out from developing and selling, or reselling, the technology for specific use cases. 

An IBM spokesperson told CNN Business that the company will now limit its visual technology to “visual object detection,” which could, for example, help manage manufacturing plants or assist farmers with crop care.