Authoritarian Tech

The peril and promise of AI-driven technology have dominated the Western tech zeitgeist in 2023 so far, with too much excitement and not enough hand-wringing around ChatGPT, the naughty little bot released late last year by Open AI, a relative newcomer with big-name Silicon Valley backers like Peter Thiel and Elon Musk. Too few questions are being asked about what Musk meant by “dangerously” when he described ChatGPT as “dangerously strong AI.” As usual, U.S. policymakers are behind the curve, watching in awe (or counting their campaign donations) as the tech moves fast into the future, with the prospect of regulation slowly plodding along behind it but thus far failing to catch up.

Across the Atlantic, though, lawmakers seem to realize that while Europe may never become a major leader in the global tech industry, it can certainly distinguish itself as the world’s de facto tech regulator. Chris Stokel-Walker wrote for us a few weeks ago about the European Union’s AI Act, a draft regulation that is intended to rein in the sale and government use of riskier AI-driven technologies. But new amendments are apparently on the table that could make the whole process voluntary and thereby defang the regulation entirely. On Twitter, Access Now AI researcher Daniel Leufer predicted that if the amendments pass, “unscrupulous” developers will simply claim that their products are not high risk and avoid the process altogether. “These amendments incentivise bad behavior [and] punish responsible developers,” he wrote.

It recently came to light that two Wikipedia editors from Saudi Arabia have been in prison since 2020. They will be there for some time. Ziad Al-Sufyani is facing eight years behind bars. Osama Khalid, a well-known Wikipedia contributor and open source software developer, has been sentenced to 32 years in prison. Both young men were also practicing physicians. There’s not much more information about Saudi Arabia’s case against either of them or why it has taken so long for their fate to become public knowledge. But the facts we do have show just how concerned Saudi authorities are with their image and the measures they’re willing to take to protect it. 

Public data experts shuddered when the Taliban retook Afghanistan and seized technical equipment that authorities used to digitize public services during the U.S. occupation. Vast stores of Afghans’ personal data were now in the hands of the Taliban. There’s a lot that we may never know about what has happened to this data or how it has been used. But what could happen might be indicated in the fact that one particular set, dating back from 2012, has found its way to Germany. Just before the new year, the New York Times reported that a digital security researcher there unknowingly purchased a database off eBay, which turned out to contain “the names, nationalities, photographs, fingerprints and iris scans of 2,632 people,” most of them from either Afghanistan or Iraq.

EAVESDROPPING IN MYANMAR

Stories of Israeli-made spyware giving states an all-access pass to people’s mobile data have become almost routine in recent years, with our attention trained mainly on NSO Group, the creator of the pernicious Pegasus spyware system that has infected the mobile devices of journalists and human rights defenders from Azerbaijan to Zambia. 

But NSO is not the only game in Tel Aviv. This week, the Israeli “investigative analytics” firm Cognyte, formerly known as Verint Systems, made headlines when court filings revealed that it won a tender to provide monitoring services for Myanmar’s state telecommunications agency, just before the 2021 military coup. Justice for Myanmar explained that the system “would allow the Myanmar military to tap calls in real time, aiding and abetting its atrocity crimes.” The sale may have been illegal to begin with — the transfer of defense equipment to Myanmar was banned in Israel 2018, largely because of the Rohingya genocide. The tender came to light after Israeli attorney Eitay Mack called for a criminal investigation of Cognyte and Israel’s defense and foreign ministries earlier this month.

The backstory here is worth a look. The 2021 coup brought a blunt end to Myanmar’s brief, fragile period of civilian rule. Foreign companies were able to enter the market, many for the first time. An early mover was Meta (then Facebook), which struck a special deal with telecom providers that effectively made Facebook a free product for anyone with a mobile phone. A technologist from Yangon once explained to me that in those early years of market liberalization, mobile phone vendors would automatically set up a Facebook account for a new customer if they were buying for the first time. For most Burmese people, she said, “Facebook is the internet.” Many readers will remember that the company then looked the other way as military leaders used Facebook to incite hatred and violence against Rohingya Muslims in the lead-up to “clearance operations” in Rakhine state. A U.N. fact-finding mission later wrote that Facebook had a “determining role” in the genocide that left an estimated 24,000 Rohingya people dead and caused more than a million to flee the country.

Another early mover in this period was Telenor, the Norwegian telecommunications company that became one of Myanmar’s leading mobile operators, giving people the option of using a service other than MPT, the state-owned telecom. But shortly after the military retook the country in the February 2021 coup, Telenor announced plans to sell its holdings there and leave altogether, citing a difficult environment. When pressed for details, an executive acknowledged that the company had been pressured to “activate intercept equipment” that went against Telenor’s policies and likely would have violated Europe’s General Data Protection Regulation. 

While Telenor eventually found its way out, and even less principled companies (including the Beirut-based M1) moved in, people in Myanmar were left with the uneasy feeling that no matter what mobile provider they chose, they could expect they were being closely watched. But until this week, there was scant public knowledge about intercept tools the state may be using on mobile networks and about how foreign tech companies might be benefiting from it along the way. It now feels like another piece of that puzzle has fallen into place. 

WHAT WE’RE READING (& LISTENING TO)

• Civil rights leader Martin Luther King, who would have turned 94 last weekend, was a top target of surveillance by the FBI, which sought to “expose, disrupt, misdirect, discredit or otherwise neutralize Dr. King and others in the movement for Black liberation.” Melissa Harris-Perry talked with the Algorithmic Justice League’s Tawanna Petty this week about King’s legacy and the fight against warrantless surveillance today. Have a listen.

• ChatGPT has apparently been asked to write a song in the style of singer-songwriter Nick Cave. On his question-and-answer blog, Cave declared that it “sucks,” calling it “a grotesque mockery of what it is to be human.”

• In a new essay for Noema, three leading researchers from the Distributed AI Research Institute offer a sobering breakdown of the human labor behind much of what we loosely refer to as AI. “Far from the sophisticated, sentient machines portrayed in media and pop culture,” they write, “so-called AI systems are fueled by millions of underpaid workers around the world, performing repetitive tasks under precarious labor conditions.”