Over the past few months, a fierce debate has erupted in Silicon Valley over whether large technology companies like Amazon, Google and Microsoft should join forces with the United States military, along with agencies like Immigration and Customs Enforcement.
The debate has largely been conducted along ethical lines.
On one side are tech executives and many government officials, who argue that at a time when advanced technologies like artificial intelligence and machine learning are poised to reshape top issues like drone warfare or border security, American tech giants have a patriotic duty to pitch in.
Jeff Bezos, Amazon’s chief executive, summed up this view last year: “If big tech companies are going to turn their back on the U.S. Department of Defense, this country is going to be in trouble.”
On the other side are groups of employees at those companies, including many anti-Trump progressives, who don’t want their tools to be used for drone warfare, immigrant detention and other projects they consider immoral. This side took a stand at Microsoft last month when a group of employees wrote an open letter to the company’s top executives demanding that they abandon an Army contract that would adapt HoloLens, the company’s augmented-reality headsets, for use by soldiers on the battlefield.
“We did not sign up to develop weapons, and we demand a say in how our work is used,” the Microsoft employees wrote.
This is a debate worth having. But there is a more pragmatic question swirling around it, one surprisingly few people are asking. Namely: Could Big Tech’s decision to pursue controversial defense and law enforcement contracts be a financial mistake?
As business deals, some of these contracts seem like no-brainers. Microsoft’s HoloLens deal is worth about $500 million — less than 1 percent of the company’s 2018 revenue, but a meaningful sum nonetheless. Amazon, IBM, Microsoft and Oracle are all battling over a Defense Department cloud-computing contract, known as Project JEDI, that will be worth as much as $10 billion to the winning bidder.
But these contracts may be less lucrative than they appear. And, in fact, they could come with enormous hidden costs in the form of damaged reputations, recruiting problems and customer boycotts that could swamp any short-term gains.
To explain why, let’s consider one of the most notorious military contracts of the past century.
In 1965, Dow Chemical, a Michigan-based chemicals manufacturer, was awarded a $5 million Department of Defense contract to produce napalm, a highly incendiary chemical used by American troops during the Vietnam War.
Listen to ‘The Daily’: Silicon Valley’s Military Dilemma
Should Big Tech partner with the Pentagon? We examine a cautionary tale.transcript
Listen to ‘The Daily’: Silicon Valley’s Military Dilemma
Hosted by Michael Barbaro, produced by Rachel Quester and Clare Toeniskoetter, and edited by Lisa Tobin
Should Big Tech partner with the Pentagon? We examine a cautionary tale.
michael barbaro
From The New York Times, I’m Michael Barbaro. This is “The Daily.” Today: Across Silicon Valley, tech companies are pursuing contracts with the U.S. military. Kevin Roose on a cautionary tale of where that can lead. It’s Wednesday, March 6.
kevin roose
So I’m a tech columnist. I spend a lot of time talking with and looking at the big tech companies and where they’re going. And one of the most interesting threads that’s come up in the past year or so is this question of how Silicon Valley should engage with the government, and with the military, specifically. And I think we’re at a point right now where the government is saying, we want the things that you’re building. We want you to be part of our modernization efforts.
michael barbaro
Mm-hmm.
kevin roose
And within some of these companies there’s a growing divide between the management of the companies and what they want for their business, and some of the people actually building this technology — the engineers who are working on these advanced systems that are in hot demand by the military and other government agencies. And we just saw this play out in a pretty meaningful way at Microsoft.
michael barbaro
What happened there, exactly? So last year Microsoft signed a contract with the Department of Defense to use a technology called HoloLens in the Army.
archived recording
Good morning, everyone. A few years ago, we started asking ourselves, could we make things so simple that technology would just disappear?
kevin roose
And HoloLens was originally developed — it’s kind of like a headset for augmented reality.
archived recording
Could we place your digital content right into your world, right into your life, with more reality than ever before?
kevin roose
I saw a demo of this a couple years ago, and they were touting it as the next generation of gaming.
michael barbaro
Mm-hmm.
archived recording
Welcome to a new era of Windows. Welcome to Windows Holographic.
kevin roose
People were wearing HoloLens, and you would be swinging your imaginary sword at imaginary monsters. But it all looked like it was happening right there in your living room.
michael barbaro
Mm-hmm.
kevin roose
And last year, Microsoft signed a contract to develop HoloLens technology for use on the battlefield — to give soldiers the ability to wear these headsets and get information about their surroundings, and to increase the lethality of the soldiers on the battlefield.
michael barbaro
So this technology will specifically be deployed within the military so that American soldiers are better at killing the enemy?
kevin roose
That’s the way it’s described in the contract, yes.
michael barbaro
And so, I’m guessing that this was something that Microsoft engineers objected to?
archived recording
And then there’s Microsoft. They’re receiving backlash from their own employees over new groundbreaking technology.
kevin roose
Well, a few of them did. But they did it pretty vocally.
archived recording
Yeah, so more than 150 Microsoft employees signed a letter demanding that the company cancel a $480 million contract to build a HoloLens for the Pentagon saying they, quote —
kevin roose
“Dear Satya Nadella and Brad Smith: We are a global coalition of Microsoft workers, and we refuse to create technology for warfare and oppression.” And it goes on. And basically, they argue that this crossed the line into weapons development — that this was basically taking something that they had built, and turned it into something that was designed to help soldiers on the battlefield kill other soldiers.
michael barbaro
Not what they decided to do.
kevin roose
Right. Not their intent, not what they were told they were going to be working on, and not something they were comfortable with.
michael barbaro
And Kevin, thinking about this, it’s hard to imagine that the U.S. military would not use U.S. technology, American-made technology like this, in warfare. That seems inevitable, doesn’t it?
kevin roose
Well, right. I mean as long as there has been a U.S. tech industry, there’s been a partnership with the U.S. government and with the military. I mean, the original internet came out of a Defense Department project.
michael barbaro
Mm-hmm.
kevin roose
And since then, lots of advances in technology have been spurred by this collaboration between the Pentagon and Silicon Valley. And a lot of these companies already provide technology to the military. I mean, a lot of military computers run Windows’s operating system. But this was a bridge too far for these employees. They say, we’re O.K. basically providing general-use technology, the same stuff that you and I could go and buy. But the specific development of our technology for use in this specific case crosses a line for us, and we don’t want to be a part of it.
michael barbaro
So Kevin, what’s interesting to you about this story?
kevin roose
Well, I’m interested in the ethical debate around the use of artificial intelligence and other technologies. But I’m also a business reporter. I think this is a decision that is not just being made along ethical lines. These companies are also thinking about their businesses and the amount of money and the profitability of developing these systems for the military. So I wanted to look at it along those lines, as well. Let’s just bracket the ethical discussion for a second and say, does this make financial sense?
michael barbaro
Well, not just bracket it, right? Isn’t a company often going to make decisions about moral dilemmas based on the implications to their finances?
kevin roose
Exactly. These are not philanthropies or think tanks. These are for-profit corporations that generally operate on a cost-benefit calculation. And so just on that plane, I wanted to see, does this make sense for them? [MUSIC] And so, I did a little bit of digging, and I went back about 50 years in history, and I found an example of a company that sold something to the military and later came to regret it.
michael barbaro
Which was what?
kevin roose
It’s the case of Dow Chemical. In 1965, Dow Chemical is this small, little-known chemicals company based in Michigan. They make lots of different things — household chemicals, agricultural chemicals.
archived recording
Now, at your favorite grocery store, the most amazing food wrap ever developed!
kevin roose
They were probably best known for making Saran Wrap.
michael barbaro
Heard of it.
archived recording (speaker 1)
Have you tried Saran Wrap? It really is amazing! And look, Saran Wrap clings like magic!
archived recording (speaker 2)
Saran Wrap is a product of the Dow Chemical Company.
archived recording (speaker 3)
[JET PLANES AND EXPLOSIONS] Vietnam. United States helicopter gunships backed up ground forces in a strong assault on a Viet Cong position —
kevin roose
Meanwhile, the Vietnam War is happening. And in 1965, Dow Chemical wins a contract with the Department of Defense to produce a new chemical called napalm B. And napalm, which is a highly flammable gel that binds and sticks to things and then burns them — had been in use. But napalm B was the new formulation, and the military was very excited in using this in Vietnam.
archived recording
Troops followed up the advantage gained by the air support to knock out the V.C.
kevin roose
They thought it had a lot of promise on the battlefield. It was hard to contain and hard to put out. It was a very effective form of weaponry — a very horrible form of weaponry. And it produced very, very bad burns on people. They would basically be burned alive. And so, there’s not much noise about this for the first little while. And then Americans start seeing images from the Vietnam War. And for a lot of people, this is their first time seeing the effects of napalm.
michael barbaro
Mm-hmm.
kevin roose
Most famously, there’s this photograph — you’ve probably seen it. It won the Pulitzer Prize. It is a photograph of a 9-year-old girl who’s been badly burned by a napalm weapon.
archived recording (martin luther king jr.)
So far we may have killed a million of them, mostly children.
kevin roose
And it sort of shocks the American public, and it galvanizes the anti-war movement.
archived recording (martin luther king jr.)
What do they think as we test out our latest weapons on them, just as the Germans tested out new medicine and new tortures in the concentration camps of Europe?
archived recording
[CROWDS CHANTING] No more napalm! No more napalm! No more napalm! No more napalm!
kevin roose
And they decide to go after the use of napalm by U.S. armed forces. And the way that they do that is by going after Dow Chemical.
archived recording
[CROWDS CHANTING]
archived recording (speaker 1)
This is Monday afternoon, November 20, on the San Jose State college campus. What you’re watching is the result of a demonstration that began about an hour ago.
kevin roose
They staged demonstrations outside the company’s headquarters, and they go to dozens of colleges where Dow was recruiting students to come work for them.
michael barbaro
Hm.
archived recording
The professors against the war and the Students for a Democratic Society had marched in front of the administration building to protest the recruitment on this campus by Dow Chemical Company of employees.
kevin roose
And they protest the recruiting sessions. They have signs like “Dow deforms children,” “Dow deals death.”
archived recording
They’re not allowing anyone in. Like, the only people they’re allowing in are those people that are connected with Dow. [PROTESTERS SHOUTING]
kevin roose
My favorite story from this period is that the Dow recruiters — the people that would go to campus to talk to the students — they would get locked up in classrooms. They would get, like, held hostage by the activists. And at one point, one of the recruiters got so used to this that he started bringing a sandwich in his briefcase, so that if the inevitable happened and he got locked in a room with a bunch of angry protesters —
michael barbaro
He would survive the day.
kevin roose
He would at least have lunch.
michael barbaro
So this sounds like a complete corporate nightmare — that your company is so associated with war and with death, and not with the household product that you make — Saran Wrap.
kevin roose
Right. And Dow was particularly offended by this because napalm was a very small part of its business. I mean, it accounted, I think, for a half of 1 percent of its annual sales. And yet it had sort of come to take over its corporate identity. So there was some internal discussion, there were some debates within the company and at the board level about what to do, whether to stop manufacturing napalm. And there were lots of reasons for that. I mean, there were customer boycotts. They did take a financial hit. And they had to do a lot to counter their corporate reputation as a warmonger. But what they really worried about was recruiting. They worried that on these visits to college campuses they weren’t going to get the best people. They weren’t going to get the scientists and the engineers who would help them in the years ahead. And as a result, they would have long-term costs, not just financial, but also in terms of what they were missing out on.
michael barbaro
The future employee with moral objections to napalm, who doesn’t come work at Dow Chemical and make the next blockbuster product.
kevin roose
Right. But ultimately, they decided to stand their ground. They said, we have a duty to produce napalm for the U.S. government. It’s our patriotic obligation, and we can withstand the controversy.
michael barbaro
Dow Chemical feels like such an extreme case, and it feels quite different from what’s going on with these tech companies today.
kevin roose
Yeah, absolutely. These are not totally comparable. I mean, the Vietnam War, at this point, was extremely unpopular. You had these horrific images coming back. There’s no good use for napalm, right? There’s no pro-social, healthy use for napalm. It’s a weapon. Whereas a lot of these technologies — A.I., facial recognition, image recognition, augmented reality — these can all be used for very productive and healthy things. It’s only when you put them into a certain context that they can be used for harm.
michael barbaro
And that would seem to be the case for most tech companies, right? They are developing products not explicitly for the military. They just happen to be used that way.
kevin roose
Exactly.
archived recording
Artificial intelligence, drones, warfare and Google — it’s a mixture that caused an uproar inside the tech giant, where the early motto was “Don’t be evil.”
kevin roose
There was a big controversy at Google last year, about this military contract known as Project Maven.
archived recording
Google is developing artificial intelligence to analyze drone video data.
kevin roose
And this was a Defense Department program that basically used A.I. and image-recognition technology to interpret video images.
michael barbaro
Mm-hmm.
kevin roose
So the same kind of A.I. that Google would use in Google Photos, or in its self-driving car unit to recognize images out on the road —
archived recording
It would apparently be used as part of a drone program.
kevin roose
— could be used to, for example, improve the targeting of drone strikes to make drone strikes more accurate, to be able to recognize certain people or certain buildings and be able to direct a drone at that target specifically.
michael barbaro
Again, not what Google engineers probably ever thought the technology would be used for.
kevin roose
Exactly, I mean, they didn’t sign up for that. And some of them didn’t even know that this was happening.
archived recording
Thousands of Google workers now protesting Google’s involvement in a Pentagon drone program.
kevin roose
So there was a big uproar. There was lots of heated internal meetings.
archived recording
At least 10, maybe 12 employees did resign.
kevin roose
Some resignations —
michael barbaro
Wow. People quit the company over it.
kevin roose
Yeah, people quit the company over it.
archived recording
Meantime, thousands of Google employees have signed a letter protesting the company’s participation in an artificial intelligence project by the Pentagon.
kevin roose
There was a petition that was signed by thousands of Google employees.
archived recording
Quote, “We believe that Google should not be in the business of war. Therefore we asked that Project Maven be canceled.”
kevin roose
Calling for the company to not only not enter into this contract, but to stop making tools of weaponry altogether. And ultimately —
archived recording
Tech giant Google will end a project with the Pentagon.
kevin roose
Google bowed to the pressure. They pulled the contract. They said they wouldn’t renew it. They put out a statement saying, we’ll still work with the military on other projects, but we’re not going to manufacture weapons and weapons-related technology. And that seemed to quiet the unrest. [MUSIC]
michael barbaro
I feel like people would be surprised that a huge tech company like Google would back down like that.
kevin roose
Yeah, I mean, it does speak to the difference between a conventional contractor like Dow Chemical and these tech companies. I think there are three main differences there. The first is that these companies are beloved. They have loyal followings and they don’t want to do anything to upset that. The second is that these companies are very idealistic. And in Silicon Valley, this is a big deal. Companies market themselves not just as profitable enterprises, but as humanitarian projects. And anything that compromises that hurts them.
michael barbaro
Right. World-changers.
kevin roose
Totally. And I think the biggest difference is that employees at these companies — especially the engineers who work on very specialized projects like A.I. development — have a ton of leverage, because there are not very many of them, they make a ton of money, and they’re really hard to recruit. If a couple of chemical engineers walk out of Dow Chemical in protest, they can probably find some more where that came from. But if hundreds or thousands of A.I. engineers are upset about a project that Google is working on, that’s a real threat to their business. It doesn’t take many people staging a protest, or threatening to walk out to really get the attention of senior leadership.
michael barbaro
So these three differences that you just laid out explain why Dow didn’t back down, despite images of dying children affected by their products, whereas Google did back down, long, long before anything like that happened to it.
kevin roose
Right. And I want to be clear: What happened at Google was the exception, not the rule. This is not happening in any vast quantity across the tech industry. At Microsoft, for example, they came right out and said, we appreciate the objections of this small group of employees. They can move elsewhere in the company if they want, but we’re committed to seeing this contract through. And they’ve defended themselves by saying, we have a patriotic duty to provide technology to the military. We think that we can advocate for the ethical use of technologies like AI if we’re at the table in these discussions. And they say that basically, these technologies could be good. These could save lives on the battlefield. The army has said it wants to use HoloLens not just for combat training, but for things like measuring vital signs, and monitoring soldiers for concussions, and having some sort of hearing protection on people’s ears. So those are the kinds of arguments they’re making to justify staying in these contracts.
michael barbaro
So the argument here being that engineers who came to these companies to do good, having their technology used by the military does not necessarily mean not doing good.
kevin roose
Yeah, I mean, that’s one of the arguments that they’re making, is that, basically, if the U.S. military doesn’t have the best technology available on the market, our adversaries will, and we will be endangered as a result.
michael barbaro
I guess the question is, do the engineers buy that argument?
kevin roose
Well, some of them probably do. But as we’ve seen, a number of them don’t. [MUSIC]
michael barbaro
So Kevin, we started by talking about this intersection of the moral and the financial as these tech companies navigate this question of, should they work with the military? What’s your understanding in Silicon Valley about how these companies are thinking about that?
kevin roose
So I think a lot of these companies have really shifted their view on this in the last couple of years. A lot of Silicon Valley companies used to just think of themselves as toolmakers, right? We build this stuff, and we put it out into the world, and people use it. And our responsibility, basically, stops at the moment of sale.
archived recording (mark zuckerberg)
To me, entrepreneurship is about creating change, not just creating companies.
kevin roose
And now, I think we’re seeing that these companies are being held responsible by the public and by their own employees, not just for the tools they’re building, but for how they’re being used in the world. And this is not just about the military.
archived recording
At this hour some 300 programmers are threatening to leave Microsoft unless the tech giant drops its contract with the U.S. Immigration and Customs Enforcement agency, known as ICE.
kevin roose
I mean, also, companies are being scrutinized for working with ICE or working with law enforcement.
archived recording
An Amazon worker is pushing the company to stop selling facial recognition technology to law enforcement.
kevin roose
Social media companies are being held responsible for bad things that happen on their platforms.
archived recording
Facebook defending Alex Jones this morning, saying the platform will continue to allow the video blogger to run video on its site, even as it claims to crack down on fake news.
kevin roose
This is really happening across technology. And as technology embeds itself into every part of our lives, we’re seeing that these companies are having to make new kinds of decisions. It’s not just a dollars-and-cents profit-and-loss calculation anymore. They also have to consider what might happen out in the world once they release these technologies.
michael barbaro
Which is another way of saying that the morality element of this is now playing a much bigger role than it has in the past, and perhaps is not beating out financial, but rivaling it?
kevin roose
It’s certainly part of the discussion now in a way it hasn’t been before. It’s not just about how much money we’ll make, or how many resources it will take. It’s about what we’re building, and why we’re building it, and who we’re selling it to. [MUSIC]
michael barbaro
Kevin, thank you very much.
kevin roose
Thank you for having me.
Dow was not known as a defense contractor — in fact, until its Pentagon contract, the business was best known for making industrial chemicals and household plastics like Saran Wrap.
But over the next few years, as Americans began seeing gruesome images of South Vietnamese children with horrific napalm burns, the antiwar movement set its sights on the company.
Activists boycotted Dow Chemical’s products, staged protests at its recruiting events on college campuses and barraged its executives with accusations of unethical war profiteering. (One demonstration outside the company’s annual shareholder meeting featured signs like “Dow Know-How in Every Drop of Napalm.”) Dow Chemical executives dug in their heels. They claimed that napalm was a small part of the company’s overall business, and that it was the company’s duty to provide necessary materials to the military.
But the company’s pleas of patriotism were drowned out by the antiwar opposition. As Robert M. Neer explains in his book “Napalm,” Dow Chemical’s reputation plummeted as a result of its napalm contract. Its recruiting ability suffered, and its marketing department was forced to embark on a long and expensive campaign to win back the public’s trust.
“We went from being a company that made Saran Wrap to keep food fresh to a kind of war machine,” the company’s former chief executive Andrew Liveris said in 2006.
Dow Chemical stopped making napalm for the military in 1969, just four years after it had begun. But the reputational damage haunted the company for decades. (Dow Chemical merged with DuPont in 2017; the company is still a defense contractor, but it produces mainly agricultural and industrial chemicals.)
All told, the $5 million napalm contract most likely cost Dow Chemical billions of dollars. And it was the kind of unforced error that could have been avoided if company executives had listened to early signs of opposition, done some risk analysis and changed course.
Today’s biggest tech companies are in a similar spot. Many of them, such as Amazon and Microsoft, are among the most beloved brands in the world. They employ lots of conscientious, idealistic engineers whose skills are highly valuable, giving them considerable leverage in discussions about company values. And they are operating in an era of heightened consumer sensitivity — in which one misstep can tarnish a brand for years.
Tech companies aren’t making anything as obviously harmful as napalm. In fact, supporters of these deals argue that some of the technologies being offered to the military, such as image-recognition algorithms that can help better target drone strikes, could save civilian lives.
But the truth is that tech companies have absolutely no idea how the government will use their products in the future — and how the political landscape might shift, throwing them into an unwanted spotlight.
In some ways, Silicon Valley tech companies face the same choice Dow Chemical faced in 1965: Accept controversial government contracts and risk a damaging backlash, or leave those deals to conventional defense contractors and protect their reputations.
Already, there are signs of trouble on the horizon. At Stanford, fliers recently appeared on campus walls urging students not to work for Amazon, Microsoft, Palantir and other companies with reported contracts with ICE and law enforcement agencies. And artificial intelligence experts caution that the stigma of being seen as a war profiteer could repel idealistic recruits for years to come.
“Top A.I. talent doesn’t want to work for Lockheed Martin,” said Jack Clark, the policy director of OpenAI, a nonprofit artificial intelligence lab.
In fact, in today’s corporate operating environment, turning down controversial military and government contracts could be a selling point.
Recently, I talked with Rana el Kaliouby, a co-founder of Affectiva, an “artificial emotional intelligence” start-up. Ms. el Kaliouby, whose software uses A.I. to track human emotions, had several early offers from government agencies, including a venture fund backed by the Central Intelligence Agency, that wanted to use the product to improve their surveillance capabilities.
Even though the company needed the money at the time, it turned down the deals. Affectiva has since raised more than $50 million from other, nongovernmental sources, and has made ethical A.I. use a core part of its brand.
“We wanted to be trusted,” Ms. el Kaliouby said. “We used the core value of integrity and respecting people’s privacy as a way to weed out use cases.”
Take note, tech giants. Turning down controversial military and government contracts won’t doom your business. In fact, in the long run, your shareholders might thank you.