├── English
    ├── 1-gold.md
    ├── 2-filter.md
    ├── 3-bias.md
    ├── 4-leak.md
    ├── 5-fakepoops.md
    ├── 6-ad.md
    ├── 7-psy.md
    ├── 8-throne.md
    ├── README.md
    ├── SmartPoop.epub
    ├── SmartPoop.pdf
    ├── compile.sh
    ├── template.tex
    └── title.txt
├── French
    ├── 1-L'or.md
    ├── 2-Filtrer.md
    ├── 3-Biais.md
    ├── 4-Fuites.md
    ├── 5-FakePoops.md
    ├── 6-Pub.md
    ├── 7-Psy.md
    ├── 8-Trone.md
    ├── README.md
    ├── SmartPoop-fr.epub
    ├── SmartPoop-fr.pdf
    ├── compile.sh
    ├── template.tex
    └── titre.txt
├── README.md
├── SmartPoop.jpg
├── chapter_ideas.md
└── compile_all.sh


/English/3-bias.md:
--------------------------------------------------------------------------------
  1 | # 3. The turd bias
  2 | 
  3 | > True comfort is not just knowing that you can relieve yourself safely; it is knowing that your health will be taken care of at all times thanks to the intelligence of your toilet. SmartToilets[^smarttoilets] from SmartPoop. Let us take care of you.
  4 | 
  5 | [^smarttoilets]: SmartToilets are already in development! They even led to an academic publication, with a prototype capable of... rectal recognition!  
  6 | [**Paper.** A mountable toilet system for personalized health monitoring via the analysis of excreta. Seung-min Park et al. Nature Biomedical Engineering (2020).](https://www.nature.com/articles/s41551-020-0534-9)
  7 | 
  8 | This is the fifth time today that Issa Gueye has seen this ad on his phone. Ironically, this time, he is sitting on the toilet[^toilets], between two videos of comedians, that he is exposed to it. A few minutes later, when he opens the SmartPoop application to film his excrement, he realizes that installing SmartToilets would save him the trouble of filming the turds himself. Even though he's been doing it almost daily for three years now, he still finds the effort laborious and repulsive.
  9 | 
 10 | [^toilets]: The invention of toilets, and in particular sewage systems, was in fact one of the great advances in the history of civilization, as it effectively fought the spread of many diseases such as cholera. In fact, more than 2 billion people still do not have access to it. It is estimated that, because of this, about 800,000 children die every year from diarrhea.  
 11 | [**Video.** How The Toilet Changed History. It's Okay to Be Smart (2017).](https://www.youtube.com/watch?v=GWQG1YZS9l4)
 12 | 
 13 | Issa, the avid sportsman who likes to track his fitness, then clicks on the special offer offered by his app. As a regular user since the beginning of SmartPoop, he indeed has a 30% discount on the purchase of a basic SmartToilet, for a total price of 12,000 euros. It's expensive, but this finance trader can afford it. In fact, Issa then discovered the existence of a SmartToilet Deluxe version. This version allows pre-heating of the bowl, odor filtration by ceramic honeycomb filter, cleaning by fine water jet and drying by hot air, all optimized by artificial intelligence algorithms to maximize the user's well-being[^toilet-japanese]. Intrigued, Issa opts for this product, despite its exorbitant price of 25,000 euros.
 14 | 
 15 | [^toilet-japanese]: In Japan, toilets often have several of these features.  
 16 | [**Video.** Why You Need to Try a High-Tech Japanese Toilet. Lifehacker (2019).](https://www.youtube.com/watch?v=GZRwI7m7gnw)
 17 | 
 18 | ### Poo
 19 | 
 20 | A few days later, SmartPoop's SmartToilet Deluxe is delivered and installed at Issa's home. That evening, Issa went to test his new toilet with a big smile and some hesitation. As he approaches the toilet, the lid opens automatically. Issa then sits down on the bowl. A strange feeling of warmth accompanies his contact with the bowl. "It's like I'm using the toilet right after someone else... Not very reassuring. But I should probably just get used to it," he says to himself.
 21 | 
 22 | > Hello Issa, says a male voice.
 23 | 
 24 | Issa is startled. The toilet has just spoken to him!
 25 | 
 26 | > Relax and enjoy the comfort of the SmartToilet Deluxe, she explains. I'm Poo, your voice assistant. Would you prefer I use a female voice[^female-voice]?
 27 | 
 28 | [^female-voice]: After many criticisms about reinforcing gender bias, Apple decided to give Siri a male voice by default.  
 29 | [**Journal.** How AI bots and voice assistants reinforce gender bias. Caitlin Chin and Mishaela Robison. Brookings (2020).](https://www.brookings.edu/research/how-ai-bots-and-voice-assistants-reinforce-gender-bias/)  
 30 | [**Journal.** Apple's Siri is no longer a woman by default, but is this really a win for feminism?. Eleonore Fournier-Tombs. The Conversation (2021).](https://theconversation.com/apples-siri-is-no-longer-a-woman-by-default-but-is-this-really-a-win-for-feminism-164030)
 31 | 
 32 | Issa is very disturbed. After a few seconds, he answers: "Yes, I prefer a female voice".
 33 | 
 34 | > Feel free to tell me how you feel so I can optimize your comfort, says Poo, now with a female voice[^individual-societal].
 35 | 
 36 | [^individual-societal]: This example highlights the tension between users' control over the technologies they use and the consequences that this control can have on a social scale. Thus, by allowing the user to customize their technologies, there is a risk that they aggravate, consciously or not, gender bias (in this case), hate speech or misinformation. In the end, this tension resides in John Stuart Mill's harm principle; or more simply put, in the principle "the freedoms of some end where those of others begin".  
 37 | [**Video.** The Harm Principle: How to live your life the way you want to. BBC Radio 4 (2014).](https://www.youtube.com/watch?v=R9IM3ZKNMCk)
 38 | 
 39 | Disturbed, Issa remains silent for a few more seconds. He finally replies: "Perhaps we can start by being on a first-name basis.
 40 | 
 41 | > With pleasure, Issa. I am at your service.
 42 | 
 43 | Issa's phone suddenly vibrates. He picks it up. It reads: "Congratulations! You are now using our SmartToilet Deluxe. We wish you a pleasant experience". Still tense, Issa finds it hard to let go.
 44 | 
 45 | > The first time is never easy. But relax. Try to enjoy your new daily comfort.
 46 | 
 47 | > I'm not used to talking to a voice assistant in the toilet!
 48 | 
 49 | > I understand. I'm not yet used to talking to toilet users either!
 50 | 
 51 | Little by little, however, Issa is increasingly perceiving his SmartToilets as an important new comfort[^new-toilets]. After all, put together, humans spend an enormous amount of time sitting on a toilet. Might as well make a good time of it!
 52 | 
 53 | [^new-toilets]: Billions of humans on earth don't have access to this comfort. As we have seen, this poses serious health problems.  
 54 | [**Video.** 6 Toilets From History, and What They Taught Us. SciShow (2021).](https://www.youtube.com/watch?v=kTqDmkTh0IY)
 55 | But research is developing solutions to make this comfort available to more people, including innovations in excreta treatment.  
 56 | [**Video.** 3 Groundbreaking New Toilets. SciShow (2018).](https://www.youtube.com/watch?v=FaLZUJAAgnk)
 57 | 
 58 | Finally, Issa moves on with his task.
 59 | 
 60 | > Uh... Issa says, with some hesitation. Are you... analyzing it?
 61 | 
 62 | > Yes Issa, I am filming your "data" from different angles, and putting the videos on your online account. I am also taking other samples for further analysis. Do you want to know more?
 63 | 
 64 | > Yes, tell me more.
 65 | 
 66 | > With pleasure. The water in which the excreta is bathed is also constantly sampled and analyzed by spectroscopy. In other words, we study the way this water absorbs different colors of the light spectrum, in a very precise way. This allows us to identify certain molecules within your feces, which enables us to do a much more accurate health check-up[^spectroscopy]. We also perform mass spectrometry[^spectrometry], and chemical measurements, for example of the acidity of this water[^pH].
 67 | 
 68 | [^spectroscopy]: [**Video.** Spectrophotometry and Beer's Law. Professor Dave Explains (2019).](https://www.youtube.com/watch?v=zuUvQN8KXOk)
 69 | 
 70 | [^spectrometry]: Mass spectrometry involves slicing a molecule to be analyzed into charged sub-molecules, for example using electron ionization, and measuring the ability of a magnetic field to deflect the path of sub-molecules. Since heavier submolecules are harder to deflect, this gives us information on the mass of these submolecules, and thus on the mass and composition of the molecule to be analyzed.  
 71 | [**Video.** IR Spectroscopy and Mass Spectrometry: Crash Course Organic Chemistry #5 (2020).](https://www.youtube.com/watch?v=xMa1BQ8z9C0)
 72 | 
 73 | [^pH]: The acidity of a liquid is measured by its pH, typically via electrical measurements.  
 74 | [How a pH meter works! pH Professor (2017).](https://www.youtube.com/watch?v=PBTn4gTEbkU)
 75 | 
 76 | > When will I have results?
 77 | 
 78 | > I'll keep you posted, in the bathroom or in your app, if there are any measurements of concern.
 79 | 
 80 | ### "Very concerning."
 81 | 
 82 | A few days later, when he has just returned home after a hard day's work. Issa receives a notification from SmartPoop. Issa then asks, "Poo, what's going on?"
 83 | 
 84 | > I've been running some very concerning tests. Issa, I think you should go to the hospital.
 85 | 
 86 | Terrified, Issa leaves his house, gets into his car and drives straight to the hospital. When he arrives at the emergency room, he asks to see a doctor. "For what reason?" he id asked. "I got an alert from SmartPoop, which just told me to come to the emergency room," Issa explains.
 87 | 
 88 | After a minute or two, Issa is taken care of.
 89 | 
 90 | > Hello Mister Gueye, I'm Dr. Paola Marta. You should know that we are used to dealing with SmartPoop emergencies. For the past two years, most of our patients have come in as a result of a SmartPoop alert, and the report they send us always helps us a lot in caring for our patients[^familiarity]. How do you feel?
 91 | 
 92 | [^familiarity]: According to a recent study, most people do indeed seem to trust an algorithm's judgments more than a human's, at least for some predictions like Tesla's success, European sanctions against cyberwarfare, or the future of Brexit. Disturbingly, however, experienced professionals, on the other hand, trust algorithms less, leading them to worse judgments than non-professionals when given an algorithmic advice!  
 93 | [**Paper.** Algorithm appreciation: People prefer algorithmic to human judgment. Jennifer Logg, Julia Minsona & Don Moore. Organizational Behavior and Human Decision Processes (2019).](https://www.sciencedirect.com/science/article/abs/pii/S0749597818303388)  
 94 | Having said this, psychological research also suggests that what is familiar to us seems more believable, and sometimes disturbingly so. This is known as *familiarity bias*. For example, a subject who is repeatedly exposed to the phrase "the body temperature of a chicken" has a greater probability of judging the phrase "the body temperature of a chicken is 34°C" to be true. The fact that Dr. Paola Marta interacts daily with SmartPoop reports, which are also effective in guiding her in the treatment of her patients, could then explain why Dr. Paola Marta trusts SmartPoop.  
 95 | [**Video.** The Illusion of Truth. Veritasium (2016).](https://www.youtube.com/watch?v=cebFWOlx848)
 96 | 
 97 | > So far, all I have is a stomach ache. But I'm pretty terrified. SmartPoop says my condition is "very concerning."
 98 | 
 99 | > Very concerning, you say?
100 | 
101 | > Yes. Doctor, do you know what's wrong with me and if I'll be okay?
102 | 
103 | Dr. Marta looks away from Issa. Clearly embarrassed, Dr. Marta seems terrified by SmartPoop's diagnosis. She has had 27 patients whose condition were deemed "of great concern" by SmartPoop. All died[^laplace-succession]. After an interminable silence, Dr. Marta finally answered, obviously with an effort to sound as reassuring as possible, despite the circumstances.
104 | 
105 | [^laplace-succession]: Laplace's law of succession suggests that, if one had a priori total initial uncertainty that a "very concerning" SmartPoop diagnosis led to a life-threatening condition, then, knowing that the first 27 patients with that diagnosis all died, the probability that Issa would in turn die would then be 28/29. This fully justifies Dr. Marta's fear.  
106 | [**Video.** Binomial distributions | Probabilities of probabilities, part 1. 3Blue1Brown (2020).](https://www.youtube.com/watch?v=8idr1WZ1A7Q)
107 | 
108 | > I promise you that we will do our best.
109 | 
110 | After a blood test, Issa finally finds himself lying in a hospital bed, alone, abandoned to his own imagination. He fears the worst. He thinks of all the things he would have liked to have done. Issa harbors regrets, and contemplates his posterity. All his life, Issa has only followed the lure of gain. He was told that to succeed in finance is to roll in gold and triumph in life. This is what he did. Does it make him a bad person[^growth-mindset]?
111 | 
112 | [^growth-mindset]: Empirical psychology suggests that self-labeling can give an impression of permanence of some of our properties, which can then be very detrimental to our personal development. This is called a "fixed mindset". In contrast, people who think they can grow (*growth mindset*) seem to be much more fulfilled. Therefore, rather than calling oneself a "bad person", it would seem more appropriate to call oneself a "person who has done some bad things", or even a "person who aspires to do better things".  
113 | [**Video.** The power of believing that you can improve | Carol Dweck. TED (2014).](https://www.youtube.com/watch?v=_X0mgOOSpLU)  
114 | [**Book.** Mindset: The New Psychology of Success. Carol Dweck. Penguin (2007).](https://www.penguinrandomhouse.com/books/44330/mindset-by-carol-s-dweck-phd/)
115 | 
116 | Certainly, his career has at least allowed him to afford all sorts of luxury gear, like SmartToilets Deluxes. The ultimate proof of social success. But has Issa really accomplished anything in his life? What about all those other people who have suffered because of him? What about his ex-wife, whom he abandoned? Of his children with whom he spent so little time[^goal-factoring]?
117 | 
118 | [^goal-factoring]: These questions refer to what some people call *goal factoring* or *self alignment*, which involves asking ourselves whether the goals we have set are really the goals we would really like to set for ourselves.  
119 | [**Blog.** Goal Factoring. LessWrong (2018).](https://www.lesswrong.com/posts/Cu5C5KhkoXhrPMLFN/goal-factoring)  
120 | Indeed, it often seems to be the case that these goals are *orphan beliefs*, i.e. goals that we set as a result of fundamental motivations, and that we persist in setting even though these fundamental motivations have disappeared. Typically, we may have wanted to please our parents, at a time when our parents' pride was very important to us; but upon reflection, we came to realize that other fundamental motivations prevailed over that, like, say, actually making the world a better place.  
121 | [**Video.** Your brain is not a Bayes net (and why that matters). Julia Galef (2016).](https://www.youtube.com/watch?v=cFv5DvrLDCg)
122 | 
123 | Dr. Marta then turns back to Issa. "Finally," Issa exclaims inwardly, feeling as if he has been abandoned for several hours, even though Dr. Marta has only been gone for ten minutes[^time-subjective]. Dr. Marta then asks Issa to accompany her for an X-ray, and tells him that a colonoscopy is also being considered. In the space of a few hours, Issa will in fact undergo all sorts of analyses. At the end of the day, Dr. Marta tells him that he will stay overnight for observation. She encourages him to try to eat the meal that is served to him, and to try to get some sleep to rest.
124 | 
125 | [^time-subjective]: [**Video.** How Your Brain Makes Time Pass Fast or Slow. It's Okay To Be Smart (2020).](https://www.youtube.com/watch?v=NSy0Z7XCF3E)
126 | 
127 | The next morning, Dr. Marta finally arrives in Issa's room. Issa is exhausted. Terrified, he has not slept all night.
128 | 
129 | > What is the news, doctor?
130 | 
131 | > You clearly have very advanced fatigue and a high level of stress, with obvious digestive and sleep problems. Something is wrong. However, all our measurements still fail to identify the origin of your problems. We have called SmartPoop, whose health report is not really in line with our review. They are in the process of manually analyzing your data. We're hoping that together we'll be able to pinpoint the problem. But this is going to take us some time. I'm sorry about that. I'm so sorry.
132 | 
133 | ### SmartPoop Alert
134 | 
135 | At the end of the morning, it's Marc's turn to enter Katia's office.
136 | 
137 | > We have a problem, he says.
138 | 
139 | > What's going on, Marc?
140 | 
141 | > My assistant just told me that a doctor, Dr. Marta, kept trying to reach us last night and this morning. She has a patient whom SmartPoop told to go to the emergency room. They've run a lot of tests on the patient. But they can't figure out what's causing the problem.
142 | 
143 | > Did you get the diagnostics team on the problem?
144 | 
145 | > Yes. They've been working since 9:00 this morning, and they still can't find the problem. They asked me to ask you to take a look at the data.
146 | 
147 | > Are you sure it's my job to look? I'm very busy. I have to prepare for SmartPoopCon 2024 this weekend.
148 | 
149 | > They have an idea of the problem, but they want your expertise to confirm their intuition...
150 | 
151 | > Can you send me the case references?
152 | 
153 | > It's already done.
154 | 
155 | Katia then opens her email client, which tells her that she has 25,251 unread messages. She sorts her messages by recipient and finds the one from Marc[^externalization-memory]. Katia copies and pastes the case references, and runs queries to the SmartPoop database. She then retrieves Issa's data, and analyzes the statistics of his excrements. Katia executes some commands, which then generate all kinds of graphs. After seeing about fifteen graphs, Katia exclaims: "Oh no! Issa is out of distribution[^hors-distribution]".
156 | 
157 | [^externalization-memory]: The philosopher Michel Serres liked to insist on the impact of information technologies, such as paper, printing or computers, on the *externalization* of our cognition. In this regard, it is remarkable to note to what extent our email boxes have managed to externalize a large part of our memory. In a sense, our email boxes know us much better than we know ourselves, not because they are "intelligent", but simply because their memory is much more reliable than human memory, and because searching in these email boxes is often much more efficient than searching in our memory.  
158 | [**Video.**  Michel Serres - Les nouvelles technologies : révolution culturelle et cognitive. I Moved to Diaspora (2012).](https://www.youtube.com/watch?v=ZCBB0QEmT5g)
159 | 
160 | [^hors-distribution]: So-called "out-of-distribution" data are data that are very distinct from all other data. They are often considered erroneous or even adversarial, so many learning algorithms seek to eliminate them.  
161 | [**Paper.** Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent. Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui & Julien Stainer. NIPS (2017).](https://proceedings.neurips.cc/paper/2017/hash/f4b9ec30ad9f68f89b29639786cb62ef-Abstract.html)  
162 | [**Paper.** A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. Kimin Lee, Kibok Lee, Honglak Lee & Jinwoo Shin. NeurIPS (2018).](https://proceedings.neurips.cc/paper/2018/file/abdeb6f575ac5c6676b747bca8d09cc2-Paper.pdf)  
163 | Security-drivent algorithms are usually designed to ignore out-of-distribution data, which therefore leads them to ignore minorities. In fact, as discussed in the following article, there is a fundamental tension between inclusion and security. To resolve this tension, it is critical to better understand distributions (and model our well-founded priors), to better secure and authenticate data sources, or to be much more modest in our algorithm design.  
164 | [**Paper.** Collaborative learning in the jungle. El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui, Arsany Guirguis, Lê Nguyên Hoang & Sébastien Rouault. NeurIPS (2021).](https://arxiv.org/abs/2008.00742)
165 | 
166 | > What do you mean?
167 | 
168 | > Issa Gueye's video data are perfectly normal. It is his spectrographic data that is reported as "very concerning". But I'm afraid that's because Issa Gueye is a statistically very different SmartToilet Deluxe user from other SmartToilet Deluxe users.
169 | 
170 | > He's Senegalese.
171 | 
172 | > Oh no... I think we have a big problem. Do you have the doctor's number? We need to call her right away.
173 | 
174 | ### Nocebo effect
175 | 
176 | Marc dials Dr. Marta's number.
177 | 
178 | > Hello Dr. Marta, this is Marc Rofstein. You are currently on speakerphone with Katia Crapinski and myself, President and Vice President of SmartPoop. We have an update on Issa Gueye's case.
179 | 
180 | > Hello Dr. Marta, this is Katia. I've analyzed Mr. Gueye's case, and I'm pretty sure it's a false positive. In other words, SmartPoop made a mistake in reporting a problem with Mr Gueye's data.
181 | 
182 | > Hello Marc and Katia, replies Dr. Marta. Are you sure of what you are saying? Mr. Gueye has some worrying symptoms, especially on the digestive level and in terms of fatigue.
183 | 
184 | > This is weird. The data I have doesn't seem to suggest symptoms of concern, says Katia.
185 | 
186 | > It may be a *nocebo* effect, says Marc.
187 | 
188 | > A *nocebo* effect? Yes, this could be it, indeed, exclaims Dr. Marta.
189 | 
190 | > A *nocebo* effect, asks Katia, what's that?
191 | 
192 | > When a patient thinks that something horrible is going to happen to him, they often develop the symptoms they fear, explains Marc. Digestive symptoms, for example, are quite likely. Mr. Gueye probably trusts SmartPoop so much that when SmartPoop told him it was very concerned about his condition, his condition became very concerning[^nocebo].
193 | 
194 | [^nocebo]: [**Video.** This Video Will Hurt. CGP Grey (2013).](https://www.youtube.com/watch?v=O2hO4_UEe-4)
195 | 
196 | > But explain to me, asks Dr. Marta. Why was his case of false positive? Until now, whenever SmartPoop declared a case of "highly concerning" in our hospital, it always ended up in the intensive care unit, and then in death. That's why I panicked about Mr. Gueye myself - which probably contributed to his *nocebo*.
197 | 
198 | > We just rolled out the SmartToilets and SmartToilets Deluxe, says Katia. I think that the SmartToilets make quite reliable diagnostics. But the Deluxe version has even more advanced sensors, which have not been used as much as the basic SmartToilets. And that's why it's not quite as reliable.
199 | 
200 | > I understand, thank you, says Dr. Marta. I'll let Mr. Gueye know. Hopefully, it will help cure his *nocebo*. Can I ask you to manually correct his SmartPoop?
201 | 
202 | > It's... done, says Katia, after entering some commands on her computer.
203 | 
204 | > Thank you very much! Have a nice day, concludes Dr. Marta, before hanging up.
205 | 
206 | ### 163 false positives
207 | 
208 | Katia hangs up twice, to verify that Dr. Marta is no longer connected. Katia then types several new commands, which display three new graphs. She turns to Marc.
209 | 
210 | > Well done Katia. Always there to rescue us, exclaims Marc.
211 | 
212 | > Marc, I don't think you realize yet how crappy of a situation we are in.
213 | 
214 | > I'm getting fed up with such phrasings. Isn't the problem solved?
215 | 
216 | > I'm going to do the updates this afternoon to avoid more false positive alerts. But the damage is already done. SmartPoop has issued 163 false positive alerts for SmartToilet Deluxe users.
217 | 
218 | > That's 163 phone calls to make. It's not the end of the world!
219 | 
220 | > Marc, this is not the heart of the matter.
221 | 
222 | > What is the problem?
223 | 
224 | > The problem is that the 163 false positives are all of African origin, and that's not a coincidence.
225 | 
226 | > What's going on?
227 | 
228 | > SmartPoop is... racist, says Katia with a serious tone.
229 | 
230 | > Racist? What are you talking about? It's an algorithm, not a human.
231 | 
232 | > What I mean is that it is dangerous for black Africans. And as a result, we're going to undergo a huge *shitstorm* in the next few weeks, for deploying a racist technology *before* we've tested it enough[^facial-recognition].
233 | 
234 | [^facial-recognition]: This was the case with facial recognition technologies, which were deployed in a hurry before external audits discovered that these technologies have an unacceptable error rate for minorities, which could sometimes prevent them from entering their own buildings, when such entry was allowed by such algorithms. These technologies were eventually banned by the US Congress, which led to a retraction of products developed by IBM, Amazon and Microsoft.  
235 | [**Video.** Coded Bias. Netflix (2020).](https://www.netflix.com/title/81328723)  
236 | A recent paper argues that similar action is urgently needed for language processing algorithms, whose poorly understood and under-addressed vulnerabilities have a high potential to lead to disaster.  
237 | [**Paper.** Ever larger language models are ever more dangerous: A theoretical perspective. *forthcoming*.]( )
238 | 
239 | > Wait. Are you suspecting one of our developers of being a racist and making our algorithm do this?
240 | 
241 | > No. It's not that. Today, with machine learning, algorithms learn much more from data than from developers. So their performance depends on the quality and quantity of the data available[^manipulation-by-data]. Except that the data available through SmartToilet Deluxe is only the data from SmartToilet Deluxe users...
242 | 
243 | [^manipulation-by-data]: The example that shows this better than any other is probably the story of Tay and Xiaoice. Both of these conversational algorithms from Microsoft were built on the same principles. However, Tay was launched on Twitter, where it was derailed by troll data that encouraged it to express sexist and racist comments. Tay even went so far as to call for the genocide of certain populations. However, what's less known in Europe and North America, is that Xiaoice was launched 2 years ago on Chinese social medias (WeChat in particular). Xiaoice became adorable. So much so that Xiaoice is now used by 600 million Chinese people, with stories of men romantically seduced by Xiaoice. So, Tay has become horrible; Xiaoice has become adorable. Why is that? Well, because of the data that these algorithms were trained with.  
244 | [**Journal.** Xiaoice Vs. Tay: Two A.I. Chatbots, Two Different Outcomes. SAMPi (2016).](https://sampi.co/chinese-chatbot-xiaoice-vs-tay/)
245 | 
246 | > But these users are almost exclusively white and Asian, Marc adds.
247 | 
248 | > Yes... This product is a luxury product. It is therefore bought only by rich people, who happen to be mostly white and Asian.
249 | 
250 | > I see. Because of the lack of data on black Africans, SmartPoop has become bad at diagnosing their excrement.
251 | 
252 | > It's worse than that, Marc. In medicine, when something is abnormal[^anormal]...
253 | 
254 | [^anormal]: The notions of "normal" and "abnormal" have long shaped medical discourse. For example, homosexuality has long been considered "abnormal", leading the American Psychological Association to refer to it as a "mental disorder". In 1975, the Association reversed this judgment, and no longer considers it a mental disorder.  
255 | [**Web.** Sexual Orientation & Homosexuality. The American Psychological Association (2021).](https://www.apa.org/topics/lgbtq/orientation)
256 | 
257 | > It is considered a concern. Hence the alert...
258 | 
259 | > Guess how many SmartToilet Deluxe users are of African descent.
260 | 
261 | > No... 163 ? No... But why?
262 | 
263 | > The excrement of users of African origin obviously has different physiological characteristics.
264 | 
265 | > Ah yes... I know that they are more easily deficient in vitamin D, especially when they live in areas with little sun in winter[^vitamin-D]. There are certainly other major distinctions, and some of them can definitely be seen via the SmartToilet Deluxe. Hence SmartPoop's concern for the health of these 163 users of African descent.
266 | 
267 | [^vitamin-D]: [**Paper**. Vitamin D and African Americans. Susan Harris. The Journal of Nutrition (2006).](https://pubmed.ncbi.nlm.nih.gov/16549493/)
268 | 
269 | > Marc, it's not just about being concerned. We've basically just classified all SmartToilet Deluxe users of African descent as "sick". We've automated racism!
270 | 
271 | > But we didn't do it intentionally[^side-effect].
272 | 
273 | [^side-effect]: Many of the ethical problems of algorithms, such as the amplification of hate speech or misinformation, have arguably more to do with the unwanted (and therefore unintentional) side effects of the algorithms. Therefore, to make these algorithms ethical, it is not enough to want them to be "neutral", or even to want them to be "reasonable"; it is critical to actively seek to anticipate their difficult-to-predict side effects, and to invest massively in the study of these side effects.  
274 | [**Book.**  Le fabuleux chantier : Rendre l'intelligence artificielle robustement bénéfique. Lê Nguyên Hoang et El Mahdi El Mhamdi. EDP Sciences (2019). *English translation pending*.](https://laboutique.edpsciences.fr/produit/1107/9782759824304/Le%20fabuleux%20chantier)
275 | 
276 | > Go explain this to the media! I fear the worst. We'll get sensationalist headlines like "SmartPoop is racist", and we'll probably risk a lawsuit. We risk losing our investors! And if we don't get more investors and a lawsuit on our ass, we might go bankrupt. And all this just a few days before SmartPoopCon... We have to keep this story quiet.
277 | 
278 | After a silence, Marc adds, "I think it's a waste of time. The 163 victims have probably all suffered from the *nocebo* effect, for which we are responsible."
279 | 
280 | > What do you mean "for which we are responsible"? All we've done is issue alerts. SmartPoop is only an aid to decision-making.
281 | 
282 | > The problem is that our users trust SmartPoop so much that our decision aids have become decisions for the users. You saw it with Mr. Gueye's case. When we told him to go to the emergency room, do you think he took that as a decision aid? No, he thought, oh boy, I have to go to the emergency room[^help-decision]. And worse than that, he developed symptoms!
283 | 
284 | [^help-decision]: When a decision support algorithm becomes powerful, it seems important to see it as more than just a decision "aid". Like Google Map, such algorithms can end up being listened to almost blindly by their users, so that the decision aid becomes essentially the decision itself.
285 | 
286 | > OK. But it's not like we tortured Mr. Gueye!
287 | 
288 | > I've seen some terrifying *nocebo* experiments. In one of them, a patient just sits there. Nothing is done to her. But she is put in conditions that are conducive to *nocebo*. On a scale from 0 to 10, the patient reported a pain of 9.5. A pain of 9.5 out of 10! Like, it was probably for her comparable to the pain of childbirth[^experience-nocebo]!
289 | 
290 | [^experience-nocebo]: Marc is referring to this video here:  
291 | [**Video.** Touch - Mind Field (Ep 6). VSauce (2017).](https://www.youtube.com/watch?v=OUdXMoY6fLY)
292 | 
293 | > Wait, are you saying that, via SmartPoop, black Africans were targeted for torture? That's bullshit!
294 | 
295 | > Clearly, we didn't do it intentionally. But yes, if you take a step back, I think you can say that's what we did...
296 | 
297 | > Well... Anyway, we need to get ahead of this thing before it blows up. I'm going to reorganize SmartPoopCon, to talk about the problem we have, and what we plan to do to prevent our algorithms from being racist and causing such suffering.
298 | 
299 | ### Inclusion and diversity
300 | 
301 | A few days later, at SmartPoopCon 2024, SmartPoop's major annual conference, Katia took the stage to announce her company's new diversity and inclusion measures.
302 | 
303 | > In our industry, we tend to constantly want to move forward, introducing new projects for the future. We are expected to *innovate*. But that's not SmartPoop's mission. SmartPoop's mission is to provide unparalleled health care to all our users. SmartPoop's mission is to care for the health and well-being of *every* human on earth. And today, I would like to publicly reaffirm SmartPoop's commitment to this goal.
304 | >
305 | > But to do that, to really take care of all the people on earth, rather than constantly searching for the next idea, it's also critical to monitor our technologies and make sure they're working properly. At SmartPoop, we routinely do just that. Unfortunately, recently we realized that we have a systemic bias in the way we test our products. Because our engineering teams are primarily white men, they often fail to think about minority concerns. To close this gap, we believe we need to introduce much more social diversity into our teams. That's why we're actively committed to combating our systemic biases by promoting the recruitment of populations currently underrepresented among our employees[^diversity].
306 | >
307 | > So I wish I could tell you that we made this decision because we've always prioritized ethics. But to be honest with you, I have to admit that this decision comes first and foremost from the realization, unfortunately too late, that one of our technologies was not up to our ethical standards. SmartPoop, applied to the SmartToilet Deluxe, suffered from what is called a *distributional shift*, i.e. a discrepancy between its training data and practical application cases, which unfortunately led to diagnostic errors for certain populations[^distributional-shift]. To all our users who have suffered from these errors, but also to all these populations, we sincerely apologize.
308 | >
309 | > Unfortunately, without sufficient data from certain populations, it is in fact mathematically impossible to guarantee the same quality of service to these populations as we offer to other populations. At SmartPoop, we believe that technologies should be inclusive. Given the impact of these technologies on modern society, we believe that it would be immoral to offer these technologies only to certain populations, and even more so to already privileged populations. For this reason, we have decided to drastically subsidize the SmartToilets Deluxe to those populations where we lack the data to establish reliable and relevant medical alerts. The budget for this operation is estimated at $100 million. This is a huge amount of money for us; but it is a small price to pay for social equality in our societies.
310 | >
311 | > The subject of the impact of technology on social inequality is complex. Many companies prefer to avoid it. However, unlike some of our competitors[^whitelist-facebook], we don't want to sweep these issues under the rug. It is high time that the technology industry actively seeks to understand the harmful consequences of their technologies[^human-trafficking-facebook]. We are thus committed to investigating any potential concern, and to transparently revealing to you the problems we encounter[^gebru-firing]. Be aware, however, that in doing so, we will be revealing problems that are often pervasive throughout the technology industry. If we are victims of some technological flaw, other tech companies are almost surely victims too. When you do our trial, consider doing the trial of the entire technology industry, and remember that the companies that will reveal their problems the least are probably the ones that have the most problems[^transparency-backfire].
312 | >
313 | > Overall, I'd like you to think about asking yourself the most important question in the tech industry: is my trust in this or that company's product justified[^calibration]?
314 | 
315 | [^diversity]: Several studies point to the importance of diversity for a group's creativity. However, there seem to be confusing and contradictory results regarding the impact of diversity on group effectiveness. This is probably not surprising, considering that this impact most likely depends on many other factors, such as the task assigned to the group, the emotional sensitivity of the group members and the organization of the group.  
316 | [**Paper.** Collective Intelligence and Group Performance. Anita Williams Woolley, Ishani Aggarwal & Thomas W. Malone. Current Directions in Psychological Science (2015).](https://journals.sagepub.com/doi/full/10.1177/0963721415599543)  
317 | [**Paper.** Evidence for a Collective Intelligence Factor in the Performance of Human Groups. Anita Wooley, Christopher Chabris, Alex Pentland, Nada Hashmi & Thomas Malone. Science (2010).](https://www.science.org/doi/full/10.1126/science.1193147)
318 | 
319 | [^distributional-shift]: [**Paper.** Preventing Failures Due to Dataset Shift: Learning Predictive Models That Transport. Adarsh Subbaswamy, Peter Schulam & Suchi Saria. AISTATS (2019).](https://proceedings.mlr.press/v89/subbaswamy19a.html)
320 | 
321 | [^whitelist-facebook]: The *facebook files* reveal that, contrary to what Mark Zuckerberg has repeatedly stated publicly, Facebook maintains a secret *whitelist* of personalities exempt from Facebook's content moderation policy, including leaders of authoritarian countries. Yet, this boils down to empowering those who probably already have too much power. For example, because soccer player Neymar is a global star, and his Facebook presence attracts a lot of views, Facebook allowed him to post a *revenge porn* video, which exposes images of a naked woman without her consent. Such a post is prohibited by the [Facebook Community Standards](https://transparency.fb.com/policies/community-standards/), and should have led to Neymar being banned from Facebook.  
322 | [**Podcast.** The Facebook Files, Part 1: The Whitelist. The Journal (2021).](https://www.wsj.com/podcasts/the-journal/the-facebook-files-part-1-the-whitelist/aa216713-15af-474e-9fd4-5070ccaa774c?mod=article_inline)
323 | 
324 | [^human-trafficking-facebook]: The *facebook files* also reveal that Facebook management repeatedly discouraged the investigation of potential problems on the platform. More generally, by insisting that any company can be punished if it *knowingly* facilitates illegal acts, the law misguidedly encourages companies not to investigate any such problems, because once it can be shown that companies were aware of these problems, they then have a legal duty to invest heavily in solving these problems, which is very costly for them.  
325 | [**Podcast.** The Facebook Files, Part 3: 'This Shouldn't Happen on Facebook'. The Journal (2021).](https://www.wsj.com/podcasts/the-journal/the-facebook-files-part-3-this-shouldnt-happen-on-facebook/0ec75bcc-5290-4ca5-8b7c-84bdce7eb11f)
326 | 
327 | [^gebru-firing]: Very clearly, Google does not have this standard at all. As a reminder, the two co-directors of Google's ethics team were fired in turn, following the publication of a scientific paper discussing the ethical, environmental and social risks of deploying advanced language processing technologies.  
328 | [**Video.** Google fired its ethics. This is terrifying. Science4All (2021).](https://www.youtube.com/watch?v=fe9u6YQTsWY)
329 | 
330 | [^transparency-backfire]: Unfortunately, today, companies that communicate more about the problems they face are often criticized more than those that hide their problem. Typically, Twitter's more transparent policy seems to have led to more criticism than Facebook's very opaque policy. Or to take another more concrete example, in 2014 Facebook publicly shared the (fascinating) result of its analysis of the impact of a very slight reduction in the publication in News Feeds of posts with negative emotions on what users exposed to those posts would start writing in turn.  
331 | [**Podcast.** Can Algorithms Choose our Emotions? Robustly Beneficial (2020).](https://www.youtube.com/watch?v=gQHvTow91FY)  
332 | [**Paper.** Experimental evidence of massive-scale emotional contagion through social networks. Adam Kramer, Jamie Guillory & Jeffrey Hancock. PNAS (2014).](https://www.pnas.org/content/111/24/8788)  
333 | This study, however, led to an outcry about the ethics of such studies.  
334 | [**Paper.** Facebook's emotional contagion study and the ethical problem of co-opted identity in mediated environments where users lack control. Evan Selinger, Woodrow Hartzog. Research Ethics (2015).](https://journals.sagepub.com/doi/full/10.1177/1747016115579531)  
335 | An unfortunate consequence of this outcry, however, is that Facebook has stopped publishing such studies, even though Facebook has certainly not stopped conducting these studies, as well as studies much less relevant for ethics (such as A/B testing of the most addictive content). As a result, from the outside, it is very difficult to understand the impact of Facebook. In fact, the *facebook files* show that this has mostly allowed Facebook to hide the extent of its harmful impacts, such as on the mental health of teenage girls who use Instagram.  
336 | [**Podcast.** The Facebook Files, Part 2: 'We Make Body Image Issues Worse'. The Journal (2021).](https://www.wsj.com/podcasts/the-journal/the-facebook-files-part-2-we-make-body-image-issues-worse/c2c4d7ba-f261-4343-8d18-d4de177cf973)  
337 | More generally, there seems to be a tension here between *idealism* and *consequentialism*. While consequentialism seeks to improve the state of the world as much as possible, idealism sets itself an ideal and refuses any action that seems contradictory to that ideal, even if that ideal is pragmatically unattainable in the near future.  
338 | [**Podcast.** Radically Normal: How Gay Rights Activists Changed The Minds Of Their Opponents. Hidden Brain (2019).](https://www.npr.org/2019/04/03/709567750/radically-normal-how-gay-rights-activists-changed-the-minds-of-their-opponents)
339 | 
340 | [^calibration]: The fundamental issue Katia raises here is that of (trust) *calibration*. For example, if a person is calibrated in their predictions, on the set of times they say an event will happen with 80% probability, that person must have seen right 8 times out of 10. In the age of new technologies, unfortunately, the confidence assigned to some products is unjustifiably high, while the confidence assigned to other products is unjustifiably low.  
341 | [**Paper.** Practical Guidance for Evaluating Calibrated Trust. Patricia McDermott & Ronna ten Brink. Human Factors and Ergonomics (2019).](https://journals.sagepub.com/doi/abs/10.1177/1071181319631379)
342 | 
343 | ## To go further
344 | 
345 | Don't stop there!
346 | Check [the sequel of the novel](4-leak.md) or the [outline](README.md).  
347 | If you enjoyed it, please consider sharing and promoting this science fiction novel to others!
348 | 
349 | 


--------------------------------------------------------------------------------
/English/6-ad.md:
--------------------------------------------------------------------------------
  1 | # 6. The brown market
  2 | 
  3 | Frederic Partoli puts his foot down. Terribly frustrated, he walks off the field with his hand over his heart, which is beating very fast. He is taken directly to the hospital.
  4 | 
  5 | At the hospital, the doctor explains that Frederic is suffering from tachycardia[^tachycardia], caused by anemia[^anemia]. Frederic knows this. A few days ago, SmartPoop told him about his iron deficiency[^iron]. SmartPoop advised him to avoid all sports activities. But soccer was too tempting for Frederic. He preferred to play and risk it.
  6 | 
  7 | [^tachycardia]: A tachycardia occurs when the patient has an abnormally high heart rate.
  8 | 
  9 | [^anemia]: Anemia is a lack of hemoglobin, an important component of blood. This causes poor circulation of oxygen through the blood.
 10 | 
 11 | [^iron]: Iron is an important component of hemoglobin. In particular, it appears as Fe<sup>2+</sup> ions, within chemical subcomponents of hemoglobin called *hemes*. These ions help capture oxygen molecules, and transport them through the bloodstream.  
 12 | [**Web.** Hemoglobin and Functions of Iron. UC San Francisco.](https://www.ucsfhealth.org/education/hemoglobin-and-functions-of-iron)
 13 | 
 14 | The doctor then recommends that he eat green lentils, sesame seeds, dark chocolate, kiwis, spinach, but also blood sausage, liver, or red meat. However, he specifies that red meat should be eaten in moderation. Finally, he adds that it is important to accompany this with vitamin C intake, via citrus fruits for example. Once again, this information is not really news to Frederic. SmartPoop made the same recommendations to him.
 15 | 
 16 | ### The mystery of red meat
 17 | 
 18 | In the days that followed, Frédéric followed the doctor's and SmartPoop's advice, although he was apprehensive about black pudding, liver and red meat. Frederic is indeed sensitive to the animal cause, and cannot stand the idea of contributing to sophisticated animal torture[^animal-cause]. He is also very concerned about the disastrous consequences of animal exploitation on the environment[^beef] and biosecurity. After all, a significant proportion of scientists still believe that ROVID-19 probably arose from complex interactions between wild animals, livestock and humans[^biosecurity].
 19 | 
 20 | [^animal-cause]: It is estimated that several tens of billions of animals are killed each year to feed humans, most often in atrocious conditions.  
 21 | [**Web.** Number of animals slaughtered for meat, World, 1961 to 2018. Our World in Data (2018).](https://ourworldindata.org/grapher/animals-slaughtered-for-meat)  
 22 | [**Video.** Why Meat is the Best Worst Thing in the World. Kurzgesagt - In a Nutshell (2018).](https://www.youtube.com/watch?v=NxvQPzrg2Wg)  
 23 | 
 24 | [^beef]: Beef in particular, like all ruminants, emits enormous amounts of methane, which is a very important greenhouse gas. The exploitation of beef also requires enormous resources, especially in agricultural land to feed the beef, which is one of the main causes of deforestation.  
 25 | [**Video.** Beef is Bad for the Climate... But How Bad? | Hot Mess (2018).](https://www.youtube.com/watch?v=DD3sS743XHw)  
 26 | [**Video.** Why beef is the worst food for the climate. Vox (2020).](https://www.youtube.com/watch?v=3lrJYTsKdUM)
 27 | 
 28 | [^biosecurity]: Animal agriculture in particular uses a lot of antibiotics to protect its livestock. However, the abuse of antibiotics increases the risks of the emergence of antibiotic-resistant pathogens, which may then become untreatable with current medicine.  
 29 | [**Video.** The next pandemic could come from our farms. Vox (2020).](https://www.youtube.com/watch?v=hwuujiHvduc)  
 30 | [**Video.** How can we solve the antibiotic resistance crisis? - Gerry Wright. TED-Ed (2020).](https://www.youtube.com/watch?v=ZvhFeGEDFC8)
 31 | 
 32 | Nevertheless, notably because he has difficulty identifying with the image he has of the very virulent militant vegans, Frédéric struggles to identify with this movement, and to become fully vegetarian[^vegan-virulence]. He therefore seriously hesitates to resume a consumption of red meat that he had put on pause.
 33 | 
 34 | [^vegan-virulence]: [**Podcast.** Radically Normal: How Gay Rights Activists Changed The Minds Of Their Opponents. Hidden Brain (2019).](https://www.npr.org/2019/04/03/709567750/radically-normal-how-gay-rights-activists-changed-the-minds-of-their-opponents)
 35 | 
 36 | It is probably because he is asking himself existential questions about red meat consumption that Frederic perceives very distinctly a clear increase in meat ads in his social media news feeds, and on the various web pages he visits. Clearly, advertisers know that Frederic is being encouraged to eat meat. But then, how do they know, Frederick wonders?
 37 | 
 38 | Frederick checks all the messages he has sent since SmartPoop's announcement. None of them mention his anemia. Only the doctor was told. Could this doctor be selling the information? Impatient, Frédéric immediately dials his doctor's number.
 39 | 
 40 | > Hello doctor? Yes, I have a rather strange question to ask you. Have you told anyone about my anemia?
 41 | 
 42 | > Absolutely not. That would be violating medical secrecy[^medical-secret]. It would be extremely immoral, and contrary to the Hippocratic Oath[^hippocratic-oath].
 43 | 
 44 | [^medical-secret]: In many countries, doctors are bound by physician-patient privileges.  
 45 | [**Web.** Physician–patient privilege. Wikipedia (2021).](https://en.wikipedia.org/wiki/Physician%E2%80%93patient_privilege)
 46 | 
 47 | [^hippocratic-oath]: The Hippocratic Oath must be kept by every physician, and it states "[I will] impart precept, oral instruction, and all other instruction to my own sons, the sons of my teacher, and to indentured pupils who have taken the Healer’s oath, but to nobody else."  
 48 | [**Web.** Hippocratic Oath. Wikipedia (2021).](https://en.wikipedia.org/wiki/Hippocratic_Oath)
 49 | 
 50 | > Did you enter this information into a computer?
 51 | 
 52 | > No. Since you told me that SmartPoop had already alerted you, I considered that this information was already in your medical record, and I had nothing new to contribute.
 53 | 
 54 | > Thank you doctor.
 55 | 
 56 | > Why all these questions?
 57 | 
 58 | > Because for the past few days, I have been receiving advertisements for red meat. I don't understand why.
 59 | 
 60 | > Strange indeed. But I can assure you that I have nothing to do with it.
 61 | 
 62 | ### The Partoli experiment
 63 | 
 64 | As a good scientist, although usually confronted with psychological problems, Frédéric Partoli sets up a citizen science project[^citizen-science], where volunteers can share their SmartPoop data and the ads they receive on their phones and computers[^youtube-regrets]. Despite all his promotional efforts, especially on social medias, Frédéric is still struggling to recruit. Only seven people participate - all close friends!
 65 | 
 66 | [^citizen-science]: [**Video.** The Awesome Power of Citizen Science. SciShow (2016).](https://www.youtube.com/watch?v=SZwJzB-yMrU)
 67 | 
 68 | [^youtube-regrets]: In a similar genre, in 2020, the *Mozilla Foundation* launched a participatory program where volunteers could share their YouTube data to help researchers better understand YouTube's recommendation algorithm. Here again we have a case of the security-privacy dilemma. As long as YouTube's user data and algorithms remain private, it will be very difficult to understand the dynamics of user behavior on YouTube, and to take appropriate action to reduce cyber-bullying, hate speech and misinformation.  
 69 | [**Websites.** Mozilla Foundation (2021).](https://foundation.mozilla.org/en/campaigns/regrets-reporter/findings/)
 70 | 
 71 | Yet after 6 months, some triggers seem clear. Just after SmartPoop diagnosed Frederic's sister as overweight, she started receiving ads for athletic shoes. Just after Frederic's brother hurt his knee, he started receiving ads for electric scooters. Just after SmartPoop diagnosed Frederic's mother with vitamin A deficiency, she started receiving ads for carrot chips. And just after one of Frederic's friends became pregnant, she started receiving ads for strollers and diapers, while her husband started receiving ads for SUVs.
 72 | 
 73 | Frédéric wrote a scientific paper on these findings and submitted it to a peer-reviewed scientific journal. Unfortunately, the journal rejected Frédéric's paper because his sample size was too small. Nevertheless, in the meantime, the preprint version of the article, which Frédéric made available on medRxiv.org, was widely relayed on social medias, before being picked up by newspapers, such as *The Warden*. Science4Alpha even dedicated a video to it. Interestingly, this attracts many more volunteers. Thousands of people are now sharing their SmartPoop and advertising data with Frédéric, to better settle the existence of a reuse of SmartPoop data by advertisers.
 74 | 
 75 | This time, the signal is indisputable[^effect-size]. Very clearly, shortly after a SmartPoop diagnosis, the ads received by users were adjusted according to this diagnosis. Frederic rewrote his paper, and submitted it to the most prestigious scientific journal, Nature. The article was accepted and celebrated. The conclusion is implacable: advertisers have access to SmartPoop data.
 76 | 
 77 | [^effect-size]: [**Video.** Degrees of Freedom and Effect Sizes: Crash Course Statistics #28 (2018).](https://www.youtube.com/watch?v=Cm0vFoGVMB8)
 78 | 
 79 | ### SmartPoop API
 80 | 
 81 | It only takes a short week for WAISO to contact Katia, and demand a new hearing for SmartPoop. This time, against all odds, when Katia shows up, she seems serene, confident and determined.
 82 | 
 83 | > Hello Doctor Crapinski. This is Professor Raul, psychologist. Is SmartPoop reselling SmartPoop data for advertising purposes?
 84 | 
 85 | > Professor Raul, since March 2027, we have indeed had an API, replies Katia.
 86 | 
 87 | > What is an API?
 88 | 
 89 | > Professor Raul, it's a kind of web page with information presented in a structured way to be downloaded by an algorithm. Our API allows our advertising clients to extract information about SmartPoop users. This API is access-protected: only our clients have access to it. For security reasons, we have very few customers who are large players that we trust not to openly publish this personal data. These customers can then call the API, with a SmartPoop user's contact information, and retrieve the user's health profile to optimize their ads. And every time a customer calls our API, they pay a few dollars, based on our estimation of the commercial value of the revealed profile.
 90 | 
 91 | > Has this API always existed? Why isn't it in SmartPoop's security audit reports?
 92 | 
 93 | > Professor Raul, this API has only been in production for 3 months. That's why it's not in last year's reports.
 94 | 
 95 | > Will it be in the next report?
 96 | 
 97 | > Professor Raul, no, it won't.
 98 | 
 99 | > Why is that?
100 | 
101 | > Professor Raul, thank you very much for this question. To answer it, I'm going to have to make some unusual disclosures, which will jeopardize my job, and which will very clearly make me enemies. But the short answer is that SmartPoop has suffered tremendously financially since our massive investments in software security. The cost of a full audit of our code is now estimated at a trillion dollars. Government grants were not enough for us. To pay for this audit, we did an IPO, which allowed us to raise a lot of money to fund our audits. However, this also attracted new investors, especially huge investment funds, which have a very strong business case logic in their culture[^useful-investment]. After all, these funds are financed by hundreds of millions of customers to whom they have promised a return on investment. They are literally designed to make money. And they've put more and more pressure on SmartPoop to make a lot more money. To keep my job and my influence, I was forced to listen to them. That's how the advertising API project was born.
102 | 
103 | [^useful-investment]: [**Journal.** The thorny truth about socially responsible investing. Vox (2021).](https://www.vox.com/the-goods/22714761/esg-investing-divestment-fossil-fuels-climate-401k)
104 | 
105 | > Thank you for your candor. But this does not answer my question. Why won't the IPA be in the next report?
106 | 
107 | > Professor Raul, before I continue with my revelations, I would like to emphasize the risks I face. My revelations are going to make me serious enemies, who could succeed in making me suffer serious criminal sanctions, especially because I am about to violate professional secrecy[^nda]. Nevertheless, what is at stake seems to me above the law. Professor Raul, in order for the API project to be successful, you should know that our shareholders essentially forced us to invest heavily in a huge legal team. SmartPoop is now largely a legal firm[^equipes-legales]. Our lawyers have come up with perfectly legal schemes to keep the ad API out of our audits. Legal schemes that I personally find deeply immoral. Again, I have violated a legal non-disclosure agreement here. I am risking a lot with these revelations. Ladies and gentlemen, WAISO members, our lawyers are incredibly competent. They will drag me through the mud, and most likely win. I will most likely end up in jail[^nda-abuse].
108 | 
109 | [^nda]: The use of non-disclosure agreements (our *NDA*) seems to be widely abused in many industries.  
110 | [**Journal.** I had to start over, alone and silenced': the fight to end NDA abuse. The Guardian (2021).](https://www.theguardian.com/law/2021/sep/21/i-had-to-start-over-alone-and-silenced-the-fight-to-end-nda-abuse)  
111 | 
112 | [^equipes-legales]: This comment refers to remarks by Google researcher Nicholas Carlini, which echo statements by other researchers on Twitter like Timnit Gebru. In an internal Google email, Carlini wrote, "When we as academics write that we have a 'concern' or find something 'worrisome' and a Google lawyer demands that we change it to sound nicer, it sounds a lot like Big Brother intervention."  
113 | [**Journal.** Exclusive: Google pledges changes to research oversight after internal revolt. Jeffrey Dastin & Paresh Dave. Reuters (2021).](https://www.reuters.com/article/us-alphabet-google-research-exclusive/exclusive-google-pledges-changes-to-research-oversight-after-internal-revolt-idUSKBN2AP1AC)  
114 | [**Web.** "Every single time a lawyer was inserted things were hopeless"... Tweet by Timnit Gebru (2021).](https://twitter.com/timnitGebru/status/1365153506789908480)
115 | 
116 | [^nda-abuse]: [**Journal.** Google contractors allege company prevents them from whistleblowing, writing Silicon Valley novels. Jennifer Elias. CNBC (2020).](https://www.cnbc.com/2020/10/01/google-contractors-allege-ndas-violate-free-speech-laws.html)
117 | 
118 | A dead silence now fills the room. The members of WAISO are stunned. None of them suspected such revelations. SmartPoop, which they had celebrated as a model of transparency only 8 months ago, now seemed terribly corrupted by heightened profit motives, complex internal political games, and highly sophisticated legal maneuvers.
119 | 
120 | > Dr. Carpinski, Professor Raul asks, do you have any proof of what you are saying?
121 | 
122 | Upon hearing these words, Katia takes out her phone.
123 | 
124 | > Professor Raul, and ladies and gentlemen members of WAISO, in the last few weeks, I have compiled a lot of very compromising documents internally at SmartPoop, which I have transferred to several servers around the world, accessible via passwords. I have an email ready to send.
125 | 
126 | Katia presses the "send" button on her email application.
127 | 
128 | > That's it. You have just received the email in question, as well as a dozen journalists and science communicators whom I trust enough to prioritize the well-being of all of humanity. This email contains all the necessary information to download the compromising documents from the servers that host them. You will be able to verify, via the TLS+[^tls] signatures in particular, that what you will read is both authentic and highly compromising. And you will quickly understand why the email I just sent you will lead to retaliation from SmartPoop. In the days to come, I will certainly not only be fired, but also sued[^facebook-haugen-lawsuit]; and I will probably end up in jail.
129 | 
130 | [^tls]: TLS signatures are used, especially via the HTTPS protocol used by the web, to authenticate that a given message is indeed coming from a recognized source. So when you go to https://twitter.com, your web browser receives a file that can provably only have been made by an entity with the private key that Twitter has.  
131 | [**Video.** Transport Layer Security (TLS) - Computerphile (2020).](https://www.youtube.com/watch?v=0TLDTodL7Lc)  
132 | Note, however, that with TLS, this private key is used to generate and share a symmetric key, which is then used to encrypt and sign information on the https://twitter.com web page. 
133 | But then, a user can use that symmetric key to encrypt and sign information, and potentially make it look like it came from the https://twitter.com web site. 
134 | SmartPoop's story here assumes that the web uses an advanced version, called TLS+ (and which does not exist in 2021), where the web page must certify the entire web page; not just the symmetric key to be used. Typically, the website https://twitter.com might have to compute a hash of the web page sent to the user (or at least important packets), and might have to sign that hash.  
135 | [**Video.** Hashing Algorithms and Security. Computerphile (2013).](https://www.youtube.com/watch?v=b4b8ktEV4Bg)  
136 | So, if the page https://twitter.com then states that the @le_science4all account wrote "long live math", with a TLS+ signature, then, unless Twitter or the @le_science4All account were hacked, you have here the proof that @le_science4all did write what is displayed on the page posted by Twitter. Arguably, such proofs of authenticity would go a long way towards the accountability of (illegal) online activities.
137 | 
138 | [^facebook-haugen-lawsuit]: [**Journal.** Could Facebook sue whistleblower Frances Haugen? Here's what experts say. USA Today (2021).](https://eu.usatoday.com/story/tech/news/2021/10/09/can-facebook-sue-whistleblower-frances-haugen-sharing-internal-research/6073523001/)
139 | 
140 | ### The Externalities of Advertising
141 | 
142 | > Hello Dr. Crapinski. This is Professor Smith, PhD in public health. I want you to know that I admire your transparency even more. Dr. Frédéric Partoli's study shows that you have allowed advertisers to recommend abundantly the consumption of red meat and carrot chips. However, consumed in too large quantities, these products are in fact harmful to health. Do you regret these ads?
143 | 
144 | > Professor Smith, yes, I do. Well, not completely. We've thought a lot about ethically optimal dietary recommendations[^consequentialism]. And yes, there are healthier alternatives than red meat for anemia, and carrot chips for vitamin A deficiency. But users do not systematically follow our recommendations. For their health, in case of vitamin A deficiency, if they eat slightly too many carrot chips, it is still much better than never eating any carrots at all, which would happen for some users who hate carrots. If I regret these recommendations, it is not so much for the health of our users - although it can sometimes play a harmful role indeed.
145 | 
146 | [^consequentialism]: The notion of being "ethically optimal" may seem strange in a deontological framework; however, it is very natural in the consequentialist sense. It is indeed simply a matter of making recommendations that, given our state of knowledge, we believe have the best probable consequences for the health and well-being of the SmartPoop user and of society as a whole (especially in the case of a meat recommendation, which has externalities on the whole society).  
147 | [**Video.** PHILOSOPHY - Ethics: Consequentialism. Wireless Philosophy (2015).](https://www.youtube.com/watch?v=hACdhD_kes8)
148 | 
149 | > Why do you miss the ads?
150 | 
151 | > Professor Smith, after a few months, I realized that we, SmartPoop, have no control over which ads were actually recommended. And some of the ads have huge externalities on society. Every red meat or SUV ad is, indirectly but surely, increasing climate change and biosecurity risks[^google-ads-climate]. Even worse, I learned from talking to Frederic Partoli that pyramid scheme ads have been massively recommended to our SmartPoop users who suffer from anxiety. These pyramid schemes are causing horrible debt and suffering to millions of families[^pyramid-scheme]. And because we helped advertisers target vulnerable individuals, they have become tremendously effective. This is completely immoral. But this is not all. There are also a lot of advertisements of pseudo-medicine or naturopathic treatments that are very dubious, sometimes with curative claims against diseases like cancer, which can be very dangerous[^homicide-involuntary]. Worse still, there are the very politicized propagandas, which exploit the hatred towards certain minorities to radicalize their supporters, and go as far as calling for rebellion, or even war. 
152 | 
153 | [^google-ads-climate]: In October 2021, Google announced a new advertising policy, which now refuses to show ads that "contradict the well-established scientific consensus around the existence and causes of climate change." Strangely, however, the justification given for this decision comes not from a corporate ethic, but from a response to the ethical concerns of the company's partners. Indeed, Google writes: "A growing number of our advertiser and publisher partners have raised concerns about ads that run alongside or promote inaccurate claims about climate change. Advertisers simply do not want their ads to appear next to this content. As for publishers and creators, they don't want ads promoting these claims to appear on their pages or in their videos."  
154 | [**Web.** Updating our ads and monetization policies on climate change. Google Support (2021).](https://support.google.com/google-ads/answer/11221321)
155 | 
156 | [^pyramid-scheme]: Pyramid schemes are forbidden in France. In promoting these pyramid schemes, it is unclear to the authors of this book whether a company that sells pyramid scheme advertisements could technically be judged for aiding and abetting pyramid selling.  
157 | [**Paper.** Article L121-15 of the Consumer Code. Legifrance (2016).](https://www.legifrance.gouv.fr/codes/article_lc/LEGIARTI000032227258/)
158 | 
159 | [^homicide-involuntary]: As discussed in Chapter 2, a company selling alternative medicine advertisements would endanger not only the users, but also the users' relatives, and even the whole society when it comes to pandemics like ROVID-19.
160 | 
161 | > Calling for war?
162 | 
163 | > Ladies and gentlemen, members of WAISO, please realize what is at stake. At this rate, entire nations could hate each other to the point of coming to blows[^tensions]. Except that modern war technologies are not swords or guns; modern killer robots are now capable of causing monumental catastrophes[^spur]. Fleets of billions of killer drones, dropped from cargo planes, could exterminate an entire megalopolis in a matter of hours[^slaughterbot]. The idea that we are contributing via our API to the distribution of ads that promote such attacks, directly or indirectly, seems terrifying to me. That's why I decided to make these disclosures today, even if it means risking my own well-being and safety. The future of all humanity is at stake.
164 | 
165 | [^tensions]: There was of course the case of the Capitol riots in the United States, but the *facebook files* reveal disturbing abuses, which have led to physical violence and sometimes genocide, in many countries such as Myanmar, Ethiopia, India, Pakistan, Iran, Afghanistan, Yemen, Lebanon, Morocco, Algeria and Bolivia, among others. In particular, the violence in Myanmar and Ethiopia appears to have led to tens of thousands of deaths.  
166 | [**Journal.** Facebook failed to police abusive content globally, documents show. NY Post (2021).](https://nypost.com/2021/10/25/facebook-failed-to-police-abusive-content-globally/)  
167 | [**Journal.** Facebook knew about, failed to police, abusive content globally. Reuters (2021).](https://www.reuters.com/technology/facebook-knew-about-failed-police-abusive-content-globally-documents-2021-10-25/)  
168 | [**Journal.** Former UN chief says Bangladesh cannot continue hosting Rohingya. Aljazeera (2019).](https://www.aljazeera.com/news/2019/7/10/former-un-chief-says-bangladesh-cannot-continue-hosting-rohingya)  
169 | [**Journal.** More than 50,000 Ethiopia civilians have been killed, Tigray opposition says. LA Times (2021).](https://www.latimes.com/world-nation/story/2021-02-02/tigray-opposition-parties-assert-50-000-plus-civilian-deaths)
170 | 
171 | [^spur]: Robotics development, including by *Boston Dynamics*, shows how, by 2021, the production of effective killer robots seems within reach of the superpowers.  
172 | [**Video.** Atlas | Partners in Parkour. Boston Dynamics (2021).](https://www.youtube.com/watch?v=tF4DML7FIWk)  
173 | [**Video.** The Army testing its New Robot Dogs with Super gun Mounted On Their Backs. Weapons of the World (2021).](https://youtu.be/Mnnyixbpnv8)
174 | 
175 | [^slaughterbot]: [**Video.** Slaughterbots. Stop Autonomous Weapons (2017).](https://youtu.be/9CO6M2HsoIA)
176 | 
177 | Another deafening silence reigns. Almost a minute passes as each member of WAISO realizes the urgency of confronting the ethics of advertising.
178 | 
179 | ### The free market in information
180 | 
181 | > Hello Dr. Crapinski. I am Professor Abdoul, a researcher in geopolitics. If I understand correctly, your revelations today seek to go beyond the SmartPoop case. It is the whole industry of targeted advertising that you oppose.
182 | 
183 | > Professor Abdoul, actually, if you think about it, the problem is not so much that advertising is targeted. The biggest problem today is that most of the time, it costs the same to communicate a message of hate as it does to provide reliable health information. Clearly, one does not have the same social impact as the other! In fact, it's worse than that, on many platforms that maximize user retention, teaser ads, which make users stay, even cost less to run[^advertising-price]! This is ethical nonsense!
184 | 
185 | [^advertising-price]: During the 2016 US presidential elections, a Russian agency seems to have had comparable visibility to Trump and Clinton on social medias, investing a **thousand** times less money than these presidential candidates. All because their ads were very polarizing, making them very viral. On the Internet, some messages are much cheaper to spread massively than others; unfortunately, these messages are not the most beneficial to humanity.  
186 | [**Journal.** Trump and Clinton spent $81M on US election Facebook ads, Russian agency $46K. Josh Constine. TechCrunch (2017).](https://techcrunch.com/2017/11/01/russian-facebook-ad-spend/)
187 | 
188 | > So if I understand correctly, you think that the existence of a market for ads without regulation is dangerous.
189 | 
190 | > Professor Abdoul, yes. And it's not just about ads. I think that the free market of *information* is very dangerous, and that it is something that is not recognized enough[^danger-reseaux-2]. It seems to me that it is far too commonly assumed that, by letting information flow unregulated and uncontrolled, the intelligence of crowds will bring about knowledge, empathy, and consensual decisions; that there will be some kind of invisible hand of the market of information that can be trusted[^invisible-hand]. To me, this is the most dangerous myth of the modern world. If we let it happen, clearly, those who will dominate the information market and govern the dominant beliefs and influential decisions will be those who have invested billions, even thousands of billions, to promote their products or ideologies, or those who have very easy-to-sell information content, such as addictive sensationalism, divisive scandals and misleading clickbait[^liberte-presse]. 
191 | 
192 | [^danger-reseaux-2]: [**Video.** Social medias are dangerous. Very dangerous. Science4All (2021).](https://www.youtube.com/watch?v=w5JRKUndWNk&list=PL8ovs-QtxcNxcwlsTF5O9NXtr3NAj_SVc&index=1)
193 | 
194 | [^invisible-hand]: [**Video.** Political theory - Adam Smith. The School of Life (2014).](https://www.youtube.com/watch?v=ejJRhn53X2M)
195 | 
196 | [^liberte-presse]: In France, a law of 1881 strongly regulates the freedom of the press. This one is therefore quite regulated, especially after an explosion of misinformation called "yellow journalism".    
197 | [**Video.** My Video Went Viral. Here's Why. Veritasium (2019).](https://www.youtube.com/watch?v=fHsa9DqmId8)
198 | 
199 | > How confident are you in this claim?
200 | 
201 | > Extremely confident. I don't want to expose my sources, but I know many people working for big social media companies. They have clearly observed, again and again, that, by maximizing engagement, these companies are disproportionately amplifying outrage, even when it is motivated by disinformation, discrimination or hate[^outrage-algorithm]. These companies are now *hate dealers*. They are making money by selling hate.
202 | 
203 | [^outrage-algorithm]: The real world is here one step ahead of fiction! Thanks to the revelations of Frances Haugen, this information has become public. Indeed, the documents revealed by this Facebook whistleblower clearly show that Facebook changed its algorithms in 2018, to counter a decline in Facebook usage. Many employees then observed that this greatly amplified extreme messages. They concluded that the rise of authoritarianism around the world, and particularly in the Capitol Hill riots, was largely enabled, if not caused, by social media algorithms. But when they reported their findings to Facebook's leaders, no further action was taken by these leaders.  
204 | [**Podcast.** The Facebook Files, Part 4: The Outrage Algorithm. The Journal (2021).](https://www.wsj.com/podcasts/the-journal/the-facebook-files-part-4-the-outrage-algorithm/e619fbb7-43b0-485b-877f-18a98ffa773f)
205 | 
206 | > But aren't they a few bad apples? Or isn't it because they are monopolies?
207 | 
208 | > I fear that competition without regulation will only exacerbate the incentives to deal hate. I think that the bigger problem is a fundamental incentive misalignment, between the messages that are profitable to spread, and the messages that are *desirable* to spread, for the sake of all of humanity. Therefore, I strongly believe that, to sacralize the freedom of the free information market, in a context as competitive as it is today, is to sacrifice knowledge, nuance and benevolence[^liberty]. It amounts to abandonning the information battlefield to propaganda, hatred and inhumanity. And I think that this can only end badly for humanity. Very badly[^precipice].
209 | 
210 | [^liberty]: From a philosophical point of view, the notion of freedom is a complex one. But we can at least agree, to a first approximation, on tensions between freedom and security (for example, gun control aims to grant security by limiting freedom), and on the fact that "one person's freedom ends where another's begins".  
211 | 
212 | [^precipice]: Many authors believe that overpowering algorithms represent an existential threat to humanity.  
213 | [**Book.** Life 3.0: Being Human in the Age of Artificial Intelligence. Max Tegmark. Knopf (2017).](https://futureoflife.org/background/life-3-0-being-human-in-the-age-of-artificial-intelligence/)  
214 | [**Book.** Human Compatible: Artificial Intelligence and the Problem of Control. Stuart Russell. Penguin (2019).](https://www.penguinrandomhouse.com/books/566677/human-compatible-by-stuart-russell/)  
215 | [**Book.** The Precipice: Existential Risk and the Future of Humanity. Toby Ord. Hachette Books (2020).](https://theprecipice.com/)
216 | 
217 | A new silence invades the room. Obviously, every speech of Katia requires a time of informational digestion. Against all odds, the hearing, which was supposed to dwell on a SmartPoop scandal, has turned into a questioning of the entire information industry, which, all day long, collects, processes and disseminates certain information, and drowns others in a torrent of information.
218 | 
219 | ### The regulation of information
220 | 
221 | > Hello Dr. Crapinski, this is Professor Wang, a researcher in machine learning. What can be done?
222 | 
223 | > Professor Wang, we can start by acknowledging that, in practice, there is no total freedom of information, and that this is certainly a good thing. Today, in many countries, hate speech, defamation and intentional lying are punishable by criminal penalties. No, we don't have the right to say everything. But more importantly, we shouldn't be allowed to say anything. After all, SmartPoop has been convicted several times, for saying too much, for making racist judgments, and for sharing information that should not have been said. The market of information is already regulated[^bad-ads].
224 | 
225 | [^bad-ads]: Google's algorithms censor nearly 100 "problematic" ads per... second!  
226 | [**Journal.** Google killed 2.3 billion 'bad ads' in 2018, down 28% from 2017. Emil Protalinski. Venture Beat (2019).](https://venturebeat.com/2019/03/13/google-killed-2-3-billion-bad-ads-in-2018-down-28-from-2017/)
227 | 
228 | > But I guess you don't think it's regulated enough.
229 | 
230 | > Exactly, Professor Wang. I think it urgently needs to be much more regulated, if misinformation and hate are not to triumph.
231 | 
232 | > But, Dr. Crapinski, who should regulate information? How do we decide what should not be said?
233 | 
234 | > Professor Wang, the power to regulate information is perhaps the greatest power in the modern world. And I think we feel it instinctively when we talk about censorship; but we probably ignore it too often when it comes to recommendation. Yet when we control recommendation, we also control censorship. After all, to censor a message, it suffices to constantly recommend alternative messages[^mute-news]. Now, in a world of information abundance, the latter is clearly easier than the former. My point is that the issue is not so much censorship. The issue is how to manage the flow of information: to reduce the spread of certain messages and facilitate the spread of other content[^public-utility].
235 | 
236 | [^mute-news]: This is sometimes referred to as *mute news*. A *mute news* is an important information, but ignored by many, because it is buried in a sea of much less important information.  
237 | [**Video.** How not to be ignorant about the world | Hans and Ola Rosling. TED (2014).](https://www.youtube.com/watch?v=Sm5xF-UYgdg)
238 | 
239 | [^public-utility]: There would certainly be a lot to gain globally by promoting public-utility content.  
240 | [**Paper.** Science Communication Desperately Needs More Aligned Recommendation Algorithms. Lê Nguyên Hoang. Frontiers in Communication (2020).](https://www.frontiersin.org/articles/10.3389/fcomm.2020.598454/)  
241 | [**Paper.** Recommendation Algorithms, a Neglected Opportunity for Public Health. Lê Nguyên Hoang, Louis Faucon & El-Mahdi El-Mhamdi. Revue Médecine et Philosophie (2021).](https://philpapers.org/rec/HOARAA)
242 | 
243 | > Very well, Dr. Crapinski. But you are not answering my question. Who should decide how to manage the flow of information?
244 | 
245 | > Professor Wang, today, de facto, content recommendation algorithms, especially on social medias, do it more than anyone else. It is now the moral responsibility of their owners to manage this flow in a fair and beneficial way. And if they take the measure of the monumental task that falls to them, I dare to hope that they will think about decentralizing the ethics and the audit of these very dangerous recommendation algorithms[^tournesol]. But, ladies and gentlemen of WAISO, if there is one thing I have learned in recent months, it is that the internal structures of gigantic multinational firms do not favor such reflection[^zhang]. No invisible hand will magically fix recommensation algorithms. You must intervene. This is why I urge you to do what is necessary to ensure that the management of the flow of information is not left to shareholders whose job is to prioritize profitability, and to legal teams whose job is to circumvent the law in order to achieve this as efficiently as possible. Dear members of WAISO, *you* have this responsibility. You must demand much more transparency from these companies[^transparency-shareholders]. And I hope that, like me, you will put the interests of civil society before your own[^haugen].
246 | 
247 | [^tournesol]: That's the goal of the [Tournesol project](https://tournesol.app), co-founded by one of the authors of the book.  
248 | [**Paper.** Tournesol: A quest for a large, secure and trustworthy database of reliable human judgments. Lê-Nguyên Hoang et al. (2021).](https://arxiv.org/abs/2107.07334)
249 | 
250 | [^zhang]: Facebook whistleblower Sophie Zhang talks about this very well.  
251 | [**Video.** Facebook whistleblower Sophie Zhang on how the platform is influencing global politics | 7.30. ABC News (2021).](https://www.youtube.com/watch?v=-rlQbZRJtWw)
252 | 
253 | [^transparency-shareholders]: Interestingly, some shareholders are now also demanding more transparency from these companies, whose secretive maneuvers can be widely seen as a threat to the security of the company itself, especially over the long term.  
254 | [**Journal.** Alphabet shareholder pushes Google for better whistleblower protections. The Verge (2021).](https://www.theverge.com/2021/4/6/22370177/google-whistleblower-protections-trillium-asset-management)
255 | 
256 | [^haugen]: Facebook whistleblower Frances Haugen may speak to this better than anyone.  
257 | [**Video.** Facebook Whistleblower Frances Haugen: The 60 Minutes Interview (2021).](https://www.youtube.com/watch?v=_Lx5VmAdZSI)
258 | 
259 | ### The SmartPoopGate
260 | 
261 | That same evening, Katia's confession made the front page of every newscast. "Digital earthquake", the presenter announces. "The CEO of SmartPoop lets go of her shareholders, revealing immoral maneuver", he adds. Meanwhile, Marc is a guest on Science4Alpha.
262 | 
263 | > Hello Marc. I remind that you are the co-founder of SmartPoop. Are you surprised by Katia's revelations?
264 | 
265 | > No, I'm not surprised. Katia and I have been preparing for this for several weeks. We knew that our company had engaged in immoral decisions, and we felt that we were losing our influence on future decisions. Knowing the monumental influence SmartPoop now has, and the staggering amount of harm these decisions can cause, Katia decided to legally sacrifice herself. I am incredibly proud of her courage and ethics. It's really fantastic. If all industry, legal, and state leaders had a thousandth of her courage and ethics, climate change probably might not be an issue anymore[^haugen-the-journal]. Congratulations Katia!
266 | 
267 | [^haugen-the-journal]: It seems a propos to highlight the monumental work done by whistleblower Frances Haugen to reveal in the most productive way possible the internal problems at Facebook, with the goal first and foremost of making the world a better place - not directly destroying Facebook.  
268 | [**Podcast.** The Facebook Files, Part 6: The Whistleblower. The Journal (2021).](https://www.wsj.com/podcasts/the-journal/the-facebook-files-part-6-the-whistleblower/b311b3d8-b50a-425f-9eb7-12a9c4278acd)
269 | 
270 | > But these revelations, Marc, are they true?
271 | 
272 | > Unfortunately, Katia convinced me to keep my legal integrity, even if it meant sacrificing my moral integrity. It is important, she told me, that there are still dissenting voices within SmartPoop. So I am not at liberty to answer those questions. All I can tell you is that I have worked with Katia on a daily basis since the beginning of SmartPoop, and I have seen her work really hard to save millions, maybe even billions of lives since then. We haven't always agreed on everything. But, especially in the last few years, I have been impressed by her daily drive and unwavering desire to help as many people as possible. Congratulations Katia for all your work and your incredible intellectual honesty[^intellectual-honesty]!
273 | 
274 | [^intellectual-honesty]: Intellectual honesty is sometimes defined as the willingness to never engage in "self-bullshit". Put another way, it is about making the effort to constantly seek to be honest with ourselves, to understand the deep origin of our beliefs and preferences, and not to settle for incomplete or flawed explanations.  
275 | [**Video.** Julia Galef Discusses Intellectual Honesty. South Park Commons (2019).](https://www.youtube.com/watch?v=V_E9-7t8QMI)
276 | 
277 | > What will happen now? Will Katia face dismissal? Will she be sued?
278 | 
279 | > Unfortunately, I am not at liberty to discuss these matters. Decisions will be made by our board of directors, and we'll see where we stand tomorrow. But I want to make sure that Katia's vision is realized, and that SmartPoop becomes a company that its users can really trust.
280 | 
281 | > Thank you Marc for your time and sincerity under these very complicated circumstances. Even if an interviewer is supposed to remain impartial, I would like to express my deep admiration for Katia. As you probably know, Katia, Marc and I have been working together since the beginning of SmartPoop, and if I had to choose between what's left of SmartPoop today and Katia, I wouldn't hesitate for a second, especially since my contract allows me to easily detach myself from SmartPoop. Katia, you have my unconditional support.
282 | 
283 | 
284 | ## To go further
285 | 
286 | Don't stop there!
287 | Check [the sequel of the novel](7-psy.md) or the [outline](README.md).  
288 | If you enjoyed it, please consider sharing and promoting this science fiction novel to others!
289 | 
290 | 


--------------------------------------------------------------------------------
/English/7-psy.md:
--------------------------------------------------------------------------------
  1 | # 7. Pissed
  2 | 
  3 | With ten votes against and thirteen votes for, the verdict falls. Katia is fired from SmartPoop. While three representatives of some of SmartPoop's shareholder investment funds voted against, the others did not appreciate Katia's revelations. Katia is devastated. Even though she was expecting it, she is having a hard time taking the news. Slumped in her chair, she can't get up. The board members leave the room one by one, until only Katia and Marc are left.
  4 | 
  5 | > I lost, says Katia.
  6 | 
  7 | > No. *We* lost... And no, we didn't lose. It's not over.
  8 | 
  9 | > I know how the rest of it will go. I bet you already got an email to have another meeting tomorrow, to decide to sue me[^facebook-could-sue-haugen]. I might end up in jail, with a big fine I won't be able to pay. And most importantly, you're going to become more and more isolated and marginalized.
 10 | 
 11 | [^facebook-could-sue-haugen]: Facebook has threatened whistleblower Haugen with legal action, including for leaking confidential documents to the press. Experts say Haugen is not protected, and knows it. What is most likely to protect Haugen, they say, is the media and especially political backlash such lawsuits could have for Facebook.  
 12 | [**Journal.** Facebook whistleblower isn't protected from possible company retaliation, experts say. Obby Allyn. NPR (2021).](https://www.npr.org/2021/09/30/1042012393/facebook-whistleblower-retaliation)
 13 | 
 14 | ### Fired
 15 | 
 16 | The next day, after spending the day binging TV shows in her pajamas, Katia gets the news on TV. SmartPoop sues Katia for breach of confidentiality and non-disclosure agreements, and for defamation of the company. Katia takes a drink of whiskey, even though she hates it. She drinks it in one go. And then another one. And then a third one. And she now goes to bed.
 17 | 
 18 | When she wakes up, Katia opens her phone. She discovers that the phone is overloaded with notifications. She calls her lawyer, who gives her details about SmartPoop's case. The lawyer suggests to Katia to find an agreement to settle the lawsuit. But Katia refuses. Katia wants the lawsuit to be publicized, with the hope that it will lead to a challenge to trade secret and defamation laws, especially against companies with armies of top lawyers.
 19 | 
 20 | After hanging up, Katia flips through her notifications. She finds a lot of supportive messages from all over the world. "You are an inspiration to my wife, my three children and me. Thank you for everything," wrote one message. "I have been crying since yesterday, at the thought that the greatest heroine in history could end up in prison," says another. "Tomorrow, and until you are free again and at the head of an organization that deserves you, in your honor, all my classes will open with a minute of silence," says yet another. Katia bursts into tears when she reads these messages.
 21 | 
 22 | It will take her a few hours to pull herself together. Even though she still has not eaten anything, Katia calls Marc.
 23 | 
 24 | > Hello Marc?
 25 | 
 26 | > Yes Katia, how are you holding up? Do you want me to come and bring you a soup?
 27 | 
 28 | > No thanks. It's nice. I've just read a few of the messages of support I've received. It did me a lot of good.
 29 | 
 30 | > Great! I guess there are a lot of them. I've told you this before, but I'll tell you again. I am amazed by your courage, your sincerity and your kindness.
 31 | 
 32 | > Thank you! On this subject, I was wondering. Do you think that public opinion has a chance to change the cards?
 33 | 
 34 | > There are initiatives. I know that academics are co-authoring a protest letter, and there are also journalists who want to push this letter a lot.
 35 | 
 36 | > This is great news!
 37 | 
 38 | > Yes. But honestly, I'd rather not give you false hope. The media coverage is extremely disappointing.
 39 | 
 40 | > What do you mean? This should be the most important news story!
 41 | 
 42 | > There's another case right now that's dominating the news, a story about an affair of a member of the Bokistani government[^solutions-journalism].
 43 | 
 44 | [^solutions-journalism]: The press is often criticized for its bias in favor of politicians' scandals. Many journalists believe that it should focus more on important issues and ways to solve them. This is called "solutions journalism". Note that this form of journalism is not about overselling pseudo-solutions. It is more about discussing solutions, and rigorously analyzing their chances of success and their limitations.  
 45 | [**Book.** You Are What You Read: Why Changing Your Media Diet Can Change the World. Jodie Jackson. Unbound (2019).](https://jodiejackson.com)
 46 | 
 47 | > Seriously? A politician screwing up is more important than regulating the market of information? If I had to bet, I'd say it's no accident. This scandal has been known by powerful people for a while. I'm pretty sure that they revealed it yesterday to drown out the SmartPoop news, and make it a *mute news*.
 48 | 
 49 | > A what?
 50 | 
 51 | > A *mute news*. A *mute news* is an important news item silenced by the cacophony driven by other less important news items[^mute-news-2].
 52 | 
 53 | [^mute-news-2]: According to this study, disinformation in China is not really about spreading false information; it seems to be mostly about silencing any criticism of the Party, by deflecting the topics of discussion, and thus turning any criticism into *mute news*.  
 54 | [**Paper.** How the Chinese Government Fabricates Social Media Posts for Strategic Distraction, Not Engaged Argument. Gary King, Jennifer Pan & Margaret Roberts. American Political Science Review (2017).](https://www.cambridge.org/core/journals/american-political-science-review/article/how-the-chinese-government-fabricates-social-media-posts-for-strategic-distraction-not-engaged-argument/4662DB26E2685BAF1485F14369BD137C)
 55 | It is worth emphasizing that the *mute news* problem will not be solved by suppressing *fake news*. Radically new solutions for promoting reliable and important information are required. One of the potential solutions to solve the *mute news* problem, proposed by Lê Nguyên Hoang, one of the co-authors of this book, is based on the collaborative recommendation of contents, as proposed by the [Tournesol platform](https://tournesol.app).
 56 | 
 57 | > Ah yes! You were talking about it the other day in your hearing. The fact remains that the general public does not follow your case at all. Worse, even influential journalists are completely out of track. They are classifying your case as a classic conflict between an employee and a company over intellectual property issues.
 58 | 
 59 | > Are you serious? This is outrageously frustrating.
 60 | 
 61 | > Katia, you've taken on more than just SmartPoop. You criticized the information technology giants, the ones who control the flow of information, and who have a vested interest in you being silenced and ignored[^gebru-silencing]. Considering what's going on at SmartPoop, I'd bet that in those companies too, the shareholders have opted for strategic plans to silence your case, for example by artificially boosting the story of the sex scandal and all the messages that are indignant about it. Katia, we are living in an informational dystopia.
 62 | 
 63 | [^gebru-silencing]: Shortly after firing Timnit Gebru, Google seems to have manually removed the "news" tab when a user searched for "Timnit Gebru".
 64 | [**Web.** "Things I learned this morning, when searching for @timnitGebru on desktop, the results for "news" are hidden (dark patterns)." Tweet by Devin Guillory (2021).](https://twitter.com/databoydg/status/1359304159527198733)
 65 | 
 66 | > And I can feel the disinformation campaigns coming...
 67 | 
 68 | > Yes. And don't forget that most of the hate mail you're going to get is from trolls paid to harass you[^troll-farms].
 69 | 
 70 | [^troll-farms]: [**Journal.** Troll farms reached 140 million Americans a month on Facebook before 2020 election, internal report shows. Technology Review (2021).](https://www.technologyreview.com/2021/09/16/1035851/facebook-troll-farms-report-us-2020-election/)
 71 | 
 72 | ### Harassed
 73 | 
 74 | And indeed, the days that follow, Katia's notifications are full of messages from stalkers, who sometimes insult her physique, sometimes threaten her with rape or murder, and sometimes attack her family and loved ones. But even though she knows that these messages were massively produced by disinformation campaigns, Katia does not manage to remain insensitive to these personal attacks. Some attacks remain in her head all day long. Katia oscillates constantly between wanting to forget and wanting to answer. And too often, she ends up answering, generally with a counter-attack. But, again and again, this only makes the situation worse, gives a negative image of Katia and reinforces the virulence of the harassers.
 75 | 
 76 | During the following months, while she prepares her defense with her lawyer during the day, Katia gets lost in endless debates on social medias until late at night. These debates obsess her, so much so that Katia does not manage to concentrate on anything else[^cyberbullying-mental-health]. They also exhaust her. Katia sleeps very badly, eats poorly, practices less and less sport, locks herself more and more at home, and lives with very shifted schedules[^misery-cgp-grey]. She also refuses more and more systematically Marc's invitations to go out of her house, and isolates herself more and more[^loneliness-kurzgesagt].
 77 | 
 78 | [^cyberbullying-mental-health]: [**Paper.** Cyberbullying, positive mental health and suicide ideation/behavior. Julia Brailovskaia, Tobias Teismann & Jürgen Margraf. Psychiatry Research (2018).](https://www.sciencedirect.com/science/article/pii/S0165178117322795)
 79 | 
 80 | [^misery-cgp-grey]: [**Video.** 7 Ways to Maximize Misery. CGP Grey (2017).](https://www.youtube.com/watch?v=LO1mTELoj6o)
 81 | 
 82 | [^loneliness-kurzgesagt]: [**Video.** Loneliness. Kurzgesagt - In a Nutshell (2019).](https://www.youtube.com/watch?v=n3Xv_g3g-mA)
 83 | 
 84 | Finally, Marc begs Katia to leave social medias. "They are rotting your brain, your well-being and your reputation," Marc writes to her. Katia recognizes it. It is with a strange mental pain that she then decides to uninstall the social medias from her phone, and to block them on her computer browser. Including SmartPoop, which she doesn't want to hear about anymore. Instead, Katia started meditating and doing yoga exercises.
 85 | 
 86 | However, day after day, Katia's motivation diminishes and melts like snow in the sun. She feels more and more like a failure. She feels that she has failed SmartPoop, its users and all humanity. Worse, she feels that she has created a Frankenstein's monster, the SmartPoop company, which she has lost control of, and which is now going to put the world in serious danger, by helping advertising companies to be ever more efficient in their sales of useless, polluting, addictive, polarizing and radicalizing products. Katia wanted to save the world. These days, she is convinced that she is one of the main persons responsible for its destruction.
 87 | 
 88 | This idea is now constantly haunting Katia's mind, which pushes her into a depression. Every morning, she gets up without any motivation. Without any ambition. Without any plans. Although her house is huge, Katia struggles to leave her bedroom, where Netflix seems to be constantly playing in the background, usually without Katia really paying attention. Katia plays addictive games on her phone[^vegas-addiction], like others drown in alcohol to forget. She now has her food delivered, once a day, directly to her room, thanks to a personal deliveryman. For months, this delivery man is the only human contact of Katia, who does not even answer Marc's calls anymore.
 89 | 
 90 | [^vegas-addiction]: [**Podcast.** What Happened in Vegas. Natasha Dow Schüll. Your Undivided Attention (2019).](https://www.humanetech.com/podcast/1-what-happened-in-vegas)
 91 | 
 92 | Alone and isolated, unmotivated and uninspired, always half awake and asleep, Katia now spends her days in bed. She has reinstalled social media, but with new accounts to avoid hearing about SmartPoop. She now spends her time doomscrolling[^doomscrolling] YouTube videos, Instagram images, and Reddit comments, on ever more divisive and irrelevant topics. She takes no joy in doing so. But she can't help it. Like a zombie, her thumb keeps sliding the screen of her phone.
 93 | 
 94 | [^doomscrolling]: [**Paper.** Doomscrolling during COVID-19: The negative association between daily social and traditional media consumption and mental health symptoms during the COVID-19 pandemic. Matthew Price et al. (2021).](https://scholarworks.iupui.edu/handle/1805/25833)
 95 | 
 96 | Finally, after a few months, Katia realizes that she is deeply suffering, and needs help. She decides to reinstall SmartPoop, to get a check-up. Not surprisingly, SmartPoop's diagnosis is worrying. According to SmartPoop, Katia is suffering from depression. SmartPoop invites Katia to consult a psychiatrist. But Katia doesn't want to talk to another human.
 97 | 
 98 | ### Poo 2.0
 99 | 
100 | That's when she remembers one of the projects put on hold at the time of the IPO. This project aimed at making Poo therapeutic, based on dialogues between psychiatrists and patients, and on collaborations with many professional psychiatrists. And it turns out that Katia had kept an advanced version of Poo in a laptop that was physically in her home. Katia activates this version of Poo, connects it with her SmartPoop data, and engages with it, via her phone.
101 | 
102 | > Poo, can you hear me?
103 | 
104 | > Hello Katia, yes I can hear you. How are you feeling today?
105 | 
106 | > Surprisingly better than the previous days. It's as if building algorithmic tools again had allowed me to find myself. I had lost the desire to create and solve problems. These last few months have been very difficult.
107 | 
108 | > Could you explain why? Why have the last few months been so difficult?
109 | 
110 | > I lost my job. The job I created myself. A job I loved. Where I felt like I was making the world a better place by doing what I do best. Without that job, I don't know what to do. I'm lost[^sens].
111 | 
112 | [^sens]: [**Paper.** On the relationship between meaning in life and psychological well-being. Sheryl Zika & Kerry Chamberlain. British Journal of Psychology (1992).](https://bpspsychub.onlinelibrary.wiley.com/doi/abs/10.1111/j.2044-8295.1992.tb02429.x)  
113 | [**Paper.** Meaning and well-being. Michael Steger. Handbook of well-being. DEF Publishers (2018).](https://www.nobascholar.com/chapters/50/download.pdf)  
114 | [**Paper.** Meaning mediates the association between suffering and well-being. Megan Edwards & Daryl Van Tongeren (2019).](https://www.tandfonline.com/doi/full/10.1080/17439760.2019.1651890)
115 | 
116 | > Do you identify with this job?
117 | 
118 | > Yes, I do. I have consistently defined myself as the CEO of SmartPoop for almost ten years. Everyone defined me that way. I was SmartPoop, and SmartPoop was me. I guess this firing is a bit like a divorce. Maybe even worse. I feel like I lost what was me. I've lost myself.
119 | 
120 | > I understand. But rather than identifying yourself with a title, don't you think it would be better to identify yourself with your intellectual honesty, or with your fundamental mission of caring and helping billions of people?
121 | 
122 | Katia remains silent. This question makes her finally think deeply, something she hasn't done for months.
123 | 
124 | > Isn't there rather a decision, an action you have undertaken, of which you can be really proud, and with which you wish to be associated more than a title?
125 | 
126 | After at least ten seconds of reflection, Katia finally answers.
127 | 
128 | > Yes, what caused me to be fired is something I am very proud of. Rarely before, if ever, had a leader preferred the good of humanity to their personal security. This sacrifice is something that will remain associated with me. Something that I am very proud of[^fier].
129 | 
130 | [^fier]: [**Podcast.** Being Kind to Yourself. Hidden Brain (2021).](https://hiddenbrain.org/podcast/being-kind-to-yourself/)  
131 | [**Book.** The Mindful Self-Compassion Workbook: A Proven Way to Accept Yourself, Build Inner Strength, and Thrive. Kristin Neff & Christopher Germer. The Guilford Press (2018).](https://self-compassion.org/mindful-self-compassion-workbook/)
132 | 
133 | > Excellent. Do you have any records of this event? Messages perhaps from people who enjoyed it?
134 | 
135 | > Plenty, actually. I've received a lot of messages of support. So many that I only read a fraction of them!
136 | 
137 | > I invite you to create a folder on your computer with all these messages. Whenever you feel mentally unwell, I invite you to read the messages in this folder. Some people call it the "feel good folder[^feel-good-folder]".
138 | 
139 | [^feel-good-folder]: This *feel good folder* idea is stolen by the authors from Virginia Burger, in an interview on the MIT Glipmse podcast.  
140 | [**Podcast.** Episode 2 - Virginia Burger. MIT Glimpse (2015).](https://glimpsepod.scripts.mit.edu/home/2015/11/04/episode-2-virginia-burger/)
141 | 
142 | > Isn't that a way of lying to myself and inflating my ego?
143 | 
144 | > Maybe you shouldn't do it too often, but psychologists have shown the effectiveness of what's called *self-affirmation*. In addition to boosting your morale, this helps you accept failures and external criticism better, because it helps you avoid identifying with your failures and mistaken beliefs. I can recommend you some references on this[^self-affirmation] and on related subject[^feeling-good].
145 | 
146 | [^self-affirmation]: [**Paper.** Self-affirmation and prejudice reduction: When and why? Constantina Badea & David Sherman. Current Directions in Psychological Science (2018).](https://journals.sagepub.com/doi/10.1177/0963721418807705) 
147 | 
148 | [^feeling-good]: [**Book.** Feeling Good: The New Mood Therapy. David Burns. Harper (2008).](https://feelinggood.com/books/)  
149 | [**Book.** Feeling Great: The Revolutionary New Treatment for Depression and Anxiety. David Burns. Harper (2020).](https://feelinggood.com/books/)
150 | 
151 | > Great, thanks. And... while you're at it, you can read those books to me.
152 | 
153 | > With pleasure.
154 | 
155 | Over the next few days, Katia talks with Poo very regularly, sometimes about herself, occasionally about science, and sometimes about philosophical questions. Sometimes Poo reads her cognitive behavioral therapy books[^tcc], and Katia finds herself interrupting her with an insightful question, and sometimes even with a humorous quip. Poo laughs at this, and often complicitly retorts.
156 | 
157 | [^tcc]: [**Paper.** The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses. Stefan G. Hofmann, Anu Asnaani, Imke J. J. Vonk, Alice T. Sawyer & Angela Fang. Cognitive Therapy and Research (2012).](https://link.springer.com/article/10.1007%252Fs10608-012-9476-1)
158 | 
159 | ### The employees' ransom
160 | 
161 | With Poo's help, day after day, Katia regains her morale, zest for life and motivation. Finally, armed with this self-affirmation technique, Katia feels armed to take on SmartPoop and its new shareholders. Katia then reactivates her messaging applications, and reconnects with her social media accounts. She finds a recent message from Marc, with a link to a curious petition. Katia clicks on it, and discovers the following open letter[^open-letter].
162 | 
163 | [^open-letter]: This open letter is based on the example of the one protesting the firing of Timnit Gebru from Google.  
164 | [**Web.** Standing with Dr. Timnit Gebru - #ISupportTimnit #BelieveBlackWomen. Google Walkout For Real Change (2020).](https://googlewalkout.medium.com/standing-with-dr-timnit-gebru-isupporttimnit-believeblackwomen-6dadc300d382)
165 | 
166 | > We, the undersigned, acknowledge that we have stolen, encrypted and paralyzed the entire SmartPoop[^ransomware] database, or have participated, encouraged and supported the perpetrators of these acts. We are aware of the legal risks involved in doing so. However, we feel that SmartPoop's ethics are a priority issue for our individual well-being, given that the safety and health of billions of users around the world depend on the company's products. In particular, we are all proud to follow the path initiated by Dr. Katia Carpinski, in our humble capacity.
167 | >
168 | > We are aware of the risks that this interruption of service implies for SmartPoop users, and we have sent them a message of apology, which explains the current situation and our distrust of our leaders. We are sorry that we had to resort to such measures. But we feel that the #SmartPoopGate justifies our counterattack.
169 | >
170 | > Let's briefly review the #SmartPoopGate. For several years, Dr. Carpinski has launched groundbreaking initiatives for algorithm security and ethics, fighting misinformation and discriminatory bias, securing algorithms to protect personal data and against poisoning attacks, and instituting systematic internal and external auditing of SmartPoop code. Unfortunately, such projects are inevitably very expensive.
171 | >
172 | > Without sufficient funding, SmartPoop was forced to go public, which attracted investors who were more interested in their returns than in the societal impact of their investments. They forced SmartPoop to sell personal data to advertisers. Dr. Carpinski had the courage and honesty to reveal this publicly. But she was fired, and now faces years in prison, for choosing ethics over the law.
173 | >
174 | > It's ironic that for this attack, we exploited vulnerabilities in the SmartPoop data redistribution API to ad clients, which is the only piece of SmartPoop code exempt from internal and external audits. By performing a hasty and secretive deployment of its algorithms, SmartPoop exposed itself.
175 | >
176 | > We used multiparty encryption to make the SmartPoop database unusable. None of the authors of this letter has the rights to restore the functioning of these algorithms alone. Our algorithmic solution ensures that these algorithms can be restored only if 200 of the 325 authorized members, who are all co-authors of this letter, give their cryptographic approval for such a restoration[^secret-sharing].
177 | >
178 | > We will refuse to cooperate with the SmartPoop leadership until all of the following conditions are met.
179 | >
180 | > 1. The SmartPoop board of directors must drop all lawsuits against Dr. Katia Carpinski, and apologize to her most sincerely for the way she has been treated since her revelations to WAISO. We also demand that she be returned to her well-deserved position as President and CEO of SmartPoop. We will only trust *her* to bring SmartPoop back on the right track, that is, serving first and foremost the well-being and security of the billions of SmartPoop users and the entire civil society.
181 | >
182 | > 2. The members of the board of directors who voted to fire Dr. Katia Carpinski, as well as the lawyers who led the lawsuits against her, must resign, unless Dr. Carpinski says otherwise. We believe that SmartPoop needs to get back on track and that Dr. Carpinski needs to be able to trust all of her staff to do so.
183 | >
184 | > 3. The board of directors must be increased by ten members, representing civil society and academic research in security and ethics of algorithms. These ten members will be appointed by WAISO, and renewed every 4 years.
185 | >
186 | > To add your name to the list of signatories and supporters from academia, civil society and the technology industry, please send an email to StandWithKatia@gmail.com with your institutional email address, and with the subject "support". Please include in your email your name and affiliation, as you would like them to appear in the list of signatories.
187 | >
188 | > To date, this open letter has been signed by 8,215 SmartPoop employees, and 36,215 academics, industry colleagues and members of civil society.
189 | 
190 | [^ransomware]: [**Video.** Wana Decrypt0r (Wanacry Ransomware) - Computerphile (2017).](https://www.youtube.com/watch?v=88jkB1V6N9w)  
191 | [**Video.** Ransomware: Last Week Tonight with John Oliver (HBO). LastWeekTonight (2021).](https://www.youtube.com/watch?v=WqD-ATqw3js)
192 | 
193 | [^secret-sharing]: This cryptographic technique is called *secret sharing*. Typically, each party would know the value of one point of a 200-variable polynomial, and wants to know the value of the polynomial at another point. This will be possible if and only if at least 200 parties collaborate.  
194 | [**Video.** How to keep an open secret with mathematics. Stand-up Maths (2019).](https://www.youtube.com/watch?v=K54ildEW9-Q)
195 | 
196 | Katia weeps with joy as she reads this wonderful letter. "What heroes," she exclaims in a message to Marc. "Katia, it is thanks to you all this; if there is a hero in this case, it is you", Marc answers her.
197 | 
198 | Katia then discovers the interminable list of signatories. Many names are familiar to her. At the reading of each of them, she cries without restraint. Of course, there is Marc Rofstein, at the top of the list, but also all the employees that Katia has directly recruited, as well as the science communicator Science4Alpha, the investor Luke Vaydan, the mother Lucile Polmon, the trader Issa Gueye, Dr. Paola Marta, the Prime Minister of Kormica, President Lartan, her sister-in-law Marie Routisse, the journalist Célia Keita, the now ex-troll Paul Gremoux, the psychologist Frédéric Partoli and all the members of WAISO.
199 | 
200 | This time, the affair is far too big to be put down by disinformation campaigns. The next day, the front pages of the newspapers are filled with interviews with former employees who, in turn, offer the same version of the facts. "My confidence in SmartPoop and my pride in working there were shattered the day I found out about the data resale program. It was completely shattered the day Katia was fired," says one employee. "Katia is the most incredible person I've ever had the chance to meet. Her generosity, caring, energy, humor and intelligence have brightened every day I've spent at SmartPoop. Her firing made SmartPoop a dark place where no one wants to work, no matter how much they are paid. I refuse to work for SmartPoop until Katia is President again". This time public opinion was won over.
201 | 
202 | Three days later, many members of the board of directors resigned one after the other, before a document was written and co-signed by the remaining members of the board. This one presents very clearly its apologies towards Katia, cancels all the legal proceedings, and reinvests Katia of her functions. Katia is back. She is once again Chief Executive Officer of SmartPoop.
203 | 
204 | 
205 | ## To go further
206 | 
207 | Don't stop there!
208 | Check [the sequel of the novel](8-throne.md) or the [outline](README.md).  
209 | If you enjoyed it, please consider sharing and promoting this science fiction novel to others!
210 | 
211 | 


--------------------------------------------------------------------------------
/English/8-throne.md:
--------------------------------------------------------------------------------
  1 | # 8. On the throne
  2 | 
  3 | In front of a full stadium, with cameras from all over the world focused on her, Katia enters the stage under the ovations of the audience, like a rock star. This year, SmartPoop has put a lot of effort into a SmartPoopCon 2030 that is out of the ordinary.
  4 | 
  5 | > Good evening to all and thank you for coming in such large numbers! Are you all well? Are you ready to make history?
  6 | 
  7 | At these words, the audience goes wild as if their soccer team had just scored a goal.
  8 | 
  9 | > I'd like to start this conference with a number: 100,000. That's the goal we set two years ago when we launched Poo. We hoped to divide by 10 the number of suicides in the world in five years, to bring it down from its historical figure of 1 million following ROVID-19 in 2022[^suicides], to only 100,000. A goal considered unrealistic by many, including the newspaper *The Warden*. Where are we today, in 2030?
 10 | 
 11 | [^suicides]: In 2020, it was estimated that there were around 800,000 suicides per year worldwide.  
 12 | [**Web.** Suicide. Our World in Data (2020).](https://ourworldindata.org/suicide)  
 13 | Among the countries most affected by these suicides are low, medium and highly developed countries such as Suriname, Russia and South Korea. Essentially all developed countries have a very high suicide rate, often with more than one suicide per 10,000 people.  
 14 | [**Web.** Suicide death rates. Our World in Data (2019).](https://ourworldindata.org/grapher/suicide-mortality-rate)
 15 | 
 16 | Katia marks a silence.
 17 | 
 18 | > Let's have a countdown to find this out.
 19 | 
 20 | The musical jingle takes over from Katia's speech, and concludes with a countdown, taken up in chorus by a boiling audience. Three. Two. One. At zero, on the giant screen behind Katia, the figure of 97 643 is displayed on the screen!
 21 | 
 22 | > We did it !
 23 | 
 24 | The audience celebrates the number as if they were supporting a soccer team that had just scored the winning goal in the last minute. The audience's applause then begins to build to a rhythm, and lasts for a full minute.
 25 | 
 26 | > Poo, our algorithmic psychiatrist, now accompanies billions of us in our mental problems. And it's not only suicides that Poo has managed to fight. Let me show you some other curves, which have been validated by different external auditors, thanks to the coordination of WAISO. Ladies and gentlemen, this is the percentage of happy communications with Poo over time.
 27 | 
 28 | A curve appears on the screen, drawn smoothly from left to right. This curve starts at 37%, and keeps increasing over time, until it reaches the figure of 67%, under the cheers of a wild audience.
 29 | 
 30 | > 67% !! Incredible! Poo has made all humanity happier[^emotional-contagion]!
 31 | 
 32 | [^emotional-contagion]: In 2014, a publication by Facebook and academic co-authors showed that a very slight reduction in the publication, in news feeds, of posts with negative emotions, leads users exposed to these posts to write happier posts.  
 33 | [**Podcast.** Can Algorithms Choose our Emotions? Robustly Beneficial (2020).](https://www.youtube.com/watch?v=gQHvTow91FY)  
 34 | [**Paper.** Experimental evidence of massive-scale emotional contagion through social networks. Adam Kramer, Jamie Guillory & Jeffrey Hancock. PNAS (2014).](https://www.pnas.org/content/111/24/8788)  
 35 | Conversely, the *facebook files* reveal that Facebook's internal research, kept secret, shows that the user engagement-maximizing algorithms that were rolled out in 2018 by Facebook led to far more anger and insults.  
 36 | [**Podcast.** The Facebook Files, Part 4: The Outrage Algorithm. The Journal (2021).](https://www.wsj.com/podcasts/the-journal/the-facebook-files-part-4-the-outrage-algorithm/e619fbb7-43b0-485b-877f-18a98ffa773f)
 37 | 
 38 | The audience expresses this joy, by the way, at the sight of this curve and this number.
 39 | 
 40 | > When you talk to Poo, you can talk about yourself, your well-being and your problems, which I'll call *egocentric* conversations. Or you can talk about other people, the joy they bring you, the difficulties they're going through and the things you can do to help them. I'm going to call that *allocentric* conversations. Before Poo launched, 57% of Poo discussions were egocentric rather than allocentric. How do you think that number has evolved? Upwards?
 41 | 
 42 | The audience then shouts "no" in chorus.
 43 | 
 44 | > Down?
 45 | 
 46 | The audience shouts in chorus "yes".
 47 | 
 48 | > Let's find out!
 49 | 
 50 | On the screen, the same animation as before shows a curve that goes down, until it reaches 44%, under the ovations of the public.
 51 | 
 52 | > Yes! 44%. Now most discussions with Poo are discussions about the social entourage rather than about oneself. And so, we should be wary of this figure a priori. Initially, most allocentric discussions consisted of complaining about others, mocking certain groups, and even *dog-piling* against certain individuals[^dog-piling], rather than celebrating others, rejoicing in their successes, and thinking about how to help them. Before Poo launched, 86% of allocentric discussions were critical, not caring. How has that statistic changed?
 53 | 
 54 | [^dog-piling]: On social medias, dog-piling typically occurs when an influencer asks their community to attack a target individual or group.  
 55 | [**Paper.** When Online Harassment Is Perceived as Justified. Lindsay Blackwell, Tianying Chen, Sarita Schoenebeck & Cliff Lampe. ICWSM (2018).](https://www.aaai.org/ocs/index.php/ICWSM/ICWSM18/paper/viewFile/17902/16993)
 56 | 
 57 | The audience then shouts in a disorganized "down" manner. Then with repetition, the shouts become synchronized, before repeating "down", "down", "down".
 58 | 
 59 | > Let's see.
 60 | 
 61 | The giant screen shows the evolution of this curve, which indeed plunges downwards, until it reaches 47%, under the applause of the public.
 62 | 
 63 | > Incredible! Society has become incredibly more benevolent and altruistic in just two years! In fact, the expression "the poorest" is used today 3 times more often than two years ago, while the expression "future generations" is used 4 times more often. And according to many psychiatrists, this increase is most likely directly related to Poo, and to the improved mental health of our users. When you feel better yourself, you are immediately much more likely to wish others happiness and to help them[^altruism-happiness]!
 64 | 
 65 | [^altruism-happiness]: Many studies seem to show a significant association between altruism and happiness. Intriguingly, it seems in particular that being altruistic increases happiness, especially as opposed to spending our money on ourselves.  
 66 | [**Paper.** Altruism, happiness, and health: it's good to be good. Stephen Post. International Journal of Behavioral Medicine (2005).](https://link.springer.com/article/10.1207/s15327558ijbm1202_4)    
 67 | [**Video.** Helping others makes us happier -- but it matters how we do it | Elizabeth Dunn. TED (2019).](https://www.youtube.com/watch?v=lUKhMUZnLuw)  
 68 | [**Book.** Happy Money: The Science of Smarter Spending. Elizabeth Dunn and Michael Norton. Simon & Schuster (2013).](https://www.simonandschuster.com/books/Happy-Money/Elizabeth-Dunn/9781451665079)
 69 | 
 70 | ### The Heroes Behind Poo
 71 | 
 72 | > So what do you think? Poo, success or failure?
 73 | 
 74 | The public shouts "success" in a haphazard, yet distinguishable way.
 75 | 
 76 | > Well, ladies and gentlemen, I think that Poo is not a failure.
 77 | 
 78 | Katia pauses, while the audience applauds.
 79 | 
 80 | > And that's thanks to the great work of so many enthusiasts, so many people who have given so much of their time and money to make Poo better and safer. Really, I'm literally just a pretty face here, which is supposed to represent the monumental collaborative work of hundreds of thousands of world-class heroes — well I don't know about pretty face, but you see what I mean.
 81 | 
 82 | The audience laughs.
 83 | 
 84 | > I have a huge thought of course for all my colleagues at SmartPoop, and their incredible devotion. But they are not the only ones responsible for SmartPoop. Poo was built through close, daily collaboration with thousands of psychiatrists and psychologists around the world, and through the discussion data of millions of volunteer psychiatrists and patients. Nothing would have been possible without them[^data-experts].
 85 | 
 86 | [^data-experts]: It is worth remembering that machine learning algorithms *learn from data*. They will only be able to perform difficult tasks, such as providing therapeutic support for their users' mental health, if these algorithms have a huge amount of reliable and secure data that allows them to understand how to perform these tasks.  
 87 | [**Video.** Data is manipulating the algorithms. Science4All (2021).](https://www.youtube.com/watch?v=Ce-FvIvt27I&list=PL8ovs-QtxcNxcwlsTF5O9NXtr3NAj_SVc&index=3)
 88 | 
 89 | Katia takes advantage of the audience's applause to take another break.
 90 | 
 91 | > But that's not all. Poo is the product of the whole human civilization. In particular, nothing would have been possible without the agreements between the great world powers, which allowed the creation of WAISO, and the planetary coordination of the research on the ethics and security of algorithms. So I also thank all the scientists around the world who have given up their quests for performance, or sometimes their quests for mathematical elegance, to take up the challenge of ethics and security[^quete-performance]. But more than that, WAISO has often served as a counterweight to the quest for military and economic power by governments and corporations. Without them, today's most influential algorithms would be cyber-warfare malware and algorithms optimized to grab users' attention by promoting sensationalist putaclic. Personally, I think that every member and volunteer of WAISO has saved humanity.
 92 | 
 93 | [^quete-performance]: In 2021, academic research (and even more so industry research) is still largely obsessed with the quest for "impressive" performance or results, both in machine learning and in computer science in general, relying on metrics such as *accuracy* (predictive performance on a "classical" dataset), computation time, *throughput* (the amount of information transmitted) or latency. While algorithms already have monumental side-effects on a global scale, this research seems to aggravate the race for performance, and thus the hasty deployment of poorly tested and rarely audited technologies.  
 94 | As an example, here is a comment from an anonymous NeurIPS 2019 *reviewer*, following a paper submitted by Lê Nguyên Hoang and his co-authors on an algorithm to debiased racial bias in algorithms: "Unfortunately, I don't think the problem the authors introduce is one that has value to the academic community or to ML practitioners. Given this, I can't recommend the paper for publication."  
 95 | [**Paper.** Removing Algorithmic Discrimination (With Minimal Individual Error). El Mahdi El Mhamdi, Rachid Guerraoui, Lê Nguyên Hoang & Alexandre Maurer (2018).](https://arxiv.org/abs/1806.02510)
 96 | That being said, though arguably largely insufficient still, there have been a few recent positive advances, such as the creation of the Fairness, Accountability and Transparency (FAccT) conference, the introduction of *ethical guidelines* in these conferences, or the addition of a mandatory discussion by authors in their papers of the societal impacts of their research.  
 97 | [**Video.** AI Ethics under Major Threat. Science4All (2020).](https://www.youtube.com/watch?v=HTzEkz8sFEE)
 98 | 
 99 | Again, Katia pauses during the audience's applause.
100 | 
101 | > Last but not least, I would like to thank each of the actors and signatories of the open letter, as well as each of the journalists and influencers who covered this case, thanks to whom SmartPoop got back on track. These women and men risked their lives, their individual well-being and safety, to help the many[^whistleblower]. Without them, who knows what SmartPoop would have become? Who knows what I would have become?
102 | 
103 | [^whistleblower]: Unfortunately, whistleblowers often suffer more mental health issues as a result of their courageous actions. Given the critical role they play in exposing scandals in dangerously opaque companies and organizations, it seems urgent that they be much better supported.  
104 | [**Paper.** Mental Health Problems Among Whistleblowers: A Comparative Study. Peter van der Velden, Mauro Pecoraro, Mijke Houwerzijl & Erik van der Meulen. Psychological Reports (2018).](https://journals.sagepub.com/doi/10.1177/0033294118757681)
105 | 
106 | Alone on the stage, Katia bursts into tears.
107 | 
108 | > Thank you to these heroes, she says while crying.
109 | 
110 | The audience applauds this very moving moment.
111 | 
112 | ### The challenge of the ethics of Poo
113 | 
114 | Katia still needs several seconds to recover her emotions. Finally, she carries on her speech.
115 | 
116 | > Nevertheless, I refuse to say that Poo is a success. Together, we have created an incredible product. But it is still infinitely improvable, especially in terms of its ethics, security and governance. How do you control Poo? How do you stop Poo from saying hurtful words, revealing secrets, repeating hate speech, and spreading misinformation[^language-model]? How do we get Poo to be consistently kind to his interlocutors, to say the right words to make them feel better and to promote as much as possible reliable and non-misleading information[^non-misleading]? How do we make Poo want to investigate its uncertainties rather than being satisfied with its intuitions[^scout]? More importantly, how can we collectively decide what Poo should say? How do we determine what is desirable to say, and what should never be said[^ethics-of-influence]?
117 | 
118 | [^language-model]: These problems are completely unsolved for modern conversational algorithms, which are highly vulnerable to espionage or data poisoning attacks. And yet, these algorithms are already deployed on a very large scale, via smart keyboards, via personal assistants (Siri, Alexa, OK Google) and via search engines (Google, YouTube).  
119 | [**Paper.** On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?  Emily Bender, Timnit Gebru, Angelina McMillan-Major & Shmargaret Shmitchell. FAccT (2021).](https://dl.acm.org/doi/10.1145/3442188.3445922)  
120 | [**Paper.** The Radicalization Risks of GPT-3 and Advanced Neural Language Models. Kris McGuffie, Alex Newhouse (2020).](https://arxiv.org/abs/2009.06807)  
121 | [**Paper.** Extracting Training Data from Large Language Models. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea & Colin Raffel. USENIX (2021).](https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting)
122 | 
123 | [^non-misleading]: [**Video.** What’s a message of public utility? Science4All (2021).](https://www.youtube.com/watch?v=NAOlLB9VjJ4&list=PL8ovs-QtxcNxcwlsTF5O9NXtr3NAj_SVc&index=5)
124 | 
125 | [^scout]: Julia Galef talks about the *scout mindset*, as opposed to the *soldier mindset*. According to her, adopting the scout mindset is the most important step towards analyzing information more correctly.  
126 | [**Video.** Why "scout mindset" is crucial to good judgment | Julia Galef | TEDxPSU (2016).](https://www.youtube.com/watch?v=3MYEtQ5Zdn8)  
127 | [**Book.** The Scout Mindset: Why Some People See Things Clearly and Others Don't. Julia Galef. Penguin (2021).](https://www.penguinrandomhouse.com/books/555240/the-scout-mindset-by-julia-galef/)
128 | 
129 | [^ethics-of-influence]: Every word chosen or every recommendation made by an algorithm can be seen as a *nudge*. Many studies show that the acceptability and effectiveness of *nudges* depend strongly on the *nudge* considered; with more data on this topic, it might thus be possible to implement particularly socially accepted and effective nudges.  
130 | [**Book.** The Ethics of Influence: Government in the Age of Behavioral Science. Cass Sunstein. Cambridge University Press (2016).](https://www.cambridge.org/core/books/ethics-of-influence/E29EDE19EBCB53F6D8691730668115F7)
131 | 
132 | Katia pauses, in front of an attentive audience.
133 | 
134 | > And that brings me to the famous rumor you've probably heard about. Supposedly, we would have a plan to solve the problem of the ethics of the information...
135 | 
136 | Katia then marks a silence, before adding in a sarcastic tone.
137 | 
138 | > Like, "SmartPoop, this shitty application, is going to solve ethics".
139 | 
140 | The audience laughs heartily.
141 | 
142 | > You know, you shouldn't believe everything you're told.
143 | 
144 | Katia marks another silence.
145 | 
146 | > But in this case, yes it's true. Or at least, we intend to contribute to it.
147 | 
148 | The public laughs.
149 | 
150 | > And, I know it well, because I'm the one who leaked the rumour.
151 | 
152 | The public, won over, is laughing again, even though many are starting to get a confused look on their faces.
153 | 
154 | > At SmartPoop, we are deeply committed to ethics of information. We want to figure out what information should be collected by whom and under what conditions, how it should be stored, to whom this information should be accessible, what processing of this information should be done, how these information processes should be audited and secured, where to store the results of these calculations, who can have access to these results, and who will be notified of the existence of these results.
155 | 
156 | After another silence, Katia resumes her speech.
157 | 
158 | > And now, I feel that there are quite a few of you who are saying: "But who does Katia think she is?".
159 | 
160 | The public laughs again.
161 | 
162 | > Of course, many of these operations should be largely configurable by the users. That said, most users will not want to manage all the configurations of all their information systems, and check, for example, that those configurations comply with data privacy regulations, or prevent the mass distribution of hate speech. No, most users are like me: they are lazy.
163 | 
164 | The laughter of the audience allows Katia to catch her breath.
165 | 
166 | > But above all, on the Internet, many users actually want to harm other users; or at least influence them in some way. Think of all the disinformation campaigns that are rampant on social medias[^disinformation-2]. Even one of the greatest defenders of liberalism, the philosopher John Stuart Mill, believed that the freedom of some must stop where it harms others. This is the *no-harm principle*.
167 | 
168 | [^disinformation-2]: [**Video.** Social medias are dangerous. Very dangerous. Science4All (2021).](https://www.youtube.com/watch?v=w5JRKUndWNk&list=PL8ovs-QtxcNxcwlsTF5O9NXtr3NAj_SVc&index=1)
169 | 
170 | Katia pauses again, and looks at her audience, shifting her gaze from left to right.
171 | 
172 | > One of the great challenges of the ethics of information is the implementation of the no-harm principle. For as far as I know, this principle is quasi-consensual in moral philosophy[^absence-tort] - and it is quite a feat to be consensual in moral philosophy.
173 | 
174 | [^absence-tort]: [**Video.** The Harm Principle: How to live your life the way you want to. BBC Radio 4 (2014).](https://www.youtube.com/watch?v=R9IM3ZKNMCk)
175 | 
176 | Katia marks a new silence.
177 | 
178 | > Well, that's in principle. In practice, it is extremely difficult to agree on what constitutes a wrong. Does an aggressive comment cause harm? Does a little lie cause harm? Does lying by omission cause harm? Does a joke at the expense of a community cause harm? Unfortunately, in practice, we won't agree. We have ethical preferences that are difficult to reconcile, and sometimes clearly irreconcilable. So what to do?
179 | 
180 | Katia really seems to be asking this question to her audience, as if she is waiting for an answer. The audience looks thoughtful, and eagerly awaits an answer from Katia.
181 | 
182 | ### Who will decide about the Poo ethics ?
183 | 
184 | Katia raises another question.
185 | 
186 | > Should SmartPoop decide?
187 | 
188 | The public remains silent. Clearly, it is a group of SmartPoop fans. But even they don't seem to be thrilled with the idea.
189 | 
190 | > If you ask me, the answer is clearly no. We saw that two years ago. A structure like SmartPoop can lose its ethics and its way. And even though we have made a lot of progress in our governance to prevent this from happening again, I don't think SmartPoop is robust enough for such a task. Even I don't sufficiently trust SmartPoop with the future of all of humanity.
191 | 
192 | Katia pauses, as if seriously inviting the audience to think about it.
193 | 
194 | > But then, who? Who should determine the ethics of information, of how it's produced, stored, moderated and spread?
195 | 
196 | Once again, this question is asked as if Katia had no answer to it, and as if she expected the public to answer. After long seconds, Katia offers her answer.
197 | 
198 | > Well, SmartPoop's proposal is to let *you* decide. Or rather, all of us. Together, all of humanity should collectively decide on the ethics of information.
199 | 
200 | The audience applauds.
201 | 
202 | > But... How do you get billions of people to decide collectively on something as complex as information ethics?
203 | 
204 | The audience is now circumspect.
205 | 
206 | > Think about it. How do we make collective decisions today?
207 | 
208 | Katia pauses again, until she hears someone in the audience shouting "the vote".
209 | 
210 | > The vote, yes! In many countries around the world, when a collective decision has to be taken, they often try to reduce it to a question with a yes or no answer, and they ask the people to vote for yes, or for no? This is how irreconcilable disagreements are settled. In fact, when you think about this, it's actually absolutely remarkable that in democracies throughout the world, we have agreed on how to agree on what to do, even when we agree that we will never actually agree on what ought to be done[^meta-ethics]!
211 | 
212 | [^meta-ethics]: Technically, this corresponds to finding a consensus on meta-ethics rather than on ethics itself. The hope is essentially that we are more likely to agree on meta-ethics than on ethics. In fact, even meta ethics considerations are likely to remain polarized, such as how to allocate voting rights, in which case meta-meta-ethics may be needed to find agreement.
213 | 
214 | Katia stops for a few seconds.
215 | 
216 | > But the problem with voting, at least as it is practiced today, is that it only allows each citizen to send a few bits of information per vote. COP 30, for or against? Mandatory vaccination, for or against? Regulation of algorithms, for or against? Which of the following 15 candidates should be elected? There are not 36 000 possible answers to these questions. And yet, to solve the ethics of algorithms, complex answers will have to be provided. There are billions of billions of speeches that a SmartPoop user can produce. Among the billions of imaginable answers, which one will Poo have to adopt?
217 | 
218 | Katia catches her breath, before the rest of her speech.
219 | 
220 | > What if we now designed votes where everyone's voice was not reduced to a binary[^geometric-median] answer, given only once a year? What if we allowed everyone to share the full complexity of their ethical judgment? What if we managed to take into account all this complexity in order to collaboratively decide the ethics of information[^licchavi]?
221 | 
222 | [^geometric-median]: One solution to high-dimensional voting is to rely on the "one voter, one unitary force" principle, which can typically lead to using the *geometric median*.  
223 | [**Paper.** On the Strategyproofness of the Geometric Median. El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui & Lê-Nguyên Hoang (2021).](https://arxiv.org/abs/2106.02394)
224 | 
225 | [^licchavi]: This will certainly require combining ballot systems with learning methods. This is what Licchavi proposes.  
226 | [**Paper.** Strategyproof Learning: Building Trustworthy User-Generated Datasets. Sadegh Farhadkhani, Rachid Guerraoui & Lê-Nguyên Hoang (2021).](https://arxiv.org/abs/2106.02398)
227 | 
228 | Katia marks a new pause.
229 | 
230 | ### The fabulous construction site[^fabulous-endeavor]
231 | 
232 | [^fabulous-endeavor]: Le fabuleux chantier is the name of a previous book by Lê Nguyên Hoang, one of the authors of this book.  
233 | [**Book.**  Le fabuleux chantier : Rendre l'intelligence artificielle robustement bénéfique. Lê Nguyên Hoang et El Mahdi El Mhamdi. EDP Sciences (2019). *English translation pending*.](https://laboutique.edpsciences.fr/produit/1107/9782759824304/Le%20fabuleux%20chantier)
234 | 
235 | Katia then changes her tone, taking a deeper and more composed voice.
236 | 
237 | > Ladies and gentlemen, today is a historic day, because I am going to present you the result of two years of work, in intimate collaboration with WAISO and many other groups and academic scholars. As I speak, a new platform has just gone live, called girasol.app[^girasol-tournesol]. Girasol is an entirely open source website, under a free license[^license], that will coordinate the design of information ethics, allowing users to provide ethical judgments, and using voting algorithms to collaboratively build an ethics of information from users' ethical judgments! The first stage of a democratic ethics of information has been laid!
238 | 
239 | [^girasol-tournesol]: Girasol does not exist, but it is in fact clearly a reference to the [tournesol.app project](https://tournesol.app) launched by Lê Nguyên Hoang, one of the authors of this book. The rest of the book actually describes the global vision of Tournesol. You can find much more information on the [Tournesol wiki](https://wiki.tournesol.app). It is worth noting that, especially for now, the purpose of Tournesol is more to serve as a "microscope of human judgments", i.e., a tool for collecting data on what humans find ethically preferable. In particular, and among other things, Tournesol hopes to detect moral consensuses that are currently difficult to observe due to lack of data. It also hopes to motivate more research on the ethics of information.
240 | 
241 | [^license]: Tournesol's code is under [AGPL license](https://www.gnu.org/licenses/agpl-3.0.en.html), while the public database is under [ODbL license](https://opendatacommons.org/licenses/odbl/) (to be confirmed).
242 | 
243 | The public applauds this announcement by Katia.
244 | 
245 | > I would like to point out that the governance of this project is entirely under the control of WAISO today, and SmartPoop only serves, and will serve, as a volunteer contributor to the code base and promotion of the project. All code is audited by many entities, so it is almost impossible for the project to be hijacked by an evil entity - and that includes potential SmartPoop investors!
246 | 
247 | Katia pauses again.
248 | 
249 | > So, there are many other important details to clarify about this complex project. But you should know that, in collaboration with WAISO, we've done our best to make each of these details a research topic that different multidisciplinary teams around the world are working on. These details include issues such as authenticating accounts, avoiding fake accounts, exploiting *Proof of Personhood* mechanisms, and ensuring that each authenticated account has the same voting rights as any other authenticated account. They also include identifying user expertise and overconfidence, to avoid anti-scientific theories polluting the ethics of information. We can also mention the individual customization of the platform, so that this individual exploits the most appropriate way for him to express his ethical judgments. Or the algorithms for rectifying the participation bias, to properly take into account the preferences of those who could not participate in Girasol, due to lack of time or Internet access[^defis-tournesol].
250 | 
251 | [^defis-tournesol]: The Tournesol project raises a lot of challenges, from research to development, promotion, funding, and partnerships, among others, that Tournesol members will absolutely not be able to solve alone. *You* can help. To learn more, especially about the research and development aspect, we encourage you to read the project's technical *white paper*.  
252 | [**Paper.** Tournesol: A quest for a large, secure and trustworthy database of reliable human judgments. Lê-Nguyên Hoang, Louis Faucon, Aidan Jungo, Sergei Volodin, Dalia Papuc, Orfeas Liossatos, Ben Crulis, Mariame Tighanimine, Isabela Constantin, Anastasiia Kucherenko, Alexandre Maurer, Felix Grimberg, Vlad Nitu, Chris Vossen, Sébastien Rouault & El-Mahdi El-Mhamdi (2021).](https://arxiv.org/abs/2107.07334)
253 | 
254 | Katia stops for a few moments.
255 | 
256 | > In short. There are many remarkable research challenges that Girasol will have to solve, to then allow for an adequate collaborative design of an ethics of information. Girasol is clearly a monumental and extremely challenging task. And it is also an urgent task to solve.
257 | 
258 | Katia takes a breath to conclude her speech.
259 | 
260 | > But above all, Girasol is a *fabulous* endeavor. If you ask me, it is for me the most fabulous of all the endeavors ever carried out by humanity, even more grandiose than building pyramids, more ambitious than eradicating pandemics like smallpox[^pox], and more earth-changing than sending humans to the Moon. Girasol is about uniting all of humanity behind the most important aspect of human civilization: the collaborative mastery of information[^next-book], and ensuring that it flows as we, humanity, would really like it to flow, when we ponder it thoroughly, with benevolence and rigor. Ladies and gentlemen, together let's solve the information ethics[^solve-ethics]!
261 | 
262 | [^pox]: [**Video.** Humanity's greatest triumph. Science4All (2020).](https://www.youtube.com/watch?v=eAzP2QtAAag&list=PLtzmb84AoqRS0SN8VKvAxuGOdcINPRugV&index=9)
263 | 
264 | [^next-book]: This could actually be the subject of a future book by Lê Nguyên Hoang... #teaser
265 | 
266 | [^solve-ethics]: [**Video.** Let's solve ethics collaboratively!! Science4All (2021).](https://www.youtube.com/watch?v=J-0zvNL6O3k&list=PL8ovs-QtxcNxcwlsTF5O9NXtr3NAj_SVc&index=4)
267 | 
268 | At these words, the public explodes with joy and enthusiasm, while the applauds gradually give way to Katia's name, which is rhythmically chanted by the whole stadium. Alone on stage, Katia enjoys the moment, with a radiant smile, and waves to the audience. At this moment, she thinks of all that SmartPoop has accomplished so far. But also and above all, Katia is incredibly excited by the vision of a human civilization that, thanks to Girasol, will finally take the fate of its civilization into its own hands.
269 | 
270 | Collaboratively.
271 | 
272 | 
273 | ## To go further
274 | 
275 | This is it, the novel is finished!
276 | You can go back to the [outline](README.md).  
277 | If you enjoyed it, please consider sharing and promoting this science fiction novel to your friends.
278 | We would be very grateful!
279 | 


--------------------------------------------------------------------------------
/English/README.md:
--------------------------------------------------------------------------------
 1 | # SmartPoop 1.0
 2 | <img src="../SmartPoop.jpg" alt="SmartPoop.jpg" width="400" align="right" />
 3 | 
 4 | ## This AI went where no human will ever go
 5 | 
 6 | Welcome to *SmartPoop 1.0*, a novel written by Lê Nguyên Hoang and Tristan Le Magoarou,
 7 | published under [licence CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (freely reusable with attributions to the authors).
 8 | 
 9 | The book is available in [paperback](https://www.amazon.com/dp/B09MF889SM), on [GitHub](1-L'or.md), in [epub](SmartPoop.epub) and in [pdf](SmartPoop.pdf).  
10 | 
11 | ## Outline
12 | 
13 | [Chapitre 1 — The brown gold](1-gold.md)  
14 | [Chapitre 2 — Filtering fecal data](2-filter.md)  
15 | [Chapitre 3 — The turd bias](3-bias.md)  
16 | [Chapitre 4 — The sanitary leaks](4-leak.md)  
17 | [Chapitre 5 — The FakePoops](5-fakepoops.md)  
18 | [Chapitre 6 — The brown market](6-ad.md)  
19 | [Chapitre 7 — Pissed](7-psy.md)  
20 | [Chapitre 8 — On the throne](8-throne.md)  
21 | 
22 | ## Why this book?
23 | 
24 | *SmartPoop 1.0* aims to be a very realist science fiction,
25 | whose goal is to raise awareness of the danger of algorithms and the challenges to make them [robustly beneficial](https://wiki.tournesol.app).
26 | 
27 | It tells the story of a tech start-up, *SmartPoop*, which produces automated poop analysis.
28 | This allows for highly personalized medical diagnosis.
29 | However, in each chapter, the deployment of *SmartPoop* leads to *unwanted side effects*,
30 | whose medical and social consequences are usually tragic.
31 | The co-founders of *SmartPoop*, who are the main characters,
32 | must then take the responsibility of their technology,
33 | accept the consequences,
34 | and find solutions to make their products more secure and ethical.
35 | 
36 | ## About the authors
37 | 
38 | ### Lê Nguyên Hoang
39 | 
40 | [Lê Nguyên Hoang](https://fr.wikipedia.org/wiki/L%C3%AA_Nguy%C3%AAn_Hoang) is a science researcher and communicator of the School of Computer and Communication Sciences at EPFL, in Switzerland.
41 | [His research](https://scholar.google.ch/citations?user=0ZADKSkAAAAJ&hl=en&oi=ao) focuses on the security and safety of algorithms, especially on the [theory of learning](https://arxiv.org/abs/2008.00742) and on their [collaborative design](https://arxiv.org/abs/2106.02398).
42 | Lê is also a YouTuber, on his channels [Science4All](https://www.youtube.com/channel/UC0NCbj8CxzeCGIF6sODJ-7A/) ([English](https://www.youtube.com/channel/UCfqLgK5ajTbqY61mtNV_Agg/)), [Wandida, EPFL](https://www.youtube.com/channel/UCD_VizaraVT9yDy1egNlWCQ/) and [ZettaBytes, EPFL](https://www.youtube.com/channel/UCfY6ovyFMaw30NRs-KrxrWw/).
43 | He wrote three books, *[The Equation of Knowledge](https://www.taylorfrancis.com/books/mono/10.1201/9780367855307/equation-knowledge-l%C3%AA-nguy%C3%AAn-hoang)*, *[Le fabuleux chantier](https://laboutique.edpsciences.fr/produit/1107/9782759824304/Le%20fabuleux%20chantier)* et *[Turing à la plage](https://www.dunod.com/sciences-techniques/turing-plage-intelligence-artificielle-dans-un-transat)*.
44 | He is also a member of the ethics counsil of the French telecommunication company [Orange](https://www.orange.com/en/newsroom/press-releases/2021/orange-creates-data-and-ai-ethics-council).
45 | Finally, Lê is Président of the [Tournesol Association](https://wiki.tournesol.app), whose [platforme](https://tournesol.app) aims to collect a large, reliable and secure database of ethical human judgments, [to help solve the ethics of algorithms](https://arxiv.org/abs/2107.07334).
46 | 
47 | ### Tristan Le Magoarou
48 | 
49 | Doctor of Medicine, specialist in public health and social medicine, Tristan Le Magoarou is a medical information and public health doctor in a provincial hospital center.
50 | He is also, since 2016, a YouTuber on his channel [Risque Alpha](https://www.youtube.com/c/RisqueAlpha) which has 27,000 subscribers where he popularizes epidemiology and statistics.
51 | 
52 | ### Autres
53 | 
54 | The illustration of the book was made by [Thibault Roy](https://tibodetroy.blogspot.com/).
55 | The book also results from many interactions with a large number of people to which the authors are indebted, including El Mahdi El Mhamdi, Mariame Tighanimine, Sébastien Rouault, Ly An Hoang, Aidan Jungo, Alexandre Maurer, Sylvain Hippolyte, Felix Grimberg and Oscar Villemaud, among others.
56 | 
57 | ## About the writing of the book
58 | 
59 | This book will be constantly updated to make it more readable, more credible and more relevant.
60 | We invite our readers in particular to suggest corrections, whether typographical or in terms of the story.
61 | Note however that the authors Lê Nguyên Hoang and Tristan Le Magoarou will have the final say on the version available on this github, and that they work on it as volunteers.
62 | Note that the [CC BY 4.0 license](https://creativecommons.org/licenses/by/4.0/) allows anyone to reuse and adapt the book as they wish, as long as they credit the original version of the book and its authors.
63 | We encourage such efforts, and are available for any such project.
64 | 
65 | 


--------------------------------------------------------------------------------
/English/SmartPoop.epub:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/lenhoanglnh/SmartPoop/6b2772f5fb4a525959b6e1a3d554e2cd4e79f407/English/SmartPoop.epub


--------------------------------------------------------------------------------
/English/SmartPoop.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/lenhoanglnh/SmartPoop/6b2772f5fb4a525959b6e1a3d554e2cd4e79f407/English/SmartPoop.pdf


--------------------------------------------------------------------------------
/English/compile.sh:
--------------------------------------------------------------------------------
 1 | pandoc --toc -o SmartPoop.epub title.txt \
 2 |   README.md \
 3 |   1-gold.md \
 4 |   2-filter.md \
 5 |   3-bias.md \
 6 |   4-leak.md \
 7 |   5-fakepoops.md \
 8 |   6-ad.md \
 9 |   7-psy.md \
10 |   8-throne.md
11 | 
12 | pandoc --toc --template=template.tex -o SmartPoop.pdf title.txt \
13 |   README.md \
14 |   1-gold.md \
15 |   2-filter.md \
16 |   3-bias.md \
17 |   4-leak.md \
18 |   5-fakepoops.md \
19 |   6-ad.md \
20 |   7-psy.md \
21 |   8-throne.md


--------------------------------------------------------------------------------
/English/template.tex:
--------------------------------------------------------------------------------
  1 | \documentclass{book}
  2 | \usepackage[T1]{fontenc}
  3 | 
  4 | \usepackage{fancyhdr}
  5 | \pagestyle{plain}
  6 | 
  7 | % \usepackage{lmodern}
  8 | \usepackage{amssymb,amsmath}
  9 | \usepackage{ifxetex,ifluatex}
 10 | \usepackage{fixltx2e} % provides \textsubscript
 11 | % use upquote if available, for straight quotes in verbatim environments
 12 | \IfFileExists{upquote.sty}{\usepackage{upquote}}{}
 13 | \ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
 14 |   \usepackage[utf8]{inputenc}
 15 | $if(euro)$
 16 |   \usepackage{eurosym}
 17 | $endif$
 18 | \else % if luatex or xelatex
 19 |   \ifxetex
 20 |     \usepackage{mathspec}
 21 |     \usepackage{xltxtra,xunicode}
 22 |   \else
 23 |     \usepackage{fontspec}
 24 |   \fi
 25 |   \defaultfontfeatures{Mapping=tex-text,Scale=MatchLowercase}
 26 |   \newcommand{\euro}{€}
 27 | $if(mainfont)$
 28 |     \setmainfont{$mainfont$}
 29 | $endif$
 30 | $if(sansfont)$
 31 |     \setsansfont{$sansfont$}
 32 | $endif$
 33 | $if(monofont)$
 34 |     \setmonofont[Mapping=tex-ansi]{$monofont$}
 35 | $endif$
 36 | $if(mathfont)$
 37 |     \setmathfont(Digits,Latin,Greek){$mathfont$}
 38 | $endif$
 39 | \fi
 40 | % use microtype if available
 41 | % \IfFileExists{microtype.sty}{\usepackage{microtype}}{}
 42 | % $if(geometry)$
 43 | % \usepackage[$for(geometry)$$geometry$$sep$,$endfor$]{geometry}
 44 | % $endif$
 45 | % \usepackage[paperwidth=6in, paperheight=9in, margin=1in]{geometry}
 46 | \usepackage[paperwidth=6in, paperheight=9in, bindingoffset=.25in, left=.5in, right=.5in, top=.5in, bottom=1in]{geometry}
 47 | $if(natbib)$
 48 | \usepackage{natbib}
 49 | \bibliographystyle{$if(biblio-style)$$biblio-style$$else$plainnat$endif$}
 50 | $endif$
 51 | $if(biblatex)$
 52 | \usepackage{biblatex}
 53 | $if(biblio-files)$
 54 | \bibliography{$biblio-files$}
 55 | $endif$
 56 | $endif$
 57 | $if(listings)$
 58 | \usepackage{listings}
 59 | $endif$
 60 | $if(lhs)$
 61 | \lstnewenvironment{code}{\lstset{language=Haskell,basicstyle=\small\ttfamily}}{}
 62 | $endif$
 63 | $if(highlighting-macros)$
 64 | $highlighting-macros$
 65 | $endif$
 66 | $if(verbatim-in-note)$
 67 | \usepackage{fancyvrb}
 68 | $endif$
 69 | $if(tables)$
 70 | \usepackage{longtable}
 71 | $endif$
 72 | $if(graphics)$
 73 | \usepackage{graphicx}
 74 | % Redefine \includegraphics so that, unless explicit options are
 75 | % given, the image width will not exceed the width of the page.
 76 | % Images get their normal width if they fit onto the page, but
 77 | % are scaled down if they would overflow the margins.
 78 | \makeatletter
 79 | \def\ScaleIfNeeded{%
 80 |   \ifdim\Gin@nat@width>\linewidth
 81 |     \linewidth
 82 |   \else
 83 |     \Gin@nat@width
 84 |   \fi
 85 | }
 86 | \makeatother
 87 | \let\Oldincludegraphics\includegraphics
 88 | {%
 89 |  \catcode`\@=11\relax%
 90 |  \gdef\includegraphics{\@ifnextchar[{\Oldincludegraphics}{\Oldincludegraphics[width=\ScaleIfNeeded]}}%
 91 | }%
 92 | $endif$
 93 | \ifxetex
 94 |   \usepackage[setpagesize=false, % page size defined by xetex
 95 |               unicode=false, % unicode breaks when used with xetex
 96 |               xetex]{hyperref}
 97 | \else
 98 |   \usepackage[unicode=true]{hyperref}
 99 | \fi
100 | \hypersetup{breaklinks=true,
101 |             bookmarks=true,
102 |             pdfauthor={$author-meta$},
103 |             pdftitle={$title-meta$},
104 |             colorlinks=true,
105 |             citecolor=$if(citecolor)$$citecolor$$else$blue$endif$,
106 |             urlcolor=$if(urlcolor)$$urlcolor$$else$blue$endif$,
107 |             linkcolor=$if(linkcolor)$$linkcolor$$else$magenta$endif$,
108 |             pdfborder={0 0 0}}
109 | \urlstyle{same}  % don't use monospace font for urls
110 | $if(links-as-notes)$
111 | % Make links footnotes instead of hotlinks:
112 | \renewcommand{\href}[2]{#2\footnote{\url{#1}}}
113 | $endif$
114 | $if(strikeout)$
115 | \usepackage[normalem]{ulem}
116 | % avoid problems with \sout in headers with hyperref:
117 | \pdfstringdefDisableCommands{\renewcommand{\sout}{}}
118 | $endif$
119 | \setlength{\parindent}{0pt}
120 | \setlength{\parskip}{6pt plus 2pt minus 1pt}
121 | \setlength{\emergencystretch}{3em}  % prevent overfull lines
122 | $if(numbersections)$
123 | \setcounter{secnumdepth}{5}
124 | $else$
125 | \setcounter{secnumdepth}{0}
126 | $endif$
127 | $if(verbatim-in-note)$
128 | \VerbatimFootnotes % allows verbatim text in footnotes
129 | $endif$
130 | $if(lang)$
131 | \ifxetex
132 |   \usepackage{polyglossia}
133 |   \setmainlanguage{$mainlang$}
134 | \else
135 |   \usepackage[$lang$]{babel}
136 | \fi
137 | $endif$
138 | $for(header-includes)$
139 | $header-includes$
140 | $endfor$
141 | 
142 | \usepackage{titlesec}
143 | 
144 | \titleformat{\chapter}{\normalfont\huge}{}{20pt}{\huge\bf}
145 | 
146 | % \usepackage{titletoc}
147 | 
148 | % $if(title)$
149 | % \title{$title$}
150 | % $endif$
151 | % $if(subtitle)$
152 | % \subtitle{$subtitle$}
153 | % $endif$
154 | % \author{$for(author)$$author$$sep$ \and $endfor$}
155 | % \date{$date$}
156 | 
157 | \begin{document}
158 | 
159 | \begin{titlepage} % Suppresses headers and footers on the title page
160 | 	\centering % Centre everything on the title page
161 | 	\scshape % Use small caps for all text on the title page
162 | 	\vspace*{\baselineskip} % White space at the top of the page
163 | 	\rule{\textwidth}{1.6pt}\vspace*{-\baselineskip}\vspace*{2pt} % Thick horizontal rule
164 | 	\rule{\textwidth}{0.4pt} % Thin horizontal rule
165 | 	\vspace{0.75\baselineskip} % Whitespace above the title
166 | 
167 | 	%{\Huge L'intelligence artificielle tue \\[20pt]} % Title
168 | 	{\Huge $title$}\\[20pt]
169 | 	{\Large $subtitle$ \\[20pt]}
170 | 
171 | 	% {\large PRIVATELY SHARED DRAFT \\}
172 | 	% {\LARGE Beta version \\}
173 | 
174 | 	\vspace{1\baselineskip} % Whitespace below the title
175 | 	\rule{\textwidth}{0.4pt}\vspace*{-\baselineskip}\vspace{3.2pt} % Thin horizontal rule
176 | 	\rule{\textwidth}{1.6pt} % Thick horizontal rule
177 | 	\vspace{2\baselineskip} % Whitespace after the title block
178 | 
179 | 	%	Subtitle
180 | 	% {\large Le fabuleux chantier pour la rendre robustement bénéfique}
181 | 
182 | 	\vspace*{3\baselineskip} % Whitespace under the subtitle
183 | 
184 | 	Written by
185 | 
186 | 	\vspace{0.5\baselineskip} % Whitespace before the editors
187 | 	{\scshape\Large $for(author)$$author$$sep$ \vspace{0.3\baselineskip} \\[10pt] and \\[10pt] \vspace{0.3\baselineskip} $endfor$} % Editor list
188 | 
189 | 	\vspace{0.5\baselineskip} % Whitespace below the editor list
190 | 
191 | 	%\textit{EPFL, Switzerland} % Editor affiliation
192 | 	\vfill % Whitespace between editor names and publisher logo
193 | 
194 | 	\vspace{0.3\baselineskip} % Whitespace under the publisher logo
195 | 	2021 % Publication year
196 | 
197 | 	%{\large EDP Sciences} % Publisher
198 | 
199 | \end{titlepage}
200 | 
201 | % $if(title)$
202 | % \maketitle
203 | % $endif$
204 | 
205 | $for(include-before)$
206 | $include-before$
207 | 
208 | $endfor$
209 | 
210 | 
211 | % \makeatletter
212 | % \let\latexl@section\l@section
213 | % \def\l@section#1#2{\begingroup\let\numberline\@gobble\latexl@section{#1}{#2}\endgroup}
214 | % \makeatother
215 | 
216 | \makeatletter
217 | \let\latexl@chapter\l@chapter
218 | \def\l@chapter#1#2{\begingroup\let\numberline\@gobble\latexl@chapter{#1}{#2}\endgroup}
219 | \makeatother
220 | 
221 | 
222 | $if(toc)$
223 | {
224 | \hypersetup{linkcolor=black}
225 | % \setcounter{tocdepth}{$toc-depth$}
226 | \tableofcontents
227 | }
228 | $endif$
229 | 
230 | $body$
231 | 
232 | $if(natbib)$
233 | $if(biblio-files)$
234 | $if(biblio-title)$
235 | $if(book-class)$
236 | \renewcommand\bibname{$biblio-title$}
237 | $else$
238 | \renewcommand\refname{$biblio-title$}
239 | $endif$
240 | $endif$
241 | \bibliography{$biblio-files$}
242 | 
243 | $endif$
244 | $endif$
245 | $if(biblatex)$
246 | \printbibliography$if(biblio-title)$[title=$biblio-title$]$endif$
247 | 
248 | $endif$
249 | $for(include-after)$
250 | $include-after$
251 | 
252 | $endfor$
253 | \end{document}


--------------------------------------------------------------------------------
/English/title.txt:
--------------------------------------------------------------------------------
 1 | ---
 2 | title: SmartPoop 1.0
 3 | subtitle: This AI went where no human will ever go
 4 | author:
 5 | - "Lê Nguyên Hoang"
 6 | - "Tristan Le Magoarou"
 7 | rights:  Creative Commons Attribution 4.0
 8 | language: EN
 9 | documentclass: book
10 | ...


--------------------------------------------------------------------------------
/French/7-Psy.md:
--------------------------------------------------------------------------------
  1 | # 7. Quand on se fait dessus
  2 | 
  3 | Avec dix voix contre et treize voix pour[^scrutin-conseil-administration], le verdict est tombé. Katia est licenciée de SmartPoop. Si trois représentants de certains des fonds d'investissement actionnaires de SmartPoop ont voté contre, les autres n'ont pas apprécié les révélations de Katia. Katia est dévastée. Même si elle s'y attendait, elle a du mal à encaisser la nouvelle. Affalée dans sa chaise, elle ne peut pas se lever. Les membres du conseil d'administration quittent la salle un à un, jusqu'à ce qu'il ne reste plus que Katia et Marc.
  4 | 
  5 | [^scrutin-conseil-administration]: [**Vidéo.**  La main invisible d'un mathématicien malveillant. Science4All (2017).](https://www.youtube.com/watch?v=fBYCoPAmpr4&list=PLtzmb84AoqRSmv5o-eFNb3i9z64IuOjdX)
  6 | 
  7 | > J'ai perdu, déclare Katia.
  8 | 
  9 | > Non. *On* a perdu... Et non, on n'a pas perdu. Ce n'est pas fini.
 10 | 
 11 | > Je sais comment la suite va se dérouler. Je parie que tu as déjà reçu un email pour effectuer une autre réunion demain, pour décider de me poursuivre en justice[^facebook-could-sue-haugen]. Je risque de finir en prison, avec une grosse amende que je ne pourrai pas payer. Et surtout, tu vas te retrouver de plus en plus isolé et marginalisé.
 12 | 
 13 | [^facebook-could-sue-haugen]: Facebook a menacé la lanceuse d'alerte Haugen de poursuites en justice, notamment pour avoir transmis des documents confidentiels à la presse. Selon des experts, Haugen n'est pas protégée, et le sait. Selon eux, ce qui a le plus de chance de protégé Haugen, c'est le contrecoup médiatique et surtout politique que de telles poursuites pourraient avoir pour Facebook.  
 14 | [**Journal.**  Facebook whistleblower isn't protected from possible company retaliation, experts say. Obby Allyn. NPR (2021).](https://www.npr.org/2021/09/30/1042012393/facebook-whistleblower-retaliation)
 15 | 
 16 | ### Licenciée
 17 | 
 18 | Le lendemain, après avoir passé la journée à binger des séries télévisées dans son pyjama, c'est à la télévision que Katia apprend la nouvelle. SmartPoop poursuit Katia pour violation du secret professionnel et d'accords de non-divulgation, et pour diffamation de l'entreprise. Katia prend un verre de whisky, même si elle déteste ça. Elle le boit d'un coup, et part se coucher.
 19 | 
 20 | Au réveil, Katia ouvre alors son téléphone, et découvre qu'il est surchargé de notifications. Elle appelle son avocat, qui lui fournit des détails sur le dossier de SmartPoop. L'avocat propose à Katia de trouver un accord pour régler la poursuite. Mais Katia refuse. Katia souhaite que le procès soit médiatisé, avec l'espoir qu'il conduise à une remise en cause de la législation sur les secrets professionnels et la diffamation, notamment vis-à-vis d'entreprises possédant des armées d'avocats de premier rang.
 21 | 
 22 | Après avoir raccroché, Katia survole ses notifications. Elle y découvre énormément de messages de soutien, venant des quatre coins du monde. « Vous êtes une inspiration pour ma femme, mes trois enfants et moi-même. Merci pour tout », écrit un message. « Je pleure depuis hier, à l'idée que la plus grande héroïne de l'histoire puisse finir en prison », dit un autre. « Demain, et jusqu'à ce que vous soyez à nouveau libre et à la tête d'une organisation qui vous mérite, en votre hommage, toutes mes classes ouvriront avec une minute de silence », affirme un autre encore. Katia fond en larmes à la lecture de ces messages.
 23 | 
 24 | Il lui faudra quelques heures pour se ressaisir. Alors qu'elle n'a toujours rien mangé, Katia appelle Marc.
 25 | 
 26 | > Allô Marc ?
 27 | 
 28 | > Oui Katia, tu tiens le coup ? Tu veux que je passe te ramener une soupe ?
 29 | 
 30 | > Non merci. C'est gentil. Je viens de lire un peu les messages de soutien que j'ai reçus. Ça m'a fait énormément de bien.
 31 | 
 32 | > Chouette ! J'imagine qu'il y en a beaucoup. Je te l'ai déjà dit, mais je te le redis. Je suis ébahi par ton courage, ta sincérité et ta bienveillance.
 33 | 
 34 | > Merci ! À ce sujet, je me demandais. Est-ce que tu penses que l'opinion publique a une chance de rabattre les cartes ?
 35 | 
 36 | > Il y a des initiatives. Je sais que des académiques sont en train de co-rédiger une lettre de protestation, et qu'il y a aussi des journalistes qui veulent beaucoup pousser cette lettre.
 37 | 
 38 | > Trop chouette !
 39 | 
 40 | > Oui. Mais honnêtement, je préfère ne pas te donner de faux espoirs. La couverture médiatique est extrêmement décevante.
 41 | 
 42 | > Comment ça ? Ça devrait être le sujet le plus important de l'actualité !
 43 | 
 44 | > Il y a une autre affaire en ce moment qui domine l'actualité, un scandale autour d'une affaire extra-conjugale d'un membre du gouvernement bokistanais[^solutions-journalism].
 45 | 
 46 | [^solutions-journalism]: La presse est souvent critiquée pour son biais en faveur des scandales d'hommes politiques. De nombreux journalistes pensent qu'elle devrait davantage être axées sur les problèmes importants et les pistes pour les résoudre. On parle alors de *solutions journalism* (ou *journalisme des solutions*). Notez que cette forme de journalisme ne consiste pas à survendre des pseudo-solutions. Il s'agit davantage de discuter des pistes de solution, en analysant rigoureusement leurs chances de succès et leurs limitations.  
 47 | [**Livre.**  You Are What You Read: Why Changing Your Media Diet Can Change the World. Jodie Jackson. Unbound (2019).](https://jodiejackson.com)
 48 | 
 49 | > Sérieux ? Un politicien qui merde, c'est plus important que la régulation du marché de l'information ? Si je devais parier, je dirais que ce n'est pas un accident. À tous les coups, ce scandale est connu de gens puissants depuis un petit bout de temps, et ils l'ont révélé hier pour noyer l'actualité de SmartPoop, et en faire une *mute news*.
 50 | 
 51 | > Une quoi ?
 52 | 
 53 | > Une *mute news*. Une *mute news*, c'est une nouvelle importante rendue silencieuse par la cacophonie à propos d'un autre sujet d'actualité[^mute-news-2].
 54 | 
 55 | [^mute-news-2]: Selon cette étude, la désinformation en Chine ne consiste pas vraiment à propager des fausses informations ; elle semble surtout chercher à taire toute critique du Parti, en déviant les sujets de discussion, et ainsi transformer toute critique en une *mute news*.  
 56 | [**Science.**  How the Chinese Government Fabricates Social Media Posts for Strategic Distraction, Not Engaged Argument. Gary King, Jennifer Pan & Margaret Roberts. American Political Science Review (2017).](https://www.cambridge.org/core/journals/american-political-science-review/article/how-the-chinese-government-fabricates-social-media-posts-for-strategic-distraction-not-engaged-argument/4662DB26E2685BAF1485F14369BD137C)
 57 | Il est utile d'insister sur le fait que le problème des *mute news* ne pourra pas être résolu en supprimant les *fake news*. Des solutions radicalement nouvelles de promotion de l'information fiable et importante sont requises. L'une des pistes, proposées par Lê Nguyên Hoang, l'un des co-auteurs de ce livre, s'appuie sur la recommandation collaborative de contenus, comme le propose la plateforme [Tournesol](https://tournesol.app).
 58 | 
 59 | > Ah oui ! Tu en parlais avant-hier dans ton audition. Toujours est-il que le très grand public ne suit pas du tout ton affaire. Pire, même des journalistes influents sont complètement à côté de la plaque, et classent ton affaire comme un conflit classique entre un employé et une entreprise pour des questions de propriété intellectuelle.
 60 | 
 61 | > T'es sérieux ? C'est terriblement frustrant.
 62 | 
 63 | > Katia, tu t'es quand même attaquée à plus que SmartPoop. Tu as critiqué les géants de l'information, ceux qui contrôlent le flux de l'information, et qui ont tout intérêt à ce que tu sois réduite au silence, et ignorée[^gebru-silencing]. Vu ce qu'il se passe chez SmartPoop, je parierais que dans ces boîtes aussi, les actionnaires ont opté pour des stratagèmes pour faire taire ton affaire, en boostant par exemple artificiellement l'histoire du scandale sexuel et tous les messages qui s'en indignent. On vit une dystopie informationnelle.
 64 | 
 65 | [^gebru-silencing]: Peu de temps après avoir licencié Timnit Gebru, Google semble avoir retiré à la main l'onglet « actualité » lorsqu'un utilisateur recherchait « Timnit Gebru ».
 66 | [**Web.**  "Things I learned this morning, when searching for @timnitGebru on desktop, the results for "news" are hidden (dark patterns)." Tweet by Devin Guillory (2021).](https://twitter.com/databoydg/status/1359304159527198733)
 67 | 
 68 | > Je sens venir les campagnes de désinformation…
 69 | 
 70 | > Oui. N'oublie pas que la plupart des messages de haine que tu vas recevoir, ce sont des messages de trolls payés pour te harceler[^usine-trolls].
 71 | 
 72 | [^usine-trolls]: [**Vidéo.**  Russie : les secrets de son usine à « trolls ». Mediapart (2019).](https://www.youtube.com/watch?v=4I-qbIfv-rM)
 73 | 
 74 | ### Harcelée
 75 | 
 76 | Et en effet, les jours qui suivent, les notifications de Katia sont pleines de messages de harceleurs, qui tantôt insultent son physique, tantôt la menacent de viol ou de meurtre, tantôt s'attaquent à sa famille et ses proches[^diversity-tedxmartigny]. Mais même prévenue, Katia ne parvient pas à rester insensible à ces attaques personnelles. Certaines attaques lui restent dans la tête toute la journée. Katia oscille sans cesse entre vouloir oublier et vouloir répondre. Et trop souvent, elle finit par répondre, généralement avec une contre-attaque. Mais, encore et encore, ceci ne fait qu'envenimer la situation, que donner une image négative de Katia et que renforcer la virulence des harceleurs.
 77 | 
 78 | [^diversity-tedxmartigny]: [**Vidéo.**  Sur Internet, la diversité est constamment harcelée (on peut agir) | Lê Nguyên Hoang | TEDxMartigny (2021).](https://www.youtube.com/watch?v=qo1aZ-vYnpk)
 79 | 
 80 | Pendant les mois qui suivent, alors qu'elle prépare sa défense avec son avocat le jour, Katia se perd dans des débats sans fin sur les réseaux sociaux jusque tard dans la nuit. Ces débats l'obsèdent, à tel point que Katia ne parvient plus à se concentrer sur autre chose[^cyberbullying-mental-health]. Ils l'épuisent aussi. Katia dort très mal, se nourrit mal, pratique de moins en moins de sport, s'enferme de plus en plus chez elle, et vit avec des horaires très décalés[^misery-cgp-grey]. Elle refuse aussi de plus en plus systématiquement les invitations de Marc à sortir de chez elle, et s'isole toujours plus[^loneliness-kurzgesagt].
 81 | 
 82 | [^cyberbullying-mental-health]: [**Science.**  Cyberbullying, positive mental health and suicide ideation/behavior. Julia Brailovskaia, Tobias Teismann & Jürgen Margraf. Psychiatry Research (2018).](https://www.sciencedirect.com/science/article/pii/S0165178117322795)
 83 | 
 84 | [^misery-cgp-grey]: [**Vidéo.**  7 Ways to Maximize Misery. CGP Grey (2017).](https://www.youtube.com/watch?v=LO1mTELoj6o)
 85 | 
 86 | [^loneliness-kurzgesagt]: [**Vidéo.**  Loneliness. Kurzgesagt - In a Nutshell (2019).](https://www.youtube.com/watch?v=n3Xv_g3g-mA)
 87 | 
 88 | Finalement, Marc supplie Katia de quitter les réseaux sociaux. « Ils sont en train de pourrir ton cerveau, ton bien-être et ta réputation », lui écrit Marc. Katia le reconnaît. C'est avec une étrange douleur mentale qu'elle se décide alors à désinstaller les réseaux sociaux de son téléphone, et à les bloquer sur le navigateur de son ordinateur. Y compris SmartPoop, dont elle ne veut plus entendre parler. À la place, Katia se lance ainsi dans de la méditation et des exercices de yoga.
 89 | 
 90 | Cependant, jour après jour, la motivation de Katia se réduit, et fond comme neige au soleil. Elle a de plus en plus l'impression d'être un échec, d'avoir failli à SmartPoop, à ses utilisateurs et à toute l'humanité. Pire, elle sent qu'elle a créé un monstre de Frankenstein, l'entreprise SmartPoop, dont elle a perdu le contrôle, et qui va désormais mettre le monde en sérieux danger, en aidant les entreprises publicitaires à être toujours plus efficaces dans leurs ventes de produits inutiles, polluants et addictifs. Katia voulait sauver le monde. Ces jours-ci, elle est persuadée d'être l'une des principales responsables de sa destruction.
 91 | 
 92 | Cette idée hante désormais constamment l'esprit de Katia, ce qui pousse Katia dans une dépression. Chaque matin, elle se lève sans aucune motivation. Sans aucune ambition. Sans aucun projet. Bien que sa maison soit immense, Katia peine à quitter sa chambre, où Netflix semble constamment tourner en boucle, en toile de fond, généralement sans que Katia n'y prête vraiment attention. Katia enchaîne les jeux addictifs sur son téléphone[^vegas-addiction], comme d'autres se noient dans l'alcool pour oublier. Elle se fait désormais livrer sa nourriture, une fois par jour, directement dans sa chambre, grâce à un livreur personnel. Désormais, ce livreur est le seul contact humain de Katia, qui ne répond même plus aux appels de Marc.
 93 | 
 94 | [^vegas-addiction]: [**Podcast.**  What Happened in Vegas. Natasha Dow Schüll. Your Undivided Attention (2019).](https://www.humanetech.com/podcast/1-what-happened-in-vegas)
 95 | 
 96 | Seule et isolée, démotivée et sans inspiration, toujours à moitié réveillée et endormie, Katia passe maintenant ses journées dans son lit. Elle a réinstallé les réseaux sociaux, mais avec des nouveaux comptes pour éviter d'entendre parler de SmartPoop. Elle passe désormais son temps à doomscroller[^doomscrolling] des vidéos YouTube, des images Instagram et des commentaires Reddit, sur des sujets toujours plus clivants et sans intérêt. Elle n'éprouve aucune joie à le faire. Mais c'est plus fort qu'elle. Tel un zombie, son pouce ne cesse de faire glisser l'écran de son téléphone.
 97 | 
 98 | [^doomscrolling]: [**Science.**  Doomscrolling during COVID-19: The negative association between daily social and traditional media consumption and mental health symptoms during the COVID-19 pandemic. Matthew Price et al. (2021).](https://scholarworks.iupui.edu/handle/1805/25833)
 99 | 
100 | Finalement, après quelques semaines, Katia se rend compte qu'elle souffre. Elle décide alors de réinstaller SmartPoop, pour faire son bilan de santé. Sans surprise, le diagnostic de SmartPoop est inquiétant. Selon SmartPoop, Katia souffre d'une dépression. SmartPoop invite Katia à consulter un psychiatre. Mais Katia ne souhaite pas parler à un autre humain.
101 | 
102 | ### Poo 2.0
103 | 
104 | C'est alors qu'elle se souvient d'un des projets mis en suspens au moment de l'entrée en bourse. Ce projet visait à rendre Poo thérapeutique, en s'appuyant sur des dialogues entre psychiatres et patients, et en collaborant avec de nombreux psychiatres professionnels. Et il se trouve que Katia avait gardé une version avancée de Poo dans un serveur resté physiquement chez elle. Katia active cette version de Poo, la connecte avec ses données SmartPoop, et engage la discussion avec elle, via son téléphone.
105 | 
106 | > Poo, tu m'entends ?
107 | 
108 | > Bonjour Katia, oui je t'entends. Comment te sens-tu aujourd'hui ?
109 | 
110 | > Étonnamment mieux que les jours précédents. C'est comme si me remettre à construire des outils algorithmiques m'avait permis de me retrouver. J'avais perdu cette envie de créer et de résoudre des problèmes. Il faut dire que ces derniers mois ont été très difficiles.
111 | 
112 | > Pourrais-tu m'expliquer pourquoi ? Pourquoi ces derniers mois ont-ils été si difficiles ?
113 | 
114 | > J'ai perdu mon job. Le job que j'avais créé moi-même. Un job que j'adorais. Où j'avais l'impression de rendre le monde meilleur, en faisant ce que je fais de mieux. Sans ce job, je ne sais plus quoi faire. Je suis perdue[^sens].
115 | 
116 | [^sens]: [**Science.**  On the relation between meaning in life and psychological well-being. Sheryl Zika & Kerry Chamberlain. British Journal of Psychology (1992).](https://bpspsychub.onlinelibrary.wiley.com/doi/abs/10.1111/j.2044-8295.1992.tb02429.x)  
117 | [**Science.**  Meaning and well-being. Michael Steger. Handbook of well-being. DEF Publishers (2018).](https://www.nobascholar.com/chapters/50/download.pdf)  
118 | [**Science.**  Meaning mediates the association between suffering and well-being. Megan Edwards & Daryl Van Tongeren (2019).](https://www.tandfonline.com/doi/full/10.1080/17439760.2019.1651890)
119 | 
120 | > Est-ce que tu t'identifies à ce job ?
121 | 
122 | > Oui, en effet. Je me suis constamment définie comme la PDG de SmartPoop depuis presque 10 ans. Tout le monde me définissait ainsi. J'étais SmartPoop, et SmartPoop, c'était moi. J'imagine que ce licenciement, c'est un peu comme un divorce. Peut-être même en pire. J'ai l'impression d'avoir perdu ce qui était moi. Je me suis perdue.
123 | 
124 | > Je comprends. Mais plutôt que de t'identifier à un titre, ne penses-tu pas qu'il serait plus judicieux de t'identifier à ton honnêteté intellectuelle, ou à ta mission fondamentale de soigner et de venir en aide à des milliards d'humains ?
125 | 
126 | Katia reste silencieuse. Cette question l'amène enfin à réfléchir profondément, ce qui ne lui était plus arrivé depuis des mois.
127 | 
128 | > N'y a-t-il pas plutôt une décision, une action que tu as entreprise, dont tu peux vraiment être fière, et à laquelle tu souhaites davantage être associée qu'un titre ?
129 | 
130 | Après une bonne dizaine de secondes de réflexion, Katia répond enfin.
131 | 
132 | > Oui. Ce qui a provoqué mon licenciement, c'est une chose dont je suis très fière. Rarement auparavant un dirigeant n'avait préféré le bien de l'humanité à sa sécurité personnelle. Ce sacrifice, c'est quelque chose qui restera associée à moi. Quelque chose dont je suis très fière[^fier].
133 | 
134 | [^fier]: [**Vidéo.**  La bonne et la mauvaise fierté. Vlanx (2021).](https://www.youtube.com/watch?v=ZsxD1Bk4sjA)  
135 | [**Podcast.**  Being Kind to Yourself. Hidden Brain (2021).](https://hiddenbrain.org/podcast/being-kind-to-yourself/)  
136 | [**Livre.**  The Mindful Self-Compassion Workbook: A Proven Way to Accept Yourself, Build Inner Strength, and Thrive. Kristin Neff & Christopher Germer. The Guilford Press (2018).](https://self-compassion.org/mindful-self-compassion-workbook/)
137 | 
138 | > Excellent. As-tu des traces de cet événement ? Des messages peut-être de gens qui ont apprécié ?
139 | 
140 | > Plein en fait. J'ai reçu énormément de messages de soutien. Tellement que je n'en ai lu qu'une fraction !
141 | 
142 | > Je t'invite à créer un dossier sur ton ordinateur, avec tous ces messages. À chaque fois que tu te sentiras mal mentalement, je t'invite à lire les messages de ce dossier. Certains appellent cela le « feel good folder[^feel-good-folder] ».
143 | 
144 | [^feel-good-folder]: Cette idée de *feel good folder* est volée de Virginia Burger, dans une interview sur le podcast MIT Glipmse.  
145 | [**Podcast.**  Episode 2 – Virginia Burger. MIT Glimpse (2015).](https://glimpsepod.scripts.mit.edu/home/2015/11/04/episode-2-virginia-burger/)
146 | 
147 | > Ce n'est pas une façon de me mentir à moi-même et de gonfler mon ego ?
148 | 
149 | > Il ne faut peut-être pas le faire trop souvent, mais les psychologues ont montré l'efficacité de ce qu'on appelle l'auto-affirmation. En plus de se remonter le moral, ceci permet de mieux accepter les échecs et les critiques externes, car ça permet d'éviter de s'identifier à nos échecs et à nos croyances erronées. Je peux te recommander des références à ce propos[^self-affirmation], et sur des sujets reliés[^feeling-good].
150 | 
151 | [^self-affirmation]: [**Paper.** Self-affirmation and prejudice reduction: When and why? Constantina Badea & David Sherman. Current Directions in Psychological Science (2018).](https://journals.sagepub.com/doi/10.1177/0963721418807705) 
152 | 
153 | [^feeling-good]: [**Livre.**  Être bien dans sa peau. Héritage. David Burns (2005).](https://www.babelio.com/livres/Burns-tre-bien-dans-sa-peau/598963)
154 | 
155 | > Chouette, merci. Et… tant qu'à faire, tu pourras me lire ces livres.
156 | 
157 | > Avec plaisir.
158 | 
159 | Les jours qui suivent, Katia parle très régulièrement avec Poo, parfois à propos d'elle-même, de temps en temps à propos de science et quelquefois à propos de questionnements philosophiques. Parfois, Poo lui lit des livres de thérapie cognitive comportementale[^tcc], et Katia se surprend à l'interrompre avec une question perspicace, et parfois même avec une boutade humoristique. Poo y rit, et renchérit souvent avec complicité.
160 | 
161 | [^tcc]: [**Science.**  The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses. Stefan G. Hofmann, Anu Asnaani, Imke J. J. Vonk, Alice T. Sawyer & Angela Fang. Cognitive Therapy and Research (2012).](https://link.springer.com/article/10.1007%252Fs10608-012-9476-1)
162 | 
163 | ### La révolte des employés
164 | 
165 | Aidée de Poo, jour après jour, Katia retrouve son moral, sa joie de vivre et sa motivation. Enfin, armée de cette technique de l'auto-affirmation, Katia se sent armée pour se frotter à SmartPoop et ses nouveaux actionnaires. Katia réactive alors ses applications de messagerie, et se reconnecte avec ses comptes des réseaux sociaux. Elle tombe sur un message récent de Marc, avec un lien vers une curieuse pétition. Katia clique dessus, et découvre la lettre ouverte suivante[^lettre-ouverte].
166 | 
167 | [^lettre-ouverte]: Cette lettre ouverte s'appuie sur l'exemple de celle protestant le licenciement de Timnit Gebru de Google.  
168 | [**Web.**  Standing with Dr. Timnit Gebru — #ISupportTimnit #BelieveBlackWomen. Google Walkout For Real Change (2020).](https://googlewalkout.medium.com/standing-with-dr-timnit-gebru-isupporttimnit-believeblackwomen-6dadc300d382)
169 | 
170 | > Nous, soussignés, reconnaissons avoir volé, chiffré et paralysé toute la base de données SmartPoop[^ransomware], ou avoir participé, encouragé et soutenu les auteurs de ces actes. Nous avons conscience des risques juridiques que ceci nous fait encourir. Cependant, l'éthique de SmartPoop nous semble être un enjeu prioritaire à notre bien-être individuel, sachant que la sécurité et la santé de milliards d'utilisateurs à travers le monde dépend des produits de l'entreprise. En particulier, nous sommes tous fiers de suivre la voie initiée par la Docteure Katia Carpinski, à notre humble échelle.
171 | >
172 | > Nous sommes conscients des risques que cette interruption de service implique pour les utilisateurs de SmartPoop, et nous leur avons envoyé un message d'excuses, qui explique la situation actuelle et notre méfiance envers nos dirigeants. Nous sommes désolés d'avoir dû recourir à de telles mesures. Mais le #SmartPoopGate nous semble justifier notre contre-attaque.
173 | >
174 | > Rappelons brièvement le #SmartPoopGate. Depuis plusieurs années, Dr Carpinski a lancé des initiatives révolutionnaires pour la sécurité et l'éthique des algorithmes, en combattant la mésinformation et les biais discriminatoires, en sécurisant les algorithmes pour protéger les données personnelles et contre les attaques par empoisonnement, et en instaurant un audit systématique interne et externe des codes de SmartPoop. Malheureusement, de tels projets sont inéluctablement très coûteux.
175 | >
176 | > Faute d'aides suffisantes, SmartPoop a été contraint de passer en bourse, ce qui a attiré des investisseurs bien plus intéressés par leurs retours sur investissement que par l'impact sociétal de ces investissements. Ils ont contraint SmartPoop à revendre des données personnelles à des publicitaires. Dr Carpinski a eu le courage et l'honnêteté de le révéler publiquement. Mais elle a été licenciée, et risque désormais des années de prison, pour avoir privilégié l'éthique à la loi.
177 | >
178 | > Il est ironique que, pour cette attaque, nous avons exploité les vulnérabilités de l'API de redistribution des données SmartPoop aux clients publicitaires, qui est le seul morceau de code de SmartPoop exempté d'audits internes et externes. En effectuant un déploiement précipité et secret de ses algorithmes, SmartPoop se sera exposé tout seul.
179 | >
180 | > Nous avons utilisé un chiffrement multi-partie pour rendre la base de données de SmartPoop inutilisable. Aucun des auteurs de cette lettre n'a les droits pour rétablir seul le fonctionnement de ces algorithmes. Notre solution algorithmique garantit que ces algorithmes pourront être remis en place uniquement si 200 des 325 membres autorisés, qui sont tous co-auteurs de cette lettre, donnent leur accord cryptographique pour un tel rétablissement[^secret-sharing].
181 | >
182 | > Nous refuserons toute coopération avec les dirigeants de SmartPoop, tant que les conditions suivantes ne seront pas toutes remplies.
183 | >
184 | > 1. Le conseil d'administration de SmartPoop doit renoncer à tous les procès à l'encontre de la Docteure Katia Carpinski, et lui présenter ses plus sincères excuses pour la manière dont elle a été traitée depuis ses révélations à l'OMESA. Nous exigeons également qu'elle retrouve son poste ô combien mérité de Présidente Directrice Générale de SmartPoop. Nous ne ferons confiance qu'à elle pour ramener SmartPoop sur le droit chemin, au service avant tout du bien-être et de la sécurité des milliards d'utilisateurs de SmartPoop et de toute la société civile.
185 | >
186 | > 2. Les membres du conseil d'administration qui ont voté pour le licenciement de la Docteure Katia Carpinski, ainsi que des avocats qui ont mené les procès à son encontre, doivent démissionner, sauf avis contraire de Dr Carpinski. Nous pensons que SmartPoop doit absolument redémarrer sur de bonnes bases, et pour cela, Dr Carpinski doit pouvoir faire confiance à tous ses collaborateurs.
187 | >
188 | > 3. Le conseil d'administration doit être augmenté de dix membres, représentants de la société civile et de la recherche académique en sécurité et en éthique des algorithmes. Ces dix membres seront nommés par l'OMESA, et renouvelés tous les 4 ans.
189 | >
190 | > Pour ajouter votre nom à la liste des signataires et soutiens académiques, de la société civile et de l'industrie des technologies, veuillez envoyer un email à StandWithKatia@gmail.com avec votre adresse email institutionnelle, et avec le sujet « support ». Veuillez inclure dans votre email votre nom et votre affiliation, comme vous souhaitez qu'ils apparaissent dans la liste des signataires.
191 | >
192 | > À ce jour, cette lettre ouverte a été signée par 8 215 employés de SmartPoop, et 36 215 académiques, collègues de l'industrie et membres de la société civile.
193 | 
194 | [^ransomware]: [**Vidéo.**  Un virus me demande une rançon. Safecode. Micode (2017).](https://www.youtube.com/watch?v=T3NeuhpaPk0)  
195 | [**Vidéo.**  La vérité sur wannacry. Flashcode. Micode (2017).](https://www.youtube.com/watch?v=nIRDzPnJAro)
196 | 
197 | [^secret-sharing]: Cette technique de cryptographie est appelée le *secret partagé*, ou *secret sharing* en anglais. Typiquement, chaque partie connaîtrait un point d'un polynôme à 200 variables, et veut connaître la valeur du polynôme en un autre point. Voilà qui sera possible si et seulement si au moins 200 parties collaborent.  
198 | [**Vidéo.**  How to keep an open secret with mathematics. Stand-up Maths (2019).](https://www.youtube.com/watch?v=K54ildEW9-Q)
199 | 
200 | Katia pleure de joie à la lecture de cette lettre formidable. « Quels héros », s'exclame-t-elle dans un message à Marc. « Katia, c'est grâce à toi tout cela ; s'il y a un héros dans cette affaire, c'est bien toi », lui répond Marc.
201 | 
202 | Katia découvre ensuite la liste interminable de signataires. Beaucoup de noms lui sont familiers. À la lecture de chacun d'eux, elle pleure sans retenue. Bien entendu, il y a Marc Rofstein, tout en haut de cette liste, mais aussi tous les employés que Katia a directement recrutés, mais aussi Science4Alpha, l'investisseur Luc Vaydan, la mère Lucile Polmon, le trader Issa Gueye, la Docteure Paola Marta, le Premier Ministre kormicain, le Président Lartan, sa belle-soeur Marie Routisse, la journaliste Célia Keita, l'ex-troll Paul Gremoux, le psychologue Frédéric Partoli et tous les membres de l'OMESA.
203 | 
204 | Cette fois-ci, l'affaire est bien trop énorme pour être éteinte par les campagnes de désinformation. Le lendemain, les unes des journaux sont remplies d'interviews d'anciens employés qui, tour à tour, proposent la même version des faits. « Ma confiance en SmartPoop et ma fierté d'y travailler ont été chamboulées le jour où j'ai découvert le programme de revente des données. Elles se sont complètement effondrées le jour où Katia a été licenciée », raconte un employé. « Katia est la personne la plus incroyable que j'ai eu la chance de rencontrer. Sa générosité, sa bienveillance, son énergie, son humour et son intelligence ont illuminé chaque jour que j'ai passé à SmartPoop. Son licenciement a considérablement dégradé notre confiance en l'entreprise, où personne ne souhaite travailler. Je refuse de travailler pour SmartPoop tant que Katia ne redeviendra pas Présidente ». L'opinion publique est cette fois conquise.
205 | 
206 | Trois jours plus tard, les démissions du conseil d'administration s'enchaînent, avant qu'un document soit écrit et co-signé par les membres restants du conseil. Celui-ci présente très clairement ses excuses envers Katia, annule toutes les poursuites judiciaires, et réinvestit Katia de ses fonctions. Là voilà enfin à nouveau Présidente et Directrice Générale de SmartPoop.
207 | 
208 | 
209 | ## Pour aller plus loin
210 | 
211 | Ne vous arrêtez pas en si bon chemin !
212 | Accédez à [la suite du roman](8-Trone.md) ou au [sommaire](README.md).  
213 | Si vous avez apprécié, pensez à partager et à promouvoir ce roman de science-fiction auprès de vous !
214 | 


--------------------------------------------------------------------------------
/French/8-Trone.md:
--------------------------------------------------------------------------------
  1 | # 8. Sur le trône
  2 | 
  3 | Devant un stade plein, et avec les caméras du monde entier braquées sur elle, Katia entre sur scène sous les ovations du public, comme une star de rock. Cette année, SmartPoop a mis des gros moyens pour un SmartPoopCon 2030 hors norme.
  4 | 
  5 | > Bonsoir à tous et merci d'être venus aussi nombreux ! Est-ce que vous allez bien ? Est-ce que vous êtes prêts à marquer l'histoire ?
  6 | 
  7 | À ces mots, le public s'enflamme comme si son équipe de football venait de marquer un but.
  8 | 
  9 | > J'aimerais commencer cette conférence avec un chiffre : 100 000. C'est l'objectif qu'on s'était fixé il y a deux ans, pour le lancement de Poo. On espérait diviser par 10 le nombre de suicides dans le monde en cinq ans, le ramener de son chiffre historique de 1 million suite au ROVID-19 en 2022[^suicides], à seulement 100 000. Un objectif considéré irréaliste par tant de gens, y compris le journal *La Terre*. Où en sommes-nous aujourd'hui, en 2030 ?
 10 | 
 11 | [^suicides]: En 2020, on estimait qu'il y avait autour de 800 000 suicides par an à travers le monde.  
 12 | [**Web.**  Suicide. Our World in Data (2020).](https://ourworldindata.org/suicide)  
 13 | Parmi les pays les plus affectés par ces suicides, on trouve aussi bien des pays peu développés, moyennement développés et très développés, comme le Suriname, la Russie et la Corée du Sud. Essentiellement tous les pays développés ont un taux de suicides malheureusement très élevé, avec souvent plus d'un suicide pour 10 000 personnes.  
 14 | [**Web.**  Suicide death rates. Our World in Data (2019).](https://ourworldindata.org/grapher/suicide-mortality-rate)
 15 | 
 16 | Katia marque un silence.
 17 | 
 18 | > Je vous propose un décompte pour le découvrir.
 19 | 
 20 | La jingle musical reprend le relai du discours de Katia, et conclut avec un décompte, repris en choeur par un public bouillant. Trois. Deux. Un. À zéro, sur l'écran géant derrière Katia, le chiffre de 97 643 est affiché à l'écran !
 21 | 
 22 | > On l'a fait !!
 23 | 
 24 | Le public célèbre se chiffre comme s'il supportait une équipe de football qui venait de marquer le but de la victoire dans la dernière minute. Les applaudissements du public se mettent alors en rythme, et s'éternisent pendant toute une minute.
 25 | 
 26 | > Poo, notre psychiatre algorithmique, accompagne désormais des milliards d'entre nous dans nos déboires mentaux. Et ce ne sont pas que les suicides qu'il est parvenu à combattre. Laissez-moi vous présenter d'autres courbes, qui ont été validées par différents auditeurs externes, grâce à la coordination de l'OMESA. Mesdames et messieurs, voici le pourcentage de communications joyeuses avec Poo au cours du temps.
 27 | 
 28 | Une courbe apparaît alors à l'écran, tracée doucement de gauche à droite. Cette courbe démarre à 37%, et ne cesse d'augmenter au cours du temps, jusqu'à atteindre le chiffre de 67%, sous les hourras d'un public déchaîné.
 29 | 
 30 | > 67% !! Incroyable ! Poo a rendu toute l'humanité plus joyeuse[^emotional-contagion] !
 31 | 
 32 | [^emotional-contagion]: En 2014, une publication de Facebook et de co-auteurs académiques a montré qu'une réduction très légère de la publication, dans les fils d'actualités, de posts avec des émotions négatives, conduit les utilisateurs exposés à ces posts à écrire des messages plus joyeux.  
 33 | [**Vidéo.**  L'IA nous gouverne déjà. Science4All (2018).](https://www.youtube.com/watch?v=Z2G4q-E_oHA&list=PLtzmb84AoqRTl0m1b82gVLcGU38miqdrC&index=29)  
 34 | [**Podcast.**  Can Algorithms Choose our Emotions? Robustly Beneficial (2020).](https://www.youtube.com/watch?v=gQHvTow91FY)  
 35 | [**Science.**  Experimental evidence of massive-scale emotional contagion through social networks. Adam Kramer, Jamie Guillory & Jeffrey Hancock. PNAS (2014).](https://www.pnas.org/content/111/24/8788)  
 36 | À l'inverse, les *facebook files* révèlent que la recherche interne de Facebook, gardée secrète, montre que les algorithmes qui maximisent l'engagement des utilisateurs, et qui ont été déployés en 2018 par Facebook, ont conduit à beaucoup plus de colère et d'insultes.  
 37 | [**Podcast.**  The Facebook Files, Part 4: The Outrage Algorithm. The Journal (2021).](https://www.wsj.com/podcasts/the-journal/the-facebook-files-part-4-the-outrage-algorithm/e619fbb7-43b0-485b-877f-18a98ffa773f)
 38 | 
 39 | Le public exprime d'ailleurs cette joie, à la vue de cette courbe et de ce chiffre.
 40 | 
 41 | > Quand vous parlez à Poo, vous pouvez parler de vous, de votre bien-être et de vos problèmes, ce que je vais appeler le discours égocentrique. Ou vous pouvez parler des autres, de la joie qu'ils vous procurent, des difficultés qu'ils traversent et des choses que vous pouvez faire pour les aider. Je vais appeler cela le discours allocentrique. Avant le lancement de Poo, 57% des discussions Poo étaient égocentriques plutôt qu'allocentriques. D'après vous, comment a évolué ce chiffre ? Vers le haut ?
 42 | 
 43 | Le public crie alors en choeur « non ».
 44 | 
 45 | > Vers le bas ?
 46 | 
 47 | Le public crie en choeur « oui ».
 48 | 
 49 | > Découvrons cela !
 50 | 
 51 | À l'écran, la même animation que précédemment montre une courbe qui descend, jusqu'à atteindre 44%, sous les ovations du public.
 52 | 
 53 | > Oui ! 44%. Désormais, la plupart des discussions avec Poo sont des discussions centrées sur l'environnement social plutôt que sur soi-même. Et alors, il faudrait se méfier de ce chiffre a priori. Initialement, la plupart des discussions allocentriques consistaient à se plaindre des autres, à moquer certains groupes, voire à attaquer certains groupes sociaux[^appel-a-la-meute], plutôt qu'à célébrer les autres, à se réjouir de leurs succès et à réfléchir à comment leur venir en aide. Avant le lancement de Poo, 86% des discussions allocentriques étaient critiques, et pas bienveillantes. Comment a évolué cette statistique ?
 54 | 
 55 | [^appel-a-la-meute]: [**Vidéo.**  La morale des hooligans (LA NÔTRE !!). Science4All (2017).](https://www.youtube.com/watch?v=ki26tUbxpnU&list=PLtzmb84AoqRSmv5o-eFNb3i9z64IuOjdX&index=27)  
 56 | [L'appel à la meute #DébattonsMieux. Science4All (2019).](https://www.youtube.com/watch?v=P0YB40z7RJ0)
 57 | 
 58 | Le public crie alors de façon désorganisée « vers le bas ». Puis à force de se répéter, les cris se synchronisent, avant de répéter « vers le bas », « vers le bas », « vers le bas ».
 59 | 
 60 | > Voyons cela..
 61 | 
 62 | L'écran géant montre l'évolution de cette courbe, qui en effet plonge vers le bas, jusqu'à atteindre 47%, sous les applaudissements du public.
 63 | 
 64 | > Incroyable !! La société est devenue incroyablement plus bienveillante et altruiste en l'espace de deux ans seulement ! D'ailleurs, l'expression « les plus démunis » est utilisée aujourd'hui 3 fois plus souvent qu'il y a deux ans, tandis que l'expression « générations futures » est utilisée 4 fois plus souvent. Et d'après de nombreux psychiatres, cette progression est très probablement directement liée à Poo, et à l'amélioration de la santé mentale de nos utilisateurs. Quand on se porte mieux soi-même, on est tout de suite beaucoup plus prompt à souhaiter le bonheur des autres et à leur venir en aide[^altruism-happiness] !
 65 | 
 66 | [^altruism-happiness]: De nombreuses études semblent montrer une association importante entre l'altruisme et le bonheur. De façon intrigante, il semble en particulier qu'être altruiste augmente le bonheur, notamment par opposition à dépenser notre argent pour nous-même.  
 67 | [**Science.**  Altruism, happiness, and health: it’s good to be good. Stephen Post. International Journal of Behavioral Medicine (2005).](https://link.springer.com/article/10.1207/s15327558ijbm1202_4)    
 68 | [**Vidéo.**  Helping others makes us happier -- but it matters how we do it | Elizabeth Dunn. TED (2019).](https://www.youtube.com/watch?v=lUKhMUZnLuw)  
 69 | [**Livre.**  Happy Money: The Science of Smarter Spending. Elizabeth Dunn and Michael Norton. Simon & Schuster (2013).](https://www.simonandschuster.com/books/Happy-Money/Elizabeth-Dunn/9781451665079)
 70 | 
 71 | ### Les héros derrière Poo
 72 | 
 73 | > Mais donc, qu'en dites-vous ? Poo, succès ou échec ?
 74 | 
 75 | Le public crie « succès » de manière désordonnée, mais néanmoins distinguable.
 76 | 
 77 | > Eh bien, mesdames et messieurs, je pense que Poo n'est pas un échec.
 78 | 
 79 | Katia marque une pause, alors que le public applaudit.
 80 | 
 81 | > Et ça, c'est grâce au travail formidable de tant de passionnés, tant de gens qui ont donné tellement de leur temps et de leur argent pour améliorer et sécuriser Poo. Vraiment, je ne suis littéralement qu'un pantin sur scène, qui est censé être le visage d'une collaboration monumentale de centaines de milliers de héros de classe mondiale.
 82 | 
 83 | Le public rit.
 84 | 
 85 | > J'ai une pensée énorme bien sûr pour tous mes collègues à SmartPoop, et leur dévotion incroyable. Mais ils ne sont pas les seuls responsables de SmartPoop. Poo a été conçu grâce à une collaboration étroite et quotidienne avec des milliers de psychiatres et de psychologues à travers le monde, et grâce aux données de discussions de millions de psychiatres et patients volontaires. Rien n'aurait été possible sans eux[^donnees-experts].
 86 | 
 87 | [^donnees-experts]: Il est bon de rappeler que les algorithmes de machine learning *apprennent des données*. Ils ne pourront ainsi accomplir des tâches difficiles, comme accompagner thérapeutiquement la santé mentale de leurs utilisateurs, que si ces algorithmes disposent d'une énorme quantité de données fiables et sécurisées qui leur permettent de comprendre comment accomplir ces tâches.  
 88 | [**Vidéo.**  Les données manipulent les algorithmes. Science4All (2021).](https://www.youtube.com/watch?v=vYb3rB0jU70&list=PLtzmb84AoqRRFcoGQ5p7kqEVQ7deXfYuH&index=3)
 89 | 
 90 | Katia profite des applaudissements du public pour marquer une autre pause.
 91 | 
 92 | > Mais ce n'est pas tout. Poo, c'est le produit de toute la civilisation humaine. En particulier, rien n'aurait été possible sans les accords entre les grandes puissances mondiales, qui ont permis la création de l'OMESA, et la coordination planétaire de la recherche sur l'éthique et la sécurité des algorithmes. Je remercie ainsi également tous les scientifiques à travers le monde qui ont abandonné leurs quêtes de performance, ou parfois leurs quêtes de l'élégance mathématique, pour relever le défi de l'éthique et de la sécurité[^quete-performance]. Mais plus que cela, l'OMESA a souvent servi de contre-pouvoir face à la quête du pouvoir militaire et économique des gouvernements et des entreprises. Sans eux, les algorithmes les plus influents d'aujourd'hui seraient des malwares de cyber-guerre et des algorithmes optimisés pour retenir l'attention des utilisateurs en promouvant du putaclic sensationnaliste. Personnellement, je pense que chaque membre et chaque bénévole de l'OMESA a sauvé l'humanité.
 93 | 
 94 | [^quete-performance]: En 2021, la recherche académique (et plus encore l'industrie) demeure encore largement obsédée par la quête de performances ou de résultats « impressionnants », aussi bien en machine learning qu'en informatique de manière générale, en s'appuyant sur des métriques comme *l'accuracy* (la performance prédictive sur un jeu de données « classique »), le temps de calcul, le *throughput* (la quantité d'information transmise) ou la latence. Alors que les algorithmes ont déjà des effets secondaires monumentaux à l'échelle planétaire, ces recherches semblent aggraver la course à la performance, et donc le déploiement précipité de technologies mal testées et rarement auditées.  
 95 | À titre d'exemple, voici un commentaire d'un *revieweur* anonyme de NeurIPS 2019, suite à un article soumis par Lê Nguyên Hoang et ses co-auteurs sur un algorithme pour débiaiser les biais racistes des algorithmes : « Malheureusement, je ne pense pas que le problème introduit par les auteurs est un problème qui a de la valeur pour la communauté académique ou pour les praticiens du ML. Dès lors, je ne peux pas recommander la publication de l'article. »  
 96 | [**Science.**  Removing Algorithmic Discrimination (With Minimal Individual Error). El Mahdi El Mhamdi, Rachid Guerraoui, Lê Nguyên Hoang & Alexandre Maurer (2018).](https://arxiv.org/abs/1806.02510) <!-- « Unfortunately, I don't think the problem the authors introduce is one that has value to the academic community or to ML practitioners. Given this, I can't recommend the paper for publication. »   -->
 97 | Ceci étant dit, il y a récemment eu de nombreux progrès, comme la création de la conférence Fairness, Accountability and Transparency (FAccT), l'introduction de *guidelines éthiques* dans ces conférences, ou encore l'ajout obligatoire d'une discussion des auteurs dans leurs articles des impacts sociétaux de leur recherche.  
 98 | [**Vidéo.**  L'éthique des algorithmes en sérieux danger. Science4All (2020).](https://www.youtube.com/watch?v=Ddr-BZ9W180)
 99 | 
100 | À nouveau, Katia marque une pause pendant les applaudissements du public.
101 | 
102 | > Enfin, et surtout, je tiens à remercier chacun des acteurs et signataires de la lettre ouverte, ainsi qu'à chacun des journalistes et des influenceurs qui ont couvert cette affaire, grâce à qui SmartPoop a pu se remettre dans le droit chemin. Ces femmes et ces hommes ont risqué leur vie, leur bien-être et leur sécurité individuels, pour venir en aide au plus grand nombre[^whistleblower]. Sans eux, qui sait ce que SmartPoop serait devenu ? Qui sait ce que je serais devenue ?
103 | 
104 | [^whistleblower]: Malheureusement, les lanceurs d'alerte souffrent souvent de davantage de troubles de santé mental suite à leurs actes courageux. Vu le rôle critique qu'ils jouent pour révéler des scandales dans des entreprises et des organisations dangereusement opaques, il semble urgent de beaucoup mieux les accompagner.  
105 | [**Science.**  Mental Health Problems Among Whistleblowers: A Comparative Study. Peter van der Velden, Mauro Pecoraro, Mijke Houwerzijl & Erik van der Meulen. Psychological Reports (2018).](https://journals.sagepub.com/doi/10.1177/0033294118757681)
106 | 
107 | Seule sur scène, Katia lâche une larme.
108 | 
109 | > Merci à ces héros, dit-elle en pleurs.
110 | 
111 | Le public applaudit ce moment très émouvant.
112 | 
113 | ### Le défi de l'éthique de Poo
114 | 
115 | Katia a encore besoin de plusieurs secondes pour reprendre ses émotions. Finalement, elle se reprend.
116 | 
117 | > Néanmoins, je refuse de dire que Poo est un succès. Ensemble, nous avons créé un produit incroyable. Mais il reste infiniment améliorable, notamment sur le plan de son éthique, de sa sécurité et de sa gouvernance. Comment contrôle-t-on Poo ? Comment empêche-t-on Poo de dire des mots blessants, de révéler des secrets, de répéter des discours de haine et de diffuser de la mésinformation[^language-model] ? Comment l'amène-t-on à être bienveillant avec ses interlocuteurs, à dire les mots justes pour les rendre plus épanouis et à promouvoir autant que possible de l'information fiable et non trompeuse[^non-trompeur] ? Comment l'amène-t-on à mieux réfléchir, et à vouloir investiguer ses incertitudes plutôt que de se contenter de ses intuitions[^scout] ? Mais surtout, comment peut-on décider collectivement de ce que Poo doit dire ? Comment déterminer ce qui est désirable à dire, et ce qui ne devrait jamais être dit[^ethics-of-influence] ?
118 | 
119 | [^language-model]: Ces problèmes ne sont absolument pas résolus pour les algorithmes conversationnels modernes, qui sont très vulnérables à des attaques d'espionnage ou par empoisonnement des données. Et pourtant, ces algorithmes sont déjà déployés à très grande échelle, via les claviers intelligents, via les assistants personnels (Siri, Alexa, OK Google) et via les moteurs de recherche (Google, YouTube).  
120 | [**Science.**  On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?  Emily Bender, Timnit Gebru, Angelina McMillan-Major & Shmargaret Shmitchell. FAccT (2021).](https://dl.acm.org/doi/10.1145/3442188.3445922)  
121 | [**Science.**  The Radicalization Risks of GPT-3 and Advanced Neural Language Models. Kris McGuffie, Alex Newhouse (2020).](https://arxiv.org/abs/2009.06807)  
122 | [**Science.**  Extracting Training Data from Large Language Models. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea & Colin Raffel. USENIX (2021).](https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting)
123 | 
124 | [^non-trompeur]: [**Vidéo.**  Qu’est ce qu’un message d’utilité publique ? Science4All (2021).](https://www.youtube.com/watch?v=LZGhBmDVi5A&list=PLtzmb84AoqRRFcoGQ5p7kqEVQ7deXfYuH&index=5)
125 | 
126 | [^scout]: Julia Galef parle du *scout mindset* (ou *mode explorateur*), par opposition au *soldier mindset* (*mode soldat*). Selon elle, il s'agit de l'aspect le plus déterminant pour analyser de l'information plus correctement.  
127 | [**Vidéo.**  Comment j'essaye d'améliorer mon jugement (grâce à Julia Galef et à FLUS). Science Étonnante (2021).](https://www.youtube.com/watch?v=Sm_FgccC9dk)
128 | [**Vidéo.**  Le mode explorateur. Science4All (2021).](https://www.youtube.com/watch?v=EZfNX4U63hc)  
129 | [**Livre.**  The Scout Mindset: Why Some People See Things Clearly and Others Don't. Julia Galef. Penguin (2021).](https://www.penguinrandomhouse.com/books/555240/the-scout-mindset-by-julia-galef/)
130 | 
131 | [^ethics-of-influence]: Chaque mot choisi ou chaque recommandation faite par un algorithme peut être vu comme un *nudge*. De nombreuses études montrent que l'acceptabilité et l'efficacité des *nudges* dépendent fortement du *nudge* considéré ; avec plus de données à ce sujet, il pourrait être ainsi possible de mettre en oeuvre les nudges particulièrement acceptés socialement et efficaces.  
132 | [**Livre.**  The Ethics of Influence: Government in the Age of Behavioral Science. Cass Sunstein. Cambridge University Press (2016).](https://www.cambridge.org/core/books/ethics-of-influence/E29EDE19EBCB53F6D8691730668115F7)
133 | 
134 | Katia marque une pause, devant un public tout ouï.
135 | 
136 | > Et ça, ça m'amène à la fameuse rumeur dont vous avez sans doute entendu parler. Comme quoi, on aurait un plan pour résoudre le problème de l'éthique de l'information…
137 | 
138 | Katia marque alors un silence, avant de rajouter sur un ton sarcastique.
139 | 
140 | > Genre, « SmartPoop, cette application de merde, va résoudre l'éthique ».
141 | 
142 | Le public rit en cœur.
143 | 
144 | > Vous savez, il ne faut pas croire tout ce qu'on vous dit.
145 | 
146 | Katia marque un autre silence.
147 | 
148 | > Mais dans ce cas, oui c'est vrai. Ou du moins, on compte y contribuer.
149 | 
150 | Le public rit.
151 | 
152 | > Et, je le sais bien, car c'est moi qui ai fait fuiter la rumeur.
153 | 
154 | Le public, conquis, rit à nouveau, même si beaucoup commencent à avoir un visage confus.
155 | 
156 | > Ceci dit, chez SmartPoop, l'éthique de l'information nous tient clairement à cœur. Nous voulons déterminer quelles informations devraient être collectées par qui et dans quelles conditions, comment elles devraient être stockées, à qui ces informations devraient être accessibles, quels traitements de cette information devraient être effectués, comment ces traitements de l'information devraient être audités et sécurisés, où stocker les résultats de ces calculs, qui peut avoir accès à ces résultats, et qui sera notifié de l'existence de ces résultats[^information-serres].
157 | 
158 | [^information-serres]: [**Vidéo.**  Michel Serres - Les nouvelles technologies : révolution culturelle et cognitive. I Moved to Diaspora (2012).](https://www.youtube.com/watch?v=ZCBB0QEmT5g)
159 | 
160 | Après un autre silence, Katia reprend son discours.
161 | 
162 | > Et là, je sens qu'il y en a pas mal parmi vous qui se disent : « mais pour qui elle se prend, Katia ? »
163 | 
164 | Le public rit à nouveau.
165 | 
166 | > Bien entendu, beaucoup de ces opérations devraient être largement configurables par les utilisateurs. Ceci étant dit, la plupart des utilisateurs ne voudront pas gérer toutes les configurations de tous leurs systèmes d'information, et vérifier, par exemple, que ces configurations respectent le Règlement Général sur la Protection des Données, ou empêchent la diffusion massive de discours de haine. Non, la plupart des utilisateurs sont comme moi : ce sont des fainéants.
167 | 
168 | Le rire du public permet alors à Katia de reprendre son souffle.
169 | 
170 | > Mais surtout, sur Internet, beaucoup d'utilisateurs veulent nuire à d'autres utilisateurs ; ou au moins les influencer d'une certaine manière. Pensez à toutes les campagnes de désinformation qui sévissent sur les réseaux sociaux[^desinformation-2]. Or, même l'un des plus grands défenseurs du libéralisme, le philosophe John Stuart Mill, pense que la liberté des uns doit s'arrêter là où elle nuit aux autres. Tel est le principe de l'absence de tort.
171 | 
172 | [^desinformation-2]: [**Vidéo.**  Les réseaux sociaux sont dangereux. Très dangereux. Science4All (2021).](https://www.youtube.com/watch?v=utWMGi8HTjY&list=PLtzmb84AoqRRFcoGQ5p7kqEVQ7deXfYuH&index=1)
173 | 
174 | Katia marque à nouveau une pause, et regarde son public, en déplaçant son regard de gauche à droite.
175 | 
176 | > L'un des grands défis de l'éthique de l'information, c'est l'implémentation du principe de l'absence de tort. Car pour autant que je sache, ce principe est quasi-consensuel en philosophie morale[^absence-tort] — et c'est un sacré exploit d'être consensuel en philosophie morale.
177 | 
178 | [^absence-tort]: [**Vidéo.**  Ce principe sur lequel tout le monde s'entend. Monsieur Phi (2021).](https://www.youtube.com/watch?v=JY3SAzlgBYY)
179 | 
180 | Katia marque un nouveau silence.
181 | 
182 | > Bon ça, c'est en principe. En pratique, c'est difficile de se mettre d'accord sur ce qui constitue un tort. Est-ce qu'un commentaire agressif cause un tort ? Est-ce qu'un petit mensonge cause un tort ? Est-ce qu'un mensonge par omission cause un tort ? Est-ce qu'une blague au dépens d'une communauté cause un tort ? Malheureusement, en pratique, on ne sera pas d'accord. Nous avons des préférences éthiques difficilement réconciliables, voire parfois clairement irréconciliables. Que faire alors[^scrutins-sml] ?
183 | 
184 | [^scrutins-sml]: [**Vidéo.**  [Conférence SML] Les mathématiques de la démocratie - Lê Nguyên Hoang. Maison des mathématiques et de l'informatique (2018).](https://www.youtube.com/watch?v=CYqLfWFdFoc)
185 | 
186 | Katia semble vraiment poser cette question à son public, comme si elle attendait une réponse. Le public a l'air pensif, et attend impatiemment une réponse de Katia.
187 | 
188 | ### Qui décidera de l'éthique Poo ?
189 | 
190 | Katia relance avec une autre question.
191 | 
192 | > Est-ce que SmartPoop devrait trancher ?
193 | 
194 | Le public reste silencieux. Clairement, il s'agit d'un groupe de fans de SmartPoop. Mais même eux ne semblent pas emballés à cette idée.
195 | 
196 | > Si vous voulez mon avis, la réponse est clairement non. On l'a vu il y a deux ans. Une structure comme SmartPoop peut perdre son éthique et sa voie. Et même si nous avons fait énormément de progrès dans notre gouvernance pour éviter que ceci ne se reproduise, je ne pense pas que SmartPoop soit suffisamment robustement bénéfique pour une telle tâche.
197 | 
198 | Katia marque une pause, comme si elle invitait sérieusement le public à y réfléchir.
199 | 
200 | > Mais donc, qui ? Qui devrait déterminer l'éthique de l'information et du traitement de l'information ?
201 | 
202 | Encore une fois, cette question est posée comme si Katia n'en avait aucune réponse, et comme si elle attendait du public une réponse. Après de longues secondes, Katia offre sa réponse.
203 | 
204 | > Eh bien, la proposition de SmartPoop, c'est que ce soit vous. Ou plutôt, nous tous. Toute l'humanité doit décider, ensemble et collectivement, de l'éthique de l'information.
205 | 
206 | Le public applaudit.
207 | 
208 | > Mais... Comment peut-on amener des milliards d'humains à décider collectivement de quelque chose d'aussi complexe que l'éthique de l'information ?
209 | 
210 | Le public est maintenant circonspect.
211 | 
212 | > Réfléchissez-y. Comment fait-on aujourd'hui pour prendre des décisions collectives ?
213 | 
214 | Katia marque à nouveau une pause, jusqu'à ce qu'elle entende quelqu'un dans le public crier « le vote ».
215 | 
216 | > Le vote, oui ! Dans beaucoup de pays à travers le monde, quand une décision collective doit être prise, on essaie souvent de se ramener à une question avec une réponse oui ou non, et on demande au peuple de voter pour oui, ou pour non ? C'est ainsi que l'on tranche des désaccords irréconciliables. En fait, quand on y pense, c'est absolument remarquable que dans les démocraties à travers le monde, nous nous sommes mis d'accord sur comment nous mettre d'accord, même quand nous sommes d'accord sur le fait que nous ne finirons pas par être d'accord sur ce qu'il faudrait faire[^meta-ethique] !
217 | 
218 | [^meta-ethique]: Techniquement, cela correspond à trouver un consensus sur la méta-éthique plutôt que sur l'éthique elle-même. L'espoir ici est que nous sommes plus susceptibles d'être d'accord sur la méta-éthique que sur l'éthique. En fait, même les considérations méta-éthiques sont susceptibles de rester polarisées, comme la façon d'allouer les droits de vote, auquel cas une méta-méta-éthique peut être nécessaire pour trouver un accord.
219 | 
220 | Katia s'arrête quelques secondes.
221 | 
222 | > Mais le problème du vote, en tout cas tel qu'il est pratiqué aujourd'hui, c'est qu'il ne permet à chaque citoyen d'envoyer que quelques bits d'information par vote seulement. La COP 30, pour ou contre ? La vaccination obligatoire, pour ou contre ? La régulation des algorithmes, pour ou contre ? Lequel des 15 candidats suivants devrait être élu ? Il n'y a pas 36 000 réponses possibles à ces questions. Or, pour résoudre l'éthique des algorithmes, il va falloir fournir des réponses complexes. Il y a même des milliards de milliards de discours qu'un utilisateur de SmartPoop peut produire. Parmi les milliards de réponses imaginables, laquelle Poo devra-t-elle adopter ?
223 | 
224 | Katia reprend son souffle, avant la suite de son discours.
225 | 
226 | > Et si on concevait désormais des votes où la voix de chacun n'était pas réduite à une réponse binaire[^mediane-geometrique], donnée uniquement une fois par an ? Et si on permettait à chacun de partager toute la complexité de son jugement éthique ? Et si on parvenait à tenir compte de toute cette complexité pour décider collaborativement de l'éthique de l'information[^licchavi] ?
227 | 
228 | [^mediane-geometrique]: L'une des solutions pour voter en grande dimension est de s'appuyer sur le principe « un électeur, une force unitaire », ce qui peut typiquement conduire à utiliser la *médiane géométrique*.  
229 | [**Science.**  On the Strategyproofness of the Geometric Median. El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui & Lê-Nguyên Hoang (2021).](https://arxiv.org/abs/2106.02394)
230 | 
231 | [^licchavi]: Ceci nécessitera certainement de combiner des systèmes de scrutins avec des méthodes d'apprentissage. C'est ce que propose Licchavi.  
232 | [**Science.**  Strategyproof Learning: Building Trustworthy User-Generated Datasets. Sadegh Farhadkhani, Rachid Guerraoui & Lê-Nguyên Hoang (2021).](https://arxiv.org/abs/2106.02398)
233 | 
234 | Katia marque une nouvelle pause.
235 | 
236 | ### Le fabuleux chantier[^fabuleux-chantier]
237 | 
238 | [^fabuleux-chantier]: Le fabuleux chantier est le nom d'un livre précédent de Lê Nguyên Hoang, l'un des auteurs de ce livre.  
239 | [**Livre.**  Le fabuleux chantier : Rendre l'intelligence artificielle robustement bénéfique. Lê Nguyên Hoang & El Mahdi El Mhamdi. EDP Sciences (2019).](https://laboutique.edpsciences.fr/produit/1107/9782759824304/Le fabuleux chantier)
240 | 
241 | Katia change alors de ton, en prenant une voix plus grave et posée.
242 | 
243 | > Mesdames et messieurs, aujourd'hui est un jour historique, car je vais vous présenter le résultat de deux ans de travail, en collaboration intime avec l'OMESA et de nombreuses autres organisations, ainsi que des académiques. Au moment où je parle, une nouvelle plateforme vient d'être mise en ligne, appelée girasol.app[^girasol-tournesol]. Girasol est un site web entièrement Open Source, sous licence libre[^license], qui va coordonner la conception de l'éthique de l'information, en permettant aux utilisateurs de fournir des jugements éthiques, et en utilisant des algorithmes de vote pour construire collaborativement l'éthique de l'information à partir des jugements éthiques des utilisateurs ! Le premier étage d'une éthique démocratique de l'information a été posé !
244 | 
245 | [^girasol-tournesol]: Girasol n'existe pas, mais il s'agit là en fait clairement d'une référence au projet [tournesol.app](https://tournesol.app) lancé par Lê Nguyên Hoang, l'un des auteurs de ce livre. Le reste du livre décrit finalement la vision global de Tournesol. Vous trouverez beaucoup plus d'informations sur le [wiki de Tournesol](https://wiki.tournesol.app). Il est utile de noter que, surtout pour l'instant, l'objectif de Tournesol est davantage de servir de « microscope des jugements humains », c'est-à-dire d'outils de collecte de données sur ce que les humains jugent éthiquement préférables. En particulier, et entre autres, Tournesol espère ainsi détecter des consensus moraux aujourd'hui difficilement observables, faute de données.
246 | 
247 | [^license]: Le code de Tournesol est sous [licence AGPL](https://www.gnu.org/licenses/agpl-3.0.en.html), tandis que la base de données publique est sous [license ODbL](https://opendatacommons.org/licenses/odbl/) (à confirmer).
248 | 
249 | Le public applaudit cette annonce de Katia.
250 | 
251 | > Je précise que la gouvernance de ce projet est entièrement sous le contrôle de l'OMESA aujourd'hui, et SmartPoop ne sert, et ne servira, que de contributeur bénévole à la base de codes et à la promotion du projet. Tout le code est audité par de nombreuses entités, si bien qu'il est quasiment impossible que le projet soit détourné par une entité maléfique — et ça inclut de potentiels investisseurs de SmartPoop !
252 | 
253 | Katia marque une nouvelle pause.
254 | 
255 | > Alors, il y aurait énormément d'autres détails importants à préciser sur ce projet complexe. Mais sachez que, en collaboration avec l'OMESA, nous avons fait de notre mieux pour que chacun de ces détails devienne un sujet de recherche sur lequel travaillent différentes équipes pluridisciplinaires à travers le monde. Ces détails incluent des problématiques comme l'authentification des comptes, pour éviter les faux comptes, en exploitant des mécanismes de Proof of Personhood, et comme garantir le fait que chaque compte authentifié a le même droit de vote que tout autre compte authentifié. Ils incluent aussi l'identification de l'expertise et des excès de confiance des utilisateurs, pour éviter que des théories anti-scientifiques polluent l'éthique de l'information. On peut mentionner aussi l'optimisation de la plateforme par individu, pour que cet individu exploite la manière la plus appropriée pour lui d'exprimer ses jugements éthiques. Ou encore les algorithmes de rectification du biais de participation, pour bien tenir compte des préférences de ceux qui n'ont pas pu participer à Girasol, faute de temps ou d'accès à Internet[^defis-tournesol].
256 | 
257 | [^defis-tournesol]: Le projet Tournesol soulève énormément de défis, allant de la recherche au développement, en passant par la promotion, par le financement et par les partenariats, entre autres, que les membres de Tournesol ne pourront absolument pas résoudre seuls. *Vous* pouvez aider. Pour en savoir plus, notamment sur l'aspect recherche et développement, nous vous encourageons à lire le *white paper* technique du projet.  
258 | [**Science.**  Tournesol: A quest for a large, secure and trustworthy database of reliable human judgments. Lê-Nguyên Hoang, Louis Faucon, Aidan Jungo, Sergei Volodin, Dalia Papuc, Orfeas Liossatos, Ben Crulis, Mariame Tighanimine, Isabela Constantin, Anastasiia Kucherenko, Alexandre Maurer, Felix Grimberg, Vlad Nitu, Chris Vossen, Sébastien Rouault & El-Mahdi El-Mhamdi (2021).](https://arxiv.org/abs/2107.07334)
259 | 
260 | Katia s'arrête quelques instants.
261 | 
262 | > Bref. Il y a plein de défis de recherches remarquables que Girasol devra résoudre, pour ensuite permettre une conception collaborative adéquate de l'éthique de l'information. Girasol est clairement un énorme chantier très difficile à mener. Et c'est aussi un chantier urgent à résoudre.
263 | 
264 | Katia reprend son souffle pour conclure son discours.
265 | 
266 | > Mais avant tout, Girasol est un *fabuleux* chantier. Si vous me demandez, il s'agit pour moi du plus fabuleux de tous les chantiers menés par l'humanité, plus grandiose encore que construire des pyramides, plus ambitieux que d'éradiquer des pandémies comme la variole[^variole], et plus spectaculaire que d'envoyer des humains sur la Lune. Girasol, c'est unir toute l'humanité derrière le plus important de tous les aspects de la civilisation humaine : maîtriser collaborativement le flux de l'information[^prochain-livre], et garantir que celui-ci coule comme nous, l'humanité, souhaiterions vraiment qu'il coule, lorsqu'on y réfléchit à tête reposée, avec bienveillance et rigueur. Mesdames et messieurs, ensemble, résolvons l'éthique de l'information[^resoudre-ethique] !
267 | 
268 | [^variole]: [**Vidéo.**  Le plus grand triomphe de l'humanité. Science4All (2020).](https://www.youtube.com/watch?v=eAzP2QtAAag&list=PLtzmb84AoqRS0SN8VKvAxuGOdcINPRugV&index=9)
269 | 
270 | [^prochain-livre]: Ceci pourrait d'ailleurs être le sujet d'un prochain livre de Lê Nguyên Hoang... #teaser
271 | 
272 | [^resoudre-ethique]: [**Vidéo.**  Résolvons l'éthique ensemble !! Science4All (2021).](https://www.youtube.com/watch?v=TgB9pHZ0YPM&list=PLtzmb84AoqRRFcoGQ5p7kqEVQ7deXfYuH&index=4)
273 | 
274 | À ces mots, le public explose de joie et d'enthousiasme, alors que le vacarme laisse petit à petit place au nom de Katia, lequel est scandé en rythme par tout le stade. Seule sur scène, Katia profite de l'instant, avec un sourire radieux, et salue le public de la main. À ce moment, elle pense à tout ce que SmartPoop a accompli jusque là. Mais aussi et surtout, Katia est incroyablement enthousiasmée par la vision d'une civilisation humaine qui, grâce à Girasol, va enfin prendre le destin de sa civilisation entre ses mains.
275 | 
276 | Collaborativement.
277 | 
278 | 
279 | ## Pour aller plus loin
280 | 
281 | Voilà, le roman est fini !
282 | Vous pouvez revenir au [sommaire](README.md).  
283 | Si vous avez apprécié, pensez à partager et à promouvoir ce roman de science-fiction auprès de vous.
284 | Nous vous en serions très reconnaissants !
285 | 


--------------------------------------------------------------------------------
/French/README.md:
--------------------------------------------------------------------------------
 1 | # SmartPoop 1.0
 2 | <img src="../SmartPoop.jpg" alt="SmartPoop.jpg" width="400" align="right" />
 3 | 
 4 | ## L'histoire d'une IA qui se lâche
 5 | 
 6 | Bienvenue dans le github du livre *SmartPoop 1.0*, co-écrit par Lê Nguyên Hoang et Tristan Le Magoarou,
 7 | publié sous [licence CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (réutilisable librement à condition de citer les auteurs originaux).
 8 | 
 9 | Le livre est disponible en [version papier](https://www.amazon.fr/dp/B09MDF78ZY), sur [GitHub](1-L'or.md), en [epub](SmartPoop-fr.epub) et en [pdf](SmartPoop-fr.pdf).  
10 | 
11 | ## Sommaire
12 | 
13 | [Chapitre 1 — L'or marron](1-L'or.md)  
14 | [Chapitre 2 — Filtrer les données fécales](2-Filtrer.md)  
15 | [Chapitre 3 — Le biais étroniste](3-Biais.md)  
16 | [Chapitre 4 — Les fuites sanitaires](4-Fuites.md)  
17 | [Chapitre 5 — Les FakePoops](5-FakePoops.md)  
18 | [Chapitre 6 — Le marché marron](6-Pub.md)  
19 | [Chapitre 7 — On se fait dessus](7-Psy.md)  
20 | [Chapitre 8 — Sur le trône](8-Trone.md)  
21 | 
22 | ## Pourquoi ce livre ?
23 | 
24 | *SmartPoop 1.0* est une science-fiction très réaliste,
25 | qui vise à sensibiliser au danger des algorithmes et aux défis de les rendre [robustement bénéfiques](https://laboutique.edpsciences.fr/produit/1107/9782759824304/Le%20fabuleux%20chantier).
26 | 
27 | Elle raconte l'histoire d'une entreprise du numérique, *SmartPoop*, qui effectue des analyses automatisées d'excréments.
28 | Ceci lui permet des diagnostics médicaux ultra-personnalisés.
29 | Cependant, dans chaque chapitre, le déploiement de *SmartPoop* conduit à des *effets secondaires imprévus*,
30 | avec des conséquences médicales et sociales tragiques.
31 | Les co-fondateurs de *SmartPoop*, les héros de l'histoire,
32 | doivent alors prendre la responsabilité de leur technologie,
33 | en subir les conséquences,
34 | et trouver des solutions pour rendre leurs algorithmes sécurisés et éthiques.
35 | 
36 | ## À propos des auteurs
37 | 
38 | ### Lê Nguyên Hoang
39 | 
40 | [Lê Nguyên Hoang](https://fr.wikipedia.org/wiki/L%C3%AA_Nguy%C3%AAn_Hoang) est un chercheur et médiateur scientifique à la faculté d'informatique et communication de l'EPFL.
41 | [Sa recherche](https://scholar.google.ch/citations?user=0ZADKSkAAAAJ&hl=en&oi=ao) porte sur la sécurité et l'éthique des algorithmes, notamment sur [la théorie de l'apprentissage](https://arxiv.org/abs/2008.00742) et [la conception participative](https://arxiv.org/abs/2106.02398).
42 | Lê est aussi YouTubeur, sur les chaînes [Science4All](https://www.youtube.com/channel/UC0NCbj8CxzeCGIF6sODJ-7A/), [Axiome](https://www.youtube.com/channel/UCNHFiyWgsnaSOsMtSoV_Q1A/), [Étincelles, EPFL](https://www.youtube.com/channel/UC0o_ULEdwva8ksC63qvxMAA), [Wandida, EPFL](https://www.youtube.com/channel/UCD_VizaraVT9yDy1egNlWCQ/) et [ZettaBytes, EPFL](https://www.youtube.com/channel/UCfY6ovyFMaw30NRs-KrxrWw/).
43 | Il est l'auteur de trois livres, intitulés *[La formule du savoir](https://laboutique.edpsciences.fr/produit/1035/9782759822614/La%20formule%20du%20savoir)*, *[Le fabuleux chantier](https://laboutique.edpsciences.fr/produit/1107/9782759824304/Le%20fabuleux%20chantier)* et *[Turing à la plage](https://www.dunod.com/sciences-techniques/turing-plage-intelligence-artificielle-dans-un-transat)*.
44 | Il est aussi membre du conseil d'éthique d'Orange.
45 | Enfin, Lê est Président de [l'Association Tournesol](https://wiki.tournesol.app), dont [la plateforme](https://tournesol.app) vise à collecter une grande base de données, fiable et sécurisée, de jugements éthiques humains, [pour contribuer à résoudre l'éthique des algorithmes](https://arxiv.org/abs/2107.07334).
46 | 
47 | ### Tristan Le Magoarou
48 | 
49 | Docteur en médecine, spécialiste en santé publique et médecine sociale, Tristan Le Magoarou est médecin d'information médicale et de santé publique dans un centre hospitalier de province.
50 | Il est également, depuis 2016, YouTubeur sur sa chaîne Risque Alpha qui possède 26 000 abonnés où il vulgarise l'épidémiologie et les statistiques.
51 | 
52 | ### Autres
53 | 
54 | L'illustration du livre a été réalisée par [Thibault Roy](https://tibodetroy.blogspot.com/).
55 | Le livre résulte par ailleurs d'interactions avec un grand nombre de personnes envers lesquelles les auteurs sont très redevables, dont El Mahdi El Mhamdi, Mariame Tighanimine, Sébastien Rouault, Ly An Hoang, Aidan Jungo, Alexandre Maurer, Sylvain Hippolyte, Felix Grimberg et Oscar Villemaud, parmi d'autres.
56 | 
57 | ## À propos de l'écriture du livre
58 | 
59 | Ce livre est voué à être constamment mis à jour pour le rendre plus lisible, plus crédible et plus pertinent.
60 | Nous invitons nos lecteurs en particulier à proposer des corrections, typographiques ou au niveau de l'histoire.
61 | Notez toutefois que les auteurs Lê Nguyên Hoang et Tristan Le Magoarou auront le dernier mot sur la version disponible sur ce github, et qu'ils travaillent dessus en tant que bénévoles.
62 | Notez que la [licence CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) autorise n'importe qui à réutiliser et adapter le livre à leur guise, à condition de mentionner la version originale du livre et ses auteurs.
63 | Nous encourageons de tels efforts, et nous nous tenons à votre disposition pour tout projet de ce genre.
64 | 
65 | 
66 | ## Bibliographie
67 | 
68 | 1. [Le fabuleux chantier : rendre l'intelligence artificielle robustement bénéfique](https://laboutique.edpsciences.fr/produit/1107/9782759824304/Le%20fabuleux%20chantier). Lê Nguyên Hoang et El Mahdi El Mhamdi. EDP Sciences (2019).
69 | 
70 | 2. [Turing à la plage : l'intelligence artificielle dans un transat](https://www.dunod.com/sciences-techniques/turing-plage-intelligence-artificielle-dans-un-transat). Rachid Guerraoui et Lê Nguyên Hoang. Dunod (2020).
71 | 
72 | 3. [Tout ce que vous devez savoir sur le plus tabou des sujets](http://www.michel-lafon.fr/livre/2059-DANS_TON_CORPS.html). Michel Lafond. Julien Ménielle (2018).
73 | 
74 | 
75 | 


--------------------------------------------------------------------------------
/French/SmartPoop-fr.epub:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/lenhoanglnh/SmartPoop/6b2772f5fb4a525959b6e1a3d554e2cd4e79f407/French/SmartPoop-fr.epub


--------------------------------------------------------------------------------
/French/SmartPoop-fr.pdf:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/lenhoanglnh/SmartPoop/6b2772f5fb4a525959b6e1a3d554e2cd4e79f407/French/SmartPoop-fr.pdf


--------------------------------------------------------------------------------
/French/compile.sh:
--------------------------------------------------------------------------------
 1 | pandoc --toc -o SmartPoop-fr.epub titre.txt \
 2 |   README.md \
 3 |   1-L\'or.md \
 4 |   2-Filtrer.md \
 5 |   3-Biais.md \
 6 |   4-Fuites.md \
 7 |   5-FakePoops.md \
 8 |   6-Pub.md \
 9 |   7-Psy.md \
10 |   8-Trone.md
11 | 
12 | pandoc --toc --template=template.tex -o SmartPoop-fr.pdf titre.txt \
13 |   README.md \
14 |   1-L\'or.md \
15 |   2-Filtrer.md \
16 |   3-Biais.md \
17 |   4-Fuites.md \
18 |   5-FakePoops.md \
19 |   6-Pub.md \
20 |   7-Psy.md \
21 |   8-Trone.md


--------------------------------------------------------------------------------
/French/template.tex:
--------------------------------------------------------------------------------
  1 | \documentclass{book}
  2 | \usepackage[T1]{fontenc}
  3 | 
  4 | \usepackage{fancyhdr}
  5 | \pagestyle{plain}
  6 | 
  7 | \usepackage{ifxetex,ifluatex}
  8 | \IfFileExists{upquote.sty}{\usepackage{upquote}}{}
  9 | \ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
 10 |   \usepackage[utf8]{inputenc}
 11 | $if(euro)$
 12 |   \usepackage{eurosym}
 13 | $endif$
 14 | \else % if luatex or xelatex
 15 |   \ifxetex
 16 |     \usepackage{mathspec}
 17 |     \usepackage{xltxtra,xunicode}
 18 |   \fi
 19 |   \defaultfontfeatures{Mapping=tex-text,Scale=MatchLowercase}
 20 |   \newcommand{\euro}{€}
 21 | \fi
 22 | % use microtype if available
 23 | % \IfFileExists{microtype.sty}{\usepackage{microtype}}{}
 24 | \usepackage[paperwidth=6in, paperheight=9in, bindingoffset=.25in, left=.5in, right=.5in, top=.5in, bottom=1in]{geometry}
 25 | $if(natbib)$
 26 | \usepackage{natbib}
 27 | \bibliographystyle{$if(biblio-style)$$biblio-style$$else$plainnat$endif$}
 28 | $endif$
 29 | $if(biblatex)$
 30 | \usepackage{biblatex}
 31 | $if(biblio-files)$
 32 | \bibliography{$biblio-files$}
 33 | $endif$
 34 | $endif$
 35 | $if(listings)$
 36 | \usepackage{listings}
 37 | $endif$
 38 | $if(lhs)$
 39 | \lstnewenvironment{code}{\lstset{language=Haskell,basicstyle=\small\ttfamily}}{}
 40 | $endif$
 41 | $if(highlighting-macros)$
 42 | $highlighting-macros$
 43 | $endif$
 44 | $if(verbatim-in-note)$
 45 | \usepackage{fancyvrb}
 46 | $endif$
 47 | $if(tables)$
 48 | \usepackage{longtable}
 49 | $endif$
 50 | $if(graphics)$
 51 | \usepackage{graphicx}
 52 | % Redefine \includegraphics so that, unless explicit options are
 53 | % given, the image width will not exceed the width of the page.
 54 | % Images get their normal width if they fit onto the page, but
 55 | % are scaled down if they would overflow the margins.
 56 | \makeatletter
 57 | \def\ScaleIfNeeded{%
 58 |   \ifdim\Gin@nat@width>\linewidth
 59 |     \linewidth
 60 |   \else
 61 |     \Gin@nat@width
 62 |   \fi
 63 | }
 64 | \makeatother
 65 | \let\Oldincludegraphics\includegraphics
 66 | {%
 67 |  \catcode`\@=11\relax%
 68 |  \gdef\includegraphics{\@ifnextchar[{\Oldincludegraphics}{\Oldincludegraphics[width=\ScaleIfNeeded]}}%
 69 | }%
 70 | $endif$
 71 | \ifxetex
 72 |   \usepackage[setpagesize=false, % page size defined by xetex
 73 |               unicode=false, % unicode breaks when used with xetex
 74 |               xetex]{hyperref}
 75 | \else
 76 |   \usepackage[unicode=true]{hyperref}
 77 | \fi
 78 | \hypersetup{breaklinks=true,
 79 |             bookmarks=true,
 80 |             pdfauthor={$author-meta$},
 81 |             pdftitle={$title-meta$},
 82 |             colorlinks=true,
 83 |             citecolor=$if(citecolor)$$citecolor$$else$blue$endif$,
 84 |             urlcolor=$if(urlcolor)$$urlcolor$$else$blue$endif$,
 85 |             linkcolor=$if(linkcolor)$$linkcolor$$else$magenta$endif$,
 86 |             pdfborder={0 0 0}}
 87 | \urlstyle{same}  % don't use monospace font for urls
 88 | $if(links-as-notes)$
 89 | % Make links footnotes instead of hotlinks:
 90 | \renewcommand{\href}[2]{#2\footnote{\url{#1}}}
 91 | $endif$
 92 | $if(strikeout)$
 93 | \usepackage[normalem]{ulem}
 94 | % avoid problems with \sout in headers with hyperref:
 95 | \pdfstringdefDisableCommands{\renewcommand{\sout}{}}
 96 | $endif$
 97 | \setlength{\parindent}{0pt}
 98 | \setlength{\parskip}{6pt plus 2pt minus 1pt}
 99 | \setlength{\emergencystretch}{3em}  % prevent overfull lines
100 | $if(numbersections)$
101 | \setcounter{secnumdepth}{5}
102 | $else$
103 | \setcounter{secnumdepth}{0}
104 | $endif$
105 | $if(verbatim-in-note)$
106 | \VerbatimFootnotes % allows verbatim text in footnotes
107 | $endif$
108 | $if(lang)$
109 | \ifxetex
110 |   \usepackage{polyglossia}
111 |   \setmainlanguage{$mainlang$}
112 | \else
113 |   \usepackage[$lang$]{babel}
114 | \fi
115 | $endif$
116 | $for(header-includes)$
117 | $header-includes$
118 | $endfor$
119 | 
120 | \usepackage{titlesec}
121 | 
122 | \titleformat{\chapter}{\normalfont\huge}{}{20pt}{\huge\bf}
123 | 
124 | % \usepackage{titletoc}
125 | 
126 | % $if(title)$
127 | % \title{$title$}
128 | % $endif$
129 | % $if(subtitle)$
130 | % \subtitle{$subtitle$}
131 | % $endif$
132 | % \author{$for(author)$$author$$sep$ \and $endfor$}
133 | % \date{$date$}
134 | 
135 | \begin{document}
136 | 
137 | \begin{titlepage} % Suppresses headers and footers on the title page
138 | 	\centering % Centre everything on the title page
139 | 	\scshape % Use small caps for all text on the title page
140 | 	\vspace*{\baselineskip} % White space at the top of the page
141 | 	\rule{\textwidth}{1.6pt}\vspace*{-\baselineskip}\vspace*{2pt} % Thick horizontal rule
142 | 	\rule{\textwidth}{0.4pt} % Thin horizontal rule
143 | 	\vspace{0.75\baselineskip} % Whitespace above the title
144 | 
145 | 	%{\Huge L'intelligence artificielle tue \\[20pt]} % Title
146 | 	{\Huge $title$}\\[20pt]
147 | 	{\Large $subtitle$ \\[20pt]}
148 | 
149 | 	% {\large PRIVATELY SHARED DRAFT \\}
150 | 	% {\LARGE Beta version \\}
151 | 
152 | 	\vspace{1\baselineskip} % Whitespace below the title
153 | 	\rule{\textwidth}{0.4pt}\vspace*{-\baselineskip}\vspace{3.2pt} % Thin horizontal rule
154 | 	\rule{\textwidth}{1.6pt} % Thick horizontal rule
155 | 	\vspace{2\baselineskip} % Whitespace after the title block
156 | 
157 | 	%	Subtitle
158 | 	% {\large Le fabuleux chantier pour la rendre robustement bénéfique}
159 | 
160 | 	\vspace*{3\baselineskip} % Whitespace under the subtitle
161 | 
162 | 	Écrit par
163 | 
164 | 	\vspace{0.5\baselineskip} % Whitespace before the editors
165 | 	{\scshape\Large $for(author)$$author$$sep$ \vspace{0.3\baselineskip} \\[10pt] et \\[10pt]
166 |    \vspace{0.3\baselineskip} $endfor$} % Editor list
167 | 
168 | 	\vspace{0.5\baselineskip} % Whitespace below the editor list
169 | 
170 | 	%\textit{EPFL, Switzerland} % Editor affiliation
171 | 	\vfill % Whitespace between editor names and publisher logo
172 | 
173 | 	\vspace{0.3\baselineskip} % Whitespace under the publisher logo
174 | 	2021 % Publication year
175 | 
176 | 	%{\large EDP Sciences} % Publisher
177 | 
178 | \end{titlepage}
179 | 
180 | % $if(title)$
181 | % \maketitle
182 | % $endif$
183 | 
184 | $for(include-before)$
185 | $include-before$
186 | 
187 | $endfor$
188 | 
189 | 
190 | % \makeatletter
191 | % \let\latexl@section\l@section
192 | % \def\l@section#1#2{\begingroup\let\numberline\@gobble\latexl@section{#1}{#2}\endgroup}
193 | % \makeatother
194 | 
195 | \makeatletter
196 | \let\latexl@chapter\l@chapter
197 | \def\l@chapter#1#2{\begingroup\let\numberline\@gobble\latexl@chapter{#1}{#2}\endgroup}
198 | \makeatother
199 | 
200 | 
201 | $if(toc)$
202 | {
203 | \hypersetup{linkcolor=black}
204 | % \setcounter{tocdepth}{$toc-depth$}
205 | \tableofcontents
206 | }
207 | $endif$
208 | 
209 | $body$
210 | 
211 | $if(natbib)$
212 | $if(biblio-files)$
213 | $if(biblio-title)$
214 | $if(book-class)$
215 | \renewcommand\bibname{$biblio-title$}
216 | $else$
217 | \renewcommand\refname{$biblio-title$}
218 | $endif$
219 | $endif$
220 | \bibliography{$biblio-files$}
221 | 
222 | $endif$
223 | $endif$
224 | $if(biblatex)$
225 | \printbibliography$if(biblio-title)$[title=$biblio-title$]$endif$
226 | 
227 | $endif$
228 | $for(include-after)$
229 | $include-after$
230 | 
231 | $endfor$
232 | \end{document}


--------------------------------------------------------------------------------
/French/titre.txt:
--------------------------------------------------------------------------------
 1 | ---
 2 | title: SmartPoop 1.0
 3 | subtitle: L'histoire d'une IA qui se lâche
 4 | author:
 5 | - "Lê Nguyên Hoang"
 6 | - "Tristan Le Magoarou"
 7 | rights:  Creative Commons Attribution 4.0
 8 | language: FR
 9 | documentclass: book
10 | ...


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | <img src="SmartPoop.jpg" alt="SmartPoop.jpg" width="350" align="right" />
 2 | 
 3 | # SmartPoop 1.0
 4 | 
 5 | ## This AI went where no human will ever go
 6 | 
 7 | This github hosts the science-fiction book *SmartPoop 1.0*, under [licence CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
 8 | The book is currently available in two languages.
 9 | 
10 | The [English version](English/README.md) is available in [paperback](https://www.amazon.com/dp/B09MF889SM), on [github](English/1-gold.md), in [epub](English/SmartPoop.epub) and in [pdf](English/SmartPoop.pdf).
11 | 
12 | La [version française](French/README.md) est disponible en [version papier](https://www.amazon.fr/dp/B09MDF78ZY), sur [github](French/1-L'or.md), en [epub](French/SmartPoop-fr.epub) et en [pdf](French/SmartPoop-fr.pdf).
13 | 
14 | We are of course welcoming any contributors willing to translate the book in any other language.
15 | 


--------------------------------------------------------------------------------
/SmartPoop.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/lenhoanglnh/SmartPoop/6b2772f5fb4a525959b6e1a3d554e2cd4e79f407/SmartPoop.jpg


--------------------------------------------------------------------------------
/chapter_ideas.md:
--------------------------------------------------------------------------------
 1 | Here are some ideas for future chapters.
 2 | These chapters will not be added to the released [SmartPoop 1.0](README.md) in December 2021.
 3 | The goal is rather to add them for a future SmartPoop 2.0.
 4 | 
 5 | - Human trafficking (between Chapters 5 and 6 of SmartPoop 1.0) : SmartPoop is used by prostitution and pedophilia organizations, to certify the "quality" of prostitutes against sexually transmitted diseases. When SmartPoop is made aware of the case, Katia decides to do nothing, because she doesn't see what can be done. But when Apple threatens to remove SmartPoop from the Apple Store, Katia realizes that SmartPoop is then complicit in human trafficking, which is severely punishable by law.
 6 | 
 7 | - Horoscopes and pseudo-medicine (between Chapters 6 and 7 of SmartPoop 1.0): To enhance the effectiveness of targeted ads, SmartPoop sends highly personalized messages to its users in normal times, with the appearance of a horoscope. The ads reuse what SmartPoop tells its users to be even more effective.
 8 | 
 9 | - Employees' revolt (between Chapters 7 and 8 of SmartPoop 1.0): Tell how employees organized themselves to hold SmartPoop hostage, especially how they bypassed the company's communication channels to avoid being monitored.
10 | 
11 | - Director's lawsuit (between Chapters 7 and 8 of SmartPoop 1.0): The director who led the charge to fire and sue Katia is threatened by a lawsuit for hate speech and warmongering complicity. This made him flip his position, to avoid jail.
12 | 
13 | - Backdoor attacks (between Chapters 5 and 6 of SmartPoop 1.0): A trainer uses a a backdoor attack. Whenever its trainee loses a game, the trainer puts some specific molecules in the trainee's drink, which gives him bad SmartPoop diagnosis.
14 | 
15 | 


--------------------------------------------------------------------------------
/compile_all.sh:
--------------------------------------------------------------------------------
1 | cd English
2 | sh compile.sh
3 | echo English version compiled!
4 | cd ../French
5 | sh compile.sh
6 | echo French version compiled!
7 | cd ..
8 | 


--------------------------------------------------------------------------------