Il "lato oscuro" del deep learning

Simone Scardapane (Sapienza)

GDG DevFest Milano (6 Ottobre 2018)

Qualcosa su di me

Assegnista di ricerca (Sapienza), con un forte interesse nella divulgazione e nella promozione del machine learning.

La classica slide sul deep learning

Deep learning nei media

IBM speech recognition is on the verge of super-human accuracy [Business Insider, 2017]

Are Computers Already Smarter Than Humans?
[Time, 2017]

Artificial Intelligence Beats 'Most Complex Game Devised by Humans' [LiveScience, 2016]

Intel is paying more than $400 million to buy deep-learning startup Nervana Systems [RECODE, 2017]

I know there's a proverb which that says 'To err is human,' but a human error is nothing to what a computer can do if it tries.

--- Agatha Christie

People worry that computers will get too smart and take over the world, but the real problem is that they're too stupid and they've already taken over the world.

--- Pedro Domingos

Sui limiti del deep learning

Il DL non è magico - è uno strumento enormemente potente per trovare regolarità nei dati rispetto ad un obiettivo prestabilito.

Corollario #1: Il DL è intelligente tanto quanto i dati che gli vengono forniti.

Corollario #2: Il DL è intelligente tanto quanto l'obiettivo che ottimizziamo.

1) Quando i dati sbagliano (e non ce ne accorgiamo)

Bias e discriminazione

Word embedding ed NLP

Esempio (Python)

Caricare un word embedding:
import gensim
model = gensim.models.KeyedVectors\
Convertire una parola:
array([ 0.32617188,  0.13085938,  0.03466797, -0.08300781,  0.08984375,
       -0.04125977, -0.19824219,  0.00689697,  0.14355469,  0.0019455 ,
        0.02880859, -0.25      , -0.08398438, -0.15136719, -0.10205078,
        0.04077148, -0.09765625,  0.05932617,  0.02978516, -0.10058594,
       -0.13085938,  0.001297  ,  0.02612305, -0.27148438,  0.06396484,

Algebra sui word embedding

Possiamo lavorare algebricamente sui word embedding!
	- model.wv['man'] 
	+ model.wv['woman']
[('queen', 0.7118191719055176),
 ('monarch', 0.6189674139022827),
 ('princess', 0.5902431011199951),
 ('prince', 0.5377321243286133),
 ('kings', 0.5236844420433044),
 ('queens', 0.5181134343147278),
 ('monarchy', 0.5087411403656006),
 ('throne', 0.5005807280540466),
 ('royal', 0.493820458650589),
 ('ruler', 0.49092763662338257)]

Bias nei word embedding

	- model.wv['man'] 
	+ model.wv['woman']
[('nurse', 0.7127888798713684),
 ('doctors', 0.6593285799026489),
 ('physician', 0.6408007144927979),
 ('pediatrician', 0.6093444228172302),
 ('midwife', 0.5823134183883667),
 ('pharmacist', 0.5700446963310242),
 ('oncologist', 0.5668959617614746),
 ('dermatologist', 0.5606269836425781),
 ('dentist', 0.5562461018562317),
 ('psychiatrist', 0.5473387241363525)]

Gli embedding sono altamente sessisti!

Bolukbasi, T., Chang, K.W., Zou, J., Saligrama, V. and Kalai, A., 2016. Quantifying and reducing stereotypes in word embeddings. arXiv preprint arXiv:1606.06121.

Centinaia di articoli sono stati pubblicati prima che questo diventasse di dominio pubblico:

Bolukbasi, T., Chang, K.W., Zou, J.Y., Saligrama, V. and Kalai, A.T., 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in Neural Information Processing Systems (pp. 4349-4357).

Questo perché il bias di genere probabilmente porta ad un aumento dell'accuratezza media.

Esistono tecniche per effettuare debiasing, ma richiedono di identificare quali possono essere i bias:

Bolukbasi, T., Chang, K.W., Zou, J.Y., Saligrama, V. and Kalai, A.T., 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in Neural Information Processing Systems (pp. 4349-4357).

AI Fairness 360: Raise AI right! [IBM Blog]

Il razzismo è pessima pubblicità!

The rise of the racist robots [New Statesman, 2016]

Oltre il problema economico

[an investigation] found that the proprietary algorithms widely used by judges to help determine the risk of reoffending are almost twice as likely to mistakenly flag black defendants than white defendants [There is a blind spot in AI research]

Oltre il problema economico (2)

AI can expose stereotypes inherent in everyday language. It can reveal uncomfortable truths [...] Making social progress and holding ourselves to account is more difficult without such hard evidence, even when it only confirms our suspicions. [Do algorithms reveal sexual orientation or just expose our stereotypes? (Medium, 2018)]

Discriminazione & fairness

Attacking discrimination with smarter machine learning [Google Research Blog]

2) Quando l'accuratezza media non è tutto

Adversarial attacks (e non solo)

Possiamo ingannare una rete neurale?

Fooling neural networks

Breaking linear classifiers on Imagenet (Andrej Karpathy blog)

Perturbazioni universali!

Moosavi-Dezfooli, S.M., Fawzi, A., Fawzi, O. and Frossard, P., 2016. Universal adversarial perturbations. arXiv preprint arXiv:1610.08401.

One-pixel perturbation

Su, J., Vargas, D.V. and Kouichi, S., 2017. One pixel attack for fooling deep neural networks. arXiv preprint arXiv:1710.08864.

Adversarial patches

Adversarial reprogramming

Elsayed, G.F., Goodfellow, I. and Sohl-Dickstein, J., 2018. Adversarial Reprogramming of Neural Networks. arXiv preprint arXiv:1806.11146..

Un attacco più sofisticato!

3) Quando essere anonimi non ci basta

Privacy breaches

Dati anonimi?

De Montjoye, Y.A., Radaelli, L. and Singh, V.K., 2015. Unique in the shopping mall: On the reidentifiability of credit card metadata. Science, 347(6221), pp.536-539.

Le reti neurali hanno sufficienti parametri per memorizzare tutto il training set: Zhang, C., Bengio, S., Hardt, M., Recht, B. and Vinyals, O., 2016. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530.

Avendo accesso ad un classificatore black-box, possiamo capire se un determinato esempio ha fatto parte del training set?

La tecnica si chiama shadow training:

Shokri, R., Stronati, M., Song, C. and Shmatikov, V., 2017, May. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP),  (pp. 3-18). IEEE.

Privacy in ambienti distribuiti

Hitaj, B., Ateniese, G. and Perez-Cruz, F., 2017. Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning. arXiv preprint arXiv:1702.07464.

4) Quando ci dimentichiamo di tutto il resto

Hidden technical debt

Il DL è solo un piccolo componente!

Hidden Technical Debt in Machine Learning Systems (NIPS 2015)

Machine learning offers a fantastically powerful toolkit for building useful complex prediction systems quickly. ... it is dangerous to think of these quick wins as coming for free. ... it is common to incur massive ongoing maintenance costs in real-world ML systems. [Risk factors include] boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns.

Hidden Technical Debt in Machine Learning Systems (NIPS 2015)

Il debito non è solo tecnico... ma anche sociale!

Recent years have brought extraordinary
advances in the technical domains of AI. Alongside such efforts, designers and researchers from a range of disciplines need to conduct what we call social-systems analyses of AI. They need to assess the impact of technologies on their social, cultural and political settings.

--- There is a blind spot in AI research, Nature, 2016

Intelligenza e senso comune

Lake, B.M., Ullman, T.D., Tenenbaum, J.B. and Gershman, S.J., 2017. Building machines that learn and think like people. Behavioral and Brain Sciences, 40.

Join us!

The future will be intensely data-driven. Creating a dynamic hub that brings together professionals, industries, and academics will be essential to achieve this vision.