Il "lato oscuro" del deep learning
Simone Scardapane (Sapienza)
Converge Conference - Milano (19 Gennaio 2018)
Qualcosa su di me
Assegnista di ricerca (Sapienza), con un forte interesse nella divulgazione e nella promozione del machine learning.
La classica slide sul deep learning
Deep learning nei media
IBM speech recognition is on the verge of super-human accuracy [Business Insider, 2017]
Are Computers Already Smarter Than Humans?
[Time, 2017]
Artificial Intelligence Beats 'Most Complex Game Devised by Humans' [LiveScience, 2016]
Intel is paying more than $400 million to buy deep-learning startup Nervana Systems [RECODE, 2017]
I know there's a proverb which that says 'To err is human,' but a human error is nothing to what a computer can do if it tries.
--- Agatha Christie
People worry that computers will get too smart and take over the world, but the real problem is that they're too stupid and they've already taken over the world.
--- Pedro Domingos
Sui limiti del deep learning
Il DL non è magico - è uno strumento enormemente potente per trovare regolarità nei dati rispetto ad un obiettivo prestabilito.
Corollario #1: Il DL è intelligente tanto quanto i dati che gli vengono forniti.
Corollario #2: Il DL è intelligente tanto quanto l'obiettivo che ottimizziamo.
Qualcosa di cui preoccuparsi #1
Bias e discriminazione
Word embedding ed NLP
Esempio (Python)
Caricare un word embedding:import gensim
model = gensim.models.KeyedVectors\
.load_word2vec_format('GoogleNews-vectors-negative300.bin')
model.wv['man']
array([ 0.32617188, 0.13085938, 0.03466797, -0.08300781, 0.08984375,
-0.04125977, -0.19824219, 0.00689697, 0.14355469, 0.0019455 ,
0.02880859, -0.25 , -0.08398438, -0.15136719, -0.10205078,
0.04077148, -0.09765625, 0.05932617, 0.02978516, -0.10058594,
-0.13085938, 0.001297 , 0.02612305, -0.27148438, 0.06396484,
Algebra sui word embedding
Possiamo lavorare algebricamente sui word embedding!model.most_similar_by_vector(
model.wv['king']
- model.wv['man']
+ model.wv['woman']
)
[('queen', 0.7118191719055176),
('monarch', 0.6189674139022827),
('princess', 0.5902431011199951),
('prince', 0.5377321243286133),
('kings', 0.5236844420433044),
('queens', 0.5181134343147278),
('monarchy', 0.5087411403656006),
('throne', 0.5005807280540466),
('royal', 0.493820458650589),
('ruler', 0.49092763662338257)]
Bias nei word embedding
Riproviamo...model.most_similar_by_vector(
model.wv['doctor']
- model.wv['man']
+ model.wv['woman']
)
[('nurse', 0.7127888798713684),
('doctors', 0.6593285799026489),
('physician', 0.6408007144927979),
('pediatrician', 0.6093444228172302),
('midwife', 0.5823134183883667),
('pharmacist', 0.5700446963310242),
('oncologist', 0.5668959617614746),
('dermatologist', 0.5606269836425781),
('dentist', 0.5562461018562317),
('psychiatrist', 0.5473387241363525)]
Gli embedding sono altamente sessisti!
Bolukbasi, T., Chang, K.W., Zou, J., Saligrama, V. and Kalai, A., 2016. Quantifying and reducing stereotypes in word embeddings. arXiv preprint arXiv:1606.06121.
Centinaia di articoli sono stati pubblicati prima che questo diventasse di dominio pubblico:
Bolukbasi, T., Chang, K.W., Zou, J.Y., Saligrama, V. and Kalai, A.T., 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in Neural Information Processing Systems (pp. 4349-4357).
Questo perché il bias di genere probabilmente porta ad un aumento dell'accuratezza media.
Esistono tecniche per effettuare debiasing, ma richiedono di identificare quali possono essere i bias:
Bolukbasi, T., Chang, K.W., Zou, J.Y., Saligrama, V. and Kalai, A.T., 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in Neural Information Processing Systems (pp. 4349-4357).
Recent years have brought extraordinary
advances in the technical domains of AI. Alongside such efforts, designers and researchers from a range of disciplines need to conduct what we call social-systems analyses of AI. They need to assess the impact of technologies on their social, cultural and political settings.--- There is a blind spot in AI research, Nature, 2016
Il razzismo è pessima pubblicità!
The rise of the racist robots [New Statesman, 2016]
Oltre il problema economico
[an investigation] found that the proprietary algorithms widely used by judges to help determine the risk of reoffending are almost twice as likely to mistakenly flag black defendants than white defendants [There is a blind spot in AI research]
Oltre il problema economico (2)
AI can expose stereotypes inherent in everyday language. It can reveal uncomfortable truths [...] Making social progress and holding ourselves to account is more difficult without such hard evidence, even when it only confirms our suspicions. [Do algorithms reveal sexual orientation or just expose our stereotypes? (Medium, 2018)]
Discrimination and fairness
Attacking discrimination with smarter machine learning [Google Research Blog]
Qualcosa di cui preoccuparsi #2
Adversarial attacks
Possiamo ingannare una rete neurale?
Fooling neural networks
Breaking linear classifiers on Imagenet (Andrej Karpathy blog)
Perturbazioni universali!
Moosavi-Dezfooli, S.M., Fawzi, A., Fawzi, O. and Frossard, P., 2016. Universal adversarial perturbations. arXiv preprint arXiv:1610.08401.
One-pixel perturbation
Su, J., Vargas, D.V. and Kouichi, S., 2017. One pixel attack for fooling deep neural networks. arXiv preprint arXiv:1710.08864..
Adversarial patches
Qualcosa di cui preoccuparsi #3
Privacy
Dati anonimi?
De Montjoye, Y.A., Radaelli, L. and Singh, V.K., 2015. Unique in the shopping mall: On the reidentifiability of credit card metadata. Science, 347(6221), pp.536-539.
Zhang, C., Bengio, S., Hardt, M., Recht, B. and Vinyals, O., 2016. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530.
Avendo accesso ad un classificatore black-box, possiamo capire se un determinato esempio ha fatto parte del training set?
La tecnica si chiama shadow training:
Shokri, R., Stronati, M., Song, C. and Shmatikov, V., 2017, May. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), (pp. 3-18). IEEE.
Privacy in ambienti distribuiti
Hitaj, B., Ateniese, G. and Perez-Cruz, F., 2017. Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning. arXiv preprint arXiv:1702.07464.
Qualcosa di cui preoccuparsi #4
Security threats
Qualcosa di cui preoccuparsi #5
Hidden technical debt
Il DL è solo un piccolo componente!
Hidden Technical Debt in Machine Learning Systems (NIPS 2015)
Machine learning offers a fantastically powerful toolkit for building useful complex prediction systems quickly. ... it is dangerous to think of these quick wins as coming for free. ... it is common to incur massive ongoing maintenance costs in real-world ML systems. [Risk factors include] boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns.
Hidden Technical Debt in Machine Learning Systems (NIPS 2015)
Join us!
http://www.iaml.it/member
The future will be intensely data-driven. Creating a dynamic hub that brings together professionals, industries, and academics will be essential to achieve this vision.