请输入您要查询的百科知识：

　

词条 Explainable artificial intelligence
释义

  1. Goals

  2. History and methods

  3. Regulation

  4. Sectors

  5. References

  6. External links

Explainable AI (XAI), Interpretable AI, or Transparent AI refer to techniques in artificial intelligence (AI) which can be trusted and easily understood by humans. It contrasts with the concept of the "black box" in machine learning where even their designers cannot explain why the AI arrived at a specific decision.[1] XAI can be used to implement a social right to explanation.[2] Some claim that transparency rarely comes for free and that there are often tradeoffs between how "smart" an AI is and how transparent it is; these tradeoffs are expected to grow larger as AI systems increase in internal complexity. The technical challenge of explaining AI decisions is sometimes known as the interpretability problem.[3] Another consideration is info-besity (overload of information), thus, full transparency may not be always possible or even required. The amount of information presented should vary based on the stakeholder interacting with the intelligent system [4]

AI systems optimize behavior to satisfy a mathematically-specified goal system chosen by the system designers, such as the command "maximize accuracy of assessing how positive film reviews are in the test dataset". The AI may learn useful general rules from the test-set, such as "reviews containing the word 'horrible'" are likely to be negative". However, it may also learn inappropriate rules, such as "reviews containing 'Daniel Day-Lewis' are usually positive"; such rules may be undesirable if they are deemed likely to fail to generalize outside the test set, or if people consider the rule to be "cheating" or "unfair". A human can audit rules in an XAI to get an idea how likely the system is to generalize to future real-world data outside the test-set.[3]

Goals

Cooperation between agents, in this case algorithms and humans, depends on trust. If humans are to accept algorithmic prescriptions, they need to trust them. Incompleteness in formalization of trust criteria is a barrier to straightforward optimization approaches. For that reason, interpretability and explainability are posited as intermediate goals for checking other criteria.[5]

AI systems sometimes learn undesirable tricks that do an optimal job of satisfying explicit pre-programmed goals on the training data, but that do not reflect the complicated implicit desires of the human system designers. For example, a 2017 system tasked with image recognition learned to "cheat" by looking for a copyright tag that happened to be associated with horse pictures, rather than learning how to tell if a horse was actually pictured.[1] In another 2017 system, a supervised learning AI tasked with grasping items in a virtual world learned to cheat by placing its manipulator between the object and the viewer in a way such that it falsely appeared to be grasping the object.[6][7]

One transparency project, the DARPA XAI program, aims to produce "glass box" models that are explainable to a "human-in-the-loop", without greatly sacrificing AI performance. Human users should be able to understand the AI's cognition (both in real-time and after the fact), and should be able to determine when to trust the AI and when the AI should be distrusted.[8][9] Other applications of XAI are knowledge extraction from black-box models and model comparisons.[10]

History and methods

Mycin, a research prototype that could explain which of its hand-coded rules contributed to a diagnosis in a specific case, was developed in the early 1970s.[11][12]

By the 1990s researchers also began studying whether it is possible to meaningfully extract the non-hand-coded rules being generated by opaque trained neural networks.[13] Researchers in clinical expert systems creating neural network-powered decision support for clinicians have sought to develop dynamic explanations that allow these technologies to be more trusted and trustworthy in practice.[2] In the 2010s public concerns about racial and other bias in the use of AI for criminal sentencing decisions and findings of creditworthiness may have led to increased demand for transparent artificial intelligence.[1] As a result, many academics and organizations are developing tools to help detect bias in their systems.[14]

Modern complex AI techniques, such as deep learning and genetic algorithms are naturally opaque.[15] There has been a development of many new methods to make new models more explainable and interpretable.[16][17][18] This includes many methods, such as Layerwise relevance propagation (LRP), a technique for determining which features in a particular input vector contribute most strongly to a neural network's output.[19][20][21] In addition, there has been work on decision trees and Bayesian networks, which are more transparent to inspection.[22] In 2018 an interdisciplinary conference called FAT* (Fairness, Accountability, and Transparency) was established to study transparency and explainability in the context of socio-technical systems, many of which include artificial intelligence.[23][24]

Regulation

As regulators, official bodies and general users come to depend on AI-based dynamic systems, clearer accountability will be required for decision making processes to ensure trust and transparency. Evidence of this requirement gaining more momentum can be seen with the launch of the first global conference exclusively dedicated to this emerging discipline, the International Joint Conference on Artificial Intelligence: Workshop on Explainable Artificial Intelligence (XAI).[25]

European Union introduced a right to explanation in General Data Protection Right (GDPR) as an attempt to deal with the potential problems stemming from the rising importance of algorithms. The implementation of the regulation began in 2018. However, the right to explanation in GDPR covers only the local aspect of interpretability, offering explanation in the form of collection of features that have contributed to a decision for particular subject.[5] In the United States, insurance companies are required to be able to explain their rate and coverage decisions.[26]

Sectors

XAI has been researched in many sectors, including:

  • Neural Network Tank imaging[27]
  • Antenna design (evolved antenna)[28]
  • Algorithmic trading (high-frequency trading)[29]
  • Medical diagnoses[30][31]
  • Autonomous vehicles[32][33]

References

1. ^1 2 {{cite news|last1=Sample|first1=Ian|title=Computer says no: why making AIs fair, accountable and transparent is crucial|url=https://www.theguardian.com/science/2017/nov/05/computer-says-no-why-making-ais-fair-accountable-and-transparent-is-crucial|accessdate=30 January 2018|work=the Guardian |date=5 November 2017|language=en}}
2. ^1 {{Cite journal|last=Edwards|first=Lilian|last2=Veale|first2=Michael|date=2017|title=Slave to the Algorithm? Why a 'Right to an Explanation' Is Probably Not the Remedy You Are Looking For|journal=Duke Law and Technology Review|volume=|pages=|ssrn=2972855}}
3. ^1 {{cite news|title=How AI detectives are cracking open the black box of deep learning |url=http://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning|accessdate=30 January 2018|work=Science {{!}} AAAS|date=5 July 2017|language=en}}.
4. ^{{Cite journal|last=Theodorou|first=Andreas|last2=Wortham|first2=Robert R.|last3=Bryson|first3=Joanna J.|date=2017|title=Designing and implementing transparency for real time inspection of autonomous robots|journal=Connection Science|volume=29|issue=3|pages=230–241|doi=10.1080/09540091.2017.1310182|url=http://opus.bath.ac.uk/55250/1/TheodorouDesigningAndImplementingTransparency.pdf|bibcode=2017ConSc..29..230T}}.
5. ^1 {{cite conference |url=http://docs.mipro-proceedings.com/dsdc/dsdc_11_4754.pdf |title=Explainable Artificial Intelligence: A Survey |last1=Dosilovic |first1=Filip |last2=Brcic |first2=Mario |last3=Hlupic |first3=Nikica |date=2018-05-25|book-title=MIPRO 2018 - 41st International Convention Proceedings |pages=210–215 |location=Opatija, Croatia |conference=MIPRO 2018 |doi=10.23919/MIPRO.2018.8400040 }}
6. ^{{cite news|title=DeepMind Has Simple Tests That Might Prevent Elon Musk’s AI Apocalypse|url=https://www.bloomberg.com/news/articles/2017-12-11/deepmind-has-simple-tests-that-might-prevent-elon-musk-s-ai-apocalypse|accessdate=30 January 2018|work=Bloomberg.com|date=11 December 2017|language=en}}
7. ^{{cite news|title=Learning from Human Preferences|url=https://blog.openai.com/deep-reinforcement-learning-from-human-preferences/|accessdate=30 January 2018|work=OpenAI Blog|date=13 June 2017}}
8. ^{{cite web|title=Explainable Artificial Intelligence (XAI)|url=https://www.darpa.mil/program/explainable-artificial-intelligence|website=DARPA|publisher=DARPA|accessdate=17 July 2017}}
9. ^{{cite arxiv|last=Holzinger|first=Andreas|last2=Plass|first2=Markus|last3=Holzinger|first3=Katharina|last4=Crisan|first4=Gloria Cerasela|last5=Pintea|first5=Camelia-M.|last6=Palade|first6=Vasile|date=2017-08-03|title=A glass-box interactive machine learning approach for solving NP-hard problems with the human-in-the-loop|eprint=1708.01104|class=cs.AI}}
10. ^{{cite journal|last=Biecek|first=Przemyslaw|title= DALEX: explainers for complex predictive models|journal=Journal of Machine Learning Research|volume=19|pages=1–5|arxiv=1806.08915|date=23 June 2018|bibcode=2018arXiv180608915B}}
11. ^Van Lent, M., Fisher, W., & Mancuso, M. (2004, July). An explainable artificial intelligence system for small-unit tactical behavior. In Proceedings of the National Conference on Artificial Intelligence (pp. 900-907). Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.
12. ^Fagan, L. M., Shortliffe, E. H., & Buchanan, B. G. (1980). Computer-based medical decision making: from MYCIN to VM. Automedica, 3(2), 97-108.
13. ^{{Cite journal|last=Tickle|first=A. B.|last2=Andrews|first2=R.|last3=Golea|first3=M.|last4=Diederich|first4=J.|date=November 1998|title=The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks|journal=IEEE Transactions on Neural Networks|volume=9|issue=6|pages=1057–1068|doi=10.1109/72.728352|pmid=18255792|issn=1045-9227}}
14. ^{{cite news|url=https://www.bloomberg.com/news/articles/2018-06-13/accenture-unveils-tool-to-help-companies-insure-their-ai-is-fair|title=Accenture Unveils Tool to Help Companies Insure Their AI Is Fair|date=June 2018|work=Bloomberg.com|accessdate=5 August 2018|language=en}}
15. ^{{cite news|last1=Mukherjee|first1=Siddhartha|title=A.I. Versus M.D.|url=https://www.newyorker.com/magazine/2017/04/03/ai-versus-md|accessdate=30 January 2018|work=The New Yorker|date=27 March 2017}}
16. ^{{cite arxiv|last=Lipton|first=Zachary C.|date=2016-06-10|title=The Mythos of Model Interpretability|eprint=1606.03490|class=cs.LG}}
17. ^{{cite arxiv|last=Murdoch|first=W. James|last2=Singh|first2=Chandan|last3=Kumbier|first3=Karl|last4=Abbasi-Asl|first4=Reza|last5=Yu|first5=Bin|date=2019-01-14|title=Interpretable machine learning: definitions, methods, and applications|eprint=1901.04592|class=stat.ML}}
18. ^{{cite arxiv|last=Doshi-Velez|first=Finale|last2=Kim|first2=Been|date=2017-02-27|title=Towards A Rigorous Science of Interpretable Machine Learning|eprint=1702.08608|class=stat.ML}}
19. ^{{cite web|url=https://dshieble.github.io/2017-04-16-deep-taylor-lrp/|title=Understanding Neural Networks with Layerwise Relevance Propagation and Deep Taylor Series|last=Shiebler|first=Dan|date=2017-04-16|website=Dan Shiebler|access-date=2017-11-03}}
20. ^{{cite journal|last=Bach|first=Sebastian|last2=Binder|first2=Alexander|last3=Montavon|first3=Grégoire|last4=Klauschen|first4=Frederick|last5=Müller|first5=Klaus-Robert|last6=Samek|first6=Wojciech|date=2015-07-10|editor-last=Suarez|editor-first=Oscar Deniz|title=On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation|journal=PLOS One|volume=10|issue=7|page=e0130140|bibcode=2015PLoSO..1030140B|doi=10.1371/journal.pone.0130140|issn=1932-6203|pmc=4498753|pmid=26161953}}
21. ^{{cite news|url=https://www.theguardian.com/science/2017/nov/05/computer-says-no-why-making-ais-fair-accountable-and-transparent-is-crucial|title=Computer says no: why making AIs fair, accountable and transparent is crucial|last1=Sample|first1=Ian|date=5 November 2017|work=the Guardian|accessdate=5 August 2018|language=en}}
22. ^Bostrom, N., & Yudkowsky, E. (2014). The ethics of artificial intelligence. The Cambridge handbook of artificial intelligence, 316-334.
23. ^{{cite web | url=https://fatconference.org/ | title=FAT* Conference }}
24. ^{{cite news |title=Computer programs recognise white men better than black women |url=https://www.economist.com/science-and-technology/2018/02/15/computer-programs-recognise-white-men-better-than-black-women |accessdate=5 August 2018 |work=The Economist |date=2018 |language=en}}
25. ^{{cite web|title=IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI)|url=http://www.intelligentrobots.org/files/IJCAI2017/IJCAI-17_XAI_WS_Proceedings.pdf|website=Earthlink|publisher=IJCAI |accessdate=17 July 2017}}
26. ^{{cite news |last1=Kahn |first1=Jeremy |title=Artificial Intelligence Has Some Explaining to Do |url=https://www.bloomberg.com/news/articles/2018-12-12/artificial-intelligence-has-some-explaining-to-do |accessdate=17 December 2018 |work=Bloomberg Businessweek |date=12 December 2018}}
27. ^{{cite web|title=Neual Network Tank image|url=https://neil.fraser.name/writing/tank/|website=Neil Fraser|publisher=Neil Fraser|accessdate=17 July 2017}}
28. ^{{cite web|title=NASA 'Evolutionary' software automatically designs antenna|url=https://www.nasa.gov/mission_pages/st-5/main/04-55AR.html|website=NASA|publisher=NASA|accessdate=17 July 2017}}
29. ^{{cite web|title=The Flash Crash: The Impact of High Frequency Trading on an Electronic Market|url=http://www.cftc.gov/idc/groups/public/@economicanalysis/documents/file/oce_flashcrash0314.pdf|website=CFTC|publisher=CFTC|accessdate=17 July 2017}}
30. ^{{Cite journal|title=Can machine-learning improve cardiovascular risk prediction using routine clinical data?|journal=PLOS One|volume=12|issue=4|pages=e0174944|doi=10.1371/journal.pone.0174944|pmid=28376093|pmc=5380334|year=2017|last1=Weng|first1=Stephen F|last2=Reps|first2=Jenna|last3=Kai|first3=Joe|last4=Garibaldi|first4=Jonathan M|last5=Qureshi|first5=Nadeem|bibcode=2017PLoSO..1274944W}}
31. ^{{cite arxiv|last=Holzinger|first=Andreas|last2=Biemann|first2=Chris|last3=Pattichis|first3=Constantinos S.|last4=Kell|first4=Douglas B.|date=2017-12-28|title=What do we need to build explainable AI systems for the medical domain?|eprint=1712.09923|class=cs.AI}}
32. ^{{cite news|title=Tesla says it has 'no way of knowing' if autopilot was used in fatal Chinese crash|url=https://www.theguardian.com/technology/2016/sep/14/tesla-fatal-crash-china-autopilot-gao-yaning|newspaper=Guardian|publisher=Guardian|accessdate=17 July 2017|date=2016-09-14}}
33. ^{{cite news|title=Joshua Brown, Who Died in Self-Driving Accident, Tested Limits of His Tesla|url=https://www.nytimes.com/2016/07/02/business/joshua-brown-technology-enthusiast-tested-the-limits-of-his-tesla.html|newspaper=New York Times|publisher=New York Times|accessdate=17 July 2017|date=July 2016|last1=Abrams|first1=Rachel|last2=Kurtz|first2=Annalyn}}

External links

  • {{ cite web | url=https://fatconference.org/ | title=FAT Conference on Fairness, Accountability, and Transparency }}
  • {{cite web | url=https://www.fatml.org/ | title=FATML Workshop on Fairness, Accountability, and Transparency in Machine Learning }}
  • {{cite web | title='Explainable Artificial Intelligence': Cracking open the black box of AI | website=Computerworld | date=2017-11-02 | url=https://www.computerworld.com.au/article/617359/explainable-artificial-intelligence-cracking-open-black-box-ai/ | ref={{sfnref | Computerworld | 2017}} | access-date=2017-11-02}}
  • {{cite journal | last=Park | first=Dong Huk | last2=Hendricks | first2=Lisa Anne | last3=Akata | first3=Zeynep | last4=Schiele | first4=Bernt | last5=Darrell | first5=Trevor | last6=Rohrbach | first6=Marcus | title=Attentive Explanations: Justifying Decisions and Pointing to the Evidence | date=2016-12-14 | arxiv=1612.04757 | ref=harv | bibcode=2016arXiv161204757H }}
  • {{cite web | title=Explainable AI: Making machines understandable for humans | website=Explainable AI: Making machines understandable for humans | url=https://explainableai.com/ | ref={{sfnref | Explainable AI: Making machines understandable for humans}} | access-date=2017-11-02}}
  • {{cite web | title=End-to-End Deep Learning for Self-Driving Cars | website=Parallel Forall | date=2016-08-17 | url=https://devblogs.nvidia.com/parallelforall/deep-learning-self-driving-cars/ | ref={{sfnref | Parallel Forall | 2016}} | access-date=2017-11-02}}
  • {{cite web | title=Explaining How End-to-End Deep Learning Steers a Self-Driving Car | website=Parallel Forall | date=2017-05-23 | url=https://devblogs.nvidia.com/parallelforall/explaining-deep-learning-self-driving-car/ | ref={{sfnref | Parallel Forall | 2017}} | access-date=2017-11-02}}
  • {{cite web | title=New isn't on its way. We're applying it right now. | website=Accenture | date=2016-10-25 | url=https://www.accenture.com/us-en/blogs/blogs-nicola-morini-bianzino-responsible-ai/ | ref={{sfnref | Accenture | 2016}} | access-date=2017-11-02}}
  • {{cite web | last=Knight | first=Will | title=DARPA is funding projects that will try to open up AI's black boxes | website=MIT Technology Review | date=2017-03-14 | url=https://www.technologyreview.com/s/603795/the-us-military-wants-its-autonomous-machines-to-explain-themselves/ | ref=harv | access-date=2017-11-02}}
  • {{cite journal | last=Alvarez-Melis | first=David | last2=Jaakkola | first2=Tommi S. | url=https://people.csail.mit.edu/tommi/papers/AlvJaa_EMNLP2017.pdf|title=A causal framework for explaining the predictions of black-box sequence-to-sequence models | volume=1707 | pages=arXiv:1707.01943 | date=2017-07-06 | arxiv=1707.01943 | ref=harv | access-date=2017-11-09| bibcode=2017arXiv170701943A }}
  • {{cite web | title=Similarity Cracks the Code Of Explainable AI | website=simMachines | date=2017-10-12 | url=http://simmachines.com/similarity-cracks-code-explainable-ai/ | ref={{sfnref | simMachines | 2017}} | access-date=2018-02-02}}
  • {{cite journal | last=Bojarski | first=Mariusz | last2=Yeres | first2=Philip | last3=Choromanska | first3=Anna | last4=Choromanski | first4=Krzysztof | last5=Firner | first5=Bernhard | last6=Jackel | first6=Lawrence | last7=Muller | first7=Urs | title=Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car | volume=1704 | pages=arXiv:1704.07911 | date=2017-04-25 | arxiv=1704.07911 | ref=harv | bibcode=2017arXiv170407911B }}
{{DEFAULTSORT:Explainable AI}}

3 : Self-driving cars|Artificial intelligence|Accountability
随便看

　

开放百科全书收录14589846条英语、德语、日语等多语种百科知识，基本涵盖了大多数领域的百科知识，是一部内容自由、开放的电子版国际百科全书。

　

Copyright © 2023 OENC.NET All Rights Reserved
京ICP备2021023879号 更新时间：2024/9/20 9:45:31