<?xml version="1.0" encoding="UTF-8"?>
<mods xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.loc.gov/mods/v3" version="3.1" xsi:schemaLocation="http://www.loc.gov/mods/v3 http://www.loc.gov/standards/mods/v3/mods-3-1.xsd">
  <titleInfo>
    <title>Auditing large language models: a three-layered approach</title>
  </titleInfo>
  <name type="personal">
    <namePart>Jakob Mökander</namePart>
    <role>
      <roleTerm authority="marcrelator" type="text">creator</roleTerm>
    </role>
    <role>
      <roleTerm type="text">author.</roleTerm>
    </role>
  </name>
  <name type="personal">
    <namePart>Jonas Schuett</namePart>
    <role>
      <roleTerm type="text">author.</roleTerm>
    </role>
  </name>
  <name type="personal">
    <namePart>Hannah Rose Kirk</namePart>
    <role>
      <roleTerm type="text">author.</roleTerm>
    </role>
  </name>
  <name type="personal">
    <namePart>Luciano Floridi</namePart>
    <role>
      <roleTerm type="text">author.</roleTerm>
    </role>
  </name>
  <typeOfResource>text</typeOfResource>
  <genre authority="">Journal articles.</genre>
  <originInfo>
    <place>
      <placeTerm type="code" authority="marccountry">xx</placeTerm>
    </place>
    <dateIssued encoding="marc">2023</dateIssued>
    <issuance>monographic</issuance>
  </originInfo>
  <language>
    <languageTerm authority="iso639-2b" type="code">eng</languageTerm>
  </language>
  <physicalDescription>
    <extent>1 online resource (1 article).</extent>
  </physicalDescription>
  <abstract>Abstract Large language models (LLMs) represent a major advance in artificial intelligence (AI) research. However, the widespread use of LLMs is also coupled with significant ethical and social challenges. Previous research has pointed towards auditing as a promising governance mechanism to help ensure that AI systems are designed and deployed in ways that are ethical, legal, and technically robust. However, existing auditing procedures fail to address the governance challenges posed by LLMs, which display emergent capabilities and are adaptable to a wide range of downstream tasks. In this article, we address that gap by outlining a novel blueprint for how to audit LLMs. Specifically, we propose a three-layered approach, whereby governance audits (of technology providers that design and disseminate LLMs), model audits (of LLMs after pre-training but prior to their release), and application audits (of applications based on LLMs) complement and inform each other. We show how audits, when conducted in a structured and coordinated manner on all three levels, can be a feasible and effective mechanism for identifying and managing some of the ethical and social risks posed by LLMs. However, it is important to remain realistic about what auditing can reasonably be expected to achieve. Therefore, we discuss the limitations not only of our three-layered approach but also of the prospect of auditing LLMs at all. Ultimately, this article seeks to expand the methodological toolkit available to technology providers and policymakers who wish to analyse and evaluate LLMs from technical, ethical, and legal perspectives.</abstract>
  <note type="statement of responsibility">Jakob Mökander; Jonas Schuett; Hannah Rose Kirk [and others].</note>
  <note>Publication type: Journal article.</note>
  <note>Linked open-access copy: published version.</note>
  <note>Open access; no subscription required for the linked open-access version. Unrestricted online access star</note>
  <note xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://openalex.org/W4381572755">Creative Commons Attribution (licence family recorded by the source; version not supplied in metadata). https://openalex.org/W4381572755</note>
  <note>In English.</note>
  <subject>
    <topic>large language models</topic>
  </subject>
  <subject>
    <topic>artificial intelligence in auditing</topic>
  </subject>
  <subject>
    <topic>AI governance</topic>
  </subject>
  <subject>
    <topic>ethical risks of AI</topic>
  </subject>
  <subject>
    <topic>AI regulation</topic>
  </subject>
  <subject>
    <topic>B. Sc. Accounting</topic>
  </subject>
  <subject>
    <topic>UMUN open access collection</topic>
  </subject>
  <subject>
    <topic>auditing</topic>
  </subject>
  <relatedItem type="host">
    <titleInfo>
      <title>AI and Ethics</title>
    </titleInfo>
    <identifier type="issn">2730-5953</identifier>
    <part>
      <text>Vol. 4, no. 4, (2023), pages/article 1085-1115</text>
    </part>
  </relatedItem>
  <identifier type="uri">https://link.springer.com/content/pdf/10.1007/s43681-023-00289-2.pdf</identifier>
  <location>
    <url displayLabel="Read open-access publication">https://link.springer.com/content/pdf/10.1007/s43681-023-00289-2.pdf</url>
  </location>
  <accessCondition type="restrictionOnAccess">Open access; no subscription required for the linked open-access version.</accessCondition>
  <accessCondition type="useAndReproduction">Creative Commons Attribution (licence family recorded by the source; version not supplied in metadata).</accessCondition>
  <recordInfo>
    <recordContentSource authority="marcorg">local</recordContentSource>
    <recordCreationDate encoding="marc">261005</recordCreationDate>
    <recordChangeDate encoding="iso8601">20261005174841.0</recordChangeDate>
    <recordIdentifier source="local">UMUN-OA-20261005-ACC-064</recordIdentifier>
    <languageOfCataloging>
      <languageTerm authority="iso639-2b" type="code">eng</languageTerm>
    </languageOfCataloging>
  </recordInfo>
</mods>
