<?xml version="1.0" encoding="UTF-8"?>
<mods xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.loc.gov/mods/v3" version="3.1" xsi:schemaLocation="http://www.loc.gov/mods/v3 http://www.loc.gov/standards/mods/v3/mods-3-1.xsd">
  <titleInfo>
    <nonSort>A </nonSort>
    <title>Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems</title>
  </titleInfo>
  <name type="personal">
    <namePart>Rafael Figueiredo Prudencio</namePart>
    <role>
      <roleTerm authority="marcrelator" type="text">creator</roleTerm>
    </role>
    <role>
      <roleTerm type="text">author.</roleTerm>
    </role>
  </name>
  <name type="personal">
    <namePart>Marcos R. O. A. Maximo</namePart>
    <role>
      <roleTerm type="text">author.</roleTerm>
    </role>
  </name>
  <name type="personal">
    <namePart>Esther Luna Colombini</namePart>
    <role>
      <roleTerm type="text">author.</roleTerm>
    </role>
  </name>
  <typeOfResource>text</typeOfResource>
  <genre authority="">Journal articles.</genre>
  <originInfo>
    <place>
      <placeTerm type="code" authority="marccountry">xx</placeTerm>
    </place>
    <dateIssued encoding="marc">2023</dateIssued>
    <issuance>monographic</issuance>
  </originInfo>
  <language>
    <languageTerm authority="iso639-2b" type="code">eng</languageTerm>
  </language>
  <physicalDescription>
    <internetMediaType>application/pdf</internetMediaType>
    <extent>1 online resource (1 article).</extent>
  </physicalDescription>
  <abstract>With the widespread adoption of deep learning, reinforcement learning (RL) has experienced a dramatic increase in popularity, scaling to previously intractable problems, such as playing complex games from pixel observations, sustaining conversations with humans, and controlling robotic agents. However, there is still a wide range of domains inaccessible to RL due to the high cost and danger of interacting with the environment. Offline RL is a paradigm that learns exclusively from static datasets of previously collected interactions, making it feasible to extract policies from large and diverse training datasets. Effective offline RL algorithms have a much wider range of applications than online RL, being particularly appealing for real-world applications, such as education, healthcare, and robotics. In this work, we contribute with a unifying taxonomy to classify offline RL methods. Furthermore, we provide a comprehensive review of the latest algorithmic breakthroughs in the field using a unified notation as well as a review of existing benchmarks' properties and shortcomings. Additionally, we provide a figure that summarizes the performance of each method and class of methods on different dataset properties, equipping researchers with the tools to decide which type of algorithm is best suited for the problem at hand and identify which classes of algorithms look the most promising. Finally, we provide our perspective on open problems and propose future research directions for this rapidly growing field.</abstract>
  <note type="statement of responsibility">Rafael Figueiredo Prudencio; Marcos R. O. A. Maximo; Esther Luna Colombini.</note>
  <note>Publication type: Journal article.</note>
  <note>Linked open-access copy: published version.</note>
  <note>Open access; no subscription required for the linked open-access version. Unrestricted online access star</note>
  <note xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://openalex.org/W4360584316">Creative Commons Attribution (licence family recorded by the source; version not supplied in metadata). https://openalex.org/W4360584316</note>
  <note>In English.</note>
  <subject>
    <topic>offline reinforcement learning</topic>
  </subject>
  <subject>
    <topic>benchmark dataset</topic>
  </subject>
  <subject>
    <topic>algorithm comparison</topic>
  </subject>
  <subject>
    <topic>B. Sc. Artificial Intelligence</topic>
  </subject>
  <subject>
    <topic>UMUN open access collection</topic>
  </subject>
  <subject>
    <topic>reinforcement learning</topic>
  </subject>
  <relatedItem type="host">
    <titleInfo>
      <title>IEEE Transactions on Neural Networks and Learning Systems</title>
    </titleInfo>
    <identifier type="issn">2162-237X</identifier>
    <part>
      <text>Vol. 35, no. 8, (2023), pages/article 10237-10257</text>
    </part>
  </relatedItem>
  <identifier type="uri">https://ieeexplore.ieee.org/ielx7/5962385/6104215/10078377.pdf</identifier>
  <location>
    <url displayLabel="Read open-access publication">https://ieeexplore.ieee.org/ielx7/5962385/6104215/10078377.pdf</url>
  </location>
  <accessCondition type="restrictionOnAccess">Open access; no subscription required for the linked open-access version.</accessCondition>
  <accessCondition type="useAndReproduction">Creative Commons Attribution (licence family recorded by the source; version not supplied in metadata).</accessCondition>
  <recordInfo>
    <recordContentSource authority="marcorg">local</recordContentSource>
    <recordCreationDate encoding="marc">261005</recordCreationDate>
    <recordChangeDate encoding="iso8601">20261005174826.0</recordChangeDate>
    <recordIdentifier source="local">UMUN-OA-20261005-AI-046</recordIdentifier>
    <languageOfCataloging>
      <languageTerm authority="iso639-2b" type="code">eng</languageTerm>
    </languageOfCataloging>
  </recordInfo>
</mods>
