A survey on missing data in machine learning /
Tlamelo Emmanuel
A survey on missing data in machine learning / Tlamelo Emmanuel; Thabiso Maupong; Dimane Mpoeleng [and others]. - 1 online resource (1 article).
Publication type: Journal article. Linked open-access copy: published version.
Open access; no subscription required for the linked open-access version.
Machine learning has been the corner stone in analysing and extracting information from data and often a problem of missing values is encountered. Missing values occur because of various factors like missing completely at random, missing at random or missing not at random. All these may result from system malfunction during data collection or human error during data pre-processing. Nevertheless, it is important to deal with missing values before analysing data since ignoring or omitting missing values may result in biased or misinformed analysis. In literature there have been several proposals for handling missing values. In this paper, we aggregate some of the literature on missing data particularly focusing on machine learning techniques. We also give insight on how the machine learning approaches work by highlighting the key features of missing values imputation techniques, how they perform, their limitations and the kind of data they are most suitable for. We propose and evaluate two methods, the k nearest neighbor and an iterative imputation method (missForest) based on the random forest algorithm. Evaluation is performed on the Iris and novel power plant fan data with induced missing values at missingness rate of 5% to 20%. We show that both missForest and the k nearest neighbor can successfully handle missing values and offer some possible future research direction.
Creative Commons Attribution. See the linked licence for the applicable terms.
In English.
10.1186/s40537-021-00516-9 doi
missing data
missing completely at random
missing at random
missing not at random
k-nearest neighbors imputation
missForest
B. Sc. Artificial Intelligence UMUN open access collection machine learning
Journal articles.
A survey on missing data in machine learning / Tlamelo Emmanuel; Thabiso Maupong; Dimane Mpoeleng [and others]. - 1 online resource (1 article).
Publication type: Journal article. Linked open-access copy: published version.
Open access; no subscription required for the linked open-access version.
Machine learning has been the corner stone in analysing and extracting information from data and often a problem of missing values is encountered. Missing values occur because of various factors like missing completely at random, missing at random or missing not at random. All these may result from system malfunction during data collection or human error during data pre-processing. Nevertheless, it is important to deal with missing values before analysing data since ignoring or omitting missing values may result in biased or misinformed analysis. In literature there have been several proposals for handling missing values. In this paper, we aggregate some of the literature on missing data particularly focusing on machine learning techniques. We also give insight on how the machine learning approaches work by highlighting the key features of missing values imputation techniques, how they perform, their limitations and the kind of data they are most suitable for. We propose and evaluate two methods, the k nearest neighbor and an iterative imputation method (missForest) based on the random forest algorithm. Evaluation is performed on the Iris and novel power plant fan data with induced missing values at missingness rate of 5% to 20%. We show that both missForest and the k nearest neighbor can successfully handle missing values and offer some possible future research direction.
Creative Commons Attribution. See the linked licence for the applicable terms.
In English.
10.1186/s40537-021-00516-9 doi
missing data
missing completely at random
missing at random
missing not at random
k-nearest neighbors imputation
missForest
B. Sc. Artificial Intelligence UMUN open access collection machine learning
Journal articles.