A semi-supervised segmentation algorithm as applied to k-means using information value

  • DG Breed PhD student, North-West University
  • T Verster North-West University
  • SE Terblanche North-West University

Abstract

Segmentation (or partitioning) of data for the purpose of enhancing predictive modelling is a well-established practice in the banking industry. Unsupervised and supervised approaches are the two main streams of segmentation and examples exist where the application of these techniques improved the performance of predictive models. Both these streams focus, however, on a single aspect (i.e. either target separation or independent variable distribution) and combining them may deliver better results in some instances. In this paper a semi-supervised segmentation algorithm is presented, which is based on k-means clustering and which applies information value for the purpose of informing the segmentation process. Simulated data are used to identify a few key characteristics that may cause one segmentation technique to outperform another. In the empirical study the newly proposed semi-supervised segmentation algorithm outperforms both an unsupervised and a supervised segmentation technique, when compared by using the Gini coefficient as performance measure of the resulting predictive models.

Author Biographies

DG Breed, PhD student, North-West University
PhD Student, Centre for Business Mathematics and Informatics, North-West University
T Verster, North-West University
Associate Professor, Centre for Business Mathematics and Informatics, North-West University
SE Terblanche, North-West University
Associate Professor, Centre for Business Mathematics and Informatics, North-West University
Published
2017-12-08
Section
Research Articles