On Classification with Pairwise Comparisons, Support Vector Machines and Feature Domain Overlapping

Most existing classification algorithms either consider all features as equally important (equal weights), or do not analyze the consistency of weights assigned to features. When features are not equally important, assigning consistent weights is not an obvious task. In general, we have two cases. The first case assumes that a given sample of data does not contain any clues about the importance of features, so the weights are provided by a pool of experts and they are usually inconsistent. The second case assumes that the given sample contains some information about feature importance, hence we can derive the weights directly from the sample. In this paper, we deal with both cases. Pairwise comparisons and weighted support vector machines (SVMs) are used for the first case. For the second case, a new approach based on the observation that the feature importance could be determined by the discrimination power of features has been proposed. For the first case, we start with pairwise comparisons to rank the importance of features, then we use distance-based inconsistency reduction to refine the weight assessment and make the comparisons more precise. Next, we calculate the weights through the fully consistent or almost consistent pairwise comparison tables. For the second case, a novel concept of feature domain overlappings has been introduced. It can measure the feature discrimination power. This model is based on the assumption that less overlapping means more discriminatory ability, and this can be used to calculate weights characterizing the importance of particular features. For both cases, weighted SVMs are used to classify the data. Both methods have been tested using two benchmark datasets, Iris and Vertebral. The results were especially superior to those obtained without weights.