Ensembling Classification Models Based on Phalanxes of Variables, With Applications in Drug Discovery
Statistical detection of a rare class of objects in a two-class classification problem can pose several challenges. Because the class of interest is rare in the training data, there is relatively little information for model building in the training-data class response labels. At the same time the available explanatory variables are often high dimensional. For instance, in the motivating application, drug discovery, compounds are active or not against a specific biological target, such as lung cancer tumour cells, and active compounds are rare. Several sets of chemical descriptor variables from computational chemistry are available to classify the active versus inactive class; each set can have up to thousands of variables characterizing molecular structure of the compounds.
The statistical challenge is to make use of the richness of the explanatory variables in the presence of scant response information. Our algorithm divides the explanatory variables into subsets data adaptively and passes each subset to a base classifier. The various base classifiers are then combined in an ensemble to produce one model to rank new objects by their estimated probabilities of belonging to the rare class of interest. The essence of the algorithm is to choose the subsets such that variables in the same group work well together; such groups are called phalanxes. The method is illustrated on biological assays from PubChem and several descriptor sets.
Joint work with Jabed Tomal and Ruben Zamar.