Analysing the importance of variables for sewer failure prediction

Abstract

The ability to adequately prioritise maintenance of sewer systems significantly increases the quality of the service provided by these systems. It is thus important to optimise decision making processes, a more feasible challenge as digital data becomes available. When defining the variables that should be used to predict sewer failure, it is important to identify the ones that mostly influence the quality of the predictions (i.e. the response variable) or to define the smallest number of variables that is adequate to conduct accurate predictions. In this study three different methods to identify the most important variables are evaluated. The first is the mutual information indicator, the second method is the stepwise search approach and the third method uses the out-of-bag samples concept, based on the random forest algorithm. The methods were applied to a real data set that consists on the categorization of sewer condition (critical, non-critical) and their physical characteristics (e.g. Length, Age, Diameter, Slope and Material). The mutual information and the stepwise search methods provided good predictions and produced similar results. The results obtained using out-of-bag samples based on random forest were somewhat different and can be justified by the lack of robustness to imbalanced class distributions.