Decision trees and decision rule systems are widely used in different applications as algorithms for problem solving, as predictors, and as a way for knowledge representation. Reducts play key role in the problem of attribute (feature) selection. The aims of this book are (i) the consideration of the sets of decision trees, rules and reducts; (ii) study of relationships among these objects; (iii) design of algorithms for construction of trees, rules and reducts; and (iv) obtaining bounds on their complexity. Applications for supervised machine learning, discrete optimization, analysis of acyclic programs, fault diagnosis, and pattern recognition are considered also. This is a mixture of research monograph and lecture notes. It contains many unpublished results. However, proofs are carefully selected to be understandable for students. The results considered in this book can be useful for researchers in machine learning, data mining and knowledge discovery, especially for those who are working in rough set theory, test theory and logical analysis of data. The book can be used in the creation of courses for graduate students.