The evaluation of the decision tree is particularly fast. The tree structure created in the model construction is used for a series of simple tests (usually 2-7). Each test is based on a single predictor. This is a membership test: IN or NOT IN a list of values (categorical predictor); or LESS THAN OR EQUAL TO A value (numeric predictor). The analysis can take into account the preference or utility function of the decision-maker (for example, the company), for example: Let`s try OneR with real data. We use the cervical cancer classification task to test the OneR algorithm. All continuous input functions have been discretized into their 5 quantiles. The following rules are created: For each function, we go through the array row by row: Each function value is the IF part of a rule; The most common class for instances with this feature value is prediction, the THEN part of the rule. For example, the size function with the small, medium and large levels gives three rules. For each feature, we calculate the total error rate of the generated rules, which is the sum of the errors. The location function has the possible wrong and good values. The most common value for homes in poor places is low and if we use low as a forecast, we make two mistakes because two homes have an average value.

The predicted value of homes in good locations is high and again, we make two mistakes because two homes have an average value. The error we make when using the tracking function is 4/10, for the size function it is 3/10 and for the pet function it is 4/10. The size function generates the rules with the least error and is used for the final OneR model: in this example, a decision tree can be drawn to illustrate the principles of decreasing return over range #1. The aim is to make d decision lists from the subsequent distribution: if-THEN rules are easy to interpret. They are probably the most interpretable of the interpretable models. This statement only applies if the number of rules is small, if the conditions of the rules are short (maximum 3 I would say) and if the rules are organized in a list of decisions or a set of decisions that do not overlap. Let`s get into more detail about the algorithm: The algorithm starts with characteristic value models pre-mining with the FP growth algorithm. BRL makes a number of assumptions about the distribution of the target and the distribution of parameters that define the distribution of the target.

(These are Bayesian statistics.) If you`re not familiar with Bayesian statistics, don`t get too caught up in the following explanations. It is important to know that the Bayesian approach is a way to combine existing knowledge or requirements (so-called priori distributions) and at the same time adapt them to the data. In the case of decision lists, the Bayesian approach makes sense, as the previous assumptions make decision lists with short rules too short. A decision list introduces an order into the decision rules. If the condition of the first rule is true for an instance, we use the prediction of the first rule. If not, let`s move on to the next rule and see if it is true, and so on. Decision lists solve the problem of overlapping rules by returning only the prediction of the first rule in the applicable list. Decision trees are often used in operations research and operations management. If, in practice, decisions are to be made online without recall under incomplete knowledge, a decision tree should be supplemented by a probability model as a best choice model or online selection model algorithm. Another use of decision trees is as a descriptive way to calculate conditional probabilities. Rules are sorted by the number of training examples assigned to each rule. For each rule, there is information about the predicted class name and the probability of prediction.

The idea is simple: first, find a good rule that applies to certain data points. Delete all data points covered by the rule. A data point is covered when the conditions apply, regardless of whether the points are correctly ranked or not. Repeat learning the rule and deleting the covered points with the remaining points until there are no more points left or another stop condition is met. The result is a list of decisions. This approach of repeatedly learning rules and removing covered data points is called «separate and conquer.» Now let`s move from a simple OneR algorithm to a more complex procedure that uses rules with more complex conditions consisting of several features: sequential coverage. Decision trees are often used in operations research, especially in business intelligence, to identify a strategy that is most likely to achieve a goal, but are also a popular tool in machine learning. Decision trees can also be thought of as generative models of induction rules from empirical data. An optimal decision tree is then defined as a tree that constitutes the most data while minimizing the number of levels (or «questions»). [5] Several algorithms have been developed to generate optimal trees such as ID3/4/5,[6] CLS, ASSISTANT and CART. A decision tree consists of three types of nodes:[1] Sequential coverage is a common procedure that repeatedly learns a single rule to create a list of decisions (or set) that covers the entire dataset on a rule-by-rule basis. Many rule learning algorithms are variants of the sequential coverage algorithm.

This chapter introduces the main recipe and uses RIPPER, a variant of the sequential coverage algorithm, for examples. The goal of the BRL algorithm is to learn an accurate decision list based on a selection of ready-made conditions while prioritizing lists with few rules and short conditions. BRL meets this objective by defining a distribution of decision lists with previous distributions for the duration of the conditions (preferably shorter rules) and the number of rules (preferably a shorter list). The subsequent probability distribution of lists makes it possible to say what is the probability of a decision list if scarcity is assumed and to what extent the list corresponds to the data. Our goal is to find the list that maximizes this likelihood of follow-up. Since it is not possible to find the best exact list directly from the distributions of the lists, BRL suggests the following recipe: 1) Generate an initial decision list that is drawn at random from the a priori distribution. 2) Edit the list iteratively by adding, modifying, or deleting rules to ensure that the resulting lists follow the back distribution of the lists. 3) Select the decision list from the sample lists with the highest probability based on the retrospective distribution.

The Scikit-Learn Decision Tree class has a export_text(). It returns the textual representation of the rules. The decision tree shows that in the sequential distribution of lifeguards, placing a first lifeguard on beach #1 would be optimal if only the budget for 1 lifeguard is available. But if there is a budget for two guards, then placing both on beach #2 would prevent further drownings overall. A decision rule is a simple IF-THEN statement that consists of a condition (also called a predecessor) and a prediction. For example: If it rains today AND if it is April (condition), THEN it will rain tomorrow (forecast). A single decision rule or a combination of multiple rules can be used to make predictions. Decision rules are robust against monotonous transformations of input functions because only the threshold in conditions changes. They are also robust against outliers, as this only depends on whether or not a condition is applied. There are many ways to present a decision tree. It can be viewed as a graphic or converted into text rendering. In MLJAR AutoML we use dtreeviz visualization and text display with a user-friendly format.

If you want to train a decision tree (or other ML algorithms), you can try MLJAR AutoML: github.com/mljar/mljar-supervised. The decision tree can be linearized into decision rules[2], where the result is the contents of the leaf node and the conditions along the path form a conjunction in the if clause. In general, the rules have the form: Let`s review the rules of DecisionTreeRegressor.