{"id":22613,"date":"2021-02-09T10:48:47","date_gmt":"2021-02-09T10:48:47","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/learning-decision-trees\/"},"modified":"2023-09-05T07:44:06","modified_gmt":"2023-09-05T07:44:06","slug":"learning-decision-trees","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/learning-decision-trees\/","title":{"rendered":"Learning Decision Trees"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In the context of supervised learning, a decision tree is a tree for predicting the output for a given input. We start from the root of the tree and ask a particular question about the input. Depending on the answer, we go down to one or another of its children. The child we visit is the root of another tree. So we repeat the process, i.e. ask another question here. Eventually, we reach a leaf, i.e. a node with no children. This node contains the final answer which we output and stop.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

This process is depicted below.<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"Learning\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Let\u2019s see an example.<\/strong><\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"A\t\t\t\t\t\t\t\t\t\t\t
A decision tree with categorical predictor variables<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

In machine learning, decision trees are of interest because they can be learned automatically from labeled data. A labeled data set is a set of pairs (x<\/strong>, y<\/em>). Here x<\/strong> is the input vector and y<\/em> the target output.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Below is a labeled data set for our example.<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Weather Temperature Action<\/strong>
Sunny   Warm        Go to park
Sunny   Cold        Stay at home
Sunny   Hot         Stay at home
Rainy   Warm        Stay at home
Rainy   Cold        Stay at home<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
Why Decision Trees<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Learned decision trees often produce good predictors. Because they operate in a tree structure, they can capture interactions among the predictor variables. (This will register as we see more examples.)<\/p>\n
The added benefit is that the learned models are transparent. That is, we can inspect them and deduce how they predict. By contrast, neural networks are opaque. Deep ones even more so.<\/p>\n
Of course, when prediction accuracy is paramount, opaqueness can be tolerated. Decision trees cover this too. Ensembles of decision trees (specifically Random Forest) have state-of-the-art accuracy.<\/p>\n
So either way, it\u2019s good to learn about decision tree learning.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
Learning Decision Trees<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Okay, let\u2019s get to it. We\u2019ll focus on binary classification as this suffices to bring out the key ideas in learning. For completeness, we will also discuss how to morph a binary classifier to a multi-class classifier or to a regressor.<\/p>\n
The pedagogical approach we take below mirrors the process of induction. We\u2019ll start with learning base cases, then build out to more elaborate ones.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
Learning Base Case 1: Single Numeric Predictor<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Consider the following problem. The input is a temperature. The output is a subjective assessment by an individual or a collective of whether the temperature is HOT or NOT.<\/p>\n
Let\u2019s depict our labeled data as follows, with – denoting NOT and + denoting HOT. The temperatures are implicit in the order in the horizontal line.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
- - - - - + + + + +<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
This data is linearly separable. All the -s come before the +s.<\/p>\n
In general, it need not be, as depicted below.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
- - - - - + - + - - - + - + + - + + - + + + + + + + +<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Perhaps the labels are aggregated from the opinions of multiple people. There might be some disagreement, especially near the boundary separating most of the -s from most of the +s.<\/p>\n
Our job is to learn a threshold that yields the best decision rule. What do we mean by \u201cdecision rule\u201d. For any threshold T, we define this as<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
IF temperature < T
  return NOT
ELSE
  return HOT<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
The accuracy of this decision rule on the training set depends on T.<\/p>\n
The objective of learning is to find the T that gives us the most accurate decision rule. Such a T is called an optimal split.<\/p>\n
In fact, we have just seen our first example of learning a decision tree. The decision tree is depicted below.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"A\t\t\t\t\t\t\t\t\t\t\t
A decision tree with one internal node<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
Learning Base Case 2: Single Categorical Predictor<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Consider season<\/em> as a predictor and sunny<\/em> or rainy<\/em> as the binary outcome. Say we have a training set of daily recordings. For each day, whether the day was sunny or rainy is recorded as the outcome to predict. The season the day was in is recorded as the predictor.<\/p>\n
This problem is simpler than Learning Base Case 1. The predictor has only a few values. The four seasons. No optimal split to be learned. It\u2019s as if all we need to do is to fill in the \u2018predict\u2019 portions of the case statement.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
case season
when summer: Predict sunny
when winter: Predict rainy
when fall: ??
when spring: ??<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
That said, we do have the issue of \u201cnoisy labels\u201d. This just means that the outcome cannot be determined with certainty. Summer can have rainy days.<\/p>\n
This issue is easy to take care of. At a leaf of the tree, we store the distribution over the counts of the two outcomes we observed in the training set. This is depicted below.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"Learning\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Say the season was summer. The relevant leaf shows 80: sunny and 5: rainy. So we would predict sunny with a confidence 80\/85.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
Learning General Case 1: Multiple Numeric Predictors<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Not surprisingly, this is more involved.<\/p>\n
We start by imposing the simplifying constraint that the decision rule at any node of the tree tests only for a single dimension of the input.<\/p>\n
Call our predictor variables X<\/em>1, \u2026, X<\/em>n. This means that at the tree\u2019s root we can test for exactly one of these. The question is, which one?<\/p>\n
We answer this as follows. For each of the n<\/em> predictor variables, we consider the problem of predicting the outcome solely from that predictor variable. This gives us n<\/em> one-dimensional predictor problems to solve. We compute the optimal splits T<\/em>1, \u2026, T<\/em>n for these, in the manner described in the first base case. While doing so we also record the accuracies on the training set that each of these splits delivers.<\/p>\n
At the root of the tree, we test for that X<\/em>i whose optimal split T<\/em>i yields the most accurate (one-dimensional) predictor. This is depicted below.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"The\t\t\t\t\t\t\t\t\t\t\t
The best predictor of the outcome from Xi alone<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
So now we need to repeat this process for the two children A and B of this root. Towards this, first, we derive training sets for A and B as follows. The training set for A (B) is the restriction of the parent\u2019s training set to those instances in which X<\/em>i is less than T<\/em> (>= T<\/em>). Let\u2019s also delete the X<\/em>i dimension from each of the training sets.<\/p>\n
Now we have two instances of exactly the same learning problem. So we recurse.<\/p>\n
And so it goes until our training set has no predictors. Only binary outcomes. The node to which such a training set is attached is a leaf. Nothing to test. The data on the leaf are the proportions of the two outcomes in the training set. This suffices to predict both the best outcome at the leaf and the confidence in it.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
Learning General Case 2: Multiple Categorical Predictors<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Here we have n<\/em> categorical predictor variables X<\/em>1, \u2026, X<\/em>n. As we did for multiple numeric predictors, we derive n<\/em> univariate prediction problems from this, solve each of them, and compute their accuracies to determine the most accurate univariate classifier. The predictor variable of this classifier is the one we place at the decision tree\u2019s root.<\/p>\n
Next, we set up the training sets for this root\u2019s children.<\/p>\n
There is one child for each value v<\/em> of the root\u2019s predictor variable X<\/em>i. The training set at this child is the restriction of the root\u2019s training set to those instances in which X<\/em>i equals v<\/em>. We also delete attribute X<\/em>i from this training set.<\/p>\n
Now we recurse as we did with multiple numeric predictors.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
Example<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Let\u2019s illustrate this learning on a slightly enhanced version of our first example, below. We\u2019ve named the two outcomes O<\/em> and I<\/em>, to denote outdoors and indoors respectively. We\u2019ve also attached counts to these two outcomes. A row with a count of o <\/em>for O<\/em> and i<\/em> for I<\/em> denotes o<\/em> instances labeled O<\/em> and i<\/em> instances labeled I<\/em>.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Weather Temperature Action
<\/strong>Sunny   Warm        O (50), I (10)
Sunny   Cold        O (10), I (40)
Sunny   Hot         O (5), I (30)
Rainy   Warm        O (10), I (25)
Rainy   Cold        O (1), I (35)
Rainy   Hot         O (2), I(7)<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
So what predictor variable should we test at the tree\u2019s root? Well, weather<\/em> being rainy<\/em> predicts I<\/em>. (That is, we stay indoors.) Weather being sunny is not predictive on its own. Now consider Temperature<\/em>. Not surprisingly, the temperature is hot or cold also predicts I<\/em>.<\/p>\n
Which variable is the winner? Not clear. Let\u2019s give the nod to Temperature<\/em> since two of its three values predict the outcome. (This is a subjective preference. Don\u2019t take it too literally.)<\/p>\n
Now we recurse.<\/p>\n
The final tree that results is below.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"A\t\t\t\t\t\t\t\t\t\t\t
A learned decision tree with two categorical predictor variables and a binary response<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
Mixed Predictor Variables<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
What if we have both numeric and categorical predictor variables? A reasonable approach is to ignore the difference. To figure out which variable to test for at a node, just determine, as before, which of the available predictor variables predicts the outcome the best. For a numeric predictor, this will involve finding an optimal split first.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
The Learning Algorithm: Abstracting Out The Key Operations<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
Let\u2019s abstract out the key operations in our learning algorithm. These abstractions will help us in describing its extension to the multi-class case and to the regression case.<\/p>\n
The key operations are<\/strong><\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
  1. Evaluate how accurately any one variable predicts the response.<\/li>
  2. Derive child training sets from those of the parent.<\/li><\/ol>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    Multi-class Case<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    What if our response variable has more than two outcomes? As an example, say on the problem of deciding what to do based on the weather and the temperature we add one more option: go to the Mall<\/em>.<\/p>\n
    Fundamentally nothing changes. Adding more outcomes to the response variable does not affect our ability to do operation 1. (The evaluation metric might differ though.) Operation 2 is not affected either, as it doesn\u2019t even look at the response.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    Regression<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    What if our response variable is numeric? Here is one example. Predict the day\u2019s high temperature from the month of the year and the latitude.<\/p>\n
    Can we still evaluate the accuracy with which any single predictor variable predicts the response? This raises a question. How do we even predict a numeric response if any of the predictor variables are categorical?<\/p>\n
    Let\u2019s start by discussing this. First, we look at<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    Base Case 1: Single Categorical Predictor Variable<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    For each value of this predictor, we can record the values of the response variable we see in the training set. A sensible prediction is the mean of these responses.<\/p>\n
    Let\u2019s write this out formally. Let X<\/em> denote our categorical predictor and y<\/em> the numeric response. Our prediction of y<\/em> when X<\/em> equals v<\/em> is an estimate of the value we expect in this situation, i.e. E[y<\/em>|X<\/em>=v<\/em>].<\/p>\n
    Let\u2019s see a numeric example. Consider the training set<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    X A A B B
    y 1 2 4 5<\/pre>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    Our predicted y<\/em>s for X<\/em> = A and X<\/em> = B are 1.5 and 4.5 respectively.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    Base Case 2: Single Numeric Predictor Variable<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    For any particular split T<\/em>, a numeric predictor operates as a boolean categorical variable. So the previous section covers this case as well.<\/p>\n
    Except that we need an extra loop to evaluate various candidate T<\/em>\u2019s and pick the one which works the best. Speaking of \u201cworks the best\u201d, we haven\u2019t covered this yet. We do this below.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    Evaluation Metric<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    Both the response and its predictions are numeric. We just need a metric that quantifies how close to the target response the predicted one is. A sensible metric may be derived from the sum of squares of the discrepancies between the target response and the predicted response.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    General Case<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    We have covered operation 1, i.e. evaluating the quality of a predictor variable towards a numeric response. Operation 2, deriving child training sets from a parent\u2019s, needs no change. As noted earlier, this derivation process does not use the response at all.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    The Response Predicted At A Leaf<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    As in the classification case, the training set attached at a leaf has no predictor variables, only a collection of outcomes. As noted earlier, a sensible prediction at the leaf would be the mean of these outcomes. So this is what we should do when we arrive at a leaf.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    Modeling Tradeoffs<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    Consider our regression example: predict the day\u2019s high temperature from the month of the year and the latitude.<\/p>\n
    Consider the month of the year. We could treat it as a categorical predictor with values January<\/em>, February<\/em>, March<\/em>, \u2026 Or as a numeric predictor with values 1<\/em>, 2<\/em>, 3<\/em>, \u2026<\/p>\n
    What are the tradeoffs? Treating it as a numeric predictor lets us leverage the order in the months. February<\/em> is near January<\/em> and far away from August<\/em>. That said, how do we capture that December<\/em> and January<\/em> are neighboring months? 12<\/em> and 1<\/em> as numbers are far apart.<\/p>\n
    Perhaps more importantly, decision tree learning with a numeric predictor operates only via splits. That would mean that a node on a tree that tests for this variable can only make binary decisions. By contrast, using the categorical predictor gives us 12 children. In principle, this is capable of making finer-grained decisions.<\/p>\n
    Now consider latitude. We can treat it as a numeric predictor. Or as a categorical one induced by a certain binning, e.g. in units of + or – 10 degrees. The latter enables finer-grained decisions in a decision tree.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t
    Summary<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    In this post, we have described learning decision trees with intuition, examples, and pictures. We have covered both decision trees for both classification and regression problems. We have also covered both numeric and categorical predictor variables.<\/p>\n
    Further Reading<\/strong><\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
    1. Decision tree learning<\/a><\/li><\/ol>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"
      In the context of supervised learning, a decision tree is a tree for predicting the output for a given input. Learned decision trees often produce good predictors.<\/p>\n","protected":false},"author":1044,"featured_media":18645,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[1324,92,1325,664],"ppma_author":[3691],"class_list":["post-22613","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-decision-trees","tag-machine-learning","tag-regression-tree","tag-supervised-learning"],"authors":[{"term_id":3691,"user_id":1044,"is_guest":0,"slug":"arun-jagota","display_name":"Arun Jagota","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2021\/05\/Arun-Jagota-150x150.jpeg","user_url":"https:\/\/www.salesforce.com\/in\/?ir=1","last_name":"Jagota","first_name":"Arun","job_title":"","description":"Arun Jagota is Director of Data Science at Salesforce.com. A PhD in computer science, he has taught undergraduate, graduate, and continuing education courses in Computer Science at many US Universities from 1992 through 2006. He has written a number of books, most available at Amazon.com, 50 academic publications and has 17+ patents issued."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22613","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/1044"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=22613"}],"version-history":[{"count":4,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22613\/revisions"}],"predecessor-version":[{"id":32255,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/22613\/revisions\/32255"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/18645"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=22613"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=22613"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=22613"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=22613"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}