Decision Tree Pruning

amitdamitd Member, University ProfessorPosts:49Maven
edited June 2021 inHelp
我有两个问题与“修剪”(post-pr有关uning) in the Decision Tree operator.

1. RM supports pessimistic pruning (i.e., top-down), but not optimistic pruning (i.e., bottom-up). Is that correct?
2. What are the precise logical steps in the pruning process?
3. When the Decision Tree is being trained using the "training set" with the pruning option enabled, which "validation set" is the classification error computed for? It cannot be the entire training set because then the classification error would be 0 in the fully-grown tree, which would always be the minimum. My understanding of pruning is that the cost complexity is computed by applying a penalty factor for tree size and the tree that minimizes the classification error for the validation set is chosen. When using the training set, how is the validation done?

I get related ideas from this previouspostand another previouspost. Also, I have looked at the RapidMiner codePessimisticPruner.java, but I am not able to parse the logic from there.

@IngoRM,@land, and others - any help would be much appreciated.



Tagged:

Answers

  • amitdamitd Member, University ProfessorPosts:49Maven
    To add to my previous comment, I think that the pruning in the Decision Tree operator is not working correctly. I have attached a simple process with the Titanic dataset. Toggling the pruning option "on" and "off" does not make any difference to the resulting tree and performance metrics. Also, changing the pruning confidence level (I tried 0.001, 0.05, 0.1, 0.5) does not matter as well. Can someone verify if there is a bug here?
    Also, where does that leave with my questions in the previous post? I would appreciate the community's help.
  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University ProfessorPosts:3,368RM Data Scientist
    Hi@avd,
    I would not say it does nothing. Have a look at the following process. Adding confidence pruning reduces the number of nodes in the lower trees.

    Best,
    Martin



























































    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University ProfessorPosts:1,751RM Founder
    Hey folks,
    Agreed, it definitely is doing something for other data sets. However, I also found it a bit weird that there does not seem to be any pruning impact for Titanic no matter what the confidence value is... I will create a ticket in our internal system to inspect this a bit closer. Most likely things are fine, but let's double check...
    Cheers,
    Ingo
  • amitdamitd Member, University ProfessorPosts:49Maven
    edited June 2021
    Thank you for looking into this. Look forward to the update.
    我和其他几位dat并测试它asets with the same result.

    Some conceptual questions that would help answering -
    (a) Since the Decision Tree operator is based on CART - does it useCost-Complexity pruningapproach during the pessimistic pruning? Some related links:link 1, andscikit-learn's Pythonimplementation.
    (b) How is the validation (internal) happening during the pruning? What portion of "training" data is internally used for validation and how?
    (c) If the CC-pruning approach is used, can the chosen penalty (alpha) parameter be informed to the user for explainability?
    (d) If the CC-pruning approach is not used, what exact method is used?
  • amitdamitd Member, University ProfessorPosts:49Maven
    edited June 2021
    While I am waiting for the pruning functionality to be double-checked by the kind RM folks in this community, upon some more digging, it seems to me that the Decision Tree operator uses “pessimistic error pruning” method (see section 2.2.5 in the linked article), not the "cost-complexity method". Can someone confirm this? In that case, it makes sense that there is no validation partition used for pruning. Are there any plans to incorporate "cost-complexity method" for pruning which is much more popular (probably because it gives more reliable results)?
Sign InorRegisterto comment.