【问题标题】:How to extract the decision rules from scikit-learn decision-tree?如何从 scikit-learn 决策树中提取决策规则?
【发布时间】:2013-12-12 00:51:30
【问题描述】:

我可以从决策树中经过训练的树中提取基础决策规则(或“决策路径”)作为文本列表吗?

类似:

if A>0.4 then if B<0.2 then if C>0.8 then class='X'

【问题讨论】:

标签: python machine-learning scikit-learn decision-tree random-forest


【解决方案1】:

我相信这个答案比这里的其他答案更正确:

from sklearn.tree import _tree

def tree_to_code(tree, feature_names):
    tree_ = tree.tree_
    feature_name = [
        feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
        for i in tree_.feature
    ]
    print "def tree({}):".format(", ".join(feature_names))

    def recurse(node, depth):
        indent = "  " * depth
        if tree_.feature[node] != _tree.TREE_UNDEFINED:
            name = feature_name[node]
            threshold = tree_.threshold[node]
            print "{}if {} <= {}:".format(indent, name, threshold)
            recurse(tree_.children_left[node], depth + 1)
            print "{}else:  # if {} > {}".format(indent, name, threshold)
            recurse(tree_.children_right[node], depth + 1)
        else:
            print "{}return {}".format(indent, tree_.value[node])

    recurse(0, 1)

这会打印出一个有效的 Python 函数。下面是一个尝试返回其输入(0 到 10 之间的数字)的树的示例输出。

def tree(f0):
  if f0 <= 6.0:
    if f0 <= 1.5:
      return [[ 0.]]
    else:  # if f0 > 1.5
      if f0 <= 4.5:
        if f0 <= 3.5:
          return [[ 3.]]
        else:  # if f0 > 3.5
          return [[ 4.]]
      else:  # if f0 > 4.5
        return [[ 5.]]
  else:  # if f0 > 6.0
    if f0 <= 8.5:
      if f0 <= 7.5:
        return [[ 7.]]
      else:  # if f0 > 7.5
        return [[ 8.]]
    else:  # if f0 > 8.5
      return [[ 9.]]

以下是我在其他答案中看到的一些绊脚石:

  1. 使用tree_.threshold == -2 来确定一个节点是否是叶子不是一个好主意。如果它是一个阈值为 -2 的真实决策节点怎么办?相反,您应该查看tree.feature 或tree.children_*。
  2. features = [feature_names[i] for i in tree_.feature] 行在我的 sklearn 版本中崩溃,因为 tree.tree_.feature 的某些值是 -2(特别是叶节点)。
  3. 递归函数中不需要多个 if 语句,一个就可以了。

【讨论】:

  • 这段代码非常适合我。但是,我有 500 多个 feature_names,所以输出代码对于人类来说几乎是不可能理解的。有没有办法让我只将我感兴趣的功能名称输入到函数中?
  • 我同意前面的评论。 IIUC,print "{}return {}".format(indent, tree_.value[node]) 应更改为 print "{}return {}".format(indent, np.argmax(tree_.value[node][0])) 以使函数返回类索引。
  • @paulkernfeld 啊,是的,我看到你可以遍历RandomForestClassifier.estimators_,但我无法弄清楚如何组合估计器的结果。
  • 我无法在 python 3 中使用它,_tree 位似乎无法正常工作,并且未定义 TREE_UNDEFINED。 This link helped me. 虽然导出的代码不能直接在 python 中运行,但它类似于 c 并且很容易翻译成其他语言:web.archive.org/web/20171005203850/http://www.kdnuggets.com/…
  • @Josiah,将 () 添加到 print 语句以使其在 python3 中工作。例如print "bla" => print("bla")
【解决方案2】:

我创建了自己的函数来从 sklearn 创建的决策树中提取规则:

import pandas as pd
import numpy as np
from sklearn.tree import DecisionTreeClassifier

# dummy data:
df = pd.DataFrame({'col1':[0,1,2,3],'col2':[3,4,5,6],'dv':[0,1,0,1]})

# create decision tree
dt = DecisionTreeClassifier(max_depth=5, min_samples_leaf=1)
dt.fit(df.ix[:,:2], df.dv)

此函数首先从节点(子数组中的 -1 标识)开始,然后递归查找父节点。我称之为节点的“血统”。在此过程中,我获取了创建 if/then/else SAS 逻辑所需的值:

def get_lineage(tree, feature_names):
     left      = tree.tree_.children_left
     right     = tree.tree_.children_right
     threshold = tree.tree_.threshold
     features  = [feature_names[i] for i in tree.tree_.feature]

     # get ids of child nodes
     idx = np.argwhere(left == -1)[:,0]     

     def recurse(left, right, child, lineage=None):          
          if lineage is None:
               lineage = [child]
          if child in left:
               parent = np.where(left == child)[0].item()
               split = 'l'
          else:
               parent = np.where(right == child)[0].item()
               split = 'r'

          lineage.append((parent, split, threshold[parent], features[parent]))

          if parent == 0:
               lineage.reverse()
               return lineage
          else:
               return recurse(left, right, parent, lineage)

     for child in idx:
          for node in recurse(left, right, child):
               print node

下面的元组集包含创建 SAS if/then/else 语句所需的一切。我不喜欢在 SAS 中使用 do 块,这就是我创建描述节点整个路径的逻辑的原因。元组后面的单个整数是路径中终端节点的 ID。所有前面的元组组合起来创建该节点。

In [1]: get_lineage(dt, df.columns)
(0, 'l', 0.5, 'col1')
1
(0, 'r', 0.5, 'col1')
(2, 'l', 4.5, 'col2')
3
(0, 'r', 0.5, 'col1')
(2, 'r', 4.5, 'col2')
(4, 'l', 2.5, 'col1')
5
(0, 'r', 0.5, 'col1')
(2, 'r', 4.5, 'col2')
(4, 'r', 2.5, 'col1')
6

【讨论】:

  • 这种类型的树是否正确,因为 col1 再次出现,一个是 col1
  • 右分支将有(0.5, 2.5]之间的记录。这些树是用递归分区制成的。没有什么可以阻止变量被多次选择。
  • 好的,你能解释一下递归部分发生了什么,因为我在我的代码中使用了它并且看到了类似的结果
  • 如何修改此代码以获取类似结构的数据框中的类和规则?
【解决方案3】:

我修改了Zelazny7提交的代码,打印了一些伪代码:

def get_code(tree, feature_names):
        left      = tree.tree_.children_left
        right     = tree.tree_.children_right
        threshold = tree.tree_.threshold
        features  = [feature_names[i] for i in tree.tree_.feature]
        value = tree.tree_.value

        def recurse(left, right, threshold, features, node):
                if (threshold[node] != -2):
                        print "if ( " + features[node] + " <= " + str(threshold[node]) + " ) {"
                        if left[node] != -1:
                                recurse (left, right, threshold, features,left[node])
                        print "} else {"
                        if right[node] != -1:
                                recurse (left, right, threshold, features,right[node])
                        print "}"
                else:
                        print "return " + str(value[node])

        recurse(left, right, threshold, features, 0)

如果您在同一个示例中调用get_code(dt, df.columns),您将获得:

if ( col1 <= 0.5 ) {
return [[ 1.  0.]]
} else {
if ( col2 <= 4.5 ) {
return [[ 0.  1.]]
} else {
if ( col1 <= 2.5 ) {
return [[ 1.  0.]]
} else {
return [[ 0.  1.]]
}
}
}

【讨论】:

  • 你能告诉我,在上面的输出中,return 语句中的 [[ 1. 0.]] 到底是什么意思。我不是 Python 人,但在做同样的事情。所以如果你能证明一些细节对我有好处,这样对我来说会更容易。
  • @user3156186 表示'0'类有1个对象,'1'类有0个对象
  • @Daniele,你知道这些课程是如何排序的吗?我猜是字母数字,但我没有在任何地方找到确认。
  • 谢谢!对于阈值实际上为-2的边缘情况,我们可能需要将(threshold[node] != -2)更改为( left[node] != -1)(类似于下面获取子节点id的方法)
  • @Daniele,知道如何让你的函数“get_code”“返回”一个值而不是“打印”它,因为我需要将它发送到另一个函数?
【解决方案4】:

Scikit learn 在 0.21 版(2019 年 5 月)中引入了一种名为 export_text 的美味新方法,用于从树中提取规则。 Documentation here。不再需要创建自定义函数。

拟合模型后,您只需要两行代码。一、导入export_text:

from sklearn.tree import export_text

其次,创建一个包含您的规则的对象。为了使规则看起来更具可读性,请使用 feature_names 参数并传递您的功能名称列表。例如,如果您的模型名为 model,并且您的特征在名为 X_train 的数据框中命名,您可以创建一个名为 tree_rules 的对象:

tree_rules = export_text(model, feature_names=list(X_train.columns))

然后打印或保存tree_rules。您的输出将如下所示:

|--- Age <= 0.63
|   |--- EstimatedSalary <= 0.61
|   |   |--- Age <= -0.16
|   |   |   |--- class: 0
|   |   |--- Age >  -0.16
|   |   |   |--- EstimatedSalary <= -0.06
|   |   |   |   |--- class: 0
|   |   |   |--- EstimatedSalary >  -0.06
|   |   |   |   |--- EstimatedSalary <= 0.40
|   |   |   |   |   |--- EstimatedSalary <= 0.03
|   |   |   |   |   |   |--- class: 1

【讨论】:

    【解决方案5】:

    0.18.0 版本中有一个新的DecisionTreeClassifier 方法decision_path。开发人员提供了一个广泛的(有据可查的)walkthrough。

    演练中打印树结构的第一部分代码似乎没问题。但是,我修改了第二部分中的代码以询问一个样本。我用# &lt;--表示的更改

    编辑在拉取请求#8653 和#10951 中指出错误后,下面代码中# &lt;-- 标记的更改已在演练链接中更新。现在更容易跟随。

    sample_id = 0
    node_index = node_indicator.indices[node_indicator.indptr[sample_id]:
                                        node_indicator.indptr[sample_id + 1]]
    
    print('Rules used to predict sample %s: ' % sample_id)
    for node_id in node_index:
    
        if leave_id[sample_id] == node_id:  # <-- changed != to ==
            #continue # <-- comment out
            print("leaf node {} reached, no decision here".format(leave_id[sample_id])) # <--
    
        else: # < -- added else to iterate through decision nodes
            if (X_test[sample_id, feature[node_id]] <= threshold[node_id]):
                threshold_sign = "<="
            else:
                threshold_sign = ">"
    
            print("decision id node %s : (X[%s, %s] (= %s) %s %s)"
                  % (node_id,
                     sample_id,
                     feature[node_id],
                     X_test[sample_id, feature[node_id]], # <-- changed i to sample_id
                     threshold_sign,
                     threshold[node_id]))
    
    Rules used to predict sample 0: 
    decision id node 0 : (X[0, 3] (= 2.4) > 0.800000011921)
    decision id node 2 : (X[0, 2] (= 5.1) > 4.94999980927)
    leaf node 4 reached, no decision here
    

    更改sample_id 以查看其他样本的决策路径。我没有向开发人员询问这些更改,只是在完成示例时看起来更直观。

    【讨论】:

    • 你我的朋友是个传奇!任何想法如何绘制该特定样本的决策树?非常感谢您的帮助
    • 感谢 Victor,最好将此作为一个单独的问题提出,因为绘图要求可能特定于用户的需求。如果您提供有关您希望输出的外观的想法,您可能会得到很好的响应。
    • 嘿,凯文,我创建了问题stackoverflow.com/questions/48888893/…
    • 您能否解释一下名为 node_index 的部分,没有得到那个部分。它有什么作用?
    【解决方案6】:
    from StringIO import StringIO
    out = StringIO()
    out = tree.export_graphviz(clf, out_file=out)
    print out.getvalue()
    

    你可以看到一个有向图树。然后,clf.tree_.feature 和clf.tree_.value 分别是节点分裂特征数组和节点值数组。你可以参考这个github source的更多细节。

    【讨论】:

    【解决方案7】:

    只是因为每个人都非常乐于助人,所以我将对 Zelazny7 和 Daniele 的精美解决方案进行修改。这个是针对 python 2.7 的,带有标签以使其更具可读性:

    def get_code(tree, feature_names, tabdepth=0):
        left      = tree.tree_.children_left
        right     = tree.tree_.children_right
        threshold = tree.tree_.threshold
        features  = [feature_names[i] for i in tree.tree_.feature]
        value = tree.tree_.value
    
        def recurse(left, right, threshold, features, node, tabdepth=0):
                if (threshold[node] != -2):
                        print '\t' * tabdepth,
                        print "if ( " + features[node] + " <= " + str(threshold[node]) + " ) {"
                        if left[node] != -1:
                                recurse (left, right, threshold, features,left[node], tabdepth+1)
                        print '\t' * tabdepth,
                        print "} else {"
                        if right[node] != -1:
                                recurse (left, right, threshold, features,right[node], tabdepth+1)
                        print '\t' * tabdepth,
                        print "}"
                else:
                        print '\t' * tabdepth,
                        print "return " + str(value[node])
    
        recurse(left, right, threshold, features, 0)
    

    【讨论】:

      【解决方案8】:

      我一直在经历这个,但我需要以这种格式编写规则

      if A>0.4 then if B<0.2 then if C>0.8 then class='X' 
      

      所以我改编了@paulkernfeld 的答案(谢谢),您可以根据自己的需要进行定制

      def tree_to_code(tree, feature_names, Y):
          tree_ = tree.tree_
          feature_name = [
              feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
              for i in tree_.feature
          ]
          pathto=dict()
      
          global k
          k = 0
          def recurse(node, depth, parent):
              global k
              indent = "  " * depth
      
              if tree_.feature[node] != _tree.TREE_UNDEFINED:
                  name = feature_name[node]
                  threshold = tree_.threshold[node]
                  s= "{} <= {} ".format( name, threshold, node )
                  if node == 0:
                      pathto[node]=s
                  else:
                      pathto[node]=pathto[parent]+' & ' +s
      
                  recurse(tree_.children_left[node], depth + 1, node)
                  s="{} > {}".format( name, threshold)
                  if node == 0:
                      pathto[node]=s
                  else:
                      pathto[node]=pathto[parent]+' & ' +s
                  recurse(tree_.children_right[node], depth + 1, node)
              else:
                  k=k+1
                  print(k,')',pathto[parent], tree_.value[node])
          recurse(0, 1, 0)
      

      【讨论】:

        【解决方案9】:

        这是你需要的代码

        我已将最喜欢的代码修改为在 jupyter notebook python 3 中正确缩进

        import numpy as np
        from sklearn.tree import _tree
        
        def tree_to_code(tree, feature_names):
            tree_ = tree.tree_
            feature_name = [feature_names[i] 
                            if i != _tree.TREE_UNDEFINED else "undefined!" 
                            for i in tree_.feature]
            print("def tree({}):".format(", ".join(feature_names)))
        
            def recurse(node, depth):
                indent = "    " * depth
                if tree_.feature[node] != _tree.TREE_UNDEFINED:
                    name = feature_name[node]
                    threshold = tree_.threshold[node]
                    print("{}if {} <= {}:".format(indent, name, threshold))
                    recurse(tree_.children_left[node], depth + 1)
                    print("{}else:  # if {} > {}".format(indent, name, threshold))
                    recurse(tree_.children_right[node], depth + 1)
                else:
                    print("{}return {}".format(indent, np.argmax(tree_.value[node])))
        
            recurse(0, 1)
        

        【讨论】:

          【解决方案10】:

          下面的代码是我在 anaconda python 2.7 下加上一个包名“pydot-ng”来制作带有决策规则的 PDF 文件的方法。希望对您有所帮助。

          from sklearn import tree
          
          clf = tree.DecisionTreeClassifier(max_leaf_nodes=n)
          clf_ = clf.fit(X, data_y)
          
          feature_names = X.columns
          class_name = clf_.classes_.astype(int).astype(str)
          
          def output_pdf(clf_, name):
              from sklearn import tree
              from sklearn.externals.six import StringIO
              import pydot_ng as pydot
              dot_data = StringIO()
              tree.export_graphviz(clf_, out_file=dot_data,
                                   feature_names=feature_names,
                                   class_names=class_name,
                                   filled=True, rounded=True,
                                   special_characters=True,
                                    node_ids=1,)
              graph = pydot.graph_from_dot_data(dot_data.getvalue())
              graph.write_pdf("%s.pdf"%name)
          
          output_pdf(clf_, name='filename%s'%n)
          

          a tree graphy show here

          【讨论】:

            【解决方案11】:

            这是一种使用 SKompiler 库将整个树转换为单个(不一定是人类可读的)python 表达式的方法:

            from skompiler import skompile
            skompile(dtree.predict).to('python/code')
            

            【讨论】:

              【解决方案12】:

              这建立在@paulkernfeld 的回答之上。如果您有一个带有您的特征的数据框 X 和一个带有您的共振的目标数据框 y 并且您想知道哪个 y 值在哪个节点中结束(以及相应地绘制它的蚂蚁),您可以执行以下操作:

                  def tree_to_code(tree, feature_names):
                      from sklearn.tree import _tree
                      codelines = []
                      codelines.append('def get_cat(X_tmp):\n')
                      codelines.append('   catout = []\n')
                      codelines.append('   for codelines in range(0,X_tmp.shape[0]):\n')
                      codelines.append('      Xin = X_tmp.iloc[codelines]\n')
                      tree_ = tree.tree_
                      feature_name = [
                          feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
                          for i in tree_.feature
                      ]
                      #print "def tree({}):".format(", ".join(feature_names))
              
                      def recurse(node, depth):
                          indent = "      " * depth
                          if tree_.feature[node] != _tree.TREE_UNDEFINED:
                              name = feature_name[node]
                              threshold = tree_.threshold[node]
                              codelines.append ('{}if Xin["{}"] <= {}:\n'.format(indent, name, threshold))
                              recurse(tree_.children_left[node], depth + 1)
                              codelines.append( '{}else:  # if Xin["{}"] > {}\n'.format(indent, name, threshold))
                              recurse(tree_.children_right[node], depth + 1)
                          else:
                              codelines.append( '{}mycat = {}\n'.format(indent, node))
              
                      recurse(0, 1)
                      codelines.append('      catout.append(mycat)\n')
                      codelines.append('   return pd.DataFrame(catout,index=X_tmp.index,columns=["category"])\n')
                      codelines.append('node_ids = get_cat(X)\n')
                      return codelines
                  mycode = tree_to_code(clf,X.columns.values)
              
                  # now execute the function and obtain the dataframe with all nodes
                  exec(''.join(mycode))
                  node_ids = [int(x[0]) for x in node_ids.values]
                  node_ids2 = pd.DataFrame(node_ids)
              
                  print('make plot')
                  import matplotlib.cm as cm
                  colors = cm.rainbow(np.linspace(0, 1, 1+max( list(set(node_ids)))))
                  #plt.figure(figsize=cm2inch(24, 21))
                  for i in list(set(node_ids)):
                      plt.plot(y[node_ids2.values==i],'o',color=colors[i], label=str(i))  
                  mytitle = ['y colored by node']
                  plt.title(mytitle ,fontsize=14)
                  plt.xlabel('my xlabel')
                  plt.ylabel(tagname)
                  plt.xticks(rotation=70)       
                  plt.legend(loc='upper center', bbox_to_anchor=(0.5, 1.00), shadow=True, ncol=9)
                  plt.tight_layout()
                  plt.show()
                  plt.close 
              

              不是最优雅的版本,但它可以完成工作......

              【讨论】:

              • 当您想要返回代码行而不是仅仅打印它们时,这是一个很好的方法。
              【解决方案13】:

              现在您可以使用 export_text。

              from sklearn.tree import export_text
              
              r = export_text(loan_tree, feature_names=(list(X_train.columns)))
              print(r)
              

              来自 [sklearn][1] 的完整示例

              from sklearn.datasets import load_iris
              from sklearn.tree import DecisionTreeClassifier
              from sklearn.tree import export_text
              iris = load_iris()
              X = iris['data']
              y = iris['target']
              decision_tree = DecisionTreeClassifier(random_state=0, max_depth=2)
              decision_tree = decision_tree.fit(X, y)
              r = export_text(decision_tree, feature_names=iris['feature_names'])
              print(r)
              

              【讨论】:

                【解决方案14】:

                这是一个函数,在 python 3 下打印 scikit-learn 决策树的规则,并带有条件块的偏移量以使结构更具可读性:

                def print_decision_tree(tree, feature_names=None, offset_unit='    '):
                    '''Plots textual representation of rules of a decision tree
                    tree: scikit-learn representation of tree
                    feature_names: list of feature names. They are set to f1,f2,f3,... if not specified
                    offset_unit: a string of offset of the conditional block'''
                
                    left      = tree.tree_.children_left
                    right     = tree.tree_.children_right
                    threshold = tree.tree_.threshold
                    value = tree.tree_.value
                    if feature_names is None:
                        features  = ['f%d'%i for i in tree.tree_.feature]
                    else:
                        features  = [feature_names[i] for i in tree.tree_.feature]        
                
                    def recurse(left, right, threshold, features, node, depth=0):
                            offset = offset_unit*depth
                            if (threshold[node] != -2):
                                    print(offset+"if ( " + features[node] + " <= " + str(threshold[node]) + " ) {")
                                    if left[node] != -1:
                                            recurse (left, right, threshold, features,left[node],depth+1)
                                    print(offset+"} else {")
                                    if right[node] != -1:
                                            recurse (left, right, threshold, features,right[node],depth+1)
                                    print(offset+"}")
                            else:
                                    print(offset+"return " + str(value[node]))
                
                    recurse(left, right, threshold, features, 0,0)
                

                【讨论】:

                  【解决方案15】:

                  您还可以通过区分它属于哪个类,甚至通过提及它的输出值来使其更具信息性。

                  def print_decision_tree(tree, feature_names, offset_unit='    '):    
                  left      = tree.tree_.children_left
                  right     = tree.tree_.children_right
                  threshold = tree.tree_.threshold
                  value = tree.tree_.value
                  if feature_names is None:
                      features  = ['f%d'%i for i in tree.tree_.feature]
                  else:
                      features  = [feature_names[i] for i in tree.tree_.feature]        
                  
                  def recurse(left, right, threshold, features, node, depth=0):
                          offset = offset_unit*depth
                          if (threshold[node] != -2):
                                  print(offset+"if ( " + features[node] + " <= " + str(threshold[node]) + " ) {")
                                  if left[node] != -1:
                                          recurse (left, right, threshold, features,left[node],depth+1)
                                  print(offset+"} else {")
                                  if right[node] != -1:
                                          recurse (left, right, threshold, features,right[node],depth+1)
                                  print(offset+"}")
                          else:
                                  #print(offset,value[node]) 
                  
                                  #To remove values from node
                                  temp=str(value[node])
                                  mid=len(temp)//2
                                  tempx=[]
                                  tempy=[]
                                  cnt=0
                                  for i in temp:
                                      if cnt<=mid:
                                          tempx.append(i)
                                          cnt+=1
                                      else:
                                          tempy.append(i)
                                          cnt+=1
                                  val_yes=[]
                                  val_no=[]
                                  res=[]
                                  for j in tempx:
                                      if j=="[" or j=="]" or j=="." or j==" ":
                                          res.append(j)
                                      else:
                                          val_no.append(j)
                                  for j in tempy:
                                      if j=="[" or j=="]" or j=="." or j==" ":
                                          res.append(j)
                                      else:
                                          val_yes.append(j)
                                  val_yes = int("".join(map(str, val_yes)))
                                  val_no = int("".join(map(str, val_no)))
                  
                                  if val_yes>val_no:
                                      print(offset,'\033[1m',"YES")
                                      print('\033[0m')
                                  elif val_no>val_yes:
                                      print(offset,'\033[1m',"NO")
                                      print('\033[0m')
                                  else:
                                      print(offset,'\033[1m',"Tie")
                                      print('\033[0m')
                  
                  recurse(left, right, threshold, features, 0,0)
                  

                  【讨论】:

                    【解决方案16】:

                    这是我提取决策规则的方法,可以直接在sql中使用,因此可以按节点对数据进行分组。 (基于之前海报的方法。)

                    结果将是后续的 CASE 子句,可以复制到 sql 语句中,例如。

                    SELECT COALESCE(*CASE WHEN <conditions> THEN > <NodeA>*, > *CASE WHEN <conditions> THEN <NodeB>*, > ....)NodeName,* > FROM <table or view>


                    import numpy as np
                    
                    import pickle
                    feature_names=.............
                    features  = [feature_names[i] for i in range(len(feature_names))]
                    clf= pickle.loads(trained_model)
                    impurity=clf.tree_.impurity
                    importances = clf.feature_importances_
                    SqlOut=""
                    
                    #global Conts
                    global ContsNode
                    global Path
                    #Conts=[]#
                    ContsNode=[]
                    Path=[]
                    global Results
                    Results=[]
                    
                    def print_decision_tree(tree, feature_names, offset_unit=''    ''):    
                        left      = tree.tree_.children_left
                        right     = tree.tree_.children_right
                        threshold = tree.tree_.threshold
                        value = tree.tree_.value
                    
                        if feature_names is None:
                            features  = [''f%d''%i for i in tree.tree_.feature]
                        else:
                            features  = [feature_names[i] for i in tree.tree_.feature]        
                    
                        def recurse(left, right, threshold, features, node, depth=0,ParentNode=0,IsElse=0):
                            global Conts
                            global ContsNode
                            global Path
                            global Results
                            global LeftParents
                            LeftParents=[]
                            global RightParents
                            RightParents=[]
                            for i in range(len(left)): # This is just to tell you how to create a list.
                                LeftParents.append(-1)
                                RightParents.append(-1)
                                ContsNode.append("")
                                Path.append("")
                    
                    
                            for i in range(len(left)): # i is node
                                if (left[i]==-1 and right[i]==-1):      
                                    if LeftParents[i]>=0:
                                        if Path[LeftParents[i]]>" ":
                                            Path[i]=Path[LeftParents[i]]+" AND " +ContsNode[LeftParents[i]]                                 
                                        else:
                                            Path[i]=ContsNode[LeftParents[i]]                                   
                                    if RightParents[i]>=0:
                                        if Path[RightParents[i]]>" ":
                                            Path[i]=Path[RightParents[i]]+" AND not " +ContsNode[RightParents[i]]                                   
                                        else:
                                            Path[i]=" not " +ContsNode[RightParents[i]]                     
                                    Results.append(" case when  " +Path[i]+"  then ''" +"{:4d}".format(i)+ " "+"{:2.2f}".format(impurity[i])+" "+Path[i][0:180]+"''")
                    
                                else:       
                                    if LeftParents[i]>=0:
                                        if Path[LeftParents[i]]>" ":
                                            Path[i]=Path[LeftParents[i]]+" AND " +ContsNode[LeftParents[i]]                                 
                                        else:
                                            Path[i]=ContsNode[LeftParents[i]]                                   
                                    if RightParents[i]>=0:
                                        if Path[RightParents[i]]>" ":
                                            Path[i]=Path[RightParents[i]]+" AND not " +ContsNode[RightParents[i]]                                   
                                        else:
                                            Path[i]=" not "+ContsNode[RightParents[i]]                      
                                    if (left[i]!=-1):
                                        LeftParents[left[i]]=i
                                    if (right[i]!=-1):
                                        RightParents[right[i]]=i
                                    ContsNode[i]=   "( "+ features[i] + " <= " + str(threshold[i])   + " ) "
                    
                        recurse(left, right, threshold, features, 0,0,0,0)
                    print_decision_tree(clf,features)
                    SqlOut=""
                    for i in range(len(Results)): 
                        SqlOut=SqlOut+Results[i]+ " end,"+chr(13)+chr(10)
                    

                    【讨论】:

                      【解决方案17】:

                      修改了 Zelazny7 的代码以从决策树中获取 SQL。

                      # SQL from decision tree
                      
                      def get_lineage(tree, feature_names):
                           left      = tree.tree_.children_left
                           right     = tree.tree_.children_right
                           threshold = tree.tree_.threshold
                           features  = [feature_names[i] for i in tree.tree_.feature]
                           le='<='               
                           g ='>'
                           # get ids of child nodes
                           idx = np.argwhere(left == -1)[:,0]     
                      
                           def recurse(left, right, child, lineage=None):          
                                if lineage is None:
                                     lineage = [child]
                                if child in left:
                                     parent = np.where(left == child)[0].item()
                                     split = 'l'
                                else:
                                     parent = np.where(right == child)[0].item()
                                     split = 'r'
                                lineage.append((parent, split, threshold[parent], features[parent]))
                                if parent == 0:
                                     lineage.reverse()
                                     return lineage
                                else:
                                     return recurse(left, right, parent, lineage)
                           print 'case '
                           for j,child in enumerate(idx):
                              clause=' when '
                              for node in recurse(left, right, child):
                                  if len(str(node))<3:
                                      continue
                                  i=node
                                  if i[1]=='l':  sign=le 
                                  else: sign=g
                                  clause=clause+i[3]+sign+str(i[2])+' and '
                              clause=clause[:-4]+' then '+str(j)
                              print clause
                           print 'else 99 end as clusters'
                      

                      【讨论】:

                        【解决方案18】:

                        我需要决策树中更人性化的规则格式。我正在构建开源 AutoML Python 包,很多时候 MLJAR 用户希望从树中查看确切的规则。

                        这就是我基于paulkernfeld答案实现一个函数的原因。

                        def get_rules(tree, feature_names, class_names):
                            tree_ = tree.tree_
                            feature_name = [
                                feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
                                for i in tree_.feature
                            ]
                        
                            paths = []
                            path = []
                            
                            def recurse(node, path, paths):
                                
                                if tree_.feature[node] != _tree.TREE_UNDEFINED:
                                    name = feature_name[node]
                                    threshold = tree_.threshold[node]
                                    p1, p2 = list(path), list(path)
                                    p1 += [f"({name} <= {np.round(threshold, 3)})"]
                                    recurse(tree_.children_left[node], p1, paths)
                                    p2 += [f"({name} > {np.round(threshold, 3)})"]
                                    recurse(tree_.children_right[node], p2, paths)
                                else:
                                    path += [(tree_.value[node], tree_.n_node_samples[node])]
                                    paths += [path]
                                    
                            recurse(0, path, paths)
                        
                            # sort by samples count
                            samples_count = [p[-1][1] for p in paths]
                            ii = list(np.argsort(samples_count))
                            paths = [paths[i] for i in reversed(ii)]
                            
                            rules = []
                            for path in paths:
                                rule = "if "
                                
                                for p in path[:-1]:
                                    if rule != "if ":
                                        rule += " and "
                                    rule += str(p)
                                rule += " then "
                                if class_names is None:
                                    rule += "response: "+str(np.round(path[-1][0][0][0],3))
                                else:
                                    classes = path[-1][0][0]
                                    l = np.argmax(classes)
                                    rule += f"class: {class_names[l]} (proba: {np.round(100.0*classes[l]/np.sum(classes),2)}%)"
                                rule += f" | based on {path[-1][1]:,} samples"
                                rules += [rule]
                                
                            return rules
                        

                        规则按分配给每个规则的训练样本数排序。对于每个规则,都有关于分类任务的预测类名称和预测概率的信息。对于回归任务,仅打印有关预测值的信息。

                        示例

                        from sklearn import datasets
                        from sklearn.tree import DecisionTreeRegressor
                        from sklearn import tree
                        
                        # Prepare the data data
                        boston = datasets.load_boston()
                        X = boston.data
                        y = boston.target
                        
                        # Fit the regressor, set max_depth = 3
                        regr = DecisionTreeRegressor(max_depth=3, random_state=1234)
                        model = regr.fit(X, y)
                        
                        # Print rules
                        rules = get_rules(regr, boston.feature_names, None)
                        for r in rules:
                            print(r)
                        

                        打印规则:

                        if (RM <= 6.941) and (LSTAT <= 14.4) and (DIS > 1.385) then response: 22.905 | based on 250 samples
                        if (RM <= 6.941) and (LSTAT > 14.4) and (CRIM <= 6.992) then response: 17.138 | based on 101 samples
                        if (RM <= 6.941) and (LSTAT > 14.4) and (CRIM > 6.992) then response: 11.978 | based on 74 samples
                        if (RM > 6.941) and (RM <= 7.437) and (NOX <= 0.659) then response: 33.349 | based on 43 samples
                        if (RM > 6.941) and (RM > 7.437) and (PTRATIO <= 19.65) then response: 45.897 | based on 29 samples
                        if (RM <= 6.941) and (LSTAT <= 14.4) and (DIS <= 1.385) then response: 45.58 | based on 5 samples
                        if (RM > 6.941) and (RM <= 7.437) and (NOX > 0.659) then response: 14.4 | based on 3 samples
                        if (RM > 6.941) and (RM > 7.437) and (PTRATIO > 19.65) then response: 21.9 | based on 1 samples
                        

                        我在文章中总结了从决策树中提取规则的方法:Extract Rules from Decision Tree in 3 Ways with Scikit-Learn and Python。

                        【讨论】:

                        • 记得导入:from sklearn.tree import _tree
                        • 和“import numpy as np”
                        • @pplonski ...我无法使您的代码适用于 xgboost 而不是 DecisionTreeRegressor。如果您能提供帮助,我将不胜感激,我是一个开始学习 Python 的 MATLAB 人。
                        • @bhamadicharef 它不适用于 xgboost。 xgboost 是树的集合。首先,您需要从 xgboost 中提取选定的树。您需要将其存储为 sklearn-tree 格式,然后您可以使用上面的代码。
                        • @pplonski 我明白你的意思,但对 sklearn-tree 格式还不是很熟悉。如果我有有用的东西,我会分享。我将简单和小的规则解析为 matlab 代码,但我的模型有 3000 棵深度为 6 的树,所以像你这样的健壮且特别递归的方法非常有用。学习...
                        【解决方案19】:

                        显然很久以前有人已经决定尝试在官方scikit的树导出函数中添加以下函数(基本上只支持export_graphviz)

                        def export_dict(tree, feature_names=None, max_depth=None) :
                            """Export a decision tree in dict format.
                        

                        这是他的完整提交:

                        https://github.com/scikit-learn/scikit-learn/blob/79bdc8f711d0af225ed6be9fdb708cea9f98a910/sklearn/tree/export.py

                        不完全确定这条评论发生了什么。但您也可以尝试使用该功能。

                        我认为这需要向 scikit-learn 的优秀人员提出严肃的文档请求,以正确记录 sklearn.tree.Tree API,这是 DecisionTreeClassifier 公开为其属性 tree_ 的底层树结构。

                        【讨论】:

                          【解决方案20】:

                          只需像这样使用 sklearn.tree 中的函数

                          from sklearn.tree import export_graphviz
                              export_graphviz(tree,
                                          out_file = "tree.dot",
                                          feature_names = tree.columns) //or just ["petal length", "petal width"]
                          

                          然后在您的项目文件夹中查找文件 tree.dot,复制所有内容并将其粘贴到此处http://www.webgraphviz.com/ 并生成您的图表 :)

                          【讨论】:

                            【解决方案21】:

                            感谢@paulkerfeld 的精彩解决方案。在他的解决方案之上,对于所有想要序列化版本树的人,只需使用tree.threshold、tree.children_left、tree.children_right、tree.feature 和tree.value。由于叶子没有分裂,因此没有特征名称和子元素,它们在tree.feature 和tree.children_*** 中的占位符是_tree.TREE_UNDEFINED 和_tree.TREE_LEAF。 depth first search 为每个拆分分配一个唯一索引。
                            注意tree.value 的形状是[n, 1, 1]

                            【讨论】:

                              【解决方案22】:

                              这是一个函数,它通过转换export_text 的输出从决策树生成 Python 代码:

                              import string
                              from sklearn.tree import export_text
                              
                              def export_py_code(tree, feature_names, max_depth=100, spacing=4):
                                  if spacing < 2:
                                      raise ValueError('spacing must be > 1')
                              
                                  # Clean up feature names (for correctness)
                                  nums = string.digits
                                  alnums = string.ascii_letters + nums
                                  clean = lambda s: ''.join(c if c in alnums else '_' for c in s)
                                  features = [clean(x) for x in feature_names]
                                  features = ['_'+x if x[0] in nums else x for x in features if x]
                                  if len(set(features)) != len(feature_names):
                                      raise ValueError('invalid feature names')
                              
                                  # First: export tree to text
                                  res = export_text(tree, feature_names=features, 
                                                      max_depth=max_depth,
                                                      decimals=6,
                                                      spacing=spacing-1)
                              
                                  # Second: generate Python code from the text
                                  skip, dash = ' '*spacing, '-'*(spacing-1)
                                  code = 'def decision_tree({}):\n'.format(', '.join(features))
                                  for line in repr(tree).split('\n'):
                                      code += skip + "# " + line + '\n'
                                  for line in res.split('\n'):
                                      line = line.rstrip().replace('|',' ')
                                      if '<' in line or '>' in line:
                                          line, val = line.rsplit(maxsplit=1)
                                          line = line.replace(' ' + dash, 'if')
                                          line = '{} {:g}:'.format(line, float(val))
                                      else:
                                          line = line.replace(' {} class:'.format(dash), 'return')
                                      code += skip + line + '\n'
                              
                                  return code
                              

                              示例用法:

                              res = export_py_code(tree, feature_names=names, spacing=4)
                              print (res)
                              

                              样本输出:

                              def decision_tree(f1, f2, f3):
                                  # DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=3,
                                  #                        max_features=None, max_leaf_nodes=None,
                                  #                        min_impurity_decrease=0.0, min_impurity_split=None,
                                  #                        min_samples_leaf=1, min_samples_split=2,
                                  #                        min_weight_fraction_leaf=0.0, presort=False,
                                  #                        random_state=42, splitter='best')
                                  if f1 <= 12.5:
                                      if f2 <= 17.5:
                                          if f1 <= 10.5:
                                              return 2
                                          if f1 > 10.5:
                                              return 3
                                      if f2 > 17.5:
                                          if f2 <= 22.5:
                                              return 1
                                          if f2 > 22.5:
                                              return 1
                                  if f1 > 12.5:
                                      if f1 <= 17.5:
                                          if f3 <= 23.5:
                                              return 2
                                          if f3 > 23.5:
                                              return 3
                                      if f1 > 17.5:
                                          if f1 <= 25:
                                              return 1
                                          if f1 > 25:
                                              return 2
                              

                              上面的例子是用names = ['f'+str(j+1) for j in range(NUM_FEATURES)]生成的。

                              一个方便的功能是它可以生成更小的文件大小和更小的间距。只需设置spacing=2。

                              【讨论】:

                                【解决方案23】:

                                从这个答案中,您可以获得一个可读且高效的表示:https://stackoverflow.com/a/65939892/3746632

                                输出如下所示。 X 是表示单个实例特征的一维向量。

                                from numba import jit,njit
                                @njit
                                def predict(X):
                                    ret = 0
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                pass
                                        else:  # if w_mexico > 0.5
                                            ret += 1
                                    else:  # if w_pizza > 0.5
                                        pass
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                pass
                                        else:  # if w_mexico > 0.5
                                            pass
                                    else:  # if w_pizza > 0.5
                                        ret += 1
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                ret += 1
                                        else:  # if w_mexico > 0.5
                                            ret += 1
                                    else:  # if w_pizza > 0.5
                                        pass
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                ret += 1
                                        else:  # if w_mexico > 0.5
                                            pass
                                    else:  # if w_pizza > 0.5
                                        ret += 1
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                pass
                                        else:  # if w_mexico > 0.5
                                            pass
                                    else:  # if w_pizza > 0.5
                                        pass
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                pass
                                        else:  # if w_mexico > 0.5
                                            ret += 1
                                    else:  # if w_pizza > 0.5
                                        ret += 1
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                pass
                                        else:  # if w_mexico > 0.5
                                            pass
                                    else:  # if w_pizza > 0.5
                                        ret += 1
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                pass
                                        else:  # if w_mexico > 0.5
                                            pass
                                    else:  # if w_pizza > 0.5
                                        pass
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                pass
                                        else:  # if w_mexico > 0.5
                                            pass
                                    else:  # if w_pizza > 0.5
                                        pass
                                    if X[0] <= 0.5: # if w_pizza <= 0.5
                                        if X[1] <= 0.5: # if w_mexico <= 0.5
                                            if X[2] <= 0.5: # if w_reusable <= 0.5
                                                ret += 1
                                            else:  # if w_reusable > 0.5
                                                pass
                                        else:  # if w_mexico > 0.5
                                            pass
                                    else:  # if w_pizza > 0.5
                                        pass
                                    return ret/10
                                

                                【讨论】:

                                  猜你喜欢
                                  • 2014-06-26
                                  • 2017-02-03
                                  • 2019-12-11
                                  • 2017-02-23
                                  相关资源
                                  最近更新 更多