Boruta特征选择
官方github地址:https://github.com/scikit-learn-contrib/boruta_py?tab=readme-ov-file
论文地址:https://www.jstatsoft.org/article/view/v036i11
官方代码:
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from boruta import BorutaPy
# load X and y
# NOTE BorutaPy accepts numpy arrays only, hence the .values attribute
X = pd.read_csv('examples/test_X.csv', index_col=0).values
y = pd.read_csv('examples/test_y.csv', header=None, index_col=0).values
y = y.ravel()
# define random forest classifier, with utilising all cores and
# sampling in proportion to y labels
rf = RandomForestClassifier(n_jobs=-1, class_weight='balanced', max_depth=5)
# define Boruta feature selection method
feat_selector = BorutaPy(rf, n_estimators='auto', verbose=2, random_state=1)
# find all relevant features - 5 features should be selected
feat_selector.fit(X, y)
# check selected features - first 5 features are selected
feat_selector.support_
# check ranking of features
feat_selector.ranking_
# call transform() on X to filter it down to selected features
X_filtered = feat_selector.transform(X)
在本地运行时出现了问题:AttributeError: module 'numpy' has no attribute 'int'. np.int
was a deprecated alias for the builtin int
.就是numpy的1.20版本以后的都不在支持np.int
,我尝试了降低numpy版本,但是报错wheel出问题了。看了github上的issues很多人都遇到了同样的问题,解决办法就是在调用boruta = BorutaPy(estimator=rf)
前加三行代码:
np.int = np.int32
np.float = np.float64
np.bool = np.bool_
boruta = BorutaPy(estimator=rf)
boruta.fit(x, y)
下面是我修改后以及适配我的需求的代码:
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from boruta import BorutaPy
import numpy as np
file_names_to_add = ['xxx', 'xxxx']
file_path2 = '../xxxx'
for file_name in file_names_to_add:
input_file_path = f"{file_path2}{file_name}.xlsx"
print(input_file_path)
sheet_name_nor = 'xxx'
y_tos = ['xxx', '...']
for y_to in y_tos:
sheet_name_uni = y_to
print(sheet_name_uni)
df = pd.read_excel(input_file_path, sheet_name=sheet_name_nor)
cols_to_pre = ['xxxxxxx', 'xxxxxx','...']
missing_cols = [col for col in cols_to_pre if col not in df.columns]
if missing_cols:
print(f"{missing_cols} not found in the, skipping.")
cols_to_pre = [col for col in cols_to_pre if col in df.columns]
# load X and y
# NOTE BorutaPy accepts numpy arrays only, hence the .values attribute
X = df[cols_to_pre].values
y = df[y_to].values
np.int = np.int32
np.float = np.float64
np.bool = np.bool_
# define random forest classifier, with utilising all cores and
# sampling in proportion to y labels
rf = RandomForestClassifier(n_jobs=-1, class_weight='balanced', max_depth=5)
# define Boruta feature selection method
feat_selector = BorutaPy(rf, n_estimators='auto', verbose=2, random_state=1)
# find all relevant features - 5 features should be selected
feat_selector.fit(X, y)
# # check selected features - first 5 features are selected
# feat_selector.support_
# # check ranking of features
# feat_selector.ranking_
# call transform() on X to filter it down to selected features
# X_filtered = feat_selector.transform(X)
selected_features = [cols_to_pre[i] for i, support in enumerate(feat_selector.support_) if support]
print('Selected features: ', selected_features)
print('Feature ranking: ', feat_selector.ranking_)
标签:features,特征选择,selected,cols,selector,np,feat,Boruta From: https://www.cnblogs.com/benbenlzw/p/18095550因为'feat_selector.support_' 放回的是一个布尔数组,当我们想打印出选出来的特征时直接打印不行,需要通过使用布尔索引来解决这个问题。
selected_features = [cols_to_pre[i] for i, support in enumerate(feat_selector.support_) if support]
上段代码遍历
cols_to_pre
列表,并且只选择feat_selector.support_
中为True
的列。