noHup - bayesianoptimization
type: channel
link: http://www.nohup.cc/tag/bayesianoptimization/
items: CatBoost贝叶斯调参程序

CatBoost贝叶斯调参程序
type: item
date: 2019-07-27T14:25:00+00:00
description: Overview1. 引入依赖包并加载数据2. 加载特征列表并处理数据3. 贝叶斯调参 Overview 之前我们记录了CatBoost一个训练的例子，这次我们更新一个CatBoost调参的例子，用的是业界比较流行的贝叶斯调参法。 1. 引入依赖包并加载数据 import pandas as pd import numpy as np from catboost import CatBoostClassifier, CatBoost, Pool, cv from bayes_opt import BayesianOptimization data_train = pd.read_csv('data/训练集.csv') data_val = pd.read_csv('data/验证集.csv') data_test = pd.read_csv('data/测试集.csv') 2. 加载特征列表并处理数据 name_list = pd.read_csv('特征列表_20190705.txt', header=None, index_col=0) my_feature_names = list(name_list.transpose()) len(my_feature_names) data_train_X = data_train[my_feature_names] data_val_X = data_val[my_feature_names] data_test_X = data_test[my_feature_names] data_train_y = data_train['label'] data_val_y = data_val['label'] data_test_y = data_test['label'] 3. 贝叶斯调参 def cat_train(bagging_temperature, reg_lambda, learning_rate): params = { 'iterations':800, 'depth':3, 'bagging_temperature':bagging_temperature, 'reg_lambda':reg_lambda, 'learning_rate':learning_rate, 'loss_function':'Logloss', 'eval_metric':'AUC', 'random_seed':696, 'verbose':30 } model = CatBoost(params) # 评价数据集是验证集，评价指标是AUC model.fit(data_train_X, data_train_y, eval_set=(data_val_X, data_val_y), plot=False, early_stopping_rounds=20) print(params) score_max = model.best_score_.get('validation').get('AUC') return score_max cat_opt = BayesianOptimization(cat_train, { 'bagging_temperature': (1, 50), 'reg_lambda': (1, 200), 'learning_rate':(0.05, 0.2) }) cat_opt.maximize(n_iter=15, init_points=5) 有了最佳参数之后，用这组最佳参数即可训练出最终的模型了。 贝叶斯调参部分，我们参考了如下文章： Bayesian methods of hyperparameter optimization Hyperparameter Optimization using bayesian optimization 以及GitHub源码：BayesianOptimization。