Overview

之前我们记录了CatBoost一个训练的例子，这次我们更新一个CatBoost调参的例子，用的是业界比较流行的贝叶斯调参法。

1. 引入依赖包并加载数据

import pandas as pd
import numpy as np
from catboost import CatBoostClassifier, CatBoost, Pool, cv
from bayes_opt import BayesianOptimization

data_train = pd.read_csv('data/训练集.csv')
data_val = pd.read_csv('data/验证集.csv')
data_test = pd.read_csv('data/测试集.csv')

2. 加载特征列表并处理数据

name_list = pd.read_csv('特征列表_20190705.txt', header=None, index_col=0)
my_feature_names = list(name_list.transpose())
len(my_feature_names)
    
data_train_X = data_train[my_feature_names]
data_val_X = data_val[my_feature_names]
data_test_X = data_test[my_feature_names]

data_train_y = data_train['label']
data_val_y  = data_val['label']
data_test_y  = data_test['label']

3. 贝叶斯调参

    def cat_train(bagging_temperature, reg_lambda, learning_rate):
        params = {
            'iterations':800,
            'depth':3,
            'bagging_temperature':bagging_temperature,
            'reg_lambda':reg_lambda,
            'learning_rate':learning_rate,
            'loss_function':'Logloss',
            'eval_metric':'AUC',
            'random_seed':696,
            'verbose':30
        }

        model = CatBoost(params)
        # 评价数据集是验证集，评价指标是AUC
        model.fit(data_train_X, data_train_y, eval_set=(data_val_X, data_val_y), plot=False, early_stopping_rounds=20) 
        
        print(params)
        score_max = model.best_score_.get('validation').get('AUC')
        return score_max

    cat_opt = BayesianOptimization(cat_train, 
                               {
                                  'bagging_temperature': (1, 50),  
                                  'reg_lambda': (1, 200),
                                  'learning_rate':(0.05, 0.2)
                                })

    cat_opt.maximize(n_iter=15, init_points=5)

有了最佳参数之后，用这组最佳参数即可训练出最终的模型了。

贝叶斯调参部分，我们参考了如下文章：
Bayesian methods of hyperparameter optimization

Hyperparameter Optimization using bayesian optimization

以及GitHub源码：BayesianOptimization。