博客

不停止训练根据epoch值更换loss function

以大于50epoch时将binary cross-entropy loss转换为focal loss为例。

1. 代码实现

Net Module

class Net(object):
    def __init__(self,epochs,img_rows,img_cols):
    	# initialize your parameters
        self.epochs = epochs
        self.img_rows = img_rows
        self.img_cols = img_cols
    def network_architecture(self):
    	# define your network architecture
    	inputs = Input(shape=(self.img_rows, self.img_cols),name='data')
    	outputs = Dense(10)
    	model = Model(inputs = [inputs], outputs = [outputs])
    	# initialize current_epoch which will be used in next training loops
    	self.current_epoch = K.variable(value=0)
    	# compile the model
    	model.compile(optimizer = Adam(learning_rate = 1e-4), loss = bce_focal_loss_consequence(change_epoch=50,current_epoch=self.current_epoch,gamma=2.,alpha=.25))
    	return model

Callback Module

class WarmUpCallback(Callback):
    def __init__(self,current_epoch):
        self.current_epoch = current_epoch

    def on_epoch_end(self, epoch, logs=None):
        K.set_value(self.current_epoch, epoch+1) 

Custom Loss Module

def bce_focal_loss_consequence(change_epoch,current_epoch,gamma,alpha):
    def bce_focal(y_true,y_pred):
        bool_case_1 = K.less(current_epoch,change_epoch)
        if bool_case_1:
            loss = binary_crossentropy(y_true,y_pred)
        else:
            loss = binary_focal_loss(gamma,alpha)(y_true,y_pred)
        return loss
    return bce_focal

Fit Module

my_callbacks = [WarmUpCallback(current_epoch=self.current_epoch)]
model.fit(training_generator, epochs=self.epochs,
			validation_data=validation_generator,callbacks=my_callbacks)

2. Loss Curve

如上图所示，如我们预期，Loss在50epoch时由于换了loss function而发生了突变，接着继续训练。

References:

https://github.com/keras-team/keras/issues/2595

https://stackoverflow.com/questions/42787181/variationnal-auto-encoder-implementing-warm-up-in-keras