chainer.functions.batch_renormalization¶
-
chainer.functions.
batch_renormalization
(x, gamma, beta, rmax, dmax, eps=2e-05, running_mean=None, running_var=None, decay=0.9, update_statistics=False)[source]¶ Batch renormalization function.
This is an extension of batch normalization, which ensures that the training and inference models generate the same outputs that depend on individual examples rather than the entire minibatch.
Note
This function does not perform in-place update to
running_mean
andrunning_var
by default, contrary tobatch_normalization()
. If the function is called, it will not be possible to access the updated running mean and variance statistics, because they are members of the function object, which cannot be accessed by the caller. If it is desired to update the running statistics, call the function withupdate_statistics=True
option.Note
For the consistency with Batch Normalization, this function intentionally ignores some of the theoretical flaws in Algorithm 1 of the Batch Renormalization paper:
F.batch_renormalization
maintains the moving average of variances \(\sigma^2\), while the original paper maintains the moving average of standard deviations \(\sigma\).F.batch_renormalization
applies Bessel’s correction to update the moving average of variances.
See: Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models
See also