chainer.functions.simplified_dropconnect¶
-
chainer.functions.
simplified_dropconnect
(x, W, b=None, ratio=0.5, train=True, mask=None, use_batchwise_mask=True)[source]¶ Linear unit regularized by simplified dropconnect.
Simplified dropconnect drops weight matrix elements randomly with probability
ratio
and scales the remaining elements by factor1 / (1 - ratio)
. It accepts two or three arguments: an input minibatchx
, a weight matrixW
, and optionally a bias vectorb
. It computes \(Y = xW^\top + b\).In testing mode, zero will be used as simplified dropconnect ratio instead of
ratio
.Notice: This implementation cannot be used for reproduction of the paper. There is a difference between the current implementation and the original one. The original version uses sampling with gaussian distribution before passing activation function, whereas the current implementation averages before activation.
Parameters: - x (chainer.Variable or
numpy.ndarray
or cupy.ndarray) – Input variable. Its first dimensionn
is assumed to be the minibatch dimension. The other dimensions are treated as concatenated one dimension whose size must beN
. - W (Variable) – Weight variable of shape
(M, N)
. - b (Variable) – Bias variable (optional) of shape
(M,)
. - ratio (float) – Dropconnect ratio.
- train (bool) – If
True
, executes simplified dropconnect. Otherwise, simplified dropconnect function works as a linear function. - mask (None or chainer.Variable or numpy.ndarray or cupy.ndarray) – If
None
, randomized dropconnect mask is generated. Otherwise, The mask must be(n, M, N)
or(M, N)
shaped array, and use_batchwise_mask is ignored. Main purpose of this option is debugging. mask array will be used as a dropconnect mask. - use_batchwise_mask (bool) – If
True
, dropped connections depend on each sample in mini-batch.
Returns: Output variable.
Return type: See also
Dropconnect
See also
Li, W., Matthew Z., Sixin Z., Yann L., Rob F. (2013). Regularization of Neural Network using DropConnect. International Conference on Machine Learning. URL
- x (chainer.Variable or