chainer.functions.black_out

chainer.functions.black_out(x, t, W, samples, reduce='mean')[source]

BlackOut loss function.

BlackOut loss function is defined as

\[-\log(p(t)) - \sum_{s \in S} \log(1 - p(s)),\]

where \(t\) is the correct label, \(S\) is a set of negative examples and \(p(\cdot)\) is likelihood of a given label. And, \(p\) is defined as

\[p(y) = \frac{\exp(W_y^\top x)}{ \sum_{s \in samples} \exp(W_s^\top x)}.\]

The output is a variable whose value depends on the value of the option reduce. If it is 'no', it holds the no loss values. If it is 'mean', this function takes a mean of loss values.

Parameters
  • x (Variable or N-dimensional array) – Batch of input vectors. Its shape should be \((N, D)\).

  • t (Variable or N-dimensional array) – Vector of ground truth labels. Its shape should be \((N,)\). Each elements \(v\) should satisfy \(0 \geq v \geq V\) or \(-1\) where \(V\) is the number of label types.

  • W (Variable or N-dimensional array) – Weight matrix. Its shape should be \((V, D)\)

  • samples (Variable) – Negative samples. Its shape should be \((N, S)\) where \(S\) is the number of negative samples.

  • reduce (str) – Reduction option. Its value must be either 'no' or 'mean'. Otherwise, ValueError is raised.

Returns

A variable object holding loss value(s). If reduce is 'no', the output variable holds an array whose shape is \((N,)\) . If it is 'mean', it holds a scalar.

Return type

Variable

See: BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabularies

See also

BlackOut to manage the model parameter W.