RNNCell

Versioned name : RNNCell-3

Category : Sequence processing

Short description : RNNCell represents a single RNN cell that computes the output using the formula described in the article.

Detailed description :

RNNCell represents a single RNN cell and is part of RNNSequence operation.

Formula:
  \*  - matrix multiplication
  ^T - matrix transpose
  f  - activation function
    Ht = f(Xt\*(Wi^T) + Ht-1\*(Ri^T) + Wbi + Rbi)

Attributes

• hidden_size

  • Description : hidden_size specifies hidden state size.

  • Range of values : a positive integer

  • Type : int

  • Required : yes

• activations

  • Description : activation functions for gates

  • Range of values : any combination of relu, sigmoid, tanh

  • Type : a list of strings

  • Default value : tanh

  • Required : no

• activations_alpha, activations_beta

  • Description : activations_alpha, activations_beta functions attributes

  • Range of values : a list of floating-point numbers

  • Type : float[]

  • Default value : None

  • Required : no

• clip

  • Description : clip specifies value for tensor clipping to be in [-C, C] before activations

  • Range of values : a positive floating-point number

  • Type : float

  • Default value : infinity that means that the clipping is not applied

  • Required : no

Inputs

• 1 : X - 2D tensor of type T [batch_size, input_size], input data. Required.

• 2 : H - 2D tensor of type T [batch_size, hidden_size], initial hidden state. Required.

• 3 : W - 2D tensor of type T [hidden_size, input_size], the weights for matrix multiplication. Required.

• 4 : R - 2D tensor of type T [hidden_size, hidden_size], the recurrence weights for matrix multiplication. Required.

• 5 : B 1D tensor of type T [hidden_size], the sum of biases (weights and recurrence weights). Required.

Outputs

• 1 : Ho - 2D tensor of type T [batch_size, hidden_size], the last output value of hidden state.

Types

• T : any supported floating-point type.

Example

<layer ... type="RNNCell" ...>
    <data hidden_size="128"/>
    <input>
        <port id="0">
            <dim>1</dim>
            <dim>16</dim>
        </port>
        <port id="1">
            <dim>1</dim>
            <dim>128</dim>
        </port>
         <port id="2">
            <dim>128</dim>
            <dim>16</dim>
        </port>
         <port id="3">
            <dim>128</dim>
            <dim>128</dim>
        </port>
         <port id="4">
            <dim>128</dim>
        </port>
    </input>
    <output>
        <port id="5">
            <dim>1</dim>
            <dim>128</dim>
        </port>
    </output>
</layer>