C program that prints operator API reference

[wangkuiyi]

The new PaddlePaddle core calls REGISTER_OP macro to fill in information (input, outputs, attributes) of an operator into a global variable of type OpProto. We need to write a program that links to the libpaddle.so, which contains the OpProto variables, and prints the registered information into JSON format, so could the documentation system reads and renders the JSON.

[wangkuiyi]

Thanks to @reyoung 's suggestion -- it seems that we can write a Python program which depends on libpaddle_pybind.so

[wangkuiyi]

Here the the current generated outputs:

[
{
 "type" : "recurrent",
 "comment" : "This is a recurrent group operator.",
 "inputs" : [ 
 { 
   "name" : "inlinks",
   "comment" : "the inputs that need to be segmented for each step.",
   "duplicable" : 1,
   "intermediate" : 0
 } ,
 { 
   "name" : "boot_memories",
   "comment" : "variables to initialize memories.",
   "duplicable" : 1,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "outlinks",
   "comment" : "the outputs that need to concated for all steps.",
   "duplicable" : 1,
   "intermediate" : 0
 } ,
 { 
   "name" : "step_scopes",
   "comment" : "step scopes",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "pre_memories",
   "type" : "string array",
   "comment" : "names of pre-memories",
   "generated" : 0
 } ,
 { 
   "name" : "memories",
   "type" : "string array",
   "comment" : "names of memories",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "cond",
 "comment" : "\nSample dependent Cond Operator:\nGiven Cond[i] as a 1/0 vector to indicate true/false\nThe equation is:\nOut[i] = subnet_t[i], if Cond[i] == true\nOut[i] = subnet_t[i], if Cond[i] == false\n",
 "inputs" : [ 
 { 
   "name" : "Cond",
   "comment" : "The condition, which is a bool vector",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Xs",
   "comment" : "Inputs of Subnets",
   "duplicable" : 1,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Outs",
   "comment" : "Outputs of Cond_Op after merge",
   "duplicable" : 1,
   "intermediate" : 0
 } ,
 { 
   "name" : "SubScopes",
   "comment" : "sub scopes for true and false branches",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "IndexTensors",
   "comment" : "Index Tensors contains indices for true/false",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "cross_entropy",
 "comment" : "\nCrossEntropy Operator.\n\nIt supports both standard cross-entropy and soft-label cross-entropy loss\ncomputation.\n1) One-hot cross-entropy:\n    soft_label = false, Label[i, 0] indicates the class index for sample i:\n\n                Y[i] = -log(X[i, Label[i]])\n\n2) Soft-label cross-entropy:\n    soft_label = true, Label[i, j] indicates the soft label of class j\n    for sample i:\n\n                Y[i] = \\sum_j{-Label[i, j] * log(X[i, j])}\n\n   Please make sure that in this case the summuation of each row of Label\n   equals one.\n\n3) One-hot cross-entropy with vecterized Input(Label):\n     As a special case of 2), when each row of Input(Label) has only one\n     non-zero element (equals 1), soft-label cross-entropy degenerates to a\n     one-hot cross-entropy with one-hot label representation.\n\nBoth the input `X` and `Label` can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input `X`.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor, default Tensor<float>), a 2-D tensor with shape N x D, where N is the batch size and D is the number of classes. This input is a probability computed by the previous operator, which is almost always the result of a softmax operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Label",
   "comment" : "(Tensor, default Tensor<int>), the ground truth which is a 2-D tensor. When soft_label is set to false, `Label` is a Tensor<int> with shape [N x 1]. When soft_label is set to true, `Label` is a Tensor<float/double> with shape [N x K].",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "(Tensor, default Tensor<float>), a 2-D tensor with shape [N x 1]. The cross entropy loss.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "soft_label",
   "type" : "bool",
   "comment" : "(bool, default false), a flag to indicate whether to interpretate the given labels as soft labels.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "softmax_with_cross_entropy",
 "comment" : "\nCross entropy loss with softmax are used as the output layer extensively. This\noperator computes the softmax normalized values for each row of the input\ntensor, after which cross-entropy loss is then computed. This provides a more\nnumerically stable gradient.\n\nBecause this operators performs a softmax on logits internally, it expects\nunscaled logits. Please do not call this op with the output of softmax operator,\nwhich will produce incorrect results.\n\nThis operators expects mutually exclusive hard labels, each sample in a batch\nis in exactly one class with probabilities 1. Each sample in the batch with one\nand only one label.\n\nEquation:\n\n1) hard label (one-hot label)\n\nLoss_j = -\\text{Logit}_{Label_j} + \\log\\left(\\sum_{i=0}^{K}\\exp(\\text{Logit}_i)\\right), j = 1, ..., K\n\n2) soft label (a distribution over all classes)\n\nLoss_j = -\\sum_{i=0}^{K}\\text{Label}_i\\left(\\text{Logit}_i-\\log\\left(\\sum_{i=0}^{K}\\exp(\\text{Logit}_i)\\right)\\right), j = 1,...,K\n\n",
 "inputs" : [ 
 { 
   "name" : "Logits",
   "comment" : "(Tensor, default: Tensor<float>), The unscaled log probabilities which is a 2-D tensor with shape [N x K]. N is the batch_size, and K is the class number.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Label",
   "comment" : "(Tensor, default: Tensor<int>), The ground truth which is a 2-D tensor. If softLable is set to 0, Label is a Tensor<int> with shape [N x 1]. If softLable is set to 1, Label is a Tensor<float/double> with shape [N x K].",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Softmax",
   "comment" : "(Tensor, default: Tensor<float>), A 2-D tensor with shape [N x K]. The outputs value of softmax activation by given the input batch, which will be used in backward calculation.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "Loss",
   "comment" : "(Tensor, default: Tensor<float>), A 2-D tensor. The cross entropy loss with shape [N x 1].",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "soft_label",
   "type" : "bool",
   "comment" : "(bool, default: false), A flag to indicate whether to interpretate the given labels as soft labels.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "adam",
 "comment" : "\nAdam Updates Operator.\n\nThis implements the Adam optimizer from Section 2 of the Adam\npaper[1]. Adam is a first-order gradient-based optimization\nmethod based on adaptive estimates of lower-order moments.\n\nAdam updates:\n\nmoment1_out = beta1 * moment1 + (1 − beta1) * grad\nmoment2_out = beta2 * moment2 + (1 − beta2) * grad * grad\nlearning_rate_t = learning_rate_t *\n                  sqrt(1 - beta2_pow) / (1 - beta1_pow)\nparam_out = param - learning_rate_t * moment1/ (sqrt(moment2) + epsilon)\n\nReferences:\n  [1] Adam: A Method for Stochastic Optimization\n      (https://arxiv.org/abs/1412.6980)\n\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "(Tensor) Input parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "(Tensor) Input gradient",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "LearningRate",
   "comment" : "(Tensor) Learning rate",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Moment1",
   "comment" : "(Tensor) Input first moment",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Moment2",
   "comment" : "(Tensor) Input second moment",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Beta1Pow",
   "comment" : "(Tensor) Input beta1 power accumulator",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Beta2Pow",
   "comment" : "(Tensor) Input beta2 power accumulator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "(Tensor) Output parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Moment1Out",
   "comment" : "(Tensor) Output first moment",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Moment2Out",
   "comment" : "(Tensor) Output second moment",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "beta1",
   "type" : "float",
   "comment" : "(float, default 0.9) Exponential decay rate for the first moment estimates.",
   "generated" : 0
 } ,
 { 
   "name" : "beta2",
   "type" : "float",
   "comment" : "(float, default 0.999) exponential decay rate for the second moment estimates.",
   "generated" : 0
 } ,
 { 
   "name" : "epsilon",
   "type" : "float",
   "comment" : "(float, default 1.0e-8) Constant for numerical stability",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "rmsprop",
 "comment" : "\n\nRMSprop\n\nMeanSquareOut = decay * MeanSquare + (1 - decay) * Grad * Grad\nMomentOut = momentum * Moment +\n            LearningRate * Grad / sqrt(MeanSquareOut + epsilon)\nParamOut = Param -  MomentOut\n\nThe original slides that proposed RMSprop: Slide 29 of\nhttp://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf)\n\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "(Tensor, default Tensor<float>) Input parameter value that has to be updated",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "MeanSquare",
   "comment" : "(Tensor, default Tensor<float>) The mean square value that gets updated",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "LearningRate",
   "comment" : "(Tensor, default Tensor<float>) The learning rate should be a tensor of size 1",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "(Tensor, default Tensor<float>) Input gradient of the parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Moment",
   "comment" : "(Tensor, default Tensor<float>) The moment that gets updated",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "(Tensor) Output updated parameter value",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "MomentOut",
   "comment" : "(Tensor) Output updated moment",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "MeanSquareOut",
   "comment" : "(Tensor) Output Mean squared updated value",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "epsilon",
   "type" : "float",
   "comment" : "(float, default 1e-10) Constant for numerical stability.",
   "generated" : 0
 } ,
 { 
   "name" : "decay",
   "type" : "float",
   "comment" : "(float, default 0.9) Discounting factor for coming gradient.",
   "generated" : 0
 } ,
 { 
   "name" : "momentum",
   "type" : "float",
   "comment" : "(float, default 0.0) Constant value",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "elementwise_div",
 "comment" : "\nLimited elementwise Div operator.The equation is: Out = Out = X / Y.\n1. The shape of Y should be same with X or\n2. Y's shape is a subset of X.\n   Y will be broadcasted to match the shape of X and axis should be dimension index Y in X.\n\n   example:\n      shape(X) = (2, 3, 4, 5), shape(Y) = (,)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (5,)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1\n      shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input X.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "\nThe first input of elementwise op, it's a tensor of any dimensions.\n",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "\nThe sencond input of elementwise op, it's a tensor and it's dimensions\nmust be small or equal to X's dimensions.\n",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of elementwise op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "axis",
   "type" : "int",
   "comment" : "\nWhen the shape(Y) does not equal the shape(X),Y will be broadcasted\nto match the shape of X and axis should be dimension index Y in X\n        ",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "decayed_adagrad",
 "comment" : "\n\nDecayed Adagrad\n\nmoment_out = decay * moment + (1 - decay) * grad * grad\nparam_out = param - learning_rate * grad / (sqrt(moment_out) + epsilon)\n\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "(Tensor) Input parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "(Tensor) Input gradient",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Moment",
   "comment" : "(Tensor) Second moment",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "LearningRate",
   "comment" : "(Tensor) Learning rate",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "(Tensor) Output parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "MomentOut",
   "comment" : "(Tensor) Output second moment",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "decay",
   "type" : "float",
   "comment" : "(float, default 0.95) Discounting factor for coming gradient",
   "generated" : 0
 } ,
 { 
   "name" : "epsilon",
   "type" : "float",
   "comment" : "(float, default 1.0e-6) Constant for numerical stability",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "accuracy",
 "comment" : "\nAccuracy. It will print accuracy rate for classification.\nThe accuracy is:\n..  math::\naccuracy = \\\\frac{NumOfCorrectPredicts}{NumOfAllSamples})\n\nBoth the input `Inference` and `Label` can carry the LoD (Level of Details)\ninformation, or not. But the output only shares the LoD with input `Inference`.\n",
 "inputs" : [ 
 { 
   "name" : "Inference",
   "comment" : "topk(indices) the network output",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Label",
   "comment" : "Label of the training data",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Accuracy",
   "comment" : "The accuracy of current batch",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "gru_unit",
 "comment" : "\nGRUUnitOp implements part calculations of the GRU unit as following:\n\n\\f[\nupdate \\ gate: u_t = actGate(xu_t + W_u * hidden_prev + bias_u) \\\\\nreset \\ gate: r_t = actGate(xr_t + W_r * hidden_prev + bias_r)  \\\\\noutput \\ candidate: {h}_t = actNode(xc_t + W_c * dot(r_t, hidden_prev) + bias_c) \\\\\noutput: h_t = dot((1-u_t), {h}_t) + dot(u_t, hidden_prev)\n\\f]\n\nThe rest of GRU unit can be completed by using FCOp's output as the input of GRUUnitOp.\n",
 "inputs" : [ 
 { 
   "name" : "Input",
   "comment" : "(Tensor) Matrix with shape [batch_size, frame_size * 3] for the input.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "HiddenPrev",
   "comment" : "(Tensor) Matrix with shape [batch_size, frame_size] for the states of previous time step.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Weight",
   "comment" : "(Tensor) Weight matrix with shape [frame_size, frame_size * 3]. The elements continuous in memory can be divided into two parts. The first part are weights of the update gate and reset gate with shape [frame_size, frame_size * 2], and the second part are weights of output candidate with shape [frame_size, frame_size]",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Bias",
   "comment" : "(Tensor) Bias vector with shape [1, frame_size * 3] concating bias of the update gate, reset gate and output candidate.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Gate",
   "comment" : "(Tensor) Matrix with shape [batch_size, frame_size * 3] for the output of update gate, reset gate and output candidate",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "ResetHiddenPrev",
   "comment" : "(Tensor) Matrix with shape [batch_size, frame_size] for the reseted hidden state of previous time step.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "Hidden",
   "comment" : "(Tensor) The GRU hidden state of the current time step with shape [batch_size, frame_size].",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "activation",
   "type" : "int",
   "comment" : "(enum int, default tanh) The activation type used for output candidate {h}_t.",
   "generated" : 0
 } ,
 { 
   "name" : "gate_activation",
   "type" : "int",
   "comment" : "(enum int, default sigmoid) The activation type used in update gate and reset gate.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "scale",
 "comment" : "Scale operator\n\nThe equation is: Out = scale*X\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input tensor of scale operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output tensor of scale operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "scale",
   "type" : "float",
   "comment" : "The scaling factor of the scale operator.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "conv_shift",
 "comment" : "\nConvShift Operator.\n\nA layer for circular convolution of two vectors,\nas used in the Neural Turing Machine: https://arxiv.org/abs/1410.5401\n\nThe equation is:\n\n  \\f[\n      Out[i] = \\sum_{j=-(N-1)/2}^{(N-1)/2} X_{i+j} * Y_{j}\n  \\f]\n\nwhere X's index is computed modulo M, and b's index is computed modulo N.\n\nBoth of the input `X` and `Y` can carry LoD (Level of Details) information.\nHowever, the output only shares the LoD information with input `X`.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor, default Tensor<float>), a 2-D tensor with shape B x M, where B is the batch size and M is the data dimension.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "(Tensor, default Tensor<float>), a 2-D tensor with shape B x N, where B is the batch size and N is the data dimension. N must be odd.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor, default Tensor<float>), a 2-D tensor with shape B x M, i.e., the same shape as X.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "interp",
 "comment" : "\n    Linear Interpolation with two inputs, used in NEURAL TURING MACHINE.\n\n    Equation:\n      Out.row[i] = X.row[i] * W[i] + Y.row[i] * (1 - W[i])\n                 = (X.row[i] - Y.row[i]) * W[i] + Y.row[i]\n\n    Example:\n      X = [[1,2],[3,4]],\n      Y = [[2,1],[4,3]],\n      W = [0.3, 0.4]\n\n      Then, Out = [[1.7,1.3],[3.6,3.4]]\n\n      where 1.7 = 1*0.3+2*(1-0.3),\n            1.3 = 2*0.3+1*(1-0.3),\n            3.6 = 3*0.4+4*(1-0.4),\n            3.4 = 4*0.4+3*(1-0.4)\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor), 2-D Matrix of shape [batch_size, data_dim]containing data samples, the first input of interp_op",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "(Tensor), 2-D Matrix of shape `[batch_size, data_dim]`containing data samples, the second input of interp_op",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "W",
   "comment" : "(Tensor), 1-D Vector of shape [batch_size],the interpolated values in the half-open interval [0.0, 1.0)",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "SubOut",
   "comment" : "(Tensor), the intermediate subtraction outputs, saving X - Y.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "MulOut",
   "comment" : "(Tensor), the intermediate multiplication outputs,saving the elementwise multiplication of (X - Y) and W.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "Out",
   "comment" : "(Tensor), the output of interp_op, same shape with X,returns the first-dimensional piecewise linear interpolant between X and Y",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "crop",
 "comment" : "\nCrop Operator.\nCrop input into output, as specified by offsets and shape.\n\nThere are two ways to set shape:\n1. referenc input: crop input X as shape as reference input.\n                    The dimension of reference input should\n                    be as same as input X.\n2. shape list: crop input X by shape described by a list<int>.\n               The size of shape list should be as same as\n               dimension size of  input X.\n\nThe input should be a k-D tensor(k > 0 and k < 7). As an example:\n\nGiven:\n\n    X = [[0, 1, 2, 0, 0]\n         [0, 3, 4, 0, 0]\n         [0, 0, 0, 0, 0]]\n\nand\n\n    offsets = [0, 1]\n\nand\n\n    shape = [2, 2]\n\nthen we get\n\n    Out = [[1, 2],\n           [3, 4]]\n\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input of pad op. The input should be a k-D tensor(k > 0 and k < 7)",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "The input used as reference for cropping with the same dimension as X. ",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of crop op with the same dimension as X.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "offsets",
   "type" : "int array",
   "comment" : "A list<int> describing offsets to be cropped.The size of offsets list should be as same as dimension size of  input X.",
   "generated" : 0
 } ,
 { 
   "name" : "shape",
   "type" : "int array",
   "comment" : "A list<int> describing the shape of output.The size of shape list should be as same as dimension size of  input X.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "lstm_unit",
 "comment" : "Lstm-Unit Operator\n\nEquation:\n  i, f, o, j = split(X)\n  C = C_prev * sigm(f + forget_bias) + sigm(i) * tanh(j)\n  H = C * sigm(o)\n\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "FC input before the non-linear activation.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "C_prev",
   "comment" : "The cell state tensor of last time-step in the Lstm Unit operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "C",
   "comment" : "The cell tensor of Lstm Unit operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "H",
   "comment" : "The hidden state tensor of Lstm Unit operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "forget_bias",
   "type" : "float",
   "comment" : "The forget bias of Lstm Unit.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "feed",
 "comment" : "feed op, it should not be configured by users directly",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input of feed op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of feed op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "col",
   "type" : "int",
   "comment" : "column of feed",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "dynamic_recurrent",
 "comment" : "This is a RNN operator for varience-length sequences.",
 "inputs" : [ 
 { 
   "name" : "inlinks",
   "comment" : "the inputs that need to be segmented for each step.",
   "duplicable" : 1,
   "intermediate" : 0
 } ,
 { 
   "name" : "boot_memories",
   "comment" : "variables to initialize memories.",
   "duplicable" : 1,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "outlinks",
   "comment" : "the outputs that need to concated for all steps.",
   "duplicable" : 1,
   "intermediate" : 0
 } ,
 { 
   "name" : "step_scopes",
   "comment" : "step scopes",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "pre_memories",
   "type" : "string array",
   "comment" : "names of pre-memories",
   "generated" : 0
 } ,
 { 
   "name" : "memories",
   "type" : "string array",
   "comment" : "names of memories",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "mean",
 "comment" : " Mean Operator\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input of mean op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of mean op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "transpose",
 "comment" : "\nThe Tensor will be permuted according to the axis values given.\nThe op is very much like the numpy.transpose function in python\nFor example:\n >> input = numpy.arange(6).reshape((2,3))\n >> input\n array([[0, 1, 2],\n        [3, 4, 5]])\n >> axis = [1, 0]\n >> output = input.transpose(axis)\n >> output\n array([[0, 3],\n        [1, 4],\n\t\t[2, 5]])\nSo, given a input tensor of shape(N, C, H, W) and the axis is {0, 2, 3, 1},\nthe output tensor shape will be (N, H, W, C)\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor)The input tensor, tensors with rank at most 6 are supported",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor)The output tensor",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "axis",
   "type" : "int array",
   "comment" : "(vector<int>)a list of values, and the size of the list should be the same with the input tensor rank, the tensor will permute the axes according the the values given",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "pad",
 "comment" : "\nPad input into output, as specified by paddings and pad_value. The input should be a k-D tensor(k > 0 and k < 7). As an example:\n\nGiven:\n\nX = [[1, 2],\n   [3, 4]]\n\nand\n\npaddings = [0, 1, 1, 2]\n\nand\n\npad_value = 0\n\nthen we get\n\nOut = [[0, 1, 2, 0, 0]\n       [0, 3, 4, 0, 0]\n       [0, 0, 0, 0, 0]]\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input of pad op. The input should be a k-D tensor(k > 0 and k < 7)",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of pad op.A tensor with the same shape as X.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "paddings",
   "type" : "int array",
   "comment" : "A list<int> to describes padding rules for each dimension. For 2-D image tensor, paddings=[0, 1, 2, 3] means padding 0 row to top, 1 row to bottom, 2 columns to left and 3 columns to right.Size of paddings should be equal to 2 * dimension size of input tensor.",
   "generated" : 0
 } ,
 { 
   "name" : "pad_value",
   "type" : "float",
   "comment" : "(float) default to 0; The value to fill padded areas.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "adagrad",
 "comment" : "\n\nAdaptive Gradient Algorithm (Adagrad).\n\nmoment_out = moment + grad * grad\nparam_out = param - learning_rate * grad / (sqrt(moment_out) + epsilon)\n\nThe original paper(http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf)\ndoes not have the epsilon attribute. It is added here for numerical stability \nby avoiding division by zero.\n\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "(Tensor) Input parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "(Tensor) Input gradient",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Moment",
   "comment" : "(Tensor) Second moment",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "LearningRate",
   "comment" : "(Tensor) Learning rate",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "(Tensor) Output parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "MomentOut",
   "comment" : "(Tensor) Output second moment",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "epsilon",
   "type" : "float",
   "comment" : "(float, default 1.0e-6) Constant for numerical stability",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "softmax",
 "comment" : "\nThe input of softmax operator is a 2-D tensor with shape N x K (N is the\nbatch_size, K is the dimension of input feature). The output tensor has the\nsame shape as the input tensor.\n\nFor each row of the input tensor, the softmax operator squashes the\nK-dimensional vector of arbitrary real values to a K-dimensional vector of real\nvalues in the range [0, 1] that add up to 1. Specifically, it computes the\nexponential of the given dimension and the sum of exponential values of all\nthe other dimensions in the K-dimensional vector input. Then the ratio of the\nexponential of the given dimension and the sum of exponential values of all\nthe other dimensions is the output of the softmax operator.\n\nFor each row `i` and each column `j` in input X, we have:\n    Y[i, j] = exp(X[i, j]) / sum_j(exp(X[i, j]))\n\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input tensor of softmax. 2-D with shape [batch_size, input_feature_dimensions].",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "The normalized values with the same shape as X.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "stanh",
 "comment" : "STanh activation operator, stanh = b * tanh(a * x)",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of STanh operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of STanh operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "scale_a",
   "type" : "float",
   "comment" : "The scale parameter of a for the input",
   "generated" : 0
 } ,
 { 
   "name" : "scale_b",
   "type" : "float",
   "comment" : "The scale parameter of b for the input",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "pow",
 "comment" : "Pow activation operator, pow(x, factor) = x^factor",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Pow operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Pow operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "factor",
   "type" : "float",
   "comment" : "The exponential factor of Pow",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "relu6",
 "comment" : "Relu6 activation operator, relu6 = min(max(0, x), 6)",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Relu6 operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Relu6 operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "threshold",
   "type" : "float",
   "comment" : "The threshold value of Relu6",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "sequence_concat",
 "comment" : "\n    The sequence_concat operator concatenates multiple LoDTensors. \n    It only supports sequence (LoD Tensor with level number is 1) \n    or a nested sequence (LoD tensor with level number is 2) as its input.\n    - Case1:\n      If the axis is other than 0(here, axis is 1 and level is 1),\n      each input should have the same LoD information and the LoD \n      information of the output keeps the same as the input.\n\n      LoD(x0) = {{0,2,4}, {0,1,2,3,4}}; Dims(x0) = (4,3,4)\n      LoD(x1) = {{0,2,4}, {0,1,2,3,4}}; Dims(x1) = (4,4,4)\n      LoD(Out) = {{0,2,4}, {0,1,2,3,4}}; Dims(Out) = (4,7,4)\n\n    - Case2:\n      If the axis is 0(here, leve is 0), the inputs are concatenated along \n      time steps, the LoD information of the output need to re-compute.\n\n      LoD(x0) = {{0,2,4}, {0,1,2,3,4}}; Dims(x0) = (4,3,4)\n      LoD(x1) = {{0,3,5}, {0,1,2,3,5}}; Dims(x1) = (5,3,4)\n      LoD(Out) = {{0,5,9}, {0,1,2,3,4,5,6,7,9}}; Dims(Out) = (9,3,4)\n\n    - Case3:\n      If the axis is 0(here, level is 1).\n\n      LoD(x0) = {{0,2,4}, {0,1,2,3,4}}; Dims(x0) = (4,3,4)\n      LoD(x1) = {{0,3,5}, {0,1,3,4,5}}; Dims(x1) = (5,3,4)\n      LoD(Out) = {{0,5,9}, {0,2,5,7,9}}; Dims(Out) = (9,3,4)\n      \n    NOTE: The levels of all the inputs should be the same.\n    ",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(A vector of LoDTensor), the input is a vector of LoDTensor, each of which is a variable-length sequence or nested sequence.",
   "duplicable" : 1,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(A LoDTensor), the variable-length output of sequence_concat Op.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "axis",
   "type" : "int",
   "comment" : "(int, default 0)The axis which the inputs will be joined with. If axis is 0, the inputs will be joined with LoD index.",
   "generated" : 0
 } ,
 { 
   "name" : "level",
   "type" : "int",
   "comment" : "(int, default 0)The level at which the inputs will be joined. If the level is 0, the inputs will be joined at the nested sequence level. If the level is 1, the inputs will be joined at the sequence level. The level should be less than the level number of inputs.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "squared_l2_distance",
 "comment" : "\n    SquaredL2DistanceOp will cacluate the squared L2 distance for\n    input and target. Number of distance value equals to the\n    first dimension of input. First dimension of target could be equal to\n    input or to 1. If the first dimension of target is 1, SquaredL2DistanceOp\n    will broadcast target's first dimension to input's first dimension.\n    You can decide whether calculate the gradient of input and target.\n\n    Both the input X and Y can carry the LoD (Level of Details) information,\n    or not. But the output only shares the LoD with input X.\n    ",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of SquaredL2DistanceOp.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "Target of SquaredL2DistanceOp.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "sub_result",
   "comment" : "Buffering substraction result which will be reused in backward.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "Out",
   "comment" : "Squared l2 distance between input and target.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "softsign",
 "comment" : "Softsign activation operator, softsign(x) = x / (1 + |x|)",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Softsign operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Softsign operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "pool2d",
 "comment" : "\nThe pooling2d operation calculates the output based on\nthe input, poolingType and ksize, strides, paddings parameters.\nInput(X) and output(Out) are in NCHW format. Where N is batch size, C is the\nnumber of channels, H and W is the height and width of feature.\nParameters(ksize, strides, paddings) are two elements.\nThese two elements represent height and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       X shape: (N, C, H_in, W_in)\n  Output:\n       Out shape: (N, C, H_out, W_out)\n       Mask shape: (N, C, H_out, W_out)\n  where\n       H_out = (H_in - ksize[0] + 2 * paddings[0]) / strides[0] + 1;\n       W_out = (W_in - ksize[1] + 2 * paddings[1]) / strides[1] + 1;\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of feature.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) The output tensor of pooling operator.The format of output tensor is also NCHW.Where N is batch size, C is the number of channels, H and W is the height and width of feature.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "pooling_type",
   "type" : "string",
   "comment" : "Pooling_type of pooling operator.Str constant equal to 'max' or 'avg'.",
   "generated" : 0
 } ,
 { 
   "name" : "ksize",
   "type" : "int array",
   "comment" : "The pooling window size(height, width) of pooling operator.If global_pooling = true, ksize is ignored and need not be specified.",
   "generated" : 0
 } ,
 { 
   "name" : "global_pooling",
   "type" : "bool",
   "comment" : "Whether to use the global_pooling.Bool constant equal to false or true.Default false.If global_pooling = true, ksize is ignored and need not be specified.",
   "generated" : 0
 } ,
 { 
   "name" : "strides",
   "type" : "int array",
   "comment" : "The strides(height, width) of pooling window.Default {1,1}.",
   "generated" : 0
 } ,
 { 
   "name" : "paddings",
   "type" : "int array",
   "comment" : "The zero padding(height, width) size on both sidesDefault {0,0}.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "square",
 "comment" : "Square activation operator, square(x) = x^2",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Square operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Square operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "uniform_random",
 "comment" : "Uniform random operator.\nUsed to initialize tensor with uniform random generator.\n",
 "inputs" : [ 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output tensor of uniform random op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "dims",
   "type" : "int array",
   "comment" : "the dimension of random tensor",
   "generated" : 0
 } ,
 { 
   "name" : "min",
   "type" : "float",
   "comment" : "Minimum value of uniform random",
   "generated" : 0
 } ,
 { 
   "name" : "max",
   "type" : "float",
   "comment" : "Maximun value of uniform random",
   "generated" : 0
 } ,
 { 
   "name" : "seed",
   "type" : "int",
   "comment" : "Random seed of uniform random. 0 means generate a seed by system",
   "generated" : 0
 } ,
 { 
   "name" : "data_type",
   "type" : "int",
   "comment" : "output tensor data type",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "fill_constant",
 "comment" : "Fill up a variable with specified constant value.",
 "inputs" : [ 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) Tensor of specified shape will be filled with the specified value",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "data_type",
   "type" : "int",
   "comment" : "(int, default 5 (FP32)) Output data type",
   "generated" : 0
 } ,
 { 
   "name" : "shape",
   "type" : "int array",
   "comment" : "(vector<int>) The shape of the output",
   "generated" : 0
 } ,
 { 
   "name" : "value",
   "type" : "float",
   "comment" : "(float, default 0) The value to be filled",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "softplus",
 "comment" : "Softplus activation operator, softplus(x) = log(1 + exp(x))",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Softplus operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Softplus operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "elementwise_mul",
 "comment" : "\nLimited elementwise Mul operator.The equation is: Out = Out = X ⊙ Y.\n1. The shape of Y should be same with X or\n2. Y's shape is a subset of X.\n   Y will be broadcasted to match the shape of X and axis should be dimension index Y in X.\n\n   example:\n      shape(X) = (2, 3, 4, 5), shape(Y) = (,)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (5,)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1\n      shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input X.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "\nThe first input of elementwise op, it's a tensor of any dimensions.\n",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "\nThe sencond input of elementwise op, it's a tensor and it's dimensions\nmust be small or equal to X's dimensions.\n",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of elementwise op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "axis",
   "type" : "int",
   "comment" : "\nWhen the shape(Y) does not equal the shape(X),Y will be broadcasted\nto match the shape of X and axis should be dimension index Y in X\n        ",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "adadelta",
 "comment" : "\nAdadelta Updates Operator.\n\nThis implements the Adadelta optimizer[1]. Adadelta is a per-dimension\nadaptive learning rate method for gradient descent.\n\nAdadelta updates:\n\navg_squared_grad_out = rho * avg_squared_grad + (1 - rho) * grad * grad\nparam_update =  - sqrt((avg_squared_update + epsilon) /\n                       (avg_squared_grad_out + epsilon)) * grad\navg_squared_update_out = rho * avg_squared_update + (1 - rho) * param_update**2\nparam_out = param + param_update\n\nReferences:\n  [1] ADADELTA: An Adaptive Learning Rate Method\n      https://arxiv.org/abs/1212.5701\n\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "(Tensor) Input parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "(Tensor) Input gradient",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "AvgSquaredGrad",
   "comment" : "(Tensor) Input expectation of squared gradient",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "AvgSquaredUpdate",
   "comment" : "(Tensor) Input expectation of squared parameter updates",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "(Tensor) Output parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "AvgSquaredGradOut",
   "comment" : "(Tensor) Output expectation of squared gradient",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "AvgSquaredUpdateOut",
   "comment" : "(Tensor) Output expectation of squared parameter updates",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "rho",
   "type" : "float",
   "comment" : "(float, default 0.95) Exponential decay rate for squared gradients.",
   "generated" : 0
 } ,
 { 
   "name" : "epsilon",
   "type" : "float",
   "comment" : "(float, default 1.0e-6) Constant for numerical stability",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "log",
 "comment" : "Log activation operator, log(x) = natural logarithm of x",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Log operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Log operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "pool3d",
 "comment" : "\nThe pooling3d operation calculates the output based on\nthe input, poolingType and ksize, strides, paddings parameters.\nInput(X) and output(Out) are in NCDHW format. Where N is batch\nsize, C is the number of channels, D, H and W is the depth, height and\nwidth of feature. Parameters(ksize, strides, paddings) are three elements.\nThese three elements represent depth, height and width, respectively.\nThe input(X) size and output(Out) size may be different.\n\nExample:\n  Input:\n       X shape: (N, C, D_in, H_in, W_in)\n  Output:\n       Out shape: (N, C, D_out, H_out, W_out)\n       Mask shape: (N, C, D_out, H_out, W_out)\n  where\n       D_out = (D_in - ksize[0] + 2 * paddings[0]) / strides[0] + 1;\n       H_out = (H_in - ksize[1] + 2 * paddings[1]) / strides[1] + 1;\n       W_out = (W_in - ksize[2] + 2 * paddings[2]) / strides[2] + 1;\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCDHW. Where N is batch size, C is the number of channels, D, H and W is the depth, height and width of feature.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) The output tensor of pooling operator.The format of output tensor is also NCDHW.Where N is batch size, C is the number of channels, D, H and W is the depth, height and width of feature.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "pooling_type",
   "type" : "string",
   "comment" : "PoolingType of pooling operator.Str constant equal to 'max' or 'avg'.",
   "generated" : 0
 } ,
 { 
   "name" : "ksize",
   "type" : "int array",
   "comment" : "The pooling window size(depth, height, width) of pooling operator.If global_pooling = true, ksize is ignored and need not be specified.",
   "generated" : 0
 } ,
 { 
   "name" : "global_pooling",
   "type" : "bool",
   "comment" : "Whether to use the global_pooling.Bool constant equal to false or true.Default false.If global_pooling = true, ksize is ignored and need not be specified.",
   "generated" : 0
 } ,
 { 
   "name" : "strides",
   "type" : "int array",
   "comment" : "Strides(depth, height, width) of pooling operator.Default {1,1,1}.",
   "generated" : 0
 } ,
 { 
   "name" : "paddings",
   "type" : "int array",
   "comment" : "Paddings(depth, height, width) of pooling operator.Default {0,0,0}.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "sgd",
 "comment" : "\n\nSimplest sgd algorithm.\n\nparam_out = param - learning_rate * grad;\n\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "Input parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "LearningRate",
   "comment" : "Learning rate of SGD",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "Input gradient",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "output parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "sum",
 "comment" : "\nSum the input tensors.\n\nAll the inputs can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with the first input.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "the input tensors of sum operator.",
   "duplicable" : 1,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "the output tensor of sum operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "cos_sim",
 "comment" : "\nCosine Similarity Operator.\n\nThe equation is: Out = X^T * Y / (sqrt(X^T * X) * sqrt(Y^T * Y)).\n\nThe input `X` and `Y` must have the same shape, except that the 1st dimension\nof input `Y` could be just 1 (different from input `X`), which will be\nbroadcasted to match the shape of input `X` before computing their cosine\nsimilarity.\n\nBoth the input `X` and `Y` can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input `X`.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The 1st input of cos_sim op.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "The 2nd input of cos_sim op.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of cos_sim op.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "XNorm",
   "comment" : "Norm of the first input, reduced along the 1st dimension.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "YNorm",
   "comment" : "Norm of the second input, reduced along the 1st dimension.",
   "duplicable" : 0,
   "intermediate" : 1
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "dropout",
 "comment" : "\nDropout Operator.\n\n'Dropout' refers to randomly dropping out units in a nerual network. It is a\nregularization technique for reducing overfitting by preventing neuron\nco-adaption during training. The dropout operator randomly set (according to\nthe given dropout probability) the outputs of some units to zero, while others\nbeing set to their inputs.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input of dropout op.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of dropout op.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Mask",
   "comment" : "The random sampled dropout mask.",
   "duplicable" : 0,
   "intermediate" : 1
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "dropout_prob",
   "type" : "float",
   "comment" : "Probability of setting units to zero.",
   "generated" : 0
 } ,
 { 
   "name" : "is_training",
   "type" : "bool",
   "comment" : "Whether in training phase.",
   "generated" : 0
 } ,
 { 
   "name" : "seed",
   "type" : "int",
   "comment" : "Dropout random seed.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "relu",
 "comment" : "Relu activation operator, relu(x) = max(x, 0)",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Relu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Relu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "elementwise_add",
 "comment" : "\nLimited elementwise add operator.The equation is: Out = Out = X + Y.\n1. The shape of Y should be same with X or\n2. Y's shape is a subset of X.\n   Y will be broadcasted to match the shape of X and axis should be dimension index Y in X.\n\n   example:\n      shape(X) = (2, 3, 4, 5), shape(Y) = (,)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (5,)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1\n      shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input X.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "\nThe first input of elementwise op, it's a tensor of any dimensions.\n",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "\nThe sencond input of elementwise op, it's a tensor and it's dimensions\nmust be small or equal to X's dimensions.\n",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of elementwise op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "axis",
   "type" : "int",
   "comment" : "\nWhen the shape(Y) does not equal the shape(X),Y will be broadcasted\nto match the shape of X and axis should be dimension index Y in X\n        ",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "multiplex",
 "comment" : "Multiplex operator\n\nMultiplex multiple tensors according to the index provided by the index tensor.\n\nIds: the index tensor.\nX[0 : N - 1]: the candidate tensors for output (N >= 2).\nFor each index i from 0 to batchSize - 1, the output is the i-th row of the\nthe (Ids[i])-th tensor.\n\nFor i-th row of the output tensor:\n\ny[i] = x_{k}[i]\n\nwhere y is the output tensor. `x_{k}` is the k-th input tensor\nand `k = Ids[i]`.\n",
 "inputs" : [ 
 { 
   "name" : "Ids",
   "comment" : "The index tensor of multiplex operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "X",
   "comment" : "The candidate tensors of multiplex operator.",
   "duplicable" : 1,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output tensor of multiplex operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "max_pool3d_with_index",
 "comment" : "\nThe maxpooling3d with index operation calculates the output and the mask\nbased on the input and ksize, strides, paddings parameters.\nInput(X) and output(Out, Mask) are in NCDHW format. Where N is batch\nsize, C is the number of channels, D, H and W is the depth, height and\nwidth of feature. Parameters(ksize, strides, paddings) are three elements.\nThese three elements represent depth, height and width, respectively.\nThe input(X) size and output(Out, Mask) size may be different.\n\nExample:\n  Input:\n       X shape: (N, C, D_in, H_in, W_in)\n  Output:\n       Out shape: (N, C, D_out, H_out, W_out)\n       Mask shape: (N, C, D_out, H_out, W_out)\n  where\n       D_out = (D_in - ksize[0] + 2 * paddings[0]) / strides[0] + 1;\n       H_out = (H_in - ksize[1] + 2 * paddings[1]) / strides[1] + 1;\n       W_out = (W_in - ksize[2] + 2 * paddings[2]) / strides[2] + 1;\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCDHW. Where N is batch size, C is the number of channels, D, H and W is the depth, height and width of image.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) The output tensor of pooling operator.The format of output tensor is also NCDHW.Where N is batch size, C is the number of channels, D, H and W is the depth, height and width of image.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Mask",
   "comment" : "(Tensor) The Mask tensor of pooling operator.The format of output tensor is also NCDHW.Where N is batch size, C is the number of channels, D, H and W is the depth, height and width of image.The value in it is the index in current feature map",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "ksize",
   "type" : "int array",
   "comment" : "The pooling window size(depth, height, width) of pooling operator.If global_pooling = true, ksize is ignored and need not be specified.",
   "generated" : 0
 } ,
 { 
   "name" : "global_pooling",
   "type" : "bool",
   "comment" : "Whether to use the global_pooling.Bool constant equal to false or true.Default false.If global_pooling = true, ksize is ignored and need not be specified.",
   "generated" : 0
 } ,
 { 
   "name" : "strides",
   "type" : "int array",
   "comment" : "Strides(depth, height, width) of pooling operator.Default {1,1,1}.",
   "generated" : 0
 } ,
 { 
   "name" : "paddings",
   "type" : "int array",
   "comment" : "Paddings(depth, height, width) of pooling operator.Default {0,0,0}.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "reshape",
 "comment" : "Reshape operator\n\nReshape Input(X) into the shape specified by Attr(shape).\n\nAn example:\nGiven a 2-D tensor X with 2 rows and 2 columns\n\n    [[1, 2], [3, 4]]\n\nwith target shape = [1, 4], the reshape operator will transform\nthe tensor X into a 1-D tensor:\n\n    [1, 2, 3, 4]\n\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input tensor of reshape operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output tensor of reshape operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "shape",
   "type" : "int array",
   "comment" : "Target shape of reshape operator.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "leaky_relu",
 "comment" : "LeakyRelu activation operator, leaky_relu = max(x, alpha * x)",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of LeakyRelu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of LeakyRelu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "alpha",
   "type" : "float",
   "comment" : "The small negative slope",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "identity",
 "comment" : "\nThe identity operator is an alias of the scale operator\nwith the attribute scale fixed to 1.0.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input tensor of identity operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "The output tensor of identity operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "logsigmoid",
 "comment" : "Logsigmoid activation operator, logsigmoid = log (1 / (1 + exp(-x)))",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of LogSigmoid operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of LogSigmoid operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "conv_cudnn",
 "comment" : "\nThe convolution operation calculates the output based on the input, filter\nand strides, paddings, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\n",
 "inputs" : [ 
 { 
   "name" : "Input",
   "comment" : "The input tensor of convolution operator. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of image.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Filter",
   "comment" : "The filter tensor of convolution operator.The format of the filter tensor is MCHW, where M is the number of output image channels, C is the number of input image channels, H and W is height and width of filter. If the groups attribute is greater than 1, C equal the number of input image channels divided by the groups.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Output",
   "comment" : "The output tensor of convolution operator.The format of output tensor is also NCHW.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "strides",
   "type" : "int array",
   "comment" : "strides of convolution operator.",
   "generated" : 0
 } ,
 { 
   "name" : "paddings",
   "type" : "int array",
   "comment" : "paddings of convolution operator.",
   "generated" : 0
 } ,
 { 
   "name" : "groups",
   "type" : "int",
   "comment" : "group size of convolution operator. Refer to grouped convolution in Alex Krizhevsky's paper: when group=2, the first half of the filters are only connected to the first half of the input channels, and the second half only connected to the second half.",
   "generated" : 0
 } ,
 { 
   "name" : "dilations",
   "type" : "int array",
   "comment" : "dilations of convolution operator.",
   "generated" : 0
 } ,
 { 
   "name" : "workspace_size_MB",
   "type" : "int",
   "comment" : "workspace size for cudnn, in MB, workspace is a section of GPU memory which will be allocated/freed each time the operator runs, larger workspace size can increase performance but also requires better hardward. This size should be carefully setted.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "proximal_gd",
 "comment" : "\n\nOptimizer that implements the proximal gradient descent algorithm.\n\nprox_param = param - learning_rate * grad\nparam = sign(prox_param) / (1 + learning_rate * l2) *\n        max { |prox_param| - learning_rate * l1 , 0 }\n\nThe paper that proposed Proximal Gradient Descent:\n(http://papers.nips.cc/paper/3793-efficient-learning-using-forward-backward-splitting.pdf)\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "(Tensor, default Tensor<float>) Input parameter value that has to be updated.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "(Tensor, default Tensor<float>) Input gradient of the parameter.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "LearningRate",
   "comment" : "(Tensor, default Tensor<float>) The learning rate should be a tensor of size 1.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "(Tensor) Output updated parameter value.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "l1",
   "type" : "float",
   "comment" : "(float, default 0.0) L1 regularization strength.",
   "generated" : 0
 } ,
 { 
   "name" : "l2",
   "type" : "float",
   "comment" : "(float, default 0.0)L2 regularization strength.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "reduce_max",
 "comment" : "\nReduceMax operator computes the max of input tensor along the given dimension. \nThe result tensor has 1 fewer dimension than the input unless `keep_dim` is true.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input tensor. Tensors with rank at most 6 are supported",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) The result tensor.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "dim",
   "type" : "int",
   "comment" : "(int, default 1) The dimension to reduce. Must be in the range [-rank(input), rank(input)). If `dim < 0`, the dim to reduce is `rank + dim`. Noting that reducing on the first dim will make the LoD info lost.",
   "generated" : 0
 } ,
 { 
   "name" : "keep_dim",
   "type" : "bool",
   "comment" : "(bool, default false) If true, retain the reduced dimension with length 1.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "concat",
 "comment" : "\n            Join the input tensors along with the axis.\n            Examples:\n              Input[0] = [[1,2],[3,4]]\n              Input[1] = [[5,6]]\n              axis = 0\n              Output = [[1,2],\n                        [3,4],\n                        [5,6]]\n        ",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "the input tensors of concat operator.",
   "duplicable" : 1,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "the output tensor of concat operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "axis",
   "type" : "int",
   "comment" : "The axis which the inputs will be joined with.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "abs",
 "comment" : "Abs activation operator, abs(x) = |x|",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Abs operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Abs operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "hard_shrink",
 "comment" : "HardShrink activation operator, hard_shrink(x) = x if x > lambdahard_shrink(x) = x if x < -lambdahard_shrink(x) = 0 otherwise",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of HardShrink operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of HardShrink operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "threshold",
   "type" : "float",
   "comment" : "The value of threshold for HardShrink",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "exp",
 "comment" : "Exp activation operator, exp(x) = e^x",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Exp operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Exp operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "soft_relu",
 "comment" : "SoftRelu activation operator, soft_relu = log(1 + exp(max(min(x, threshold), threshold)))",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of SoftRelu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of SoftRelu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "threshold",
   "type" : "float",
   "comment" : "The threshold value of SoftRelu",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "sigmoid_cross_entropy_with_logits",
 "comment" : "\nSigmoidCrossEntropyWithLogits Operator.\n\nThis measures the elementwise probability error in discrete classification tasks\nin which each class is independent. This can be thought of as predicting labels\nfor a data-point that are not mutually exclusive. For example, a news article\ncan be about politics, technology or sports at the same time or none of these.\n\nThe logistic loss is given as follows:\n\n       loss = -Labels * log(sigmoid(X)) - (1 - Labels) * log(1 - sigmoid(X))\n\nWe know that sigmoid(X) = (1 / (1 + exp(-X))). By substituting this we get\n\n       loss = X - X * Labels + log(1 + exp(-X))\n\nFor stability and to prevent overflow of exp(-X) when X < 0,\nwe can reformulate the loss as follows:\n\n       loss = max(X, 0) - X * Labels + log(1 + exp(-abs(X)))\n\nBoth the input `X` and `Labels` can carry the LoD (Level of Details) information.\nHowever the output only shares the LoD with input `X`.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor, default Tensor<float>), a 2-D tensor with shape N x D, where N is the batch size and D is the number of classes. This input is a tensor of logits computed by the previous  operator. Logits are unscaled log probabilities given as log(p/(1-p)).",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Labels",
   "comment" : "(Tensor, default Tensor<float>), a 2-D tensor of the same type and shape as X. This input is a tensor of probabalistic labels for each logit",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor, default Tensor<float>), a 2-D tensor with shape N x D  of elementwise logistic losses.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "brelu",
 "comment" : "BRelu activation operator, brelu = max(min(x, t_min), t_max)",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of BRelu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of BRelu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "t_min",
   "type" : "float",
   "comment" : "The min marginal value of BRelu",
   "generated" : 0
 } ,
 { 
   "name" : "t_max",
   "type" : "float",
   "comment" : "The max marginal value of BRelu",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "tanh",
 "comment" : "Tanh activation operator, tanh = (exp(x) - exp(-x)) / (exp(x) + exp(-x))",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Tanh operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Tanh operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "gaussian_random",
 "comment" : "\nGaussianRandom operator.\nUse to initialize tensor with gaussian random generator.\n",
 "inputs" : [ 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "output matrix of random op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "dims",
   "type" : "int array",
   "comment" : "The dimension of random tensor.",
   "generated" : 0
 } ,
 { 
   "name" : "mean",
   "type" : "float",
   "comment" : "mean of random tensor.",
   "generated" : 0
 } ,
 { 
   "name" : "std",
   "type" : "float",
   "comment" : "std of random tensor.",
   "generated" : 0
 } ,
 { 
   "name" : "seed",
   "type" : "int",
   "comment" : "Random seed of generator.0 means use system wide seed",
   "generated" : 0
 } ,
 { 
   "name" : "data_type",
   "type" : "int",
   "comment" : "output data type",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "fc",
 "comment" : "\nFully Connected Operator, known as Fully Connected Layer or Inner Product Layer\nin Convolutional Neural Networks. Neurons in a fully connected layer have\nfull connections to all activations in the previous layer.\nIt computes an inner product of a set of\nlearned weights with a matrix multiplication followed by a bias offset\n(optionally).\n\nEquation:\n  Out = Act(sum_n{X_i * W_i} + B)\n\nwhere X_i is Tensor that will be reshaped to a 2-D matrix of size (M x K),\nusually M is the minibatch size and K is the number of input features.\nW_i is a 2-D matrix of size (K x N), where N means the number of neurons\nin the fully connected layer. B is a 1-D vector of size N.\nThus, the output Out is a 2-D matrix of size (M x N).\nActivation type can be set to `identity` (default), `sigmoid` or `softmax`.\n\nAll the inputs can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with first input (`X[0]`).\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(A vector of Tensors) each input Tensor can be of arbitrary dimension, and will be reshaped to a 2-D matrix of size (minibatch, number_of_input_features) according to attribute xNumColDims.",
   "duplicable" : 1,
   "intermediate" : 0
 } ,
 { 
   "name" : "W",
   "comment" : "(A vector of Tensors) the weights of FC operator, a vector of 2-D matrix of size (number_of_input_features, number_of_neurons).",
   "duplicable" : 1,
   "intermediate" : 0
 } ,
 { 
   "name" : "B",
   "comment" : "(Tensor) the bias of FC operator, a 1-D vector of size number_of_neurons.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) the activated output matrix of FC operator, a 2-D matrix of size (minibatch, number_of_neurons).",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "MulOut",
   "comment" : "(A vector of Tensors) the intermediate outputs of FC operator, each Tensor saving the product of X_i * W_i.",
   "duplicable" : 1,
   "intermediate" : 1
 } ,
 { 
   "name" : "SumOut",
   "comment" : "(Tensor) the intermediate output of FC operator, saving the sum of the products of X and W, that is sum{X_i * W_i}.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "AddOut",
   "comment" : "(Tensor) the non-actived output of FC operator, saving sum{X_i * W_i} + B.",
   "duplicable" : 0,
   "intermediate" : 1
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "activation",
   "type" : "string",
   "comment" : "(string, default identity) the activation type of FC operator.",
   "generated" : 0
 } ,
 { 
   "name" : "xNumColDims",
   "type" : "int array",
   "comment" : "(std::vector<int>) The inputs Tensors of FC operator can be of more than 2 dimensions. In that case, each input Tensor `X_i` will be reshaped to a 2-D matrix. The matrix's first dimension (the length of column) will be the product of `X_i`'s last `xNumColDims_i` dimensions, that is `X_i.dims[0] x ... x X_i.dims[xNumColDims_i - 1]`. The matrix's second dimension (the length of row) will be the product of `X_i`'s first `rank - xNumColDims_i` dimensions, that is `X_i.dims[xNumColDims_i] x ... x X_i.dims[rank - 1]`)",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "sigmoid",
 "comment" : "Sigmoid activation operator, sigmoid = 1 / (1 + exp(-x))",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Sigmoid operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Sigmoid operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "conv2d",
 "comment" : "\nThe convolution operation calculates the output based on the input, filter\nand strides, paddings, groups parameters. The size of each dimension of the\nparameters is checked in the infer-shape.\n",
 "inputs" : [ 
 { 
   "name" : "Input",
   "comment" : "The input tensor of convolution operator. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of image.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Filter",
   "comment" : "The filter tensor of convolution operator.The format of the filter tensor is MCHW, where M is the number of output image channels, C is the number of input image channels, H and W is height and width of filter. If the groups attribute is greater than 1, C equal the number of input image channels divided by the groups.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Output",
   "comment" : "The output tensor of convolution operator.The format of output tensor is also NCHW.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "strides",
   "type" : "int array",
   "comment" : "strides of convolution operator.",
   "generated" : 0
 } ,
 { 
   "name" : "paddings",
   "type" : "int array",
   "comment" : "paddings of convolution operator.",
   "generated" : 0
 } ,
 { 
   "name" : "groups",
   "type" : "int",
   "comment" : "group size of convolution operator. Refer to grouped convolution in Alex Krizhevsky's paper: when group=2, the first half of the filters are only connected to the first half of the input channels, and the second half only connected to the second half.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "softshrink",
 "comment" : "Softshrink activation operator, softshrink = x - lambda, if x > lambda; x + lambda, if x < lambda; 0 otherwise",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Softshrink operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Softshrink operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "lambda",
   "type" : "float",
   "comment" : "non-negative offset",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "sqrt",
 "comment" : "Sqrt activation operator, sqrt(x) = x^(1/2)",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Sqrt operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Sqrt operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "split",
 "comment" : "\n      Split the input tensor into multiple sub-tensors.\n      Example:\n        Input = [[1,2],\n                 [3,4],\n                 [5,6]]\n        sections = [2,1]\n        axis = 0\n        Output[0] = [[1,2],\n                     [3,4]]\n        Output[1] = [[5,6]]\n\n    ",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "the input tensor of split operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "the output tensors of split operator.",
   "duplicable" : 1,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "sections",
   "type" : "int array",
   "comment" : "the length for eachoutput along with the specify axis.",
   "generated" : 0
 } ,
 { 
   "name" : "num",
   "type" : "int",
   "comment" : "number of the sub-tensors, it must evenly divide Input.dims()[axis]",
   "generated" : 0
 } ,
 { 
   "name" : "axis",
   "type" : "int",
   "comment" : "The axis which the input will be splited on.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "rank_loss",
 "comment" : "RankLoss operator\n\nRank loss operator for RankNet[1]. RankNet is a pairwise ranking model with\none training sample consisting of a pair of doc A and B, and the label P\nindicating that A is ranked higher than B or not:\n\nP = {0, 1} or {0, 0.5, 1}, where 0.5 means no information about the rank of\nthe input pair.\n\nThe RankLoss operator contains three inputs: Left (o_i), Right (o_j) and Label\n(P_{i,j}), which represent the output of RankNet for two docs and the label\nrespectively, and yields the rank loss C_{i,j} by following the expression\n\n\\f[\n  C_{i,j} = -\\tilde{P_{ij}} * o_{i,j} + log(1 + e^{o_{i,j}}) \\\\\n  o_{i,j} =  o_i - o_j  \\\\\n  \\tilde{P_{i,j}} = \\left \\{0, 0.5, 1 \\right \\} \\ or \\ \\left \\{0, 1 \\right \\}\n\\f]\n\nThe operator can take inputs of one sample or in batch.\n\n[1]. Chris Burges, Tal Shaked, Erin Renshaw, et al. Learning to\n     Rank using Gradient Descent.\n     http://icml.cc/2015/wp-content/uploads/2015/06/icml_ranking.pdf\n",
 "inputs" : [ 
 { 
   "name" : "Label",
   "comment" : "The label indicating A ranked higher than B or not, row vector.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Left",
   "comment" : "The output of RankNet for doc A, vector.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Right",
   "comment" : "The output of RankNet for doc B, vetor",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output loss of RankLoss operator, vector.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "reciprocal",
 "comment" : "Reciprocal activation operator, reciprocal(x) = 1 / x",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of Reciprocal operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of Reciprocal operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "thresholded_relu",
 "comment" : "ThresholdedRelu activation operator, thresholded_relu = x for x > threshold, thresholded_relu = 0 otherwise.",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of ThresholdedRelu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of ThresholdedRelu operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "threshold",
   "type" : "float",
   "comment" : "The threshold location of activation",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "prelu",
 "comment" : "PRelu operator\n\nThe equation is:\n\n  f(x) = alpha * x , for x < 0\n  f(x) = x         , for x >= 0\n\nThe input `X` can carry the LoD (Level of Details) information,\nor not. And the output shares the LoD with input `X`.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input tensor of prelu operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Alpha",
   "comment" : "The alpha weight of PRelu operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output tensor of PRelu operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "fill_zeros_like",
 "comment" : "\nFill up a vriable with zeros.\n\nThe output will have the same size with input.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input of fill-zeros-like op.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "The varibale will be filled up with zeros.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "hard_sigmoid",
 "comment" : "\nHard Sigmoid activation operator.\n\nSegment-wise linear approximation of sigmoid[1].\nThis is much faster than sigmoid.\n\nhard_sigmoid = max(0, min(1, slope * x + shift))\n\nThe slope should be positive. The offset can be either positive or negative.\nThe default slope and shift are set from [1].\nIt is recommended to use the defaults for this activation.\n\nReferences:\n  [1] Noisy Activation Functions\n      (https://arxiv.org/abs/1603.00391)\n\n    ",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of HardSigmoid operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of HardSigmoid operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "slope",
   "type" : "float",
   "comment" : "Slope for linear approximation of sigmoid",
   "generated" : 0
 } ,
 { 
   "name" : "offset",
   "type" : "float",
   "comment" : "Offset for linear approximation of sigmoid",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "lookup_table",
 "comment" : "\nThis operator is used to perform lookups on the parameter W,\nthen concatenated into a dense tensor.\n\nThe input `Ids` can carry the LoD (Level of Details) information,\nor not. And the output only shares the LoD with input `Ids`.\n",
 "inputs" : [ 
 { 
   "name" : "W",
   "comment" : "An input represents embedding tensors, which is a learnable parameter.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Ids",
   "comment" : "An input with type int32 or int64contains the ids to be looked up in W.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The lookup results, which have the same type with W.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "sequence_pool",
 "comment" : "\n    SequencePoolOp pools features of all time-steps of each instance.\n\n    For a mini-batch of 3 variable-length sentences, containing 2, 3, and 2 time-steps:\n\n    Assume X is a [7,M,N] LoDTensor, and X->lod()[0] = [0, 2, 5, 7], 7=2+3+2.\n    Besides, for the sake of simplicity, we assume M=1 and N=1,\n    and the value of X = [[1, 3], [2, 4, 6], [5, 1]].\n\n    Thus, Out is a [3,1,1] Tensor without LoD infomation.\n    And for different strategy, the value of Out is as follows:\n\n    - AVERAGE: [2, 4, 3], where 2=(1+3)/2, 4=(2+4+6)/3, 3=(5+1)/2\n    - SUM: [4, 12, 6], where 4=1+3, 12=2+4+6, 6=5+1\n    - SQRT: [2.82, 6.93, 4.24], where 2.82=(1+3)/sqrt(2),\n           6.93=(2+4+6)/sqrt(3), 4.24=(5+1)/sqrt(2)\n    - MAX: [3, 6, 5], where 3=max(1,3), 6=max(2,4,6), 5=max(5,1)\n    - LAST: [3, 6, 1], where 3=last(1,3), 6=last(2,4,6), 1=last(5,1)\n    - FIRST: [1, 2, 5], where 1=first(1,3), 2=first(2,4,6), 5=first(5,1)\n    ",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(LoDTensor), the variable-length input of SequencePoolOp",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor), output of SequencePoolOp, which does not contain LoD infomation.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "strategy",
   "type" : "int",
   "comment" : "(int, default AVERAGE) the pooling strategy of SequencePoolOp.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "max_pool2d_with_index",
 "comment" : "\nThe maxPooling2d with index operation calculates the output and the mask\nbased on the input and ksize, strides, paddings parameters. Input(X) and\noutput(Out, Mask) are in NCHW format. Where N is batch size, C is the\nnumber of channels, H and W is the height and width of feature.\nParameters(ksize, strides, paddings) are two elements.\nThese two elements represent height and width, respectively.\nThe input(X) size and output(Out, Mask) size may be different.\n\nExample:\n  Input:\n       X shape: (N, C, H_in, W_in)\n  Output:\n       Out shape: (N, C, H_out, W_out)\n       Mask shape: (N, C, H_out, W_out)\n  where\n       H_out = (H_in - ksize[0] + 2 * paddings[0]) / strides[0] + 1;\n       W_out = (W_in - ksize[1] + 2 * paddings[1]) / strides[1] + 1;\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input tensor of pooling operator. The format of input tensor is NCHW. Where N is batch size, C is the number of channels, H and W is the height and width of image.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) The output tensor of pooling operator.The format of output tensor is also NCHW.Where N is batch size, C is the number of channels, H and W is the height and width of image.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Mask",
   "comment" : "(Tensor) The Mask tensor of pooling operator.The format of output tensor is also NCHW.Where N is batch size, C is the number of channels, H and W is the height and width of image.The value in it is the index in current feature map",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "ksize",
   "type" : "int array",
   "comment" : "The pooling window size(height, width) of pooling operator.If global_pooling = true, ksize is ignored and need not be specified.",
   "generated" : 0
 } ,
 { 
   "name" : "global_pooling",
   "type" : "bool",
   "comment" : "Whether to use the global_pooling.Bool constant equal to false or true.Default false.If global_pooling = true, ksize is ignored and need not be specified.",
   "generated" : 0
 } ,
 { 
   "name" : "strides",
   "type" : "int array",
   "comment" : "The strides(height, width) of pooling window.Default {1,1}.",
   "generated" : 0
 } ,
 { 
   "name" : "paddings",
   "type" : "int array",
   "comment" : "The zero padding(height, width) size on both sidesDefault {0,0}.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "adamax",
 "comment" : "\nAdamax Updates Operator.\n\nThis implements the Adamax optimizer from Section 7 of the Adam\npaper[1]. Adamax is a variant of the\nAdam algorithm based on the infinity norm.\n\nAdamax updates:\n\nmoment_out = beta1 * moment + (1 - beta1) * grad\ninf_norm_out = max(beta2 * inf_norm + epsilon, abs(grad))\nlearning_rate_t = learning_rate/(1 - beta1_pow)\nparam_out = param - learning_rate_t * moment_out/inf_norm_out\n\nThe original paper does not have an epsilon attribute.\nHowever, it is added here for numerical stability\nby preventing divide by 0.\n\nReferences:\n  [1] Adam: A Method for Stochastic Optimization\n      (https://arxiv.org/abs/1412.6980)\n\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "(Tensor) Input parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "(Tensor) Input gradient",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "LearningRate",
   "comment" : "(Tensor) Learning rate",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Moment",
   "comment" : "(Tensor) First moment",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "InfNorm",
   "comment" : "(Tensor) Input exponentially weighted infinity norm",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Beta1Pow",
   "comment" : "(Tensor) Input beta1 power accumulator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "(Tensor) Output parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "MomentOut",
   "comment" : "(Tensor) Output first moment",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "InfNormOut",
   "comment" : "(Tensor) Output exponentially weighted infinity norm",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "beta1",
   "type" : "float",
   "comment" : "(float, default 0.9) Exponential decay rate for the 1st moment estimates.",
   "generated" : 0
 } ,
 { 
   "name" : "beta2",
   "type" : "float",
   "comment" : "(float, default 0.999) exponential decay rate for the weighted infinity norm estimates.",
   "generated" : 0
 } ,
 { 
   "name" : "epsilon",
   "type" : "float",
   "comment" : "(float, default 1.0e-8) Constant for numerical stability",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "tanh_shrink",
 "comment" : "TanhShrink activation operator, tanhshrink(x) = x - tanh(x)",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "Input of TanhShrink operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "Output of TanhShrink operator",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "reduce_sum",
 "comment" : "\nReduceSum operator computes the sum of input tensor along the given dimension. \nThe result tensor has 1 fewer dimension than the input unless `keep_dim` is true.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input tensor. Tensors with rank at most 6 are supported",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) The result tensor.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "dim",
   "type" : "int",
   "comment" : "(int, default 1) The dimension to reduce. Must be in the range [-rank(input), rank(input)). If `dim < 0`, the dim to reduce is `rank + dim`. Noting that reducing on the first dim will make the LoD info lost.",
   "generated" : 0
 } ,
 { 
   "name" : "keep_dim",
   "type" : "bool",
   "comment" : "(bool, default false) If true, retain the reduced dimension with length 1.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "reduce_mean",
 "comment" : "\nReduceMean operator computes the mean of input tensor along the given dimension. \nThe result tensor has 1 fewer dimension than the input unless `keep_dim` is true.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input tensor. Tensors with rank at most 6 are supported",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) The result tensor.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "dim",
   "type" : "int",
   "comment" : "(int, default 1) The dimension to reduce. Must be in the range [-rank(input), rank(input)). If `dim < 0`, the dim to reduce is `rank + dim`. Noting that reducing on the first dim will make the LoD info lost.",
   "generated" : 0
 } ,
 { 
   "name" : "keep_dim",
   "type" : "bool",
   "comment" : "(bool, default false) If true, retain the reduced dimension with length 1.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "fetch",
 "comment" : "fetch op, it should not be configured by users directly",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input of fetch op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of fetch op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "col",
   "type" : "int",
   "comment" : "column of fetch",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "clip",
 "comment" : "\nClip operator limits the given input within an interval. The interval is\nspecified with arguments 'min' and 'max'.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor)The input of clip op.The number of dimensions must be between [1, 9].",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor)The output of clip op with shape as input(X)",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "min",
   "type" : "float",
   "comment" : "(float)Minimum value, under which element is replaced by min.",
   "generated" : 0
 } ,
 { 
   "name" : "max",
   "type" : "float",
   "comment" : "(float)Maximum value, above which element is replaced by max",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "matmul",
 "comment" : "\nThe MatMul operator is used to perform (batched) matrix multiplication\nover the last two dimensions of the input tensors `X` and `Y`.\n\nIf a transpose flag is specified, the last two dimensions of the\ntensor are transposed. If the tensor is rank-1 of shape [D], then\nfor `X` it is treated as [1, D] in nontransposed form and as [D, 1]\nin transposed form, whereas for `Y` it is the opposite: It is treated\nas [D, 1] in nontransposed form and as [1, D] in transposed form.\n\nExamples without transpose:\n- X: [K], Y: [K] => Out: [1]\n- X: [K], Y: [K, N] => Out: [N]\n- X: [B, M, K], Y: [K] => Out: [B, M]\n- X: [M, K], Y: [B, K, N] => Out: [B, M, N]\n- X: [B, M, K], Y: [B, K, N] => Out: [B, M, N]\n\nThe behavior is designed to be similar to the `numpy.matmul` function.\nThe differences are:\n- Currently only rank 1 to rank 3 input tensors are supported.\n- We add `transpose_X` and `transpose_Y` flags.\n\nBoth the input `X` and `Y` can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input `X`.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The first input of MatMul op",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "The second input of MatMul op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of MatMul op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "transpose_X",
   "type" : "bool",
   "comment" : "If true, use the transpose of `X`.\n        ",
   "generated" : 0
 } ,
 { 
   "name" : "transpose_Y",
   "type" : "bool",
   "comment" : "If true, use the transpose of `Y`.\n        ",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "minus",
 "comment" : "Minus Operator\n\nEquation:\n\n    Out = X - Y\n\nBoth the input `X` and `Y` can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input `X`.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The left tensor of minus operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "The right tensor of minus operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output tensor of minus operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "scatter",
 "comment" : "\nScatter Operator by selecting from the first axis,\n\nOut = Ref\nOut[Index] = Ref[Index] + Updates\n",
 "inputs" : [ 
 { 
   "name" : "Ref",
   "comment" : "The source input of scatter op",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Index",
   "comment" : "The index input of scatter op where Ref will be updated",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Updates",
   "comment" : "The updated value of updates op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of add op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "momentum",
 "comment" : "\n\nMomentum Algorithm (momentum).\n\nvelocity = mu * velocity + gradient\nparam = param - learning_rate * velocity\n\n",
 "inputs" : [ 
 { 
   "name" : "Param",
   "comment" : "(Tensor, default Tensor<float>) Input parameter that has to be updated",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Grad",
   "comment" : "(Tensor, default Tensor<float>) Input gradient of the parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Velocity",
   "comment" : "(Tensor, default Tensor<float>) Input velocity (corresponding to the parameter) that has to be updated",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "LearningRate",
   "comment" : "(Tensor, default Tensor<float>) Input learning rate",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "ParamOut",
   "comment" : "(Tensor) Output updated parameter",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "VelocityOut",
   "comment" : "(Tensor) Output updated velocity",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "mu",
   "type" : "float",
   "comment" : "(float) Momentum coefficient",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "margin_rank_loss",
 "comment" : "\n\nMarginRankLoss operator measures the loss given a pair of training sample\n{`X1`, `X2`} and the `Label` with attribute `margin`, where `Label = +1` \nindicating X1 is ranked higher than `X2`, otherwise `Label = -1`. The loss \nturns out\n\nloss(X1, X2, Label) = max(0, -Label * (X1 - X2) + margin).\n\nThe attribute `margin` involved here helps make the predictions more robust.\nDenote the item ranked higher as the positive sample, otherwise the negative \nsample. If the score of the two samples satisfies \n\npositive sample - negative sample < margin,\n\nthe pair of samples will contribute to the final loss, which will backpropogate \nand train the ranking model to enlarge the difference of the two score.\n\nFor batch input with size `batch_size`, `X1`, `X2` and `Label`\nall have the same shape [batch_size x 1].\n\n",
 "inputs" : [ 
 { 
   "name" : "X1",
   "comment" : "(2-D tensor with shape [batch_size x 1]) The score for one item X1 to be ranked, from pairwise ranking model.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "X2",
   "comment" : "(2-D tensor with shape [batch_size x 1]) The score for another item X2 to be ranked, from pairwise ranking model.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Label",
   "comment" : "(2-D tensor with shape [batch_size x 1]) The label indicating X1 ranked higher than X2 or not, can only be +1 or -1.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Activated",
   "comment" : "(2-D tensor with shape [batch_size x 1]) Intermediate tensor to indicate whether each element of Output(Out) is activated.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "Out",
   "comment" : "(2-D tensor with shape [batch_size x 1]) The output loss of MarginRankLoss operator.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "margin",
   "type" : "float",
   "comment" : "(scalar, default 0) Margin for MarginRankLossOp.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "mul",
 "comment" : "\nMul operator is used to perform matrix multiplication for input X and Y.\n\nThe equation is:\n\n    Out = X * Y\n\nBoth the input `X` and `Y` can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input `X`.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The first input of mul op",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "The second input of mul op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of mul op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "x_num_col_dims",
   "type" : "int",
   "comment" : "mul_op can take tensors with more than two dimensions as input `X`,\n            in that case, tensors will be reshaped to a matrix. The matrix's first\n            dimension(column length) will be the product of tensor's last\n            `num_col_dims` dimensions, and the matrix's second dimension(row length)\n            will be the product of tensor's first `rank - num_col_dims` dimensions.\n        ",
   "generated" : 0
 } ,
 { 
   "name" : "y_num_col_dims",
   "type" : "int",
   "comment" : "mul_op can take tensors with more than two dimensions as input `Y`,\n             in that case, tensors will be reshaped to a matrix. Just like input `X`.\n        ",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "elu",
 "comment" : "\n        ELU activation operator. It applies this element-wise computation on\n        the input: f(x) = max(0, x) + min(0, alpha * (exp(x) - 1)).\n        Check .. _Link: https://arxiv.org/abs/1511.07289 for more details.",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input of ELU operator, it shouldn't be empty. Input is flattened and treated as a 1D array.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Y",
   "comment" : "(Tensor) The output of ELU operator. It has the same shape as the input.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "alpha",
   "type" : "float",
   "comment" : "(float, default 1.0) Alpha value in the elu formulation.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "sequence_softmax",
 "comment" : "\nSequenceSoftmaxOp computes softmax activation among all time-steps for each\nsequence. The dimension of each time-step should be 1. Thus, the shape of\ninput Tensor can be either [N, 1] or [N], where N is the sum of all sequences'\nlengths.\n\nEquation:\n    for i-th sequence in a mini-batch:\n        Out(X[lod[i]:lod[i+1]], :) =\n            exp(X[lod[i]:lod[i+1], :]) / sum(exp(X[lod[i]:lod[i+1], :]))\n\nFor example, for a mini-batch of 3 sequences with variable-length,\neach containing 2, 3, 2 time-steps, the lod of which is [0, 2, 5, 7],\nthen softmax will be computed among X[0:2, :], X[2:5, :], X[5:7, :]\nand N turns out to be 7.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(LoDTensor) 1-D or 2-D input LoDTensor with the 2-nd dimension of length 1.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(LoDTensor) 1-D or 2-D output LoDTensor with the 2-nd dimension of length 1.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "elementwise_sub",
 "comment" : "\nLimited elementwise Sub operator.The equation is: Out = Out = X - Y.\n1. The shape of Y should be same with X or\n2. Y's shape is a subset of X.\n   Y will be broadcasted to match the shape of X and axis should be dimension index Y in X.\n\n   example:\n      shape(X) = (2, 3, 4, 5), shape(Y) = (,)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (5,)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (4, 5)\n      shape(X) = (2, 3, 4, 5), shape(Y) = (3, 4), with axis=1\n      shape(X) = (2, 3, 4, 5), shape(Y) = (2), with axis=0\n\nBoth the input X and Y can carry the LoD (Level of Details) information,\nor not. But the output only shares the LoD with input X.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "\nThe first input of elementwise op, it's a tensor of any dimensions.\n",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "\nThe sencond input of elementwise op, it's a tensor and it's dimensions\nmust be small or equal to X's dimensions.\n",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of elementwise op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "axis",
   "type" : "int",
   "comment" : "\nWhen the shape(Y) does not equal the shape(X),Y will be broadcasted\nto match the shape of X and axis should be dimension index Y in X\n        ",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "top_k",
 "comment" : "If the input is a vector (1d tensor), finds the k largest entries in the vector and outputs their values and indices as vectors. Thus values[j] is the j-th largest entry in input, and its index is indices[j].\n\n    For matrices, computes the top k entries in each row. ",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input of Topk op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output tensor of Topk op",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Indices",
   "comment" : "The indices of Topk elements of input",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "k",
   "type" : "int",
   "comment" : "Number of top elements to look for along the last dimension (along each row for matrices).",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "modified_huber_loss",
 "comment" : "\nModified huber loss is used in binary classification problem. The shape of\ninput X and target Y are both [N, 1] and so is the shape of output loss.\nSince target Y is not differentiable, cacluating gradient for Y is illegal.\nThe formulation of modified huber loss is:\n\nL(y, f(x)) = max(0, 1 - yf(x))^2  for yf(x) >= -1,\n             -4yf(x)              otherwise.\n\nMake sure the values of target label Y are in {0, 1} here. The operator will\nscale values of Y to {-1, +1} when computing losses and gradients.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input tensor of modified huber loss op.X is 2-D tensor with shape [batch_size, 1].",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "The target labels of modified huber loss op.The shape of Y is same as X. Values of Y must be 0 or 1.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "IntermediateVal",
   "comment" : "Variable to save intermediate result which will be reused in backward processing.",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "Out",
   "comment" : "Classification loss for X.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "smooth_l1_loss",
 "comment" : "\nCompute smooth l1 loss for input and target. The operator take the 1st\ndimension of input as batch size. For each instance, it will compute\nsmooth l1 loss element by element first and sum all losses to one value.\nSo the output shape is [batch_size, 1].\n\nThe equation is:\nloss = 0.5 * (sigma * (x-y))^2    if abs(x - y) < 1 / sigma^2\n       abs(x - y) - 0.5 / sigma^2 otherwise\n\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The input tensor of smooth l1 loss op.The rank should be greater or equal to 2 with shape [batch_size, value_dim1, value_dim2, ..., value_dimN]",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Y",
   "comment" : "The target tensor of smooth l1 loss op with the same shape as X.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "InsideWeight",
   "comment" : "Optional input tensor of smooth l1 loss op with the same shape as X. If provided, the result of (X - Y) will be multiplied by this tensor element by element.",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "OutsideWeight",
   "comment" : "Optinal input of smooth l1 loss op with the same shape as X.If provided, the output smooth l1 loss will be multiplied by this tensor element by element.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Diff",
   "comment" : "Intermediate variable to cache InsideWeight*(X-Y).",
   "duplicable" : 0,
   "intermediate" : 1
 } ,
 { 
   "name" : "Out",
   "comment" : "Smooth l1 loss.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "sigma",
   "type" : "float",
   "comment" : "Hyper parameter of smooth l1 loss op.A float scalar with default value 3.0.",
   "generated" : 0
 } 
 ] 
},
{
 "type" : "gather",
 "comment" : "\nGather Operator by selecting from the first axis,\n\nOut = X[Index]\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "The source input of gather op",
   "duplicable" : 0,
   "intermediate" : 0
 } ,
 { 
   "name" : "Index",
   "comment" : "The index input of gather op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "The output of add op",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 ] 
},
{
 "type" : "reduce_min",
 "comment" : "\nReduceMin operator computes the min of input tensor along the given dimension. \nThe result tensor has 1 fewer dimension than the input unless `keep_dim` is true.\n",
 "inputs" : [ 
 { 
   "name" : "X",
   "comment" : "(Tensor) The input tensor. Tensors with rank at most 6 are supported",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "outputs" : [ 
 { 
   "name" : "Out",
   "comment" : "(Tensor) The result tensor.",
   "duplicable" : 0,
   "intermediate" : 0
 } 
 ], 
 "attrs" : [ 
 { 
   "name" : "dim",
   "type" : "int",
   "comment" : "(int, default 1) The dimension to reduce. Must be in the range [-rank(input), rank(input)). If `dim < 0`, the dim to reduce is `rank + dim`. Noting that reducing on the first dim will make the LoD info lost.",
   "generated" : 0
 } ,
 { 
   "name" : "keep_dim",
   "type" : "bool",
   "comment" : "(bool, default false) If true, retain the reduced dimension with length 1.",
   "generated" : 0
 } 
 ] 
}
]