Implement the type of data in channel

[chengduoZH]

Currently, the function of Channel of Fluid has been nearly completed. But to use Channel in Fluid, we also need define the data type of Channel.
I think the data type of it should be n-tuple, just like std::tuple. For example, we want to send (image, label) using one channel, rather than using multiple channels.
The following is what I complete ChanDesc, it includes the base-data-type and advanced-data-type.

message MetaData {
  enum DataType {
    BOOL = 1;
    INT16 = 2;
    INT32 = 3;
    INT64 = 4;
    FP16 = 5;
    FP32 = 6;
    FP64 = 7;
    TENSOR = 8;
    LOD_TENSOR = 9;
    SELECTED_ROWS = 10;
    LOD_TENSOR_ARRAY = 11;
    // or others 
  }
  required DataType base_type = 1;
  optional bool data_bool = 2;
  optional int32 data_int32 = 3;
  optional int64 data_int64 = 4;
  optional float data_float = 5;
  optional LoDTensorDesc lod_tensor = 6;
  optional TensorDesc selected_rows = 7;
  optional LoDTensorArrayDesc tensor_array = 8;
}

message ChanEleDesc { repeated MetaData meta_data = 1; }

message ChanDesc {
  repeated ChanEleDesc channel_type = 1;
  required int32 cap = 2 [ default = 0 ];
  required string name = 3;
}

[chengduoZH]

We can replace MetaData with VarDesc, and VarDesc includes ChanDesc.

message ChanEleDesc { repeated VarDesc meta_data = 1; }

message ChanDesc {
  repeated ChanEleDesc channel_type = 1;
  required int32 cap = 2 [ default = 0 ];
  required string name = 3;
}

message VarDesc {
  enum VarType {
    LOD_TENSOR = 1;
    SELECTED_ROWS = 2;
    FEED_MINIBATCH = 3;
    FETCH_LIST = 4;
    STEP_SCOPES = 5;
    LOD_RANK_TABLE = 6;
    LOD_TENSOR_ARRAY = 7;
    PLACE_LIST = 8;
    READER = 9;
    CHAN = 10;
  }
  required string name = 1;
  required VarType type = 2;
  optional bool persistable = 3 [ default = false ];
  optional LoDTensorDesc lod_tensor = 4;
  optional TensorDesc selected_rows = 5;
  optional LoDTensorArrayDesc tensor_array = 6;
  optional ReaderDesc reader = 7;
  optional ChanDesc chan = 8;
}

[helinwang]

Thanks for initiating the discussion!

Maybe rename channel_type to element_type, since it's about the type of each element in the tuple, not the type of channel.

And, what are

  optional bool data_bool = 2;
  optional int32 data_int32 = 3;
  optional int64 data_int64 = 4;
  optional float data_float = 5;

used for? I thought the MetaData is only for describing the channel elements, not for storing the data.

[chengduoZH]

Thanks for your comment!

I thought the MetaData is only for describing the channel elements, not for storing the data.

You are right, this issue is too old. To support the type of Channel, Fluid's type system should be redefined, this PR(#8352) is working for it.