GraphNet样本信息数据库

[lixinqi]

GraphSample :=
	DataBaseRecord
  * $uuid str
  * $repo_type str # github, huggingface
  * $repo_name str # GraphNet, ...
  * $repo_url str # https://github.com/PaddlePaddle/GraphNet, ...
  * $relative_model_path str # indexing
  * $sample_type str # full_graph, typical_graph, fusible_graph, sole_op_graph
  * $num_ops int 
  * $graph_hash str # indexing
  * $order_value int # indexing
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
SubgraphSource :=
	DataBaseRecord
  * $subgraph_uuid str # indexing
  * $full_graph_uuid str # indexing
  * $range_start int
  * $range_end int
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
DimensionGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $order_value int
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime

DataTypeGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $data_type str # float16, bfloat16
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  

[lixinqi]

AI4C的样本需要基于DimensionGeneralizationSource和DataTypeGeneralizationSource这两个表格。具体来说，我们把源自于同一个original_graph_uuid的多个generalized_graph_uuid聚合成一个ai4c样本。

[Honglei-Qiu]

所以DimensionGeneralizationSource这里存的是泛化后的子图，每个泛化后的子图形状shape信息得加上去吧

[Honglei-Qiu]

我子图和DimensionGeneralizationSource应该有一个关联，表示我这个子图的维度泛化信息是什么，他的shape是什么，这个应该指的是子图original_graph_uuid，然后在子图中在对应整图

[Honglei-Qiu]

GraphSample（里面把整图和子图全部放在一起了）下分了sample_type，路径上应该也要体现出来还是拼接路径（这些不同类型的子图应该分不同文件夹）
为什么不能整图子图分开，然后SubgraphSource直接改为子图信息，然后子图整图做一个外键，可以加一个subgraph_range,方便查看

[Honglei-Qiu]

GraphSample这个命名就很有诱惑性，考虑分开SubgraphSample

[Honglei-Qiu]

所以正确查找应该是我查找SubgraphSample信息，然后根据里面的整图uuid可以找到整图，根据对应的维度泛化信息可以找到维度泛化的shape

[Honglei-Qiu]

然后整图里有个List对应着所有他出来的子图的信息

[Honglei-Qiu]

大体流程和四个table

[lixinqi]

现在咱们不涉及流程。
你在这里用ADT描述的你设计方案。

[Honglei-Qiu]

GraphSample :=
	DataBaseRecord
  * $uuid str
  * $repo_type str # github, huggingface
  * $repo_name str # GraphNet, ...
  * $repo_url str # https://github.com/PaddlePaddle/GraphNet, ...
  * $relative_model_path str # indexing
  * $sample_type str # full_graph, typical_graph, fusible_graph, sole_op_graph+后续判断是否添加泛化整图类型
  * $num_ops int 
  * $graph_hash str # indexing
  * $order_value int # indexing
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  * $is_subgraph bool
  * $full_graph_uuid # indexing default=-1
  * $is_fullgraph bool
  * $subgraph_uuid List[]
  * $range_start int
  * $range_end int
  * $generalized_graph_uuid str
  * generalized_graph_uuid str
  
  
DimensionGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $order_value int
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime

DataTypeGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $data_type str # float16, bfloat16
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime

[lixinqi]

Repo :=
	DataBaseRecord
  * $repo_uid str # 不是uuid，当前简单地设置为offical就行
  * $repo_type str # github, huggingface
  * $repo_name str # GraphNet, ...
  * $repo_url str #   

GraphSample :=
	DataBaseRecord
  * $uuid str
  * $repo_uid str
  * $relative_model_path str # indexing
  * $sample_type str # full_graph, typical_graph, fusible_graph, sole_op_graph
  * $num_ops int 
  * $graph_hash str # indexing
  * $order_value int # indexing
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
SubgraphSource :=
	DataBaseRecord
  * $subgraph_uuid str # indexing
  * $full_graph_uuid str # indexing
  * $range_start int
  * $range_end int
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
DimensionGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $order_value int
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime

DataTypeGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $data_type str # float16, bfloat16
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
  

这里把repo信息单独提出来了。

[Honglei-Qiu]

repo没问题
GraphSample可以加一个is_subgraph bool，其他无异议

[lixinqi]

Repo :=
	DataBaseRecord
  * $repo_uid str # 不是uuid
  * $repo_type str # github, huggingface
  * $repo_name str # GraphNet, ...
  * $repo_url str #   

GraphSample :=
	DataBaseRecord
  * $uuid str
  * $repo_uid str
  * $relative_model_path str # indexing
  * $sample_type str # full_graph, typical_graph, fusible_graph, sole_op_graph
  * $is_subgraph bool
  * $num_ops int 
  * $graph_hash str # indexing
  * $order_value int # indexing
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
SubgraphSource :=
	DataBaseRecord
  * $subgraph_uuid str # indexing
  * $full_graph_uuid str # indexing
  * $range_start int
  * $range_end int
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
DimensionGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $order_value int
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime

DataTypeGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $data_type str # float16, bfloat16
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  

[Honglei-Qiu]

在这里DimensionGeneralizationSource特别想添加shape信息，但是是List，其他没问题

[lixinqi]

可以填一个total_element_size吗？取代order_value

[Honglei-Qiu]

这样做不如加一个batch_size 和 size？total对于不同维度会不会相同？batch_size * 3 * size * size
order_value的含义是什么

[lixinqi]

这样做不如加一个batch_size 和 size

不好，不是所有算子都有batch_size语义。

total对于不同维度会不会相同

会相同，但无所谓。只是由于排序

order_value的含义是什么

呈现的时候用于排序。

[Honglei-Qiu]

可以的，呈现的时候group by和order by一起用，支持total_element_size取代order_value

[lixinqi]

Repo :=
	DataBaseRecord
  * $repo_uid str # 不是uuid
  * $repo_type str # github, huggingface
  * $repo_name str # GraphNet, ...
  * $repo_url str #   

GraphSample :=
	DataBaseRecord
  * $uuid str
  * $repo_uid str
  * $relative_model_path str # indexing
  * $sample_type str # full_graph, typical_graph, fusible_graph, sole_op_graph
  * $is_subgraph bool
  * $num_ops int 
  * $graph_hash str # indexing
  * $order_value int # indexing
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
SubgraphSource :=
	DataBaseRecord
  * $subgraph_uuid str # indexing
  * $full_graph_uuid str # indexing
  * $range_start int
  * $range_end int
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  
DimensionGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $total_element_size int64 # indexing
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime

DataTypeGeneralizationSource :=
	DataBaseRecord
  * $generalized_graph_uuid str # indexing
  * $original_graph_uuid str # indexing
  * $data_type str # float16, bfloat16
  * $create_at datatime
  * $deleted bool # default False
  * $delete_at datatime
  

[Honglei-Qiu]

-- SQLite
-- create Repo table
CREATE TABLE Repo (
    repo_uid VARCHAR(255) NOT NULL PRIMARY KEY,
    repo_type VARCHAR(50) NOT NULL,
    repo_name VARCHAR(255) NOT NULL,
    repo_url TEXT
);

-- create GraphSample table
CREATE TABLE GraphSample (
    uuid VARCHAR(255) NOT NULL PRIMARY KEY,
    repo_uid VARCHAR(255) NOT NULL,
    relative_model_path TEXT NOT NULL,
    sample_type VARCHAR(50) NOT NULL,
    is_subgraph BOOLEAN DEFAULT FALSE,
    num_ops INTEGER,
    graph_hash VARCHAR(255) NOT NULL,
    order_value INTEGER,
    create_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    deleted BOOLEAN DEFAULT FALSE,
    delete_at DATETIME,
    FOREIGN KEY (repo_uid) REFERENCES Repo(repo_uid)
);
CREATE INDEX idx_relative_model_path ON GraphSample (relative_model_path);
CREATE INDEX idx_graph_hash ON GraphSample (graph_hash);
CREATE INDEX idx_order_value ON GraphSample (order_value);

-- create SubgraphSource table
CREATE TABLE SubgraphSource (
    subgraph_uuid VARCHAR(255) NOT NULL PRIMARY KEY,
    full_graph_uuid VARCHAR(255) NOT NULL,
    range_start INTEGER NOT NULL,
    range_end INTEGER NOT NULL,
    create_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    deleted BOOLEAN DEFAULT FALSE,
    delete_at DATETIME,
    FOREIGN KEY (full_graph_uuid) REFERENCES GraphSample(uuid)
);
CREATE INDEX idx_subgraph_uuid ON SubgraphSource (subgraph_uuid);
CREATE INDEX idx_full_graph_uuid ON SubgraphSource (full_graph_uuid);

-- create DimensionGeneralizationSource table
CREATE TABLE DimensionGeneralizationSource (
    generalized_graph_uuid VARCHAR(255) NOT NULL,
    original_graph_uuid VARCHAR(255) NOT NULL,
    total_element_size INTEGER NOT NULL,
    create_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    deleted BOOLEAN DEFAULT FALSE,
    delete_at DATETIME,
    FOREIGN KEY (generalized_graph_uuid) REFERENCES GraphSample(uuid),
    FOREIGN KEY (original_graph_uuid) REFERENCES GraphSample(uuid)
);
CREATE INDEX idx_generalized_graph_uuid ON DimensionGeneralizationSource (generalized_graph_uuid);
CREATE INDEX idx_original_graph_uuid ON DimensionGeneralizationSource (original_graph_uuid);
CREATE INDEX idx_total_element_size ON DimensionGeneralizationSource (total_element_size);

-- create DataTypeGeneralizationSource table
CREATE TABLE DataTypeGeneralizationSource (
    generalized_graph_uuid VARCHAR(255) NOT NULL,
    original_graph_uuid VARCHAR(255) NOT NULL,
    data_type VARCHAR(50) NOT NULL,
    create_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    deleted BOOLEAN DEFAULT FALSE,
    delete_at DATETIME,
    FOREIGN KEY (generalized_graph_uuid) REFERENCES GraphSample(uuid),
    FOREIGN KEY (original_graph_uuid) REFERENCES GraphSample(uuid)
);
CREATE INDEX idx_generalized_graph_uuid ON DataTypeGeneralizationSource (generalized_graph_uuid);
CREATE INDEX idx_original_graph_uuid ON DataTypeGeneralizationSource (original_graph_uuid);

$repo_uid str # 不是uuid 这里不是uuid，能否举个例子
$repo_type str # github, huggingface 这个写github还是huggingface
$repo_url 现在repo_url也无具体值
$order_value 是否使用自增
$num_ops int 是否必须

[lixinqi]

$repo_uid str # 不是uuid 这里不是uuid，能否举个例子

github_torch_samples

你暂时在代码里直接写死当前github torch samples对应的repo_uid为github_torch_samples。对应paddle的样本会使用github_paddle_samples。

$repo_type str # github, huggingface 这个写github还是huggingface

github

$repo_url 现在repo_url也无具体值

https://github.com/PaddlePaddle/GraphNet

$order_value 是否使用自增

可以。就顺着python list dir的顺序来。

$num_ops int 是否必须
非必须。默认值为-1

[Honglei-Qiu]

收到

[lixinqi]

schema迁移管理

Migrate :=
	DataBaseRecord
  * $id (AUTO_INCREMENT int)
  * $script_name str # 对应GraphNet/sqlite/migrates/$script_name.py脚本
  * $apply_at datatime
  * $applied bool # default false
  * $create_at datetime
  * $deleted bool

[lixinqi]

GraphNet/sqlite/下一定要有一个init_db.py的脚本，它负责创建数据库和migrate表。

[lixinqi]

MigrateSchema :=
	void
  <- $script_name str
  <- $modify_tables (void <- $script_name str)
  <- $is_applied (bool <- $script_name str)
  <- $update_migrate_table (void <- $script_name str)

[fangfangssj]

反向图关联表，反向图可以和整图，经典子图，可融合子图，单算子子图一一对应，产出对应的样本为前向 + 反向

BackwardGraphSource :=
    DataBaseRecord
  * $forward_graph_uuid str # indexing, GraphSample (Fwd)
  * $backward_graph_uuid str # indexing, GraphSample (Bwd)
  * $original_graph_uuid str # indexing, GraphSample（original）
  * $create_at datetime
  * $deleted bool           # default False
  * $delete_at datetime

[Honglei-Qiu]

收到

[Honglei-Qiu]

需要一些产物，以
samples/timm/regnety_080_tv.tv2_in1k
samples/timm/mobilenetv4_conv_aa_large.e230_r384_in12k
为例子，生成前向图和返向图例子，我插入数据集的时候用来验证