Skip to content

Conversation

@zhiboniu
Copy link
Contributor

@zhiboniu zhiboniu commented Jun 16, 2021

PR types

New features

PR changes

Others

Describe

add new api ci check file

背景:
API相关CI检查的更新和完善
目前发现到的问题:
这样的代码也合入到了2.1版本中:https://github.com/PaddlePaddle/Paddle/blob/release/2.1/python/paddle/device.py#L28

为了防止带有错误字段的pr合入,更新ci递归遍历所有paddle下模块,依次检查每一项功能的可访问性,找出不具有可访问性的功能。

本pr增加了对paddle模块下的各级子模块的可访问性检查

新的检查机制能够发现的问题,例如:
__all__里少了逗号导致两个字符拼接到一起了。
或者仍然保留的无效的api接口

特殊说明:
skiplist = [
'paddle.vision.datasets.DatasetFolderImageFolder', 'paddle.truncdigamma'
]
这两个问题先加入skip列表暂时跳过,本次修复后,下次pr就可以清空skip列表了。否则旧版paddle会导致ci错误无法通过。

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

只有这里调用format_spec函数,是否上面的 format_spec 和 get_functools_partial_spec 函数也可以去掉了

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@wadefelix wadefelix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可验证下和get_all_api的结果是否有差异

ErrorSet = set()
IdSet = set()
skiplist = [
'paddle.vision.datasets.DatasetFolderImageFolder', 'paddle.truncdigamma'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

此二不已经在__init__.py予以更正了么,就不需要保留此list了

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

明白了。否则在对旧版执行这段程序都会出错

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

print_signatures.py会在本次pr合入前的paddle上也跑一遍,旧版paddle还没有更正,会导致ci失败。这里的skiplist主要为了在旧版上先跳过这两个问题。

IdSet.add(instance_id)
member_dict[cur_name] = "({}, ('document', '{}'))".format(
cur_name, doc_md5)
if hasattr(instance,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

前面L113 if instance_id in IdSet已对alias做出判别了。此处的逻辑可去除

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@wadefelix
Copy link
Contributor

wadefelix commented Jun 17, 2021

sampcd_processor.py文件在--full-test状态时,会调用本文件的get_all_api来搜索全部api。计划如此实施。
当前使用get_all_api_from_modulelist大约它能找出八百多api出来。
image

使用get_all_api则能找出一千多个
image

print(
"Found alias API, alias name is: {}, original name is: {}".
format(member_name, instance.__name__),
file=sys.stderr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

此py文件已引入了loggging日志记录,可用logger.warn 等输出过程日志

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

此处已经移除

@zhiboniu
Copy link
Contributor Author

sampcd_processor.py文件在--full-test状态时,会调用本文件的get_all_api来搜索全部api。计划如此实施。
当前使用get_all_api_from_modulelist大约它能找出八百多api出来。
image

使用get_all_api则能找出一千多个
image

使用本次pr修改后的程序可以找出2889个

Copy link
Contributor

@jzhang533 jzhang533 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm
不过最好把tools/print_signatures.py这个脚本弄得更正式一些。比如:

  • -h参数给出使用说明。
  • importlib, functools,这些没用到的import给去掉。

Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhiboniu
Copy link
Contributor Author

lgtm
不过最好把tools/print_signatures.py这个脚本弄得更正式一些。比如:

  • -h参数给出使用说明。
  • importlib, functools,这些没用到的import给去掉。

下个pr清理skiplist时修改

@XiaoguangHu01 XiaoguangHu01 merged commit 50f885f into PaddlePaddle:develop Jun 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants