Support inference for GCU #14142
Support inference for GCU #14142
Conversation
EnflameGCU
force-pushed
the
support_gcu
branch
from
6f3a1aa
to
f2269b9
Compare
tools/infer/utility.py
Outdated
| @@ -41,6 +41,7 @@ def init_args(): | |||
| parser.add_argument("--use_xpu", type=str2bool, default=False) | |||
| parser.add_argument("--use_npu", type=str2bool, default=False) | |||
| parser.add_argument("--use_mlu", type=str2bool, default=False) | |||
| parser.add_argument("--use_gcu", type=str2bool, default=False) | |||
There was a problem hiding this comment.
please include a short message to help user understand its purpose.
To me, at the first glance, I am guessing gcu stands for GraphCore Unit, realized it's Enflame's device after searching.
Other arguments(i.e.: uie_xpu, use_npu, use_mlu) also need help message to clarify the respective devices, but could be addressed in separate PRs.
There was a problem hiding this comment.
OK, thanks for pointing that out. I'll add comments to GCU first.
I am afraid that our description of other arguments may not be accurate, and they may need to be described by the developers of the corresponding hardware manufacturers.
There was a problem hiding this comment.
I am afraid that our description of other arguments may not be accurate, and they may need to be described by the developers of the corresponding hardware manufacturers.
agree
EnflameGCU
force-pushed
the
support_gcu
branch
from
f2269b9
to
e293196
Compare
tools/infer/utility.py
Outdated
| parser.add_argument( | ||
| "--use_gcu", type=str2bool, default=False | ||
| ) # Use Enflame GCU(General Compute Unit) |
There was a problem hiding this comment.
| parser.add_argument( | |
| "--use_gcu", type=str2bool, default=False | |
| ) # Use Enflame GCU(General Compute Unit) | |
| parser.add_argument( | |
| "--use_gcu", type=str2bool, default=False, | |
| help="Use Enflame GCU(General Compute Unit)", | |
| ) |
There was a problem hiding this comment.
Thank you for your suggestion.
tools/infer/utility.py
Outdated
| @@ -41,6 +41,7 @@ def init_args(): | |||
| parser.add_argument("--use_xpu", type=str2bool, default=False) | |||
| parser.add_argument("--use_npu", type=str2bool, default=False) | |||
| parser.add_argument("--use_mlu", type=str2bool, default=False) | |||
| parser.add_argument("--use_gcu", type=str2bool, default=False) | |||
There was a problem hiding this comment.
I am afraid that our description of other arguments may not be accurate, and they may need to be described by the developers of the corresponding hardware manufacturers.
agree
EnflameGCU
force-pushed
the
support_gcu
branch
from
e293196
to
893ec61
Compare
| @@ -293,6 +299,28 @@ def create_predictor(args, mode, logger): | |||
| config.enable_custom_device("mlu") | |||
| elif args.use_xpu: | |||
| config.enable_xpu(10 * 1024 * 1024) | |||
| elif args.use_gcu: # for Enflame GCU(General Compute Unit) | |||
| import paddle_custom_device.gcu.passes as gcu_passes | |||
There was a problem hiding this comment.
Should we first check whether the paddle_custom_device package is present? If it’s not, remind the user to install it first?
There was a problem hiding this comment.
Thank you for your suggestion.
| if paddle.framework.use_pir_api(): | ||
| config.enable_new_ir(True) | ||
| config.enable_new_executor(True) | ||
| kPirGcuPasses = gcu_passes.inference_passes( |
There was a problem hiding this comment.
Why do we need to build passes based on "PaddleOCR" or "PaddleXXX"?
There was a problem hiding this comment.
We need to optimize performance based on hardware features and even network structure, and the processing here is to increase this flexibility.
EnflameGCU
force-pushed
the
support_gcu
branch
from
893ec61
to
9fef9a9
Compare
配置
GCU设备的推理流程