Fix op_builder against PyTorch nightly

[malfet]

After pytorch/pytorch#100557 PyTorch is free to use C++17 feature in it's headers, such as std::optional, if constexpr etc.

This update language standard for DS extensions to use C++17 standard by default

[stas00]

@tjruwase, For context please see the breakage reported when trying to build deepspeed with pt-nightly as of last week:
pytorch/pytorch#102082

[stas00]

@malfet, would your proposed changes work with older pytorch versions? if I'm not mistaken deepspeed needs to work with pt-1.9 or higher.

[malfet]

@malfet, would your proposed changes work with older pytorch versions? if I'm not mistaken deepspeed needs to work with pt-1.9 or higher.

@stas00 I have not checked with pt-1.9, but I don't see why not (it should clearly work with 1.13/2.0), as C++17 is backward compatible with C++14. It might print a few warnings (for example because older PyTorch builds use std::iterator

[stas00]

Perfect, thank you for validating that, @malfet!

[stas00]

fyi @jeffra

[malfet]

Looking into formatting failure, but will not push following diff until all checks are finished:

diff --git a/op_builder/builder.py b/op_builder/builder.py
index b8e9f13b..e2902918 100644
--- a/op_builder/builder.py
+++ b/op_builder/builder.py
@@ -673,8 +673,8 @@ class CUDAOpBuilder(OpBuilder):
             cuda_major, _ = installed_cuda_version()
             args += [
                 '-allow-unsupported-compiler' if sys.platform == "win32" else '', '--use_fast_math',
-                '-std=c++17' if cuda_major > 10 else '-std=c++14',
-                '-U__CUDA_NO_HALF_OPERATORS__', '-U__CUDA_NO_HALF_CONVERSIONS__', '-U__CUDA_NO_HALF2_OPERATORS__'
+                '-std=c++17' if cuda_major > 10 else '-std=c++14', '-U__CUDA_NO_HALF_OPERATORS__',
+                '-U__CUDA_NO_HALF_CONVERSIONS__', '-U__CUDA_NO_HALF2_OPERATORS__'
             ]
             if os.environ.get('DS_DEBUG_CUDA_BUILD', '0') == '1':
                 args.append('--ptxas-options=-v')

[jeffra]

Thank you @malfet! 🚀

[fecet]

Still cannot build, could you please provide the command and environment while building?

[dainey94]

yes sir !! Thanks Sir Tải xuống Outlook dành cho Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Xie Zejian ***@***.***> Sent: Sunday, July 2, 2023 8:54:37 PM To: microsoft/DeepSpeed ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [microsoft/DeepSpeed] Fix op_builder against PyTorch nightly (PR #3596) Still cannot build, could you please provide the command and environment while building? — Reply to this email directly, view it on GitHub<#3596 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A2C74IU63OUK7KXC5A5662TXOF4R3ANCNFSM6AAAAAAYMJQA3U>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>