aio: enable -O3 optimization

[adammoody]

This updates the aio op to build with -O3 rather than -O0 optimization.

It looks like all ops either build with -O2 or -O3, but aio uses -O0 for some reason. Is that intentional?

[tjruwase]

@adammoody, aio op is IO-bound and thus gets little benefit from compiler optimizations. So we choose -O0 for better debugging experience.

[adammoody]

Ok, thanks. I had noticed that some of aio includes some SIMD code like here:

DeepSpeed/csrc/aio/py_lib/deepspeed_py_copy.cpp

Lines 107 to 109 in 6ba9628

	SIMD_STORE(dest + i, src_4[0].data);
	SIMD_STORE(dest + i + SIMD_WIDTH, src_4[1].data);
	SIMD_STORE(dest + i + (SIMD_WIDTH << 1), src_4[2].data);

On PowerPC, those SIMD-tuned algorithms aren't available, so it falls back to the non-vectorized copy method here:

DeepSpeed/csrc/aio/py_lib/deepspeed_py_copy.cpp

Lines 46 to 49 in 6ba9628

	if (param_size > rounded_size) {
	#pragma omp parallel for
	for (size_t k = rounded_size; k < param_size; k++) { dest[k] = src[k]; }
	}

I thought the compiler might help in that case, and that maybe that would be useful since someone took the time to create the SIMD versions.

I do understand performance being bound by much slower I/O, though.

Mind if I replace this PR with one that adds a comment to note why -O0 is used?

[tjruwase]

Ok, thanks. I had noticed that some of aio includes some SIMD code like here:

Yes, you are correct. That was a failed attempt to create fast copy routines, but was abandoned due to poor progress. The routines are currently not used, and these codes should be ideally be deleted. Sorry for the confusion.

[tjruwase]

Mind if I replace this PR with one that adds a comment to note why -O0 is used?

Thanks, that would be very much appreciated.