Skip to content

Sync with upstream main branch #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 307 commits into from

Conversation

joshpeterson
Copy link

This is an automatically generated pull request to merge changes from the upstream main branch.

elinor-fung and others added 30 commits February 8, 2022 11:57
* Update Android to r23b image

* Don't use flag unavailable in r23b
* Bump version of C# compiler

This should be reverted when arcade brings in a new enough compiler.

* Fix a few new warnings from updating compiler

* Enable IDE0190 to employ parameter null checking

* Initial roll out of !!

* Fix more compiler warnings from upgrade, plus some test fixes / reverts

* Fix a few more tests

* Disable profiling test
* Add `runtime-wasm` pipeline

This adds support for a `runtime-wasm` pipeline, which uses
`runtime-extra-platforms.yml`.

The wasm jobs have been moved to `runtime-extra-platforms-wasm.yml`, and
the rest to `runtime-extra-platforms-other.yml`. And the two files are
included conditionally.

Effectively, this still runs all these jobs when
`runtime-extra-platforms` is run (same as before). But it allows running
just the wasm jobs, triggered manually with:

    `/azp run runtime-wasm`

When `runtime-wasm` is manually run:

- LibraryTests will run all the tests on linux+windows
- LibraryTests_EAT will run all the tests on linux
- LibraryTests_AOT will run all the tests on linux+windows
- Wasm.Build.Tests, and Wasm debugger tests will run on linux+windows
- Runtime tests will run on linux

When `runtime-extra-platforms` is manually run:

- LibraryTests will run all the tests, if paths changed
- LibraryTests_EAT will run all the tests on linux, if paths changed
- LibraryTests_AOT will run all the tests on linux+windows, if paths changed
- Wasm.Build.Tests, and Wasm debugger tests will never run
- Runtime tests will never run

On rolling builds, `runtime-extra-platforms` runs by default:

- LibraryTests will run all the tests
- LibraryTests_EAT will run all the tests on linux
- LibraryTests_AOT will run all the tests on linux+windows
- Wasm.Build.Tests, and Wasm debugger tests will never run
- Runtime tests will never run

With feedback from @steveisok, and @safern.

* Flow `isExtraPlatformsBuild` to the new *-wasm, and *-other templates

```
/mnt/vss/_work/1/s/.packages/microsoft.dotnet.helix.sdk/7.0.0-beta.22080.1/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(18,5): error : You must specify at least one target queue to send a job to helix. Use the HelixTargetQueues property or HelixTargetQueue items. [/mnt/vss/_work/1/s/src/libraries/sendtohelixhelp.proj]
```

Fix suggested by @safern

* Update condition, based on feedback

* Fix typo

* Rename the var `RunSmokeTestsOnly` to *not* the match msbuild property

.. used to control smoke tests. The variables in yml seem to get
converted to environment variables which can affect the build. Like
causing unrelated builds to get `RUNSMOKETESTSONLY=true`, and breaking
the build.
* Remove NativeAOT testing added in dotnet#62704 to make room for the different strategy.
* Build NativeAOT as part of test build, not test run
…entation (dotnet#63315)

* Convert ContextStack to use generics

* Convert AttributeCollection to generics

Also replaced a double dictionary lookup with the `TryGet` variant to
increase performance.

* Convert ComponentResourceManager to generics

* Convert LicInfoHelperLicenseContext to generics

* Convert PropertyDescriptor to generics
This particular change doesn't yet attempt to merge platform-variant tests into
fewer source files mostly because the affected tests are mostly ILPROJ tests and
it's not yet clear to me how to apply platform variations to them without substantial
changes in the source code. For now I have just aligned the assembly vs. project
naming so that the tests work in the merged scenario.

Thanks

Tomas
…under JIT/Methodical (dotnet#64839)

* Fix System.Runtime reference public key token in delegatetail.il

* Three more IL sources needing explicit System.Runtime PKT

* Clean up several project file names

* Typo

* Fix assembly name for sin_cs_il* tests
* improve SslStream tests on misconfigured systems

* feedback from review
…net#64959)

* Override DictionaryEntry.ToString to match KeyValuePair.ToString

* Add a couple more test cases and test case comments

* Update src/libraries/Common/tests/System/Collections/DictionaryEntry.Tests.cs

Co-authored-by: Stephen Toub <stoub@microsoft.com>
* Add a new wrapper `runtime-wasm.yml` to be used from azdo

Currently, we have `runtime-wasm` in azdo pointing to
`runtime-extra-platforms.yml`, which has some scheduled triggers
defined. azdo seems to use these for running `runtime-wasm` too.

Instead, in azdo use a new `runtime-wasm.yml` for `runtime-wasm` pipeline, so
we can explicitly avoid the scheduled triggers. This yml just uses the
existing `runtime-extra-platforms-wasm.yml`.

* Add missing variables template. thanks to @safern
* improve SslStream tests on server 2022

* Update src/libraries/System.Net.Security/tests/FunctionalTests/SslStreamNetworkStreamTest.cs

Co-authored-by: Anton Firszov <antonfir@gmail.com>

Co-authored-by: Anton Firszov <antonfir@gmail.com>
* Fix mono native warnings from wasm targets

* Add back Werror option

* Use tabs for indent
The newly added NativeAOT libraries testing is very unstable. System.Collections are probably a less intense test, so run that one instead.

This will of course be revisited since we want to run more libraries tests but I want to stop turning people's PR's red.

If this is not sufficient, we'll have to pull the entire leg until it's root caused.
- Move all SetScriptCommands to a target, so they are placed after scenario in the generated script.
- Add WasmXHarnessArgsCli as an alternative variable that can be used from cli.
- Fix NodeJS symbolic links on Helix.
- Copy TestEchoMiddleware and RemoteLoopMiddleware even for Scenario=NodeJS.
- Enable System.Net.WebSockets.Client.Tests on NodeJS.
…otnet#64994)

* Add intrinsic for Vector64/128 Ceiling

* Add intrinsic for Vector64/128 Floor

Co-authored-by: Simon Rozsival <srozsival@xam-emag-01.redmond.corp.microsoft.com>
…#64747)

The two buffers have been merged together since they are never used in parallel (i.e. receciving appdata is not allowed during renegotiation etc.).
…time-assets dotnet/roslyn-analyzers dotnet/hotreload-utils dotnet/llvm-project (dotnet#64982)

[main] Update dependencies from dotnet/icu dotnet/xharness dotnet/runtime-assets dotnet/roslyn-analyzers dotnet/hotreload-utils dotnet/llvm-project


 - Merge branch 'main' into darc-main-51defe84-5fd5-4263-9073-7190ffd6b499
Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>
dotnet#65062)

* Add conditional select vector intrinsic

* Remove unrelated code change
…otnet#65070)

There's a problem with exit code detection on the 12.00 queue.
…otnet#64679)

There used to be a bug in Android libc implementation of `clock_nanosleep`. The return value should be `errno` on errors  but instead in it returns `-1` and sets `errno`. The libc (Bionic) bug [has been fixed](https://android-review.googlesource.com/c/platform/bionic/+/110652/) since Android 6 and newer but it causes problems to [customers who are unable to update Android on their devices](dotnet/android#6600 (comment)).

Fixes dotnet/android#6600

* Account for incorrect implementation of clock_nanosleep in older Android libc

* Shorten comments

* Add g_clock_nanosleep function

* Add remap definition

* Fix build

* Make sure the extra check runs only on Android

* Make Windows builds happy

* Try making wasm builds happy
steveisok and others added 26 commits February 17, 2022 22:29
This change improves the inner devloop experience by making sure to cache the .aotdata file when it's generated. Otherwise, if you clean your artifacts directory and no aot changes have been made, it will not be generated again.  The result being an error when building an app.

I believe this mostly impacts iOS configurations.
* Use CopyContext to restore saved context on X86

* PR feedback

* more PR feedback
…dotnet#64906)

* FileSystemWatcher.Linux: handle races while adding child directories.

This handles the child directory getting removed or replaced by a file
while we are adding a watch for it or enumerating its subdirectories.

* Pass other enumeration exceptions to the user.
* [LoongArch64] add ToolBox directory about jitinterace for getting ABI-info. (dotnet#59561)

Co-authored-by: Loongson's .NET-teams

* [LoongArch64] add new interace for getting ABI-info. (dotnet#59561)

* [LoongArch64] add the linking page for LoongArch64 ABI-info. (dotnet#59561)

* [LoongArch64] moved ThunkInput.txt to dotnet#62885.

* [LoongArch64] moved vm/jitinterface.cpp to dotnet#62885.

* remove the JIT/EE interface back from dotnet#62885..

* [LoongArch64] Fix the compiling error after merge.

* [LoongArch64] add comments for the returned value of `getFieldTypeByHnd`.

* [LoongArch64] rename getFieldTypeByHnd to getFieldSizeClassificationByHnd.
Also add macro define for returned value of `getFieldSizeClassificationByHnd`.

* [LoongArch64] Delete the interface `getArgType2`.
And refactor the returned values of `getFieldSizeClassificationByHnd`.

* [LoongArch64] delete `GetArgType` within `ToolBox/superpmi`.

* [LoongArch64] rename `getFieldSizeClassificationByHnd` to
`getLoongArch64PassStructInRegisterFlags`.

* [LoongArch64] amend the floating-ABI for native-struct.

* [LoongArch64] update all related `GetFieldSizeClassificationByHnd`
by `GetLoongArch64PassStructInRegisterFlags` and amend some comments.

* [LoongArch64] replace `LookupApproxFieldTypeHandle()`
by `GetFieldTypeHandleThrowing()`.

* [LoongArch64] implements the crossgen2 for LoongArch64.

* Revert "[LoongArch64] implements the crossgen2 for LoongArch64."

This reverts commit b05a2b9.

The crossgen2 for LoongArch64 will be submitted by a new PR.

* [LoongArch64] update the `GUID JITEEVersionIdentifier`.
Also delete some unused comments.

Co-authored-by: qiaopengcheng <qiaopengcheng-hf@loongson.cn>
* Fix XNodeReaderFunctionalTests

* cleanup Flush/Dispose

* cleanup few more flushes
* Support resumable serialization in NullableConverter<T>

* use null instead of default
Allocate the buffer outside of the loop so we don't hit OOM issues.

While looking at the test I noticed that it actually had a bug too:
Nothing was asserting that we indeed throw an InvalidOperationException when the writer is completed. Adding `Assert.ThrowsAsync` revealed that the current loop iteration count was too low to hit the exception reliably
within the 10ms delay so instead check for the execution time.

Fixes dotnet#64930
* Rationalize wasm jobs run on CI

Essentially:
- For rolling builds, we want to run all the wasm jobs, unconditionally
   - And `runtime-extra-platforms` runs the tests that were not in `runtime`, and `runtime-staging`
- For PRs, we use the various path checks

Changes:
- split wasm jobs into various templates
- combine, and simplify some jobs

* `runtime-extra-platforms`, and `runtime-wasm` run only when manually
  triggered with a comment - `/azp run <pipeline-name>`
* `runtime`, and `runtime-staging`, run jobs only when relevant paths
  change. And for `EAT`, and `AOT`, only smoke tests are run.
* And when `runtime-wasm` is triggered manually, it runs *all* the wasm
  jobs completely

| .                 | runtime               | runtime-staging         | runtime-extra-platforms(manual only) | runtime-wasm (manual only) |
| ----------------- | --------------------  | ---------------         | ------------------------------------ | -------                    |
| libtests          | linux: all,   only-pc | windows: all,   only-pc | linux+windows: all, only-pc          | linux+windows: all, always |
| libtests eat      | linux: smoke, only-pc | -                       | linux:         all, only-pc          | linux:         all, always |
| libtests aot      | linux: smoke, only-pc | windows: smoke, only-pc | linux+windows: all, only-pc          | linux+windows: all, always |
|                   |                       |                         |                                      |                            |
| Wasm.Build.Tests  | linux:        only-pc | windows:        only-pc | linux+windows: only-pc               | linux+windows              |
| Debugger tests    | -                     | linux+windows:  only-pc | linux+windows: only-pc               | linux+windows              |
| Runtime tests     | linux:        only-pc | -                       | linux: only-pc                       | linux                      |

* `runtime`, and `runtime-staging`, run all the wasm jobs unconditionally, but `EAT`, and `AOT` still run only smoke tests.
* `runtime-extra-platforms` also runs by default. And it runs only the cases not covered by the above two pipelines.

* jobs w/o `only-pc` are always run

| .                 | runtime                   | runtime-staging       | runtime-extra-platforms (always run) | runtime-wasm (manual only) |
| ----------------- | -------------             | ---------------       | ------------------------------------ | ------                     |
| libtests          | linux: all(v8/chr/node)   | windows: all          | linux+windows: all with node         | N/A                        |
| libtests eat      | linux: smoke              | -                     | linux: all                           |                            |
| libtests aot      | linux: smoke              | windows: smoke        | linux+windows: all                   |                            |
|                   |                           |                       |                                      |                            |
| Wasm.Build.Tests  | linux: always             | windows: always       | none                                 |                            |
| Debugger tests    | -                         | linux+windows: always | none                                 |                            |
| Runtime tests     | linux: always             | -                     | none                                 |                            |

* [wasm] Run tests with node only on runtime-wasm
We only require the "greater than" condition on x86.
* Add checking for zero-offset FldSeq addition

* Add a test

* Fix ADDR(LCL_VAR) Zero [FldSeq]

* Always add NotAField field sequences

* NotAField printing improvements
* Fix Vector256.IsHardwareAccelerated in R2R binaries
- If a function had a dependency on Vector256<T> but not on any Avx2 functionality then we would fail to produce an R2R image using a debug version of the compiler, and would generate an incorrect function where the result of calling Vector256.IsHardwareAccelerated would produce the wrong result.
- To fix this we now annotate the VectorXXX instruction sets so that if there is a negative depenedency on them during R2R generation, that the function is not available for use if the underlying actual instruction set is available to use at runtime.
- Adjust test Runtime_34587 to test instruction set manner in a way that allows testing crossgen2 instruction set variation
  - Move instruction set issupported checks into standalone functions to isolate the tests in the JIT/Crossgen2 compiler to better see exactly what is happening
  - Add diagnostic output to test, for better diagnosis of future failures.
* Initial work

* Added a comma to display

* Cleanup

* Fixing build

* More cleanup

* Update comment

* Update comment

* Added CompareEqual Vector64/128 with Zero tests

* Do not contain op1 for now

* Wrong intrinsic id used

* Removing generated tests

* Removing generated tests

* Added CompareEqual tests

* Supporting containment for first operand

* Fix test build

* Passing correct register

* Check IsVectorZero before not allocing a register

* Update comment

* Fixing test

* Minor format change

* Fixed formatting

* Renamed test

* Adding AdvSimd_Arm64 tests:

* Adding support for rest of 'cmeq' and 'fcmeq' instructions

* Removing github csproj

* Minor test fix

* Fixed tests

* Fix print

* Minor format change

* Fixing test

* Initial commit for Vector.Create to Vector.Zero normalization

* Added some emitter tests

* Feedback

* Update emitarm64.cpp

* Feedback

* Handling variations of Vector.Create

* Use Operands iterator instead of edges

* Fix condition

* Simplify

* format

* Fixed IsFloatPositiveZero

* Uncomment

* Updated tests to include Vector64.Create/Vector128.Create for ARM64

* Making implementation of IsFloatPositiveZero explicit

* Update src/coreclr/jit/gentree.cpp

Co-authored-by: SingleAccretion <62474226+SingleAccretion@users.noreply.github.com>

* Feedback

* Update comment

* Update comment

* Do not perform optimization when VN CSE phase

* use ResetHWIntrinsicId

* Assert !optValnumCSE_phase

* Simplify IsVectorZero

* Simplify IsVectorZero

* Simplify some uses of Vector*_get_Zero

* Added another test

* Fixed formatting

* Revert lowering removal

* Initial work for optimizations on VectorZero value numbering

* Allowing all Vector.Zero to be constant prop'ed. Added VNFuncSimdTypeInfo.

* Update gentree.h

* Quick rename

* Removed extra variable

* Added default case

* Format

* Fixed vnDumpSimdType to take into account CorInfoType

* Fixed gtNewSimdZeroNode to produce the right Vector*_get_Zero based on simdSize

* Formatting

* Feedback and a loop test

* Added another test. Formatting fixes

* Added GetSimdBaseJitPreciseType

* Feedback

* Minor fix

* Minor comment update

* Added another comment

* Added another comment

* Added another comment

* Update comment

* Formatting

* Feedback

* Fixing build

* Feedback

* Update assertionprop.cpp

* Formatting

Co-authored-by: SingleAccretion <62474226+SingleAccretion@users.noreply.github.com>
…5531)

* Skip RegexOptions.NonBacktracking phase 3 for some patterns

If a pattern doesn't have any captures and if any match of that pattern will always be the same length, we can skip the Phase 3 computation as, given the computed starting position of the match, we know exactly where it's going to end.

* Add example to the phase comments
* Clean-up slngen related logic

* Update dotnet-tools.json

Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
1. Be more robust to temp directory removal failure

If we fail to remove a temporary directory, don't crash. Log the failure
and the set of directories and files still remaining, and continue.

We have seen this failure in at least one Linux x64 PMI coreclr_tests
SuperPMI collection:
```
[Errno 39] Directory not empty: '/datadisks/disk1/work/B18E0979/t/tmpov3b4_qa'
```

2. Limit the length of file names created by `make_safe_filename`. We were creating
file names out of full path names, where those paths contained long temporary directory
paths, and thus we were exceeding the maximum allowed file name component.
- Read first directory name under node instead of hardcoded version.
* Make the sanitize pass less aggressive
…an… (dotnet#65451)

* [wasm] Trigger all wasm jobs, if eng/Version.Details.xml has any changes, so we run a full check on emsdk updates

* fix yml

* Fix merge

* [wasm] Pass isRollingBuild to runtime-extra-platforms-wasm template, to correctly trigger the EAT/AOT builds

* .. and pass isWasmOnlyBuild also
@joshpeterson
Copy link
Author

I'll close this and try again next week.

@joshpeterson joshpeterson deleted the bot-upstream-main-merge-2022-02-19 branch March 10, 2022 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.