ASP.NET Core CSharp benchmarks have disappeared from previous rounds

[sebastienros]

We can find some VB results for asp.net core, but the Frameworks/CSharp/aspnetcore ones have disappeared. And these ones were the best.

@DamianEdwards @roji

[NateBrady23]

@msmith-techempower this is likely because of a name change as we're (unfortunately) using the latest file to do the mappings. Can you check it out when you have a chance? We should be able to get that restored.

[sebastienros]

The config file changed last week for sure, as we wanted to improve the names. But even the docker files changed a year ago, so I am not sure what we can do.

[msmith-techempower]

Appears to have occurred when #8498 was merged. The benchmark_config.json key is one that the metadata relies on ... for legacy reasons.

I will look into this more tomorrow, but this is a systemic issue that we really need a better handle on fixing moving forward. It should be recoverable; however, it will take some time.

[remittor]

And these ones were the best.

Is rating really that important?
This code is just disgusting!

FrameworkBenchmarks/frameworks/CSharp/aspnetcore/src/Platform/BenchmarkApplication.Json.cs

Lines 20 to 23 in c82a834

	"HTTP/1.1 200 OK\r\n"u8 +
	"Server: K\r\n"u8 +
	"Content-Type: application/json\r\n"u8 +
	"Content-Length: 27"u8;

[NateBrady23]

@remittor if you think that's an issue please open a separate ticket. working on something else here.

[msmith-techempower]

Just as an update: I have been working on this - it hasn't fallen off our radar.

Full disclosure: the TFB Results website has been a hodgepodge over spaghetti code over the years, and has basically become unmaintainable due to hand-offs. I have 2 feature branches currently.

The first is ready for review, but I need to sit down with the team and work through all the myriad user stories to ensure I haven't inadvertently broken anything while fixing the inadvertently broken metadata issue. All it does is attempts to break out each Round's metadata so they are standalone. This had received push-back over the years for [reasons], but it's time. We're at an inflection point and it's time to just have each round self sufficient and not based on, or munged by, previous rounds.

The short version explanation is that the tooling for identifying, matching, and rebuilding the metadata on which the entire site operates has had subtle bugs introduced over time that made it past QA. Each round would hide more and more tests because of edge cases until Round 23, which saw several hundred frameworks hidden. As I pointed out above, the change in benchmark_config.json from "aspnet" to "aspnetcore" caused our internal tooling to identify "aspnet" as having been deleted while a new "aspnetcore" has been added. There may have been a way to reconcile this type of change, but that's lost to time. As such, the metadata was updated to reflect that deletion and addition, which is why the old results are missing - all rounds use the same metadata. Again, the first branch I'm working on aims to isolate rounds so that each will have its own metadata. There will be some drawbacks to this, but the upside is that each round will exist in isolation so that the next round doesn't accidentally hide - and I want to stress that the results are simply hidden and not deleted - 8 years of tests.

The second is an entire spring-cleaning updating of the TFB results website. This one is taking much longer because it is much more involved. Ultimately, we have wanted to get the TFB Results website open sourced so the community could help out with it, but again it has been such a hodgepodge over the years (and some clutched a little too hard) that it simply did not represent code we were proud of. This update is 90% done and I do hope that, while I'm still not 100% proud of the code, it will be in a better state and suitable to open sourcing to improve the experience for the community.

[msmith-techempower]

I've opened up #9789 to field anything else, but I have just deployed the new Round 23 which unbreaks Rounds [..22].

I have confirmed that aspcore is correctly returned for Round 22

As the new issues suggests, feel free to comment on anything I may have broken as a part of this process. Closing this issue in lieu of the new one.

Thanks all!

[sebastienros]

Thanks @msmith-techempower