Fix DataFrame to allow to store columns with size more than 2 Gb #6710
Conversation
ghost
added
the
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #6710 +/- ##
=======================================
Coverage 68.87% 68.88%
=======================================
Files 1216 1216
Lines 250915 250916 +1
Branches 26259 26259
=======================================
+ Hits 172825 172841 +16
+ Misses 71265 71258 -7
+ Partials 6825 6817 -8
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
/azp run |
|
Commenter does not have sufficient privileges for PR 6710 in repo dotnet/machinelearning |
|
@michaelgsharp, tests are failing due to "Unable to load DLL 'lib_lightgbm' or one of its dependencies: The specified module could not be found. (0x8007007E)" in ML.Fairlearn.Tests.GridSearchTest.TestGridSearchTrialRunner2. It doesn't look to be somehow related to changes done in this PR. More over, I see that the same error happens from time to time during build in many other my PRs. Is it possible to disable this test temporaly (until the fix for the test is found)? |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
| @@ -188,6 +188,28 @@ public void TestArrowStringColumnClone() | |||
| Assert.Null(clone[i]); | |||
| } | |||
|
|
|||
| /* Don't run tests during build as they fail, because build if build machine doesn't have enought memory | |||
There was a problem hiding this comment.
@asmirnov82 are you sure this is the reason?
Can we add a flag so these still run locally?
There was a problem hiding this comment.
We should be able to do a #if and then set a flag in build definition
There was a problem hiding this comment.
@asmirnov82 can you open an issue. I don't want to block on this.
@tannergooding said this might actually be a netfx thing? There might need to be something set to support > 2gb. Comment below.
"For the more than 2GB one, it will be allowed but will then fail on .NET Framework anyways due to GC limitations
You have to opt in with gcAllowVeryLargeObjects iirc, and that's only for 64-bit"
There was a problem hiding this comment.
@JakeRadMSFT I fixed tests using X64FactAttribute. Please take a look at #6758 and #6759
Fixes #6699