Skip to content

Introduce VBufferEditor and hide VBuffer.Count #1580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Nov 16, 2018
Merged

Introduce VBufferEditor and hide VBuffer.Count #1580

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
4149cdc
Implement VBuffer master plan WIP #1
eerhardt Oct 11, 2018
d031fa1
Getting everything to build and tests passing
eerhardt Nov 7, 2018
8adb2af
Keep moving to the master plan of VBuffer.
eerhardt Nov 8, 2018
2eff656
Remove out bools on VBufferMutationContext.Create.
eerhardt Nov 8, 2018
35a0ccc
Remove the rest of the VBuffer.Count usages in ML.Data
eerhardt Nov 8, 2018
001f7d8
Remove the rest of the VBuffer.Count usages and make VBuffer.Count pr…
eerhardt Nov 8, 2018
01654a7
Clean up VBufferMutationContext to match the spec.
eerhardt Nov 8, 2018
d9905f3
Fix two failing tests.
eerhardt Nov 8, 2018
4cafe93
Fix FastTreeBinaryClassificationCategoricalSplitTest by remembering t…
eerhardt Nov 12, 2018
1d786ef
Fix up VectorWhitening after rebase.
eerhardt Nov 13, 2018
1cb1411
Respond to PR feedback.
eerhardt Nov 15, 2018
28d48a1
Rename CreateBuffer to Commit and CommitTruncated.
eerhardt Nov 15, 2018
faeb7d4
Rename VBufferMutationContext to VBufferEditor.
eerhardt Nov 15, 2018
e16ab68
Rename mutation to editor.
eerhardt Nov 15, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions src/Microsoft.ML.Core/Data/MetadataUtils.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,9 @@ public static void GetSlotNames(RoleMappedSchema schema, RoleMappedSchema.Column

IReadOnlyList<ColumnInfo> list;
if ((list = schema?.GetColumns(role)) == null || list.Count != 1 || !schema.Schema.HasSlotNames(list[0].Index, vectorSize))
slotNames = new VBuffer<ReadOnlyMemory<char>>(vectorSize, 0, slotNames.Values, slotNames.Indices);
{
VBufferUtils.Resize(ref slotNames, vectorSize, 0);
}
else
schema.Schema.GetMetadata(Kinds.SlotNames, list[0].Index, ref slotNames);
}
Expand Down Expand Up @@ -447,21 +449,22 @@ public static bool TryGetCategoricalFeatureIndices(Schema schema, int colIndex,
{
int previousEndIndex = -1;
isValid = true;
for (int i = 0; i < catIndices.Values.Length; i += 2)
var catIndicesValues = catIndices.GetValues();
for (int i = 0; i < catIndicesValues.Length; i += 2)
{
if (catIndices.Values[i] > catIndices.Values[i + 1] ||
catIndices.Values[i] <= previousEndIndex ||
catIndices.Values[i] >= columnSlotsCount ||
catIndices.Values[i + 1] >= columnSlotsCount)
if (catIndicesValues[i] > catIndicesValues[i + 1] ||
catIndicesValues[i] <= previousEndIndex ||
catIndicesValues[i] >= columnSlotsCount ||
catIndicesValues[i + 1] >= columnSlotsCount)
{
isValid = false;
break;
}

previousEndIndex = catIndices.Values[i + 1];
previousEndIndex = catIndicesValues[i + 1];
}
if (isValid)
categoricalFeatures = catIndices.Values.Select(val => val).ToArray();
categoricalFeatures = catIndicesValues.ToArray();
}
}

Expand Down
Loading