Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a sample for Onnx conversion #5195

Merged
merged 7 commits into from
Jun 8, 2020
Merged

Add a sample for Onnx conversion #5195

merged 7 commits into from
Jun 8, 2020

Conversation

wangyems
Copy link
Contributor

@wangyems wangyems commented Jun 2, 2020

No description provided.

@wangyems wangyems requested a review from a team as a code owner June 2, 2020 23:16
@wangyems wangyems marked this pull request as draft June 3, 2020 00:46
@wangyems wangyems marked this pull request as ready for review June 4, 2020 00:02
@codecov
Copy link

codecov bot commented Jun 4, 2020

Codecov Report

Merging #5195 into master will increase coverage by 0.00%.
The diff coverage is 0.00%.

@@           Coverage Diff           @@
##           master    #5195   +/-   ##
=======================================
  Coverage   73.31%   73.32%           
=======================================
  Files        1007     1007           
  Lines      188075   188124   +49     
  Branches    20244    20246    +2     
=======================================
+ Hits       137892   137934   +42     
- Misses      44658    44661    +3     
- Partials     5525     5529    +4     
Flag Coverage Δ
#Debug 73.32% <0.00%> (+<0.01%) ⬆️
#production 69.07% <0.00%> (-0.01%) ⬇️
#test 87.41% <ø> (+0.01%) ⬆️
Impacted Files Coverage Δ
...c/Microsoft.ML.SamplesUtils/SamplesDatasetUtils.cs 40.00% <0.00%> (-0.68%) ⬇️
...c/Microsoft.ML.FastTree/Utils/ThreadTaskManager.cs 79.48% <0.00%> (-20.52%) ⬇️
....ML.AutoML/PipelineSuggesters/PipelineSuggester.cs 84.03% <0.00%> (-2.53%) ⬇️
src/Microsoft.ML.Maml/MAML.cs 24.75% <0.00%> (-1.46%) ⬇️
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs 84.32% <0.00%> (-0.85%) ⬇️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs 89.13% <0.00%> (-0.16%) ⬇️
src/Microsoft.ML.OnnxTransformer/OnnxTransform.cs 91.97% <0.00%> (+0.06%) ⬆️
...osoft.ML.OnnxTransformerTest/OnnxTransformTests.cs 97.67% <0.00%> (+0.19%) ⬆️
...soft.ML.Transforms/Text/WordEmbeddingsExtractor.cs 86.87% <0.00%> (+1.13%) ⬆️
...rosoft.ML.AutoML/ColumnInference/TextFileSample.cs 62.25% <0.00%> (+2.64%) ⬆️
... and 1 more

@wangyems wangyems requested a review from harishsk June 4, 2020 21:10
}

public static void Example()
{
Copy link
Contributor

@harishsk harishsk Jun 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this sample is getting more complicated than necessary. Can we simplify it just to demonstrate basic Onnx export and exporting to a different opset? The scenarios of using the whole pipeline versus partial pipeline is more advanced and can be left out of this sample. #Resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure


In reply to: 436857134 [](ancestors = 436857134)

@wangyems wangyems requested a review from harishsk June 8, 2020 17:43
{
// Create data featurizing pipeline
var pipeline = mlContext.Transforms.CopyColumns("Label", "IsOver50K")
// Convert categorical features to one-hot vectors
Copy link
Contributor

@harishsk harishsk Jun 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you still need these changes? #Resolved

Copy link
Contributor Author

@wangyems wangyems Jun 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, because the input data for the pipeline is raw data instead of featurizedData, and the onnx model can only be fed with raw data. So I refactor the SamplesDatasetUtils.cs to have a specific function to get raw data #Resolved

Copy link
Contributor

@harishsk harishsk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@wangyems wangyems merged commit 9244e68 into dotnet:master Jun 8, 2020
@wangyems wangyems deleted the wangye/sample branch June 9, 2020 20:18
@ghost ghost locked as resolved and limited conversation to collaborators Mar 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants