Removing Logging Line from ch.Info (#5598)

[ankitasankars]

• Merge from main repository (Get a working build #1)

• update tensorflow.net to 0.20.0 (update tensorflow.net to 0.20.0 #5404)

• upgrade to 3.1

• write inline data using invariantCulture

• upodate tensorflow

• update Microsoft.ML.Vision

• fix test && comment

• udpate tensorflow.net to 0.20.1

• update tf major version

• downgrade tf runtime to 1.14.1

• Update Dependencies.props

• Update Dependencies.props

• update tffact to stop running test on linux with glibc < 2.3)

• fix TensorFlowTransformInputShapeTest

• use tf.v1 api

• fix comment:

• fix building error

• fix test

• fix nit

• remove linq

Co-authored-by: BigBigMiao BigBigMiao@github.com

• ProduceWordBags Onnx Export Fix (ProduceWordBags Onnx Export Fix  #5435)

• fix for issue

• fix documentation

• aligning test

• adding back line

• aligning fix

Co-authored-by: Keren Fuentes kedejesu@microsoft.com

• [SrCnnEntireAnomalyDetector] Upgrade boundary calculation and expected value calculation ([SrCnnEntireAnomalyDetector] Upgrade boundary calculation and expected value calculation #5436)

• adjust expected value

• update boundary calculation

• fix boundary

• adjust default values

• fix percent case

• fix error in anomaly score calculation

Co-authored-by: yuyi@microsoft.com Yuanxiang.Ying@microsoft.com

• Update OnnxRuntime to 1.5.2 (Update OnnxRuntime to 1.5.2 #5439)

• Added prerelease feed and updated to 1.5.2

• Remove prerelease feed

• Updated docs

• Update doc

• Fixed MacOS CI Pipeline builds (Fixed MacOS CI Pipeline builds #5457)

• Added MacOS Homebrew bug fix

• nit fix

• Improving error message (Improving error message  #5444)

• better error fix

• revisions

Co-authored-by: Keren Fuentes kedejesu@microsoft.com

• Fixed MacOS daily & nightly builds due to Homebrew bug (Fixed MacOS daily & nightly builds due to Homebrew bug #5467)

• Fixed MacOS nightly builds due to Homebrew bug

• Edit workaround

• Remove untapping of python2

• Nit edit

• Remove installation of mono-libgdiplus

• try installing mono-libgdiplus

• unlink python 3.8

• Auto.ML: Fix issue when parsing float string fails on pl-PL culture set using Regression Experiment (Auto.ML: Fix issue when parsing float string fails on pl-PL culture set using Regression Experiment #5163)

• Fix issue when parsing float string fails on pl-PL culture set

• Added InvariantCulture float parsing as per CodeReview request

• Update src/Microsoft.ML.AutoML/Sweepers/SweeperProbabilityUtils.cs

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Update Parameters.cs

• Added PL test

• Added multiple cultures

• debugging CI failure

• Debug runSpecific

• Revert "Debug runSpecific"

This reverts commit 95b7280.

• Removed LightGBM and addressed comments

• Increased time

• Increase time

• Increased time

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com
Co-authored-by: Antonio Velazquez anvelazq@microsoft.com

• handle exception during GetNextPipeline for AutoML (handle exception during GetNextPipeline for AutoML #5455)

• handle exception during GetNextPipeline for AutoML

• take comments

• Changing LoadRawImages Sample (Changing LoadRawImages Sample #5460)

replacing example

Co-authored-by: Keren Fuentes kedejesu@microsoft.com

• Use Timer and ctx.CancelExecution() to fix AutoML max-time experiment bug (Use Timer and ctx.CancelExecution() to fix AutoML max-time experiment bug #5445)

• Use ctx.CalncelExecution() to fix AutoML max-time experiment bug

• Added unit test for checking canceled experiment

• Nit fix

• Different run time on Linux

• Review

• Testing four ouput

• Used reflection to test for contexts being canceled

• Reviews

• Reviews

• Added main MLContext listener-timer

• Added PRNG on _context, held onto timers for avoiding GC

• Addressed reviews

• Unit test edits

• Increase run time of experiment to guarantee probabilities

• Edited unit test to check produced schema of next run model's predictions

• Remove scheme check as different CI builds result in varying schemas

• Decrease max experiment time unit test time

• Added Timers

• Increase second timer time, edit unit test

• Added try catch for OperationCanceledException in Execute()

• Add AggregateException try catch to slow unit tests for parallel testing

• Reviews

• Final reviews

• Added LightGBMFact to binary classification test

• Removed extra Operation Stopped exception try catch

• Add back OperationCanceledException to Experiment.cs

• fix issue 5020, allow ML.NET to load tf model with primitive input and output column (fix issue 5020, allow ML.NET to load tf model with primitive input and output column #5468)

• handle exception during GetNextPipeline for AutoML

• take comments

• Enable TesnflowTransformer take primitive type as input column

• undo unnecessary changes

• add test

• update on test

• remove unnecessary line

• take comments

• maxModels instead of time for AutoML unit test (MaxModels exit criteria for AutoML unit test #5471)

Uses the internal maxModels parameter instead of MaxExperimentTimeInSeconds for the exit criteria of AutoML.

This is to increase the test stability in case the test is run on a slower machine.

• Disabling AutoFitMaxExperimentTimeTest

Disabling AutoFitMaxExperimentTimeTest

• Fix AutoFitMaxExperimentTimeTest (Fix AutoFitMaxExperimentTimeTest #5506)

*Fixed test
Co-authored-by: Antonio Velazquez anvelazq@microsoft.com

• Fix SR anomaly score calculation at beginning (Fix SR anomaly score calculation at beginning #5502)

• adjust expected value

• update boundary calculation

• fix boundary

• adjust default values

• fix percent case

• fix error in anomaly score calculation

• adjust score calculation for first & second points

• fix sr do not report anomaly at beginning

• fix a issue in batch process

• remove a unused parameter

Co-authored-by: yuyi@microsoft.com Yuanxiang.Ying@microsoft.com

• Merge arcade to master (Merge arcade to master #5525)

• Initial commit for Arcade migration

• Added omitted files

• Changed strong name signing to use the same key for shipping and test assemblies

• arcade linux build (arcade linux build #5423)

• arcade linux build

• put file execution permission change into source control

• The -test command for windows. Nuget packages (The -test command for windows. Nuget packages #5464)

• working on testing

• testing updates

• tests almost working

• build changes

• all tests should be working

• changes from PR comments

• fixes for .net 3.1

• Fixed extension check. Removed where not needed

• Removed pkg folder and updated paths.

• Added test key. (Added test key. #5475)

• Added test key.

• Update PublicKey.cs

Removed extra newline.

• Update ComponentCatalog.cs

Fixed 3 spaces to 4.

• Windows CI working (Windows CI working #5477)

• ci testing changes

• comments from pr

• Added Linux & Mac changes for Arcade (Added Linux & Mac changes for Arcade #5479)

• Initial Windows, Linux, Macos builds test

• Add Linux/MacOS specific CI requirements

• Run Arcade CI tests on MacOS/Linux

• Fix final package building

• Add benchmark download to benchmars .csporj file

• Print detailed status of each unit test

• Install CentOS & Ubuntu build dependencies

• Use container names to differenciate between Ubuntu & CentOS

• Remove sudo usage in CentOS

• Fix Linux build dependencies

• Add -y param to apt install

• Remove installation of Linux dependencies

• Minor additions

• Rename Benchmarks to PerformanceTests for Arcade

• Changes

• Added benchmark doc changes

• Pre-merge changes

• Fixing failing Arcade Windows Builds (Fixing failing Arcade Windows Builds #5482)

• Try Windows build single quote fix

• Remove %20

• Added variable space value

• Using variables for spacing

• Added space values as job parameters

• Try conditional variables again

• fix official builds

• Revert "fix official builds"

This reverts commit 7dbbdc7.

• fixing tensorflow rebase issue

• Fixes for many of the CI builds. (Fixes for many of the CI builds. #5496)

• yml log changes

• Fix NetFX builds by ensuring assembly version is set correctly and not to Arcade default of 42.42.42.42 (Fix NetFX builds by ensuring assembly version is set correctly #5503)

• Fixed official builds for Arcade SDK (Fixed official builds for Arcade SDK #5512)

• Added fixes for official builds

• Make .sh files executable

• fix mkl nuget issue

Co-authored-by: Frank Dong frdong@microsoft.com

• fix code generator tests failure (fix code generator tests failure #5520)

• Added fixes for official builds

• Make .sh files executable

• fix mkl nuget issue

• fix code generate test fails

• only add necessary dependency

Co-authored-by: Mustafa Bal 5262061+mstfbl@users.noreply.github.com

• Fixed memory leaks from OnnxTransformer (Fixed memory leaks from OnnxTransformer #5518)

• Fixed memory leak from OnnxTransformer and related x86 build fixes

• Reverting x86 build related fixes to focus only on the memory leaks

• Updated docs

• Reverted OnnxRuntimeOutputCatcher to private class

• Addressed code review comments

• Refactored OnnxTransform back to using MapperBase based on code review comments

• Handle integration tests and nightly build testing (Handle integration tests and nightly build testing #5509)

• Make -integrationTests work

• Update .yml file

• Added the TargetArchitecture properties

• Try out -integrationTest

• Missed -integrationTest flag

• Renamed FunctionalTestBaseClass to IntegrationTestBaseClass

• Missed rename

• Modified tests to make them more stable

• Fixed leak in object pool (Fixed leak in object pool #5521)

Co-authored-by: frank-dong-ms 55860649+frank-dong-ms@users.noreply.github.com
Co-authored-by: Michael Sharp 51342856+michaelgsharp@users.noreply.github.com
Co-authored-by: Mustafa Bal 5262061+mstfbl@users.noreply.github.com
Co-authored-by: Frank Dong frdong@microsoft.com
Co-authored-by: Michael Sharp misharp@microsoft.com
Co-authored-by: Antonio Velázquez 38739674+antoniovs1029@users.noreply.github.com

• fix benchmark test timeout issue (fix benchmark test timeout issue #5530)

• removed old build stuff (removed old build stuff #5531)

• Fixes Code Coverage in Arcade (Fixes Code Coverage in Arcade #5528)

• arcade code coverage changes

• adding Michael's changes

• updating path

Co-authored-by: Keren Fuentes kedejesu@microsoft.com

• Removed CODEOWNERS file to unify review process (Removed CODEOWNERS file to unify review process #5535)

• Fix publishing problems (Fix publishing problems #5538)

• Removed our dependency to BuildTools by using the NugetCommand Azure Task.

• We should publish a nuget named "SampleUtils", but we were publishing it with the name "SamplesUtils"

• The naming conventions of our published nugets didn't match the ones described on arcade's docs: Versioning.md. I've also added the option so that when queuing the publishing build, we can pass the VERSIONKIND variable with value "release", so that it produces the nugets with arcade's conventions for "Release official build" nugets (as opposed to the "Daily official build" naming convention that's going to be used now by our CI that publishes nightly nugets).

• Updated prerelease label (Updated prerelease label #5540)

• Fix warnings from CI Build (Fix warnings from CI Build #5541)

• fix warnings

• also add conditional copy asset to native.proj

• test fix warnings

• supress nuget warning 5118

• supress other warning

• remove unnecessary change

• put skip warning at Directory.Buil.props

• Updated build instructions (Updated build instructions #5534)

• Updated build instructions

• Adressed reviews

• Reviews

• removed the rest of the old pkg references: (Removed the rest of the old pkg references: #5537)

• Perf improvement for TopK Accuracy and return all topK in Classification Evaluator (Perf improvement for TopK Accuracy and return all topK in Classification Evaluator #5395)

• Fix for issue 744

• cleanup

• fixing report output

• fixedTestReferenceOutputs

• Fixed test reference outputs for NetCore31

• change top k acc output string format

• Ranking algorithm now uses first appearance in dataset rather than worstCase

• fixed benchmark

• various minor changes from code review

• limit TopK to OutputTopKAcc parameter

• top k output name changes

• make old TopK readOnly

• restored old baselineOutputs since respecting outputTopK param means no topK in most test output

• fix test fails, re-add names parameter

• Clean up commented code

• that'll teach me to edit from the github webpage

• use existing method, fix nits

• Slight comment change

• Comment change / Touch to kick off build pipeline

• fix whitespace

• Added new test

• Code formatting nits

• Code formatting nit

• Fixed undefined rankofCorrectLabel and trailing whitespace warning

• Removed _numUnknownClassInstances and added test for unknown labels

• Add weight to seenRanks

• Nits

• Removed FastTree import

Co-authored-by: Antonio Velazquez anvelazq@microsoft.com
Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Fixed Spelling on stopwords (Fixed Spelling on stopwords #5524)

• Changes to onnx export. (Changes to onnx export. #5544)

• Add back missing test project from running on arcade (Add back missing test project from running on arcade #5545)

• add back test result upload and add missing test project from running

• fix identification

• filter out performance test result files to avoid warnings

• [CodeGenerator] Fix MLNet.CLI build error. ([CodeGenerator] Fix MLNet.CLI build error. #5546)

• upgrade to 3.1

• write inline data using invariantCulture

• fix mlnet build error

• Fixed AutoML CrossValSummaryRunner for TopKAccuracyForAllK (Fixed AutoML CrossValSummaryRunner for TopKAccuracyForAllK #5548)

• Fixed bug

• Tensorflow fix (Tensorflow fix #5547)

• fix tensorflow issue on sample repo

• add comments

• Update to OnnxRuntime 1.6.0 and fixed bug with sequences outputs (Update to OnnxRuntime 1.6.0 and fixed bug with sequences outputs #5529)

• Use onnx prerelease

• Upgrade to onnx 1.6.0

• Updated docs

• Fixed problem with sequences

• added in DcgTruncationLevel to AutoML api (added in DcgTruncationLevel to AutoML api #5433)

• added in DcgTruncationLevel to automl api

• changed default to 10

• updated basline output

• fixed failing tests and baselines

• Changes from PR comments.

• Update src/Microsoft.ML.AutoML/Experiment/MetricsAgents/RankingMetricsAgent.cs

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Changes based on PR comments.

• Fix ranking test.

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Created release notes for v1.5.3 (Created release notes for v1.5.3 #5543)

• Created release notes for v1.5.3

• Updated with review comments

• Updated with review comments

• Updated release notes with latest PRs

• Fixed typo

• Forward logs of Experiment's sub MLContexts to main MLContext (Forward logs of Experiment's sub MLContexts to main MLContext #5554)

• Forward logs of Experiment's sub MLContexts to main MLContext

• Adressed reviews

• Update Stale docs (Update Stale docs #5550)

• Updated OnnxMl.md

• Updated MlNetMklDeps docs

• Typo

• typo

• continueOnError on Brew Workaround (continueOnError on Brew Workaround #5555)

• continueOnError:true

• Fix publishing symbols (Fix publishing symbols #5556)

• Disable Portable PDB conversion

• Push packages to artifacts

• Fix symbols issues

• Added note about Microsoft.ML.dll

• try out just packing

• Return Build=false, but actually use configuration

• Added missing TargetArchitecture

• add back tests

• Added missing flags

• Updated version to 1.5.4 (Updated version to 1.5.4 #5557)

• Fixed version numbers in the right place (Fixed version numbers in the right place #5558)

• Updated version to 1.5.4

• Updated version to 1.5.4

• eng (Updated version to 1.5.5 and 0.17.5 #5560)

• Renamed release notes file (Renamed release notes file #5561)

• Renamed release notes file

• Updated version number in release notes

• Add SymSgdNative reference to AutoML.Tests.csproj (Add SymSgdNative reference to AutoML.Tests.csproj #5559)

• runSpecific in YAML

• RunSpecific in test

• Add SymSgdNative reference

• Revert "RunSpecific in test"

This reverts commit fed12b2.

• Revert "runSpecific in YAML"

This reverts commit f9f328d.

• Nuget.config url fix for roslyn compilers (Nuget.config url fix for roslyn compilers #5584)

• fixed nuget url, versions, and failing tests

• changes from pr comments and MacOS changes

• MacOS homebrew bug workaround

• removed unnused nuget url

• added in note that PredictionEngine is not thread safe (Added in note in the documentation that the PredictionEngine is not thread safe. #5583)

• Onnx Export for ValueMapping estimator (Onnx Export for ValueMapping estimator #5577)

• Fixed Averaged Perceptron default value (Fixed Averaged Perceptron default value #5586)

• fixed missed averaged perceptron default value

• fixed extension api

• fixed test baselines

• fixing official build (Fixing official build #5596)

• Release/1.5.4 fix ( Release/1.5.4 fix #5599)

• Nuget.config url fix for roslyn compilers (Nuget.config url fix for roslyn compilers #5584)

• fixed nuget url, versions, and failing tests

• changes from pr comments and MacOS changes

• MacOS homebrew bug workaround

• removed unnused nuget url

• fixing official build (Fixing official build #5596)

• Remove references to Microsoft.ML.Scoring (Remove references to Microsoft.ML.Scoring #5602)

This was the very first ONNX .NET bindings, it was replaced with Microsoft.ML.OnnxRuntime
then Microsoft.ML.OnnxRuntime.Managed.

• Make ColumnInference serializable (Make ColumnInference serializable #5611)

• upgrade to 3.1

• write inline data using invariantCulture

• make column inference serializable

• add test json

• add approvaltests

• fixerd nuget.config (fixed nuget.config multi-feed #5614)

• Fix issue in SRCnnEntireAnomalyDetector (Fix issue in SRCnnEntireAnomalyDetector #5579)

• update

• refine codes

• update comments

• update for nit

Co-authored-by: yuyi@microsoft.com Yuanxiang.Ying@microsoft.com

• Offer suggestions for possibly mistyped label column names in AutoML (Improve usability of AutoML column not found error #5574) (Offer suggestions for possibly mistyped label column names in AutoML (#5574) #5624)

• Offer suggestions for possibly mistyped label column names

• review changes

• TimeSeries - fix confidence parameter type for some detectors (DetectSpikeBy(Ssa|Iid) confidence param should be double #4058) (TimeSeries - fix confidence parameter type for some detectors (#4058) #5623)

• TimeSeries - fix confidence parameter type for some detectors.

• The public API exposed confidence parameters as int even though it's internally implemented as double
• There was no workaround since all classes where double is used are internal
• This caused major issues for software requiring high precision predictions
• This change to API should be backwards compatible since int can be passed to parameter of type double

• TimeSeries - reintroduce original methods with confidence parameter of type int (to not break the API).

• TimeSeries - make catalog API methods with int confidence parameter deprecated.

• Tests adjusted to not use the deprecated methods

• Update Conversion.cs (Update Conversion.cs #5627)

• Documentation updates (Documentation updates #5635)

• documentation updates

• fixed spelling error

• Update docs/building/unix-instructions.md

Co-authored-by: Santiago Fernandez Madero safern@microsoft.com

Co-authored-by: Santiago Fernandez Madero safern@microsoft.com

• AutoML aggregate exception (AutoML aggregate exception #5631)

• added check for aggregate exception

• Update src/Microsoft.ML.AutoML/Experiment/Experiment.cs

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• Update src/Microsoft.ML.AutoML/Experiment/Experiment.cs

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• pulled message out to private variable so its not duplicated

• Update src/Microsoft.ML.AutoML/Experiment/Experiment.cs

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com
Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Treat TensorFlow output as non-batched. (Treat TensorFlow output as non-batched. #5634)

• Can now not treat output as batched.

• updated comments based on PR comments.

• Fixing saving/loading with new parameter.

• Updates based on PR comments

• Update src/Microsoft.ML.TensorFlow/TensorflowUtils.cs

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• reverted accidental test changes

• fixes based on PR comments

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• Added in release notes for 1.5.5 (Added in release notes for 1.5.5 #5639)

• added in release notes

• Update release-1.5.5.md

Removed incorrect PR.

• Update docs/release-notes/1.5.5/release-1.5.5.md

Co-authored-by: Eric StJohn ericstj@microsoft.com

• Update docs/release-notes/1.5.5/release-1.5.5.md

Co-authored-by: Eric StJohn ericstj@microsoft.com

• Update release-1.5.5.md

Co-authored-by: Eric StJohn ericstj@microsoft.com

• updating version after release (Updating version after release #5642)

• Move DataFrame to machinelearning (Move DataFrame to machinelearning #5641)

• Change namespace to Microsoft.Data.Analysis (Sample for ReplaceMissingValues. #2773)

• Update namespace to Microsoft.Data.Analysis

• Remove "DataFrame" from the test project name

• APIs for reversed binary operators (BinaryClassificationMetrics is not sealed - but equivalent classes for regression and multiclass are #2769)

• Support reverse binary operators

• Fix file left behind in a rebase

• Fix whitespace

• Throw for incompatible inPlace (Possible Bug Tensorflow in v0.11 - Invalid argument: input must be 4-dimensional[224,224,3] #2778)

• Throw if inPlace is set and types mismatch

• Unit test

• Better error message

• Remove empty lines

• Version, Tags and Description for Nuget (OneHotEncoding sample #2779)

• Version, Tags and Description for Nuget

• sq

• Flags for release (Sample for ConvertType transform estimator #2781)

• Publish packages to artifacts

• Flags for release

• Fix the Description method to not throw (Rename Microsoft.ML.StandardLearners to Microsoft.ML.StandardTrainers. #2786)

• Fix the Description method to not crash
  Adds an Info method

• sq

• Address feddback

• Last round of feedback

• Use dataTypes if it passed in to LoadCsv (Error when using OnnxTransformer in two AppDomains #2791)

• Fix LoadCsv to use dataType if it passed in

• sq

• Don't read the full file after guessRows lines have been read

• Address feedback

• Last round of feedback

• Creating a Rows property, similar to Columns (SamplesUtils is not searchable on doc site and makes ML.NET learning curve sharp #2794)

• Rows collection, similar to Columns

• Doc

• Some minor clean up

• Make DataFrameRow a view into the DataFrame

• sq

• Address feedback

• Remove DataFrame.RowCount

• More row count changes

• sq

• Address feedback

• Merge upstream

• DataFrame.LoadCsv throws an exception on projects targeting < netcore3.0 (Refactor cancellation mechanism and make it internal, accessible via experimental nuget. #2797)

Fixing by passing in an encoding and a default buffer size.

Also, get our tests running on .NET Framework.

Fix #2783

• Params constructor on DataFrame (Rename HalLearners assembly and nuget to Microsoft.ML.Mkl.Components. #2800)

• Params constructor on DataFrame

• Delete redundant constructors

• Remove T : unmanaged constraint from DataFrameColumn.BinaryOperations (TextFeaturizer API is non-standard #2801)

• Remove T : unmanaged constraint from DataFrameColumn.BinaryOperations

• Address feedback

• Rename the value version of the APIs

• sq

• Fix build

• Address feedback

• Remove Value from the APIs

• sq

• Address feedback

• Bump version to 0.2.0 (Add V1 Scenario tests for data transformation #2803)

• Add Applymethod to PrimitiveDataFrameColumn (Make accessor of linear coefficients unique to the public #2807)

• Add Apply method to PrimitiveDataFrameColumn and its container

• Add TestApply test

• Remove unused df variable in DataFrameTests

• Add xml doc comments to Apply method

• Add additional tests for ReadCsv (Read and write binary file documentation #2811)

• Add additional tests for ReadCsv

• Update asserts

• Add empty row and skip test pending another fix

• Remove test for another issue

• Added static factory methods to DataFrameColumn (Scrub changes for LightGBM #2808)

• Added static factory methods to DataFrameColumn where they make sense (for the overloads where its possible to infer the column's type).

• Remove regions

• Update some parts of the unit tests to use static factory methods to create DataFrameColumns.

• Remove errant {T} on StringDataFrameColumn.

• PR feedback

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• Append rows to a DataFrame (can not register #2823)

• Append rows to a DataFrame

• Unit test

• Update unit tests and doc

• Need to perfrom a type check every time

• sq

• Update unit test

• Address comments

• Move corefxlab to arcade (Refactor cancellation mechanism and make it internal, accessible via experimental nuget. #2795)

• Add eng folder

• First cut of moving corefxlab to arcade

• Move arcade symbol validation inside official buil

• Move base yml file to root

• Arcade will build, publish packages and symbols

• UpdateXlf. Review this

• Arcade Update to version 5.0.0-beta.19575.4 to include Experimental Channel

• Remove property that was causing the build to fail

• Moving global properties to the main Yaml instead of step in order to unblock publishing

• Committing xlfs and changing the build script to not update Xlf on build

• clean up corefxlab-base.yml

• sq

• Delete unused files and scripts

• Get rid of all the xlf stuff

• Remove UpdateXlfOnBuild for non-NT builds

• Minor cleanup

• More cleanup

• update eng\build.sh permission

• Rename to Nuget.config

• sq

• Remove the runtime spec from global.json

• Don't publish test projs

• Typo

• Move version prefix to versions.props
  Change prereleaselabel to alpha

• Increment version number to list as the latest package
  Increment version number of Microsoft.Experimental.Collections to list as the latest package
  Turn off graph generation

• Update the Readme

• Test removing the scripts folder

• Touch readme to force a change

• Address Jose's comments

• Typo

• Move versions to eng/versions.props

• Benchmark.proj needs to refer to xunit

• Clean up dependencies.props

• Remove dependencies.props

Co-authored-by: Jose Perez Rodriguez joperezr@microsoft.com

• Rename Sort to OrderBy (Add sample for IndicateMissingValues #2814)

• Rename sort to orderby and add orderbydescending method

• Add doc strings

• Update bench mark test

• Update tests

• Update DataFrameColumn to use orderby

• Update doc comment

• Additions to sortby

• Revert "Additions to sortby"

This reverts commit 3931d4e2a72ce44a539be7c27b2592395f3efd35.

• Revert "Update doc comment"

This reverts commit 192f7797fe2b77625486637badf77046162fedbf.

• Revert "Update DataFrameColumn to use orderby"

This reverts commit 8f94664c5fd18570cd2b601535e816ca5dd5e3c4.

• Explode column types and generate converters (GAM parameter names are non-standard #2857)

• Explode column types and generate converters

• Clean this

• sq

• sq

• Cherry pick for next commit

• sq

• Undo unnecessary change

• Address remaining concerns from the 2nd DataFrame API Review (Update release for 0.11 #2861)

• Move string indexer to Columns

• API changes from the 2nd API review

• Unit tests

• Address comments

• Add binary operations and operators on the exploded columns (Updating MKL #2867)

• Generate combinations of binary operations and Add

• Numeric Converters and CloneAsNumericColumns

• Binary, Comparison and Shift operations

• Clean up and bug fix

• Fix the binary op apis to not be overridden

• Internal constructors for exploded types

• Proper return types for exploded types

• Update unit tests

• Update csproj

• Revert "Fix the binary op apis to not be overridden"

This reverts commit 2dc2240c9449930139c1492d1388d5e1f8ba5fa1.

• Bug fix and unit test

• Constructor that takes in a container

• Unit tests

• Call the implementation where possible

• Review sq

• sq

• Cherry pick for next commit

• sq

• Undo unnecessary change

• Rename to the system namespace column types

• Address comments

• Push to pull locally

• Mimic C#'s arithmetic grammar in DataFrame

• Address feedback

• Reduce the number of partial column definitions

• Address feedback

• Add APIs to get the strongly typed columns from a DataFrame (Make array values intended to be immutable IReadOnlyList #2878)

• CP

• sq

• sq

• Improve docs

• Enable xml docs for Data.Analysis (Non-standard naming in L-BFGS Learners (LogisticRegression, PoissonRegression) #2882)

• Enable xml docs for Data.Analysis

• Fix /// summary around inheritdoc

• Minor doc changes

• sq

• sq

• Address feedback

• Add Apply to ArrowStringDataFrameColumn (Temporarily disable myget #2889)

• Support for Exploded columns types in Arrow and IO scenarios (Main namespace types2445 #2885)

• Support for Exploded columns types in Arrow and IO scenarios

• Unit tests

• Address feedback

• Bump version (Scrub Latent Dirichlet Allocation Transform (Just Renaming) #2890)

• Fix versioning to allow for individual stable packages (Scrub word embedding transform #2891)

• Fix versioning to allow for individual stable packages

• sq

• Bump Microsoft.Data.Analysis version to 0.4.0 (Scrubbing online learners #2892)

• Bump Microsoft.Data.Analysis version to 0.4.0

• Fix Passing in DataTypes to DataFrame.LoadCsv loses column name information corefxlab#2906 (TrainTestSplit should be inside MLContext.Data #2907)

• Fix Passing in DataTypes to DataFrame.LoadCsv loses column name information corefxlab#2906

• Improvements and unit tests

• sq

• Better fix

• sq

• Improve LoadCsv to handle null values when deducing the column types (Polish char- and word-level tokenizers & stopword removers #2916)

• Unit test to repro

• Fix DataFrame.LoadCsv doesn't handle null values when it attempts to guess the column types corefxlab#2915

Append a null value to a column when encountering it instead of changing the column type to a StringDataFrameColumn

• Update src/Microsoft.Data.Analysis/DataFrame.IO.cs

Co-authored-by: Günther Foidl gue@korporal.at

• Update src/Microsoft.Data.Analysis/DataFrame.cs

Co-authored-by: Günther Foidl gue@korporal.at

• Feedback

Co-authored-by: Günther Foidl gue@korporal.at

• Create a 0.4.0 package (Scrub text normalizer #2918)

• Revert "Create a 0.4.0 package (Scrub text normalizer #2918)" (One name for MulticlassClassification #2919)

This reverts commit 0bef531.

• Produce a 0.4.0 build (OVA Multiclass Classification can be instantiated for variety of sub-trainer training tasks #2920)

• Default Length for StringDataFrameColumn (Adding functional tests for all training scenarios #2921) (Logistic Regression NumberOfIterations to MaximumNumberOfIterations #2923)

• Increment version and stop producing stable packages (Logistic Regression: NumberOfIterations should be MaximumNumberOfIterations #2922)

• Increment version and stop producing stable packages

• Add DataFrame object formatter. (Train FieldAwareFactorizationMachines without providing arguments #2931)

• Add DataFrame object formatter.

• Update nuget dependencies.

• Apply CR fixes.

• Fix a bug in InsertColumn

• Add Microsoft.Data.Analysis.nuget project (Knowing what to cast the model to is hard #2933)

• Add DataFrame object formatter.

• Update nuget dependencies.

• Apply CR fixes.

• Remove ReferenceOutputAssembly added to from Microsoft.Data.Analysys.csproj.

• Add Microsoft.Data.Analysis.nuget project.

• Move project to src. Fix nuget project settings.

• Remove NoBuild property from project.

• Remove IncludeBuildOutput and IncludeSymbols from project.

• Add VersionPrefix to project.

• Add IncludeBuildOutput property.

• Add unit tests.

• Downgrade from netcoreapp3.1 to netcoreapp3.0

• Upgrade from netcoreapp3.0 to netcoreapp3.1 (dotnet interactive is not compatible with 3.0)

• Add netcoreapp3.1 to global settings

• Add dotnet 3.1.5 runtime to global settings

• Build fixes

• Moving MDAI into interactive-extensions folder of the package

• Minor refactoring

• Respond to PR feedback

Co-authored-by: Prashanth Govindarajan prgovi@microsoft.com
Co-authored-by: Jose Perez Rodriguez joperezr@microsoft.com
Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• ColumnName indexer on DataFrame (Hiding of ColumnOptions #2959)

• ColumnName indexer on DataFrame

Fixes dotnet/corefxlab#2934

• Unit tests

• Null column name

• Implement FillNulls() for ArrowStringDataFrameColumn with inPlace: false (Updating OVA tests #2956)

• implement FillNulls method for ArrowStringDataFrameColumn

• additional asserts for testcase

• Prevent DataFrame.Sample() method from returning duplicated rows (CpuMath Enhancement: Make bound checking of loops in hardware intrinsics more efficient #2939)

• resolves My data view has a label column named 'Score' #2806

• replace forloop with ArraySegment

• reduce shuffle loop operations from O(Rows.Count) to O(numberOfRows)

• Add WriteCsv plus unit tests. (FeaturizeText: Add instructions to turn off char- or word-gram generation to the tooltip. #2947)

• Add WriteCsv plus unit tests.

• Add CultureInfo to WriteCsv. Remove index column param. Update unit tests.

• Add CR changes. CultureInfo. Separator.

• Format decimal types individually. Fix culture info. Fix unit tests.

• Format decimal types individually. Fix culture info. Fix unit tests.

• Missing values default to a StringDataFrameColumn (ONNX Transform changes output types #2982)

• Make LoadCsv more robust

• Test empty string column

• Retain prev guess where possible

• Update FromArrowRecordBatches for dotnet-spark (Public API for SchemaDefinition class #2978)

• Support for RecordBatches with StructArrays

• Sq

• Address comments

• Nits

• Nits

• Implement DataFrame.LoadCsvFromString (Load a model by path #2988)

• Implement DataFrame.LoadCsvFromString

• Address comments

• Part 1 of porting the csv reader (Add support for cancelling of training #2997)

• Move to the test folder

• Suppress warnings

• Move extensions reference out of props

Make MDA.test use the props defined TFM
Comment out 2 unit tests

• Address feedback

• Address feedback

• Default to preview version

• Update nuget.config

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com
Co-authored-by: Haytam Zanid 34218324+zHaytam@users.noreply.github.com
Co-authored-by: Jon Wood jwood803@users.noreply.github.com
Co-authored-by: Sam 1965570+MgSam@users.noreply.github.com
Co-authored-by: Jose Perez Rodriguez joperezr@microsoft.com
Co-authored-by: Günther Foidl gue@korporal.at
Co-authored-by: Rhys Parry rhys@i-think22.net
Co-authored-by: daniel costea dcostea@users.noreply.github.com
Co-authored-by: Ramon 56896136+RamonWill@users.noreply.github.com

• Update to the latest Microsoft.DotNet.Interactive (Update to the latest Microsoft.DotNet.Interactive #5710)

• Update to the latest Microsoft.DotNet.Interactive

• Add System.CommandLine nuget feed

• Fix Data.Analysis.Interactive test

• added main branch to yml files (Added main branch to yml files #5715)

• Renamed master to main (Renamed master to main #5717)

• renamed master to main

• Update vsts-ci.yml

• updated urls

• renamed master to main (Renamed master to main #5719)

• IDataView to DataFrame (IDataView to DataFrame #5712)

• IDataView -> DataFrame

Implement the virtual function

• More APIs and unit tests

• ANother unit test

• Address feedback

• Last bit of feedback

• Fix some stuff and unit tests

• sq

• Move RowCursor back

• Remove unused param

Docs
maxRows
More unit tests
Fixed ArrowStringDataFrameColumn construction in the unit test

• Improve csv parsing (Improve csv parsing #5711)

• Part 2 of TextFieldParser.

Next up is hooking up ReadCsv to use TextFieldParser

• Make LoadCsv use TextFieldParser

• More unit tests

• cleanup

• Address feedback

• Last bit of feedback

• Remove extra var

• Remove duplicate file

• Rename strings.resx to Strings.resx

• rename the designer.cs file too

• Fix doc markdown (Fix doc markdown Fixes #5731 #5732)

Fixed documentation markdown remarks for

• MulticlassClassificationMetrics.LogLoss
• MulticlassClassificationMetrics.LogLossReduction

Signed-off-by: Robin Windey ro.windey@gmail.com

• Use Official package for SharpZipLib (Use Official package for SharpZipLib #5735)

Co-authored-by: Xiaoyun Zhang bigmiao.zhang@gmail.com
Co-authored-by: BigBigMiao BigBigMiao@github.com
Co-authored-by: Keren Fuentes dkeren@seas.upenn.edu
Co-authored-by: Keren Fuentes kedejesu@microsoft.com
Co-authored-by: Yuanxiang Ying yingyuanxiang34@sina.com
Co-authored-by: yuyi@microsoft.com Yuanxiang.Ying@microsoft.com
Co-authored-by: Antonio Velázquez 38739674+antoniovs1029@users.noreply.github.com
Co-authored-by: Mustafa Bal 5262061+mstfbl@users.noreply.github.com
Co-authored-by: Piotr Telman ptelman@users.noreply.github.com
Co-authored-by: Justin Ormont justinormont@users.noreply.github.com
Co-authored-by: Antonio Velazquez anvelazq@microsoft.com
Co-authored-by: frank-dong-ms 55860649+frank-dong-ms@users.noreply.github.com
Co-authored-by: Harish Kulkarni harishsk@users.noreply.github.com
Co-authored-by: Michael Sharp 51342856+michaelgsharp@users.noreply.github.com
Co-authored-by: Frank Dong frdong@microsoft.com
Co-authored-by: Michael Sharp misharp@microsoft.com
Co-authored-by: Jason DeBoever github@deboever.us
Co-authored-by: Leo Gaunt 36968548+LeoGaunt@users.noreply.github.com
Co-authored-by: Keren Fuentes kerenfuentes313@gmail.com
Co-authored-by: Eric StJohn ericstj@microsoft.com
Co-authored-by: Ivan Agarský agarskyivan@gmail.com
Co-authored-by: Andrej Kmetík akmetik@gmail.com
Co-authored-by: Phan Tấn Tài 37982283+4201104140@users.noreply.github.com
Co-authored-by: Santiago Fernandez Madero safern@microsoft.com
Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com
Co-authored-by: Prashanth Govindarajan prgovi@microsoft.com
Co-authored-by: Haytam Zanid 34218324+zHaytam@users.noreply.github.com
Co-authored-by: Jon Wood jwood803@users.noreply.github.com
Co-authored-by: Sam 1965570+MgSam@users.noreply.github.com
Co-authored-by: Jose Perez Rodriguez joperezr@microsoft.com
Co-authored-by: Günther Foidl gue@korporal.at
Co-authored-by: Rhys Parry rhys@i-think22.net
Co-authored-by: daniel costea dcostea@users.noreply.github.com
Co-authored-by: Ramon 56896136+RamonWill@users.noreply.github.com
Co-authored-by: Robin Windey ro.windey@gmail.com

• Actually merge from main (Fixed the syntax of cited example. #2)

• update tensorflow.net to 0.20.0 (update tensorflow.net to 0.20.0 #5404)

• upgrade to 3.1

• write inline data using invariantCulture

• upodate tensorflow

• update Microsoft.ML.Vision

• fix test && comment

• udpate tensorflow.net to 0.20.1

• update tf major version

• downgrade tf runtime to 1.14.1

• Update Dependencies.props

• Update Dependencies.props

• update tffact to stop running test on linux with glibc < 2.3)

• fix TensorFlowTransformInputShapeTest

• use tf.v1 api

• fix comment:

• fix building error

• fix test

• fix nit

• remove linq

Co-authored-by: BigBigMiao BigBigMiao@github.com

• ProduceWordBags Onnx Export Fix (ProduceWordBags Onnx Export Fix  #5435)

• fix for issue

• fix documentation

• aligning test

• adding back line

• aligning fix

Co-authored-by: Keren Fuentes kedejesu@microsoft.com

• [SrCnnEntireAnomalyDetector] Upgrade boundary calculation and expected value calculation ([SrCnnEntireAnomalyDetector] Upgrade boundary calculation and expected value calculation #5436)

• adjust expected value

• update boundary calculation

• fix boundary

• adjust default values

• fix percent case

• fix error in anomaly score calculation

Co-authored-by: yuyi@microsoft.com Yuanxiang.Ying@microsoft.com

• Update OnnxRuntime to 1.5.2 (Update OnnxRuntime to 1.5.2 #5439)

• Added prerelease feed and updated to 1.5.2

• Remove prerelease feed

• Updated docs

• Update doc

• Fixed MacOS CI Pipeline builds (Fixed MacOS CI Pipeline builds #5457)

• Added MacOS Homebrew bug fix

• nit fix

• Improving error message (Improving error message  #5444)

• better error fix

• revisions

Co-authored-by: Keren Fuentes kedejesu@microsoft.com

• Fixed MacOS daily & nightly builds due to Homebrew bug (Fixed MacOS daily & nightly builds due to Homebrew bug #5467)

• Fixed MacOS nightly builds due to Homebrew bug

• Edit workaround

• Remove untapping of python2

• Nit edit

• Remove installation of mono-libgdiplus

• try installing mono-libgdiplus

• unlink python 3.8

• Auto.ML: Fix issue when parsing float string fails on pl-PL culture set using Regression Experiment (Auto.ML: Fix issue when parsing float string fails on pl-PL culture set using Regression Experiment #5163)

• Fix issue when parsing float string fails on pl-PL culture set

• Added InvariantCulture float parsing as per CodeReview request

• Update src/Microsoft.ML.AutoML/Sweepers/SweeperProbabilityUtils.cs

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Update Parameters.cs

• Added PL test

• Added multiple cultures

• debugging CI failure

• Debug runSpecific

• Revert "Debug runSpecific"

This reverts commit 95b7280.

• Removed LightGBM and addressed comments

• Increased time

• Increase time

• Increased time

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com
Co-authored-by: Antonio Velazquez anvelazq@microsoft.com

• handle exception during GetNextPipeline for AutoML (handle exception during GetNextPipeline for AutoML #5455)

• handle exception during GetNextPipeline for AutoML

• take comments

• Changing LoadRawImages Sample (Changing LoadRawImages Sample #5460)

replacing example

Co-authored-by: Keren Fuentes kedejesu@microsoft.com

• Use Timer and ctx.CancelExecution() to fix AutoML max-time experiment bug (Use Timer and ctx.CancelExecution() to fix AutoML max-time experiment bug #5445)

• Use ctx.CalncelExecution() to fix AutoML max-time experiment bug

• Added unit test for checking canceled experiment

• Nit fix

• Different run time on Linux

• Review

• Testing four ouput

• Used reflection to test for contexts being canceled

• Reviews

• Reviews

• Added main MLContext listener-timer

• Added PRNG on _context, held onto timers for avoiding GC

• Addressed reviews

• Unit test edits

• Increase run time of experiment to guarantee probabilities

• Edited unit test to check produced schema of next run model's predictions

• Remove scheme check as different CI builds result in varying schemas

• Decrease max experiment time unit test time

• Added Timers

• Increase second timer time, edit unit test

• Added try catch for OperationCanceledException in Execute()

• Add AggregateException try catch to slow unit tests for parallel testing

• Reviews

• Final reviews

• Added LightGBMFact to binary classification test

• Removed extra Operation Stopped exception try catch

• Add back OperationCanceledException to Experiment.cs

• fix issue 5020, allow ML.NET to load tf model with primitive input and output column (fix issue 5020, allow ML.NET to load tf model with primitive input and output column #5468)

• handle exception during GetNextPipeline for AutoML

• take comments

• Enable TesnflowTransformer take primitive type as input column

• undo unnecessary changes

• add test

• update on test

• remove unnecessary line

• take comments

• maxModels instead of time for AutoML unit test (MaxModels exit criteria for AutoML unit test #5471)

Uses the internal maxModels parameter instead of MaxExperimentTimeInSeconds for the exit criteria of AutoML.

This is to increase the test stability in case the test is run on a slower machine.

• Disabling AutoFitMaxExperimentTimeTest

Disabling AutoFitMaxExperimentTimeTest

• Fix AutoFitMaxExperimentTimeTest (Fix AutoFitMaxExperimentTimeTest #5506)

*Fixed test
Co-authored-by: Antonio Velazquez anvelazq@microsoft.com

• Fix SR anomaly score calculation at beginning (Fix SR anomaly score calculation at beginning #5502)

• adjust expected value

• update boundary calculation

• fix boundary

• adjust default values

• fix percent case

• fix error in anomaly score calculation

• adjust score calculation for first & second points

• fix sr do not report anomaly at beginning

• fix a issue in batch process

• remove a unused parameter

Co-authored-by: yuyi@microsoft.com Yuanxiang.Ying@microsoft.com

• Merge arcade to master (Merge arcade to master #5525)

• Initial commit for Arcade migration

• Added omitted files

• Changed strong name signing to use the same key for shipping and test assemblies

• arcade linux build (arcade linux build #5423)

• arcade linux build

• put file execution permission change into source control

• The -test command for windows. Nuget packages (The -test command for windows. Nuget packages #5464)

• working on testing

• testing updates

• tests almost working

• build changes

• all tests should be working

• changes from PR comments

• fixes for .net 3.1

• Fixed extension check. Removed where not needed

• Removed pkg folder and updated paths.

• Added test key. (Added test key. #5475)

• Added test key.

• Update PublicKey.cs

Removed extra newline.

• Update ComponentCatalog.cs

Fixed 3 spaces to 4.

• Windows CI working (Windows CI working #5477)

• ci testing changes

• comments from pr

• Added Linux & Mac changes for Arcade (Added Linux & Mac changes for Arcade #5479)

• Initial Windows, Linux, Macos builds test

• Add Linux/MacOS specific CI requirements

• Run Arcade CI tests on MacOS/Linux

• Fix final package building

• Add benchmark download to benchmars .csporj file

• Print detailed status of each unit test

• Install CentOS & Ubuntu build dependencies

• Use container names to differenciate between Ubuntu & CentOS

• Remove sudo usage in CentOS

• Fix Linux build dependencies

• Add -y param to apt install

• Remove installation of Linux dependencies

• Minor additions

• Rename Benchmarks to PerformanceTests for Arcade

• Changes

• Added benchmark doc changes

• Pre-merge changes

• Fixing failing Arcade Windows Builds (Fixing failing Arcade Windows Builds #5482)

• Try Windows build single quote fix

• Remove %20

• Added variable space value

• Using variables for spacing

• Added space values as job parameters

• Try conditional variables again

• fix official builds

• Revert "fix official builds"

This reverts commit 7dbbdc7.

• fixing tensorflow rebase issue

• Fixes for many of the CI builds. (Fixes for many of the CI builds. #5496)

• yml log changes

• Fix NetFX builds by ensuring assembly version is set correctly and not to Arcade default of 42.42.42.42 (Fix NetFX builds by ensuring assembly version is set correctly #5503)

• Fixed official builds for Arcade SDK (Fixed official builds for Arcade SDK #5512)

• Added fixes for official builds

• Make .sh files executable

• fix mkl nuget issue

Co-authored-by: Frank Dong frdong@microsoft.com

• fix code generator tests failure (fix code generator tests failure #5520)

• Added fixes for official builds

• Make .sh files executable

• fix mkl nuget issue

• fix code generate test fails

• only add necessary dependency

Co-authored-by: Mustafa Bal 5262061+mstfbl@users.noreply.github.com

• Fixed memory leaks from OnnxTransformer (Fixed memory leaks from OnnxTransformer #5518)

• Fixed memory leak from OnnxTransformer and related x86 build fixes

• Reverting x86 build related fixes to focus only on the memory leaks

• Updated docs

• Reverted OnnxRuntimeOutputCatcher to private class

• Addressed code review comments

• Refactored OnnxTransform back to using MapperBase based on code review comments

• Handle integration tests and nightly build testing (Handle integration tests and nightly build testing #5509)

• Make -integrationTests work

• Update .yml file

• Added the TargetArchitecture properties

• Try out -integrationTest

• Missed -integrationTest flag

• Renamed FunctionalTestBaseClass to IntegrationTestBaseClass

• Missed rename

• Modified tests to make them more stable

• Fixed leak in object pool (Fixed leak in object pool #5521)

Co-authored-by: frank-dong-ms 55860649+frank-dong-ms@users.noreply.github.com
Co-authored-by: Michael Sharp 51342856+michaelgsharp@users.noreply.github.com
Co-authored-by: Mustafa Bal 5262061+mstfbl@users.noreply.github.com
Co-authored-by: Frank Dong frdong@microsoft.com
Co-authored-by: Michael Sharp misharp@microsoft.com
Co-authored-by: Antonio Velázquez 38739674+antoniovs1029@users.noreply.github.com

• fix benchmark test timeout issue (fix benchmark test timeout issue #5530)

• removed old build stuff (removed old build stuff #5531)

• Fixes Code Coverage in Arcade (Fixes Code Coverage in Arcade #5528)

• arcade code coverage changes

• adding Michael's changes

• updating path

Co-authored-by: Keren Fuentes kedejesu@microsoft.com

• Removed CODEOWNERS file to unify review process (Removed CODEOWNERS file to unify review process #5535)

• Fix publishing problems (Fix publishing problems #5538)

• Removed our dependency to BuildTools by using the NugetCommand Azure Task.

• We should publish a nuget named "SampleUtils", but we were publishing it with the name "SamplesUtils"

• The naming conventions of our published nugets didn't match the ones described on arcade's docs: Versioning.md. I've also added the option so that when queuing the publishing build, we can pass the VERSIONKIND variable with value "release", so that it produces the nugets with arcade's conventions for "Release official build" nugets (as opposed to the "Daily official build" naming convention that's going to be used now by our CI that publishes nightly nugets).

• Updated prerelease label (Updated prerelease label #5540)

• Fix warnings from CI Build (Fix warnings from CI Build #5541)

• fix warnings

• also add conditional copy asset to native.proj

• test fix warnings

• supress nuget warning 5118

• supress other warning

• remove unnecessary change

• put skip warning at Directory.Buil.props

• Updated build instructions (Updated build instructions #5534)

• Updated build instructions

• Adressed reviews

• Reviews

• removed the rest of the old pkg references: (Removed the rest of the old pkg references: #5537)

• Perf improvement for TopK Accuracy and return all topK in Classification Evaluator (Perf improvement for TopK Accuracy and return all topK in Classification Evaluator #5395)

• Fix for issue 744

• cleanup

• fixing report output

• fixedTestReferenceOutputs

• Fixed test reference outputs for NetCore31

• change top k acc output string format

• Ranking algorithm now uses first appearance in dataset rather than worstCase

• fixed benchmark

• various minor changes from code review

• limit TopK to OutputTopKAcc parameter

• top k output name changes

• make old TopK readOnly

• restored old baselineOutputs since respecting outputTopK param means no topK in most test output

• fix test fails, re-add names parameter

• Clean up commented code

• that'll teach me to edit from the github webpage

• use existing method, fix nits

• Slight comment change

• Comment change / Touch to kick off build pipeline

• fix whitespace

• Added new test

• Code formatting nits

• Code formatting nit

• Fixed undefined rankofCorrectLabel and trailing whitespace warning

• Removed _numUnknownClassInstances and added test for unknown labels

• Add weight to seenRanks

• Nits

• Removed FastTree import

Co-authored-by: Antonio Velazquez anvelazq@microsoft.com
Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Fixed Spelling on stopwords (Fixed Spelling on stopwords #5524)

• Changes to onnx export. (Changes to onnx export. #5544)

• Add back missing test project from running on arcade (Add back missing test project from running on arcade #5545)

• add back test result upload and add missing test project from running

• fix identification

• filter out performance test result files to avoid warnings

• [CodeGenerator] Fix MLNet.CLI build error. ([CodeGenerator] Fix MLNet.CLI build error. #5546)

• upgrade to 3.1

• write inline data using invariantCulture

• fix mlnet build error

• Fixed AutoML CrossValSummaryRunner for TopKAccuracyForAllK (Fixed AutoML CrossValSummaryRunner for TopKAccuracyForAllK #5548)

• Fixed bug

• Tensorflow fix (Tensorflow fix #5547)

• fix tensorflow issue on sample repo

• add comments

• Update to OnnxRuntime 1.6.0 and fixed bug with sequences outputs (Update to OnnxRuntime 1.6.0 and fixed bug with sequences outputs #5529)

• Use onnx prerelease

• Upgrade to onnx 1.6.0

• Updated docs

• Fixed problem with sequences

• added in DcgTruncationLevel to AutoML api (added in DcgTruncationLevel to AutoML api #5433)

• added in DcgTruncationLevel to automl api

• changed default to 10

• updated basline output

• fixed failing tests and baselines

• Changes from PR comments.

• Update src/Microsoft.ML.AutoML/Experiment/MetricsAgents/RankingMetricsAgent.cs

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Changes based on PR comments.

• Fix ranking test.

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Created release notes for v1.5.3 (Created release notes for v1.5.3 #5543)

• Created release notes for v1.5.3

• Updated with review comments

• Updated with review comments

• Updated release notes with latest PRs

• Fixed typo

• Forward logs of Experiment's sub MLContexts to main MLContext (Forward logs of Experiment's sub MLContexts to main MLContext #5554)

• Forward logs of Experiment's sub MLContexts to main MLContext

• Adressed reviews

• Update Stale docs (Update Stale docs #5550)

• Updated OnnxMl.md

• Updated MlNetMklDeps docs

• Typo

• typo

• continueOnError on Brew Workaround (continueOnError on Brew Workaround #5555)

• continueOnError:true

• Fix publishing symbols (Fix publishing symbols #5556)

• Disable Portable PDB conversion

• Push packages to artifacts

• Fix symbols issues

• Added note about Microsoft.ML.dll

• try out just packing

• Return Build=false, but actually use configuration

• Added missing TargetArchitecture

• add back tests

• Added missing flags

• Updated version to 1.5.4 (Updated version to 1.5.4 #5557)

• Fixed version numbers in the right place (Fixed version numbers in the right place #5558)

• Updated version to 1.5.4

• Updated version to 1.5.4

• eng (Updated version to 1.5.5 and 0.17.5 #5560)

• Renamed release notes file (Renamed release notes file #5561)

• Renamed release notes file

• Updated version number in release notes

• Add SymSgdNative reference to AutoML.Tests.csproj (Add SymSgdNative reference to AutoML.Tests.csproj #5559)

• runSpecific in YAML

• RunSpecific in test

• Add SymSgdNative reference

• Revert "RunSpecific in test"

This reverts commit fed12b2.

• Revert "runSpecific in YAML"

This reverts commit f9f328d.

• Nuget.config url fix for roslyn compilers (Nuget.config url fix for roslyn compilers #5584)

• fixed nuget url, versions, and failing tests

• changes from pr comments and MacOS changes

• MacOS homebrew bug workaround

• removed unnused nuget url

• added in note that PredictionEngine is not thread safe (Added in note in the documentation that the PredictionEngine is not thread safe. #5583)

• Onnx Export for ValueMapping estimator (Onnx Export for ValueMapping estimator #5577)

• Fixed Averaged Perceptron default value (Fixed Averaged Perceptron default value #5586)

• fixed missed averaged perceptron default value

• fixed extension api

• fixed test baselines

• fixing official build (Fixing official build #5596)

• Release/1.5.4 fix ( Release/1.5.4 fix #5599)

• Nuget.config url fix for roslyn compilers (Nuget.config url fix for roslyn compilers #5584)

• fixed nuget url, versions, and failing tests

• changes from pr comments and MacOS changes

• MacOS homebrew bug workaround

• removed unnused nuget url

• fixing official build (Fixing official build #5596)

• Remove references to Microsoft.ML.Scoring (Remove references to Microsoft.ML.Scoring #5602)

This was the very first ONNX .NET bindings, it was replaced with Microsoft.ML.OnnxRuntime
then Microsoft.ML.OnnxRuntime.Managed.

• Make ColumnInference serializable (Make ColumnInference serializable #5611)

• upgrade to 3.1

• write inline data using invariantCulture

• make column inference serializable

• add test json

• add approvaltests

• fixerd nuget.config (fixed nuget.config multi-feed #5614)

• Fix issue in SRCnnEntireAnomalyDetector (Fix issue in SRCnnEntireAnomalyDetector #5579)

• update

• refine codes

• update comments

• update for nit

Co-authored-by: yuyi@microsoft.com Yuanxiang.Ying@microsoft.com

• Offer suggestions for possibly mistyped label column names in AutoML (Improve usability of AutoML column not found error #5574) (Offer suggestions for possibly mistyped label column names in AutoML (#5574) #5624)

• Offer suggestions for possibly mistyped label column names

• review changes

• TimeSeries - fix confidence parameter type for some detectors (DetectSpikeBy(Ssa|Iid) confidence param should be double #4058) (TimeSeries - fix confidence parameter type for some detectors (#4058) #5623)

• TimeSeries - fix confidence parameter type for some detectors.

• The public API exposed confidence parameters as int even though it's internally implemented as double
• There was no workaround since all classes where double is used are internal
• This caused major issues for software requiring high precision predictions
• This change to API should be backwards compatible since int can be passed to parameter of type double

• TimeSeries - reintroduce original methods with confidence parameter of type int (to not break the API).

• TimeSeries - make catalog API methods with int confidence parameter deprecated.

• Tests adjusted to not use the deprecated methods

• Update Conversion.cs (Update Conversion.cs #5627)

• Documentation updates (Documentation updates #5635)

• documentation updates

• fixed spelling error

• Update docs/building/unix-instructions.md

Co-authored-by: Santiago Fernandez Madero safern@microsoft.com

Co-authored-by: Santiago Fernandez Madero safern@microsoft.com

• AutoML aggregate exception (AutoML aggregate exception #5631)

• added check for aggregate exception

• Update src/Microsoft.ML.AutoML/Experiment/Experiment.cs

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• Update src/Microsoft.ML.AutoML/Experiment/Experiment.cs

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• pulled message out to private variable so its not duplicated

• Update src/Microsoft.ML.AutoML/Experiment/Experiment.cs

Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com
Co-authored-by: Justin Ormont justinormont@users.noreply.github.com

• Treat TensorFlow output as non-batched. (Treat TensorFlow output as non-batched. #5634)

• Can now not treat output as batched.

• updated comments based on PR comments.

• Fixing saving/loading with new parameter.

• Updates based on PR comments

• Update src/Microsoft.ML.TensorFlow/TensorflowUtils.cs

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• reverted accidental test changes

• fixes based on PR comments

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• Added in release notes for 1.5.5 (Added in release notes for 1.5.5 #5639)

• added in release notes

• Update release-1.5.5.md

Removed incorrect PR.

• Update docs/release-notes/1.5.5/release-1.5.5.md

Co-authored-by: Eric StJohn ericstj@microsoft.com

• Update docs/release-notes/1.5.5/release-1.5.5.md

Co-authored-by: Eric StJohn ericstj@microsoft.com

• Update release-1.5.5.md

Co-authored-by: Eric StJohn ericstj@microsoft.com

• updating version after release (Updating version after release #5642)

• Move DataFrame to machinelearning (Move DataFrame to machinelearning #5641)

• Change namespace to Microsoft.Data.Analysis (Sample for ReplaceMissingValues. #2773)

• Update namespace to Microsoft.Data.Analysis

• Remove "DataFrame" from the test project name

• APIs for reversed binary operators (BinaryClassificationMetrics is not sealed - but equivalent classes for regression and multiclass are #2769)

• Support reverse binary operators

• Fix file left behind in a rebase

• Fix whitespace

• Throw for incompatible inPlace (Possible Bug Tensorflow in v0.11 - Invalid argument: input must be 4-dimensional[224,224,3] #2778)

• Throw if inPlace is set and types mismatch

• Unit test

• Better error message

• Remove empty lines

• Version, Tags and Description for Nuget (OneHotEncoding sample #2779)

• Version, Tags and Description for Nuget

• sq

• Flags for release (Sample for ConvertType transform estimator #2781)

• Publish packages to artifacts

• Flags for release

• Fix the Description method to not throw (Rename Microsoft.ML.StandardLearners to Microsoft.ML.StandardTrainers. #2786)

• Fix the Description method to not crash
  Adds an Info method

• sq

• Address feddback

• Last round of feedback

• Use dataTypes if it passed in to LoadCsv (Error when using OnnxTransformer in two AppDomains #2791)

• Fix LoadCsv to use dataType if it passed in

• sq

• Don't read the full file after guessRows lines have been read

• Address feedback

• Last round of feedback

• Creating a Rows property, similar to Columns (SamplesUtils is not searchable on doc site and makes ML.NET learning curve sharp #2794)

• Rows collection, similar to Columns

• Doc

• Some minor clean up

• Make DataFrameRow a view into the DataFrame

• sq

• Address feedback

• Remove DataFrame.RowCount

• More row count changes

• sq

• Address feedback

• Merge upstream

• DataFrame.LoadCsv throws an exception on projects targeting < netcore3.0 (Refactor cancellation mechanism and make it internal, accessible via experimental nuget. #2797)

Fixing by passing in an encoding and a default buffer size.

Also, get our tests running on .NET Framework.

Fix #2783

• Params constructor on DataFrame (Rename HalLearners assembly and nuget to Microsoft.ML.Mkl.Components. #2800)

• Params constructor on DataFrame

• Delete redundant constructors

• Remove T : unmanaged constraint from DataFrameColumn.BinaryOperations (TextFeaturizer API is non-standard #2801)

• Remove T : unmanaged constraint from DataFrameColumn.BinaryOperations

• Address feedback

• Rename the value version of the APIs

• sq

• Fix build

• Address feedback

• Remove Value from the APIs

• sq

• Address feedback

• Bump version to 0.2.0 (Add V1 Scenario tests for data transformation #2803)

• Add Applymethod to PrimitiveDataFrameColumn (Make accessor of linear coefficients unique to the public #2807)

• Add Apply method to PrimitiveDataFrameColumn and its container

• Add TestApply test

• Remove unused df variable in DataFrameTests

• Add xml doc comments to Apply method

• Add additional tests for ReadCsv (Read and write binary file documentation #2811)

• Add additional tests for ReadCsv

• Update asserts

• Add empty row and skip test pending another fix

• Remove test for another issue

• Added static factory methods to DataFrameColumn (Scrub changes for LightGBM #2808)

• Added static factory methods to DataFrameColumn where they make sense (for the overloads where its possible to infer the column's type).

• Remove regions

• Update some parts of the unit tests to use static factory methods to create DataFrameColumns.

• Remove errant {T} on StringDataFrameColumn.

• PR feedback

Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• Append rows to a DataFrame (can not register #2823)

• Append rows to a DataFrame

• Unit test

• Update unit tests and doc

• Need to perfrom a type check every time

• sq

• Update unit test

• Address comments

• Move corefxlab to arcade (Refactor cancellation mechanism and make it internal, accessible via experimental nuget. #2795)

• Add eng folder

• First cut of moving corefxlab to arcade

• Move arcade symbol validation inside official buil

• Move base yml file to root

• Arcade will build, publish packages and symbols

• UpdateXlf. Review this

• Arcade Update to version 5.0.0-beta.19575.4 to include Experimental Channel

• Remove property that was causing the build to fail

• Moving global properties to the main Yaml instead of step in order to unblock publishing

• Committing xlfs and changing the build script to not update Xlf on build

• clean up corefxlab-base.yml

• sq

• Delete unused files and scripts

• Get rid of all the xlf stuff

• Remove UpdateXlfOnBuild for non-NT builds

• Minor cleanup

• More cleanup

• update eng\build.sh permission

• Rename to Nuget.config

• sq

• Remove the runtime spec from global.json

• Don't publish test projs

• Typo

• Move version prefix to versions.props
  Change prereleaselabel to alpha

• Increment version number to list as the latest package
  Increment version number of Microsoft.Experimental.Collections to list as the latest package
  Turn off graph generation

• Update the Readme

• Test removing the scripts folder
  https://github.com/ankitasankars/machinelearning/blob/main/src/Microsoft.ML.Mkl.Components/SymSgdClassificationTrainer.cs

• Touch readme to force a change

• Address Jose's comments

• Typo

• Move versions to eng/versions.props

• Benchmark.proj needs to refer to xunit

• Clean up dependencies.props

• Remove dependencies.props

Co-authored-by: Jose Perez Rodriguez joperezr@microsoft.com

• Rename Sort to OrderBy (Add sample for IndicateMissingValues #2814)

• Rename sort to orderby and add orderbydescending method

• Add doc strings

• Update bench mark test

• Update tests

• Update DataFrameColumn to use orderby

• Update doc comment

• Additions to sortby

• Revert "Additions to sortby"

This reverts commit 3931d4e2a72ce44a539be7c27b2592395f3efd35.

• Revert "Update doc comment"

This reverts commit 192f7797fe2b77625486637badf77046162fedbf.

• Revert "Update DataFrameColumn to use orderby"

This reverts commit 8f94664c5fd18570cd2b601535e816ca5dd5e3c4.

• Explode column types and generate converters (GAM parameter names are non-standard #2857)

• Explode column types and generate converters

• Clean this

• sq

• sq

• Cherry pick for next commit

• sq

• Undo unnecessary change

• Address remaining concerns from the 2nd DataFrame API Review (Update release for 0.11 #2861)

• Move string indexer to Columns

• API changes from the 2nd API review

• Unit tests

• Address comments

• Add binary operations and operators on the exploded columns (Updating MKL #2867)

• Generate combinations of binary operations and Add

• Numeric Converters and CloneAsNumericColumns

• Binary, Comparison and Shift operations

• Clean up and bug fix

• Fix the binary op apis to not be overridden

• Internal constructors for exploded types

• Proper return types for exploded types

• Update unit tests

• Update csproj

• Revert "Fix the binary op apis to not be overridden"

This reverts commit 2dc2240c9449930139c1492d1388d5e1f8ba5fa1.

• Bug fix and unit test

• Constructor that takes in a container

• Unit tests

• Call the implementation where possible

• Review sq

• sq

• Cherry pick for next commit

• sq

• Undo unnecessary change

• Rename to the system namespace column types

• Address comments

• Push to pull locally

• Mimic C#'s arithmetic grammar in DataFrame

• Address feedback

• Reduce the number of partial column definitions

• Address feedback

• Add APIs to get the strongly typed columns from a DataFrame (Make array values intended to be immutable IReadOnlyList #2878)

• CP

• sq

• sq

• Improve docs

• Enable xml docs for Data.Analysis (Non-standard naming in L-BFGS Learners (LogisticRegression, PoissonRegression) #2882)

• Enable xml docs for Data.Analysis

• Fix /// summary around inheritdoc

• Minor doc changes

• sq

• sq

• Address feedback

• Add Apply to ArrowStringDataFrameColumn (Temporarily disable myget #2889)

• Support for Exploded columns types in Arrow and IO scenarios (Main namespace types2445 #2885)

• Support for Exploded columns types in Arrow and IO scenarios

• Unit tests

• Address feedback

• Bump version (Scrub Latent Dirichlet Allocation Transform (Just Renaming) #2890)

• Fix versioning to allow for individual stable packages (Scrub word embedding transform #2891)

• Fix versioning to allow for individual stable packages

• sq

• Bump Microsoft.Data.Analysis version to 0.4.0 (Scrubbing online learners #2892)

• Bump Microsoft.Data.Analysis version to 0.4.0

• Fix Passing in DataTypes to DataFrame.LoadCsv loses column name information corefxlab#2906 (TrainTestSplit should be inside MLContext.Data #2907)

• Fix Passing in DataTypes to DataFrame.LoadCsv loses column name information corefxlab#2906

• Improvements and unit tests

• sq

• Better fix

• sq

• Improve LoadCsv to handle null values when deducing the column types (Polish char- and word-level tokenizers & stopword removers #2916)

• Unit test to repro

• Fix DataFrame.LoadCsv doesn't handle null values when it attempts to guess the column types corefxlab#2915

Append a null value to a column when encountering it instead of changing the column type to a StringDataFrameColumn

• Update src/Microsoft.Data.Analysis/DataFrame.IO.cs

Co-authored-by: Günther Foidl gue@korporal.at

• Update src/Microsoft.Data.Analysis/DataFrame.cs

Co-authored-by: Günther Foidl gue@korporal.at

• Feedback

Co-authored-by: Günther Foidl gue@korporal.at

• Create a 0.4.0 package (Scrub text normalizer #2918)

• Revert "Create a 0.4.0 package (Scrub text normalizer #2918)" (One name for MulticlassClassification #2919)

This reverts commit 0bef531.

• Produce a 0.4.0 build (OVA Multiclass Classification can be instantiated for variety of sub-trainer training tasks #2920)

• Default Length for StringDataFrameColumn (Adding functional tests for all training scenarios #2921) (Logistic Regression NumberOfIterations to MaximumNumberOfIterations #2923)

• Increment version and stop producing stable packages (Logistic Regression: NumberOfIterations should be MaximumNumberOfIterations #2922)

• Increment version and stop producing stable packages

• Add DataFrame object formatter. (Train FieldAwareFactorizationMachines without providing arguments #2931)

• Add DataFrame object formatter.

• Update nuget dependencies.

• Apply CR fixes.

• Fix a bug in InsertColumn

• Add Microsoft.Data.Analysis.nuget project (Knowing what to cast the model to is hard #2933)

• Add DataFrame object formatter.

• Update nuget dependencies.

• Apply CR fixes.

• Remove ReferenceOutputAssembly added to from Microsoft.Data.Analysys.csproj.

• Add Microsoft.Data.Analysis.nuget project.

• Move project to src. Fix nuget project settings.

• Remove NoBuild property from project.

• Remove IncludeBuildOutput and IncludeSymbols from project.

• Add VersionPrefix to project.

• Add IncludeBuildOutput property.

• Add unit tests.

• Downgrade from netcoreapp3.1 to netcoreapp3.0

• Upgrade from netcoreapp3.0 to netcoreapp3.1 (dotnet interactive is not compatible with 3.0)

• Add netcoreapp3.1 to global settings

• Add dotnet 3.1.5 runtime to global settings

• Build fixes

• Moving MDAI into interactive-extensions folder of the package

• Minor refactoring

• Respond to PR feedback

Co-authored-by: Prashanth Govindarajan prgovi@microsoft.com
Co-authored-by: Jose Perez Rodriguez joperezr@microsoft.com
Co-authored-by: Eric Erhardt eric.erhardt@microsoft.com

• ColumnName indexer on DataFrame (Hiding of ColumnOptions #2959)

• ColumnName indexer on DataFrame

Fixes dotnet/corefxlab#2934

• Unit tests

• Null column name

• Implement FillNulls() for ArrowStringDataFrameColumn with inPlace: false (Updating OVA tests #2956)

• implement FillNulls method for ArrowStringDataFrameColumn

• additional asserts for testcase

• Prevent DataFrame.Sample() method from returning duplicated rows (CpuMath Enhancement: Make bound checking of loops in hardware intrinsics more efficient #2939)

• resolves My data view has a label column named 'Score' #2806

• replace forloop with ArraySegment

• reduce shuffle loop operations from O(Rows.Count) to O(numberOfRows)

• Add WriteCsv plus unit tests. (FeaturizeText: Add instructions to turn off char- or word-gram generation to the tooltip. #2947)

• Add WriteCsv plus unit tests.

• Add CultureInfo to WriteCsv. Remove index column param. Update unit tests.

• Add CR changes. CultureInfo. Separator.

• Format decimal types individually. Fix culture info. Fix unit tests.

• Format decimal types individually. Fix culture info. Fix unit tests.

• Missing values default to a StringDataFrameColumn (ONNX Transform changes output types #2982)

• Make LoadCsv more robust

• Test empty string column

• Retain prev guess where possible

• Update FromArrowRecordBatches for dotnet-spark (Public API for SchemaDefinition class #2978)

• Support for RecordBatches with StructAr…

We are excited to review your PR.

So we can do the best job, please check:

• There's a descriptive title that will make sense to other developers some time from now.
• There's associated issues. All PR's should have issue(s) associated - unless a trivial self-evident change such as fixing a typo. You can use the format Fixes #nnnn in your description to cause GitHub to automatically close the issue(s) when your PR is merged.
• Your change description explains what the change does, why you chose your approach, and anything else that reviewers should know.
• You have included any necessary tests in the same PR.

[dnfadmin]

All CLA requirements met.