tag:github.com,2008:https://github.com/iris-hep/func_adl_servicex/releases Release notes from func_adl_servicex 2023-02-15T06:52:41Z tag:github.com,2008:Repository/291337510/2.2 2023-02-15T06:54:22Z Fixing how the backend uses data types <h2>Highlights</h2> <ul> <li>The new <code>ServiceX</code> background is better about paying attention to the <code>results-format</code> tag - which was not being properly handled by the front-end stack. This fixes that up.</li> <li>Small fixes to metadata, package version limits, etc.</li> </ul> <h2>What's Changed</h2> <ul> <li>Be consistent about root file type by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1584495108" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/57" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/57/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/57">#57</a></li> <li>New Release Prep by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1585308823" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/58" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/58/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/58">#58</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/2.1...2.2"><tt>2.1...2.2</tt></a></p> gordonwatts tag:github.com,2008:Repository/291337510/2.2b1 2023-02-14T22:18:48Z Using file types properly <p>Fixing up how we propagate file types through the various layers. Should have no impact for anyone using this.</p> gordonwatts tag:github.com,2008:Repository/291337510/2.1 2022-10-30T00:03:35Z Getting AsParquetFiles to work correctly with the uproot backend <p>The main goal of this release is to make sure that <code>AsParquetFiles</code> has the right semantics and is compatible with the <code>uproot</code> transformer.</p> <p>As <code>servicex</code> moves towards having more backend file flexibility, this interface may change (and generate some depreciated warnings). Note that best practices (which allows for most flexibility) is to rewrite something like this:</p> <div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import pandas as pd from func_adl_servicex import ServiceXSourceUpROOT dataset_name = &quot;data15_13TeV:data15_13TeV.00282784.physics_Main.deriv.DAOD_PHYSLITE.r9264_p3083_p4165_tid21568807_00&quot; src = ServiceXSourceUpROOT(dataset_name, &quot;CollectionTree&quot;) data = src.Select(&quot;lambda e: e['AnalysisJetsAuxDyn.pt']&quot;) \ .AsParquetFiles('junk.parquet', ['JetPT']) \ .value() df = pd.read_parquet(data[0]) print(df)"><pre><span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span> <span class="pl-k">from</span> <span class="pl-s1">func_adl_servicex</span> <span class="pl-k">import</span> <span class="pl-v">ServiceXSourceUpROOT</span> <span class="pl-s1">dataset_name</span> <span class="pl-c1">=</span> <span class="pl-s">"data15_13TeV:data15_13TeV.00282784.physics_Main.deriv.DAOD_PHYSLITE.r9264_p3083_p4165_tid21568807_00"</span> <span class="pl-s1">src</span> <span class="pl-c1">=</span> <span class="pl-en">ServiceXSourceUpROOT</span>(<span class="pl-s1">dataset_name</span>, <span class="pl-s">"CollectionTree"</span>) <span class="pl-s1">data</span> <span class="pl-c1">=</span> <span class="pl-s1">src</span>.<span class="pl-c1">Select</span>(<span class="pl-s">"lambda e: e['AnalysisJetsAuxDyn.pt']"</span>) \ .<span class="pl-c1">AsParquetFiles</span>(<span class="pl-s">'junk.parquet'</span>, [<span class="pl-s">'JetPT'</span>]) \ .<span class="pl-c1">value</span>() <span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-c1">read_parquet</span>(<span class="pl-s1">data</span>[<span class="pl-c1">0</span>]) <span class="pl-en">print</span>(<span class="pl-s1">df</span>)</pre></div> <p>as this:</p> <div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import pandas as pd from func_adl_servicex import ServiceXSourceUpROOT dataset_name = &quot;data15_13TeV:data15_13TeV.00282784.physics_Main.deriv.DAOD_PHYSLITE.r9264_p3083_p4165_tid21568807_00&quot; src = ServiceXSourceUpROOT(dataset_name, &quot;CollectionTree&quot;) data = src.Select(&quot;lambda e: {'JetPT': e['AnalysisJetsAuxDyn.pt']}&quot;) \ .AsParquetFiles('junk.parquet') \ .value() df = pd.read_parquet(data[0]) print(df)"><pre><span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span> <span class="pl-k">from</span> <span class="pl-s1">func_adl_servicex</span> <span class="pl-k">import</span> <span class="pl-v">ServiceXSourceUpROOT</span> <span class="pl-s1">dataset_name</span> <span class="pl-c1">=</span> <span class="pl-s">"data15_13TeV:data15_13TeV.00282784.physics_Main.deriv.DAOD_PHYSLITE.r9264_p3083_p4165_tid21568807_00"</span> <span class="pl-s1">src</span> <span class="pl-c1">=</span> <span class="pl-en">ServiceXSourceUpROOT</span>(<span class="pl-s1">dataset_name</span>, <span class="pl-s">"CollectionTree"</span>) <span class="pl-s1">data</span> <span class="pl-c1">=</span> <span class="pl-s1">src</span>.<span class="pl-c1">Select</span>(<span class="pl-s">"lambda e: {'JetPT': e['AnalysisJetsAuxDyn.pt']}"</span>) \ .<span class="pl-c1">AsParquetFiles</span>(<span class="pl-s">'junk.parquet'</span>) \ .<span class="pl-c1">value</span>() <span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-c1">read_parquet</span>(<span class="pl-s1">data</span>[<span class="pl-c1">0</span>]) <span class="pl-en">print</span>(<span class="pl-s1">df</span>)</pre></div> <p>In short - use dictionaries in your query to label the columns that come back.</p> <h2>What's Changed</h2> <ul> <li>Fix markdown syntax in readme by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/klieret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/klieret">@klieret</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1215989903" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/48" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/48/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/48">#48</a></li> <li>Make sure AsParquetFiles works correctly with uproot by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1428433672" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/53" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/53/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/53">#53</a></li> <li>Unclear sentence in readme by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1428509539" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/54" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/54/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/54">#54</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/klieret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/klieret">@klieret</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1215989903" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/48" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/48/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/48">#48</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/2.0...2.1"><tt>2.0...2.1</tt></a></p> gordonwatts tag:github.com,2008:Repository/291337510/2.1b1 2022-10-29T23:32:56Z Uproot and AsParquetFiles interoperability <h2>What's Changed</h2> <ul> <li>Fix markdown syntax in readme by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/klieret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/klieret">@klieret</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1215989903" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/48" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/48/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/48">#48</a></li> <li>Make sure AsParquetFiles works correctly with uproot by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1428433672" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/53" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/53/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/53">#53</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/klieret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/klieret">@klieret</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1215989903" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/48" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/48/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/48">#48</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/2.0...2.1b1"><tt>2.0...2.1b1</tt></a></p> gordonwatts tag:github.com,2008:Repository/291337510/2.0 2022-04-19T02:35:11Z Typed Datasets, Local Executor <p>This is a new release! Major improvements:</p> <ul> <li>Datasets can be typed, allowing type checkers like <code>pylance</code> and <code>mypy</code> to follow types through the system. This should aid in the editing experience.</li> <li>A local executor for xAOD's has been added.</li> </ul> <h2>What's Changed</h2> <ul> <li>Typed Datasets &amp; Local Executor by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1028159784" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/37" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/37/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/37">#37</a></li> <li>No Col Specified, No Col to access by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1177905005" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/46" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/46/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/46">#46</a></li> <li>Fix up the README by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1177903683" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/45" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/45/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/45">#45</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/1.1.3...2.0"><tt>1.1.3...2.0</tt></a></p> gordonwatts tag:github.com,2008:Repository/291337510/2.0b2 2022-03-24T22:45:13Z Better Single Column Access <h2>What's Changed</h2> <ul> <li>No Col Specified, No Col to access by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1177905005" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/46" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/46/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/46">#46</a></li> <li>Fix up the README by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1177903683" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/45" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/45/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/45">#45</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/2.0b1...2.0b2"><tt>2.0b1...2.0b2</tt></a></p> gordonwatts tag:github.com,2008:Repository/291337510/2.0b1 2022-03-14T01:44:15Z Typed Datasets, Local Executor <h2>What's Changed</h2> <ul> <li>Typed Datasets &amp; Local Executor by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1028159784" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/37" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/37/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/37">#37</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/1.1.3...2.0b1"><tt>1.1.3...2.0b1</tt></a></p> gordonwatts tag:github.com,2008:Repository/291337510/1.1.3 2021-12-09T19:29:43Z Support Python 3.10 <p>Add Python 3.10 support</p> gordonwatts tag:github.com,2008:Repository/291337510/1.2b2 2021-10-16T22:28:15Z Typed Datasets and MetaData <ul> <li>Typed datasets</li> <li>Metadata can be sent down to the backend</li> </ul> gordonwatts tag:github.com,2008:Repository/291337510/1.2b1 2021-08-20T21:42:27Z 1.2b1: Uproot transformer can't deal with explicit column names (#33) <p>Fixes <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="974690065" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/32" data-hovercard-type="issue" data-hovercard-url="/iris-hep/func_adl_servicex/issues/32/hovercard" href="https://github.com/iris-hep/func_adl_servicex/issues/32">#32</a></p> gordonwatts