tag:github.com,2008:https://github.com/iris-hep/func_adl_servicex/releasesRelease notes from func_adl_servicex2023-02-15T06:52:41Ztag:github.com,2008:Repository/291337510/2.22023-02-15T06:54:22ZFixing how the backend uses data types<h2>Highlights</h2>
<ul>
<li>The new <code>ServiceX</code> background is better about paying attention to the <code>results-format</code> tag - which was not being properly handled by the front-end stack. This fixes that up.</li>
<li>Small fixes to metadata, package version limits, etc.</li>
</ul>
<h2>What's Changed</h2>
<ul>
<li>Be consistent about root file type by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1584495108" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/57" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/57/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/57">#57</a></li>
<li>New Release Prep by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1585308823" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/58" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/58/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/58">#58</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/2.1...2.2"><tt>2.1...2.2</tt></a></p>gordonwattstag:github.com,2008:Repository/291337510/2.2b12023-02-14T22:18:48ZUsing file types properly<p>Fixing up how we propagate file types through the various layers. Should have no impact for anyone using this.</p>gordonwattstag:github.com,2008:Repository/291337510/2.12022-10-30T00:03:35ZGetting AsParquetFiles to work correctly with the uproot backend<p>The main goal of this release is to make sure that <code>AsParquetFiles</code> has the right semantics and is compatible with the <code>uproot</code> transformer.</p>
<p>As <code>servicex</code> moves towards having more backend file flexibility, this interface may change (and generate some depreciated warnings). Note that best practices (which allows for most flexibility) is to rewrite something like this:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import pandas as pd
from func_adl_servicex import ServiceXSourceUpROOT
dataset_name = "data15_13TeV:data15_13TeV.00282784.physics_Main.deriv.DAOD_PHYSLITE.r9264_p3083_p4165_tid21568807_00"
src = ServiceXSourceUpROOT(dataset_name, "CollectionTree")
data = src.Select("lambda e: e['AnalysisJetsAuxDyn.pt']") \
.AsParquetFiles('junk.parquet', ['JetPT']) \
.value()
df = pd.read_parquet(data[0])
print(df)"><pre><span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>
<span class="pl-k">from</span> <span class="pl-s1">func_adl_servicex</span> <span class="pl-k">import</span> <span class="pl-v">ServiceXSourceUpROOT</span>
<span class="pl-s1">dataset_name</span> <span class="pl-c1">=</span> <span class="pl-s">"data15_13TeV:data15_13TeV.00282784.physics_Main.deriv.DAOD_PHYSLITE.r9264_p3083_p4165_tid21568807_00"</span>
<span class="pl-s1">src</span> <span class="pl-c1">=</span> <span class="pl-en">ServiceXSourceUpROOT</span>(<span class="pl-s1">dataset_name</span>, <span class="pl-s">"CollectionTree"</span>)
<span class="pl-s1">data</span> <span class="pl-c1">=</span> <span class="pl-s1">src</span>.<span class="pl-c1">Select</span>(<span class="pl-s">"lambda e: e['AnalysisJetsAuxDyn.pt']"</span>) \
.<span class="pl-c1">AsParquetFiles</span>(<span class="pl-s">'junk.parquet'</span>, [<span class="pl-s">'JetPT'</span>]) \
.<span class="pl-c1">value</span>()
<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-c1">read_parquet</span>(<span class="pl-s1">data</span>[<span class="pl-c1">0</span>])
<span class="pl-en">print</span>(<span class="pl-s1">df</span>)</pre></div>
<p>as this:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import pandas as pd
from func_adl_servicex import ServiceXSourceUpROOT
dataset_name = "data15_13TeV:data15_13TeV.00282784.physics_Main.deriv.DAOD_PHYSLITE.r9264_p3083_p4165_tid21568807_00"
src = ServiceXSourceUpROOT(dataset_name, "CollectionTree")
data = src.Select("lambda e: {'JetPT': e['AnalysisJetsAuxDyn.pt']}") \
.AsParquetFiles('junk.parquet') \
.value()
df = pd.read_parquet(data[0])
print(df)"><pre><span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>
<span class="pl-k">from</span> <span class="pl-s1">func_adl_servicex</span> <span class="pl-k">import</span> <span class="pl-v">ServiceXSourceUpROOT</span>
<span class="pl-s1">dataset_name</span> <span class="pl-c1">=</span> <span class="pl-s">"data15_13TeV:data15_13TeV.00282784.physics_Main.deriv.DAOD_PHYSLITE.r9264_p3083_p4165_tid21568807_00"</span>
<span class="pl-s1">src</span> <span class="pl-c1">=</span> <span class="pl-en">ServiceXSourceUpROOT</span>(<span class="pl-s1">dataset_name</span>, <span class="pl-s">"CollectionTree"</span>)
<span class="pl-s1">data</span> <span class="pl-c1">=</span> <span class="pl-s1">src</span>.<span class="pl-c1">Select</span>(<span class="pl-s">"lambda e: {'JetPT': e['AnalysisJetsAuxDyn.pt']}"</span>) \
.<span class="pl-c1">AsParquetFiles</span>(<span class="pl-s">'junk.parquet'</span>) \
.<span class="pl-c1">value</span>()
<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-c1">read_parquet</span>(<span class="pl-s1">data</span>[<span class="pl-c1">0</span>])
<span class="pl-en">print</span>(<span class="pl-s1">df</span>)</pre></div>
<p>In short - use dictionaries in your query to label the columns that come back.</p>
<h2>What's Changed</h2>
<ul>
<li>Fix markdown syntax in readme by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/klieret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/klieret">@klieret</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1215989903" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/48" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/48/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/48">#48</a></li>
<li>Make sure AsParquetFiles works correctly with uproot by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1428433672" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/53" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/53/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/53">#53</a></li>
<li>Unclear sentence in readme by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1428509539" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/54" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/54/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/54">#54</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/klieret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/klieret">@klieret</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1215989903" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/48" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/48/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/48">#48</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/2.0...2.1"><tt>2.0...2.1</tt></a></p>gordonwattstag:github.com,2008:Repository/291337510/2.1b12022-10-29T23:32:56ZUproot and AsParquetFiles interoperability<h2>What's Changed</h2>
<ul>
<li>Fix markdown syntax in readme by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/klieret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/klieret">@klieret</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1215989903" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/48" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/48/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/48">#48</a></li>
<li>Make sure AsParquetFiles works correctly with uproot by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1428433672" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/53" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/53/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/53">#53</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/klieret/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/klieret">@klieret</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1215989903" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/48" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/48/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/48">#48</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/2.0...2.1b1"><tt>2.0...2.1b1</tt></a></p>gordonwattstag:github.com,2008:Repository/291337510/2.02022-04-19T02:35:11ZTyped Datasets, Local Executor<p>This is a new release! Major improvements:</p>
<ul>
<li>Datasets can be typed, allowing type checkers like <code>pylance</code> and <code>mypy</code> to follow types through the system. This should aid in the editing experience.</li>
<li>A local executor for xAOD's has been added.</li>
</ul>
<h2>What's Changed</h2>
<ul>
<li>Typed Datasets & Local Executor by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1028159784" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/37" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/37/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/37">#37</a></li>
<li>No Col Specified, No Col to access by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1177905005" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/46" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/46/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/46">#46</a></li>
<li>Fix up the README by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1177903683" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/45" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/45/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/45">#45</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/1.1.3...2.0"><tt>1.1.3...2.0</tt></a></p>gordonwattstag:github.com,2008:Repository/291337510/2.0b22022-03-24T22:45:13ZBetter Single Column Access<h2>What's Changed</h2>
<ul>
<li>No Col Specified, No Col to access by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1177905005" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/46" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/46/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/46">#46</a></li>
<li>Fix up the README by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1177903683" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/45" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/45/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/45">#45</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/2.0b1...2.0b2"><tt>2.0b1...2.0b2</tt></a></p>gordonwattstag:github.com,2008:Repository/291337510/2.0b12022-03-14T01:44:15ZTyped Datasets, Local Executor<h2>What's Changed</h2>
<ul>
<li>Typed Datasets & Local Executor by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gordonwatts/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gordonwatts">@gordonwatts</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1028159784" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/37" data-hovercard-type="pull_request" data-hovercard-url="/iris-hep/func_adl_servicex/pull/37/hovercard" href="https://github.com/iris-hep/func_adl_servicex/pull/37">#37</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/iris-hep/func_adl_servicex/compare/1.1.3...2.0b1"><tt>1.1.3...2.0b1</tt></a></p>gordonwattstag:github.com,2008:Repository/291337510/1.1.32021-12-09T19:29:43ZSupport Python 3.10<p>Add Python 3.10 support</p>gordonwattstag:github.com,2008:Repository/291337510/1.2b22021-10-16T22:28:15ZTyped Datasets and MetaData<ul>
<li>Typed datasets</li>
<li>Metadata can be sent down to the backend</li>
</ul>gordonwattstag:github.com,2008:Repository/291337510/1.2b12021-08-20T21:42:27Z1.2b1: Uproot transformer can't deal with explicit column names (#33)<p>Fixes <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="974690065" data-permission-text="Title is private" data-url="https://github.com/iris-hep/func_adl_servicex/issues/32" data-hovercard-type="issue" data-hovercard-url="/iris-hep/func_adl_servicex/issues/32/hovercard" href="https://github.com/iris-hep/func_adl_servicex/issues/32">#32</a></p>gordonwatts