[SUGGESTION] Implement the pipeline operator from P2011 & P2672

[bluetarpmedia]

Suggestion
Cpp2 could support the pipeline operator |> as proposed in P2011 and further explored in P2672.

Specifically, the pipeline operator with the "placeholder model" with mandatory placeholder (e.g. $), as described in P2672 section 6 Disposition. Both papers explain the problem and motivation for the new operator, as well as discussing options for the placeholder token.

Circle uses $ as its token.

The proposed operator enables a simpler left-to-right style as opposed to an inside-out style.

Conor Hoekstra (code_report) has various talks about ranges and pipelines and explains how the pipeline operator can make the code simpler and more readable. The following is one of his examples:

Without the operator:

auto filter_out_html_tags(std::string_view sv) {
    auto angle_bracket_mask = 
        sv | rv::transform([](auto e) { return e == '<' or e == '>'; });
    return rv::zip(rv::zip_with(std::logical_or{}, 
            angle_bracket_mask, 
            angle_bracket_mask | rv::partial_sum(std::not_equal_to{})), sv)
        | rv::filter([](auto t) { return not std::get<0>(t); })
        | rv::transform([](auto t) { return std::get<1>(t); })
        | ranges::to<std::string>;
}

With the operator:

auto filter_out_html_tags(std::string_view sv) {
    return sv 
        |> transform($, [](auto e) { return e == '<' or e == '>'; })
        |> zip_transform(std::logical_or{}, $, scan_left($, true, std::not_equal_to{}))
        |> zip($, sv)
        |> filter($, [](auto t) { return not std::get<0>(t); })
        |> values($)
        |> ranges::to<std::string>($);
}

Will your feature suggestion eliminate X% of security vulnerabilities of a given kind in current C++ code?
No

Will your feature suggestion automate or eliminate X% of current C++ guidance literature?
No

Describe alternatives you've considered.
Alternatives are discussed at length in the two papers referenced above.

[JohelEGP]

The talk: "New Algorithms in C++23 - Conor Hoekstra - C++ on Sea 2023".
The recommended version: "Guide to the New Algorithms in C++23 - Conor Hoekstra - CppNorth 2023".

[hsutter]

The proposed operator enables a simpler left-to-right style as opposed to an inside-out style.

As soon as I see this, I think "hmm, like UFCS does?", and then I saw the example which looks suspiciously like UFCS for most of the cases.

Clarifying questions, just so I understand the question (I haven't had time to watch the talk).

Is the $ is the placeholder for where to put the left-hand side of the operator, so that this code (ignoring for now the zip_transform one where it's not in the first location):

    return sv 
        |> transform($, [](auto e) { return e == '<' or e == '>'; })
        |> zip($, sv)
        |> filter($, [](auto t) { return not std::get<0>(t); })
        |> values($)
        |> ranges::to<std::string>($);

would be the same as this using UFCS, which I think would work in Cpp2 now:

    return sv 
        .transform(:(e) e == '<' || e == '>';)
        .zip(sv)
        .filter(:(t) !std::get<0>(t);)
        .values()
        .ranges::to<std::string>();

or something like that, modulo any late-night typos I wrote?

[JohelEGP]

That's right.

Including zip_transform, I have confirmed that this translates correctly (https://cpp2.godbolt.org/z/7Wdbf5o1Y, https://compiler-explorer.com/z/hcx9ex4j4 [formatted]):

#include <algorithm>
#include <ranges>
using namespace std::views;
// auto filter_out_html_tags(std::string_view sv) {
//     return sv 
//         |> transform($, [](auto e) { return e == '<' or e == '>'; })
//         |> zip_transform(std::logical_or{}, $, scan_left($, true, std::not_equal_to{}))
//         |> zip($, sv)
//         |> filter($, [](auto t) { return not std::get<0>(t); })
//         |> values($)
//         |> ranges::to<std::string>($);
// }
filter_out_html_tags_cpp2: (sv: std::string_view) -> _ = {
  (a := sv
        .transform(:(e) e == '<' || e == '>';))
  return zip_transform(std::logical_or(), a, scan_left(a, true, std::not_equal_to()))
        .zip(sv)
        .filter(:(t) !std::get<0>(t);)
        .values()
        .to<std::string>();
}
main: () = { }

Can you remind me where scan_left comes from?

[bluetarpmedia]

Can you remind me where scan_left comes from?

scan_left is from code_report's example.

auto scan_left(auto rng, auto init, auto op) {
    return transform(rng, [first = true, acc = init, op](auto e) mutable {
            if (first) first = false; 
            else acc = op(acc, e);
            return acc;
        });
}

[hsutter]

Wow, that was fast. I was just trying to write the code for zip_transform a different way, but I see a statement parameter worked.

What about expressing this

    |> zip_transform(std::logical_or{}, $, scan_left($, true, std::not_equal_to{}))

as this

    . :(x) zip_transform(std::logical_or(), x, scan_left(x, true, std::not_equal_to())) ()

?

Then it's all . ?

(I haven't tried to compile the code though.)

[bluetarpmedia]

There's some overlap between UFCS and this proposed pipeline operator.

Conor's presentation shows off various range pipeline examples, but the P2011 and P2672 papers discuss the motivation and problem space.

P2011 also discusses how it's different to UFCS. I think the key difference for Cpp2 is allowing the placeholder to appear in different argument positions in order to compose the range algorithms and views.

[JohelEGP]

Do you have a test?
This compiles: https://cpp2.godbolt.org/z/n8673xofn.

I had to add to, because Libstdc++ doesn't have ranges::to.
Also, Libc++ doesn't implement zip_transform.
(https://en.cppreference.com/w/cpp/compiler_support).

[JohelEGP]

By the way, I just took the parameter of to by in.
The error message was hideous.
I compiled locally with #506 (and some other things)
and quickly found out

x.cpp2:4:104: error: no match for call to ‘(to<std::__cxx11::basic_string<char>, std::ranges::elements_view<…

(… inserted by me).
These are the sizes of the error outputs:

$ ls out-* -lh
-rw-r--r-- 1 johel johel 60K Oct  9 21:28 out-main
-rw-r--r-- 1 johel johel 23K Oct  9 21:27 out-waarudo

[JohelEGP]

Do you have a test?

I added one.
It seems to output characters rather than strings.
https://cpp2.godbolt.org/z/s5crPezaj.

[JohelEGP]

. :(x) zip_transform(std::logical_or(), x, scan_left(x, true, std::not_equal_to())) ()

That's not valid grammar (https://cpp2.godbolt.org/z/9avjYh6sb):

main.cpp2...
main.cpp2(25,10): error: '.' must be followed by a valid member name (at '(')

[hsutter]

Yes -- in a racing update I was updating the comment to say the following, but I'll make it a separate reply instead:

Right, that code is currently rejected because . must be followed by a name.
So perhaps have a general helper like call:(f) :(x) f(x); to enable writing

    .call(:(x) zip_transform(std::logical_or(), x, scan_left(x, true, std::not_equal_to()));)

modulo typos and bugs? Or maybe name it curry? Anyway, signing off for tonight, but a very interesting question! Thanks.

[bluetarpmedia]

You're right that UFCS is very close already, and arguably the dot syntax is just as nice as the new operator, but with UFCS I'd still prefer a token to make the syntax simpler. (Perhaps _ is better than Circle's choice of $ since there's already precedent in Cpp2.)

.zip_transform(std::logical_or{}, _, scan_left(_, true, std::not_equal_to{}))

[JohelEGP]

Excellent.
It works again with call:(forward o, forward f) f(o); (https://cpp2.godbolt.org/z/evj8PnzE7).
Circle:

auto filter_out_html_tags(std::string_view sv) {
    return sv 
        |> transform($, [](auto e) { return e == '<' or e == '>'; })
        |> zip_transform(std::logical_or{}, $, scan_left($, true, std::not_equal_to{}))
        |> zip($, sv)
        |> filter($, [](auto t) { return not std::get<0>(t); })
        |> values($)
        |> ranges::to<std::string>($);
}

Cpp2 (colored):

Text

filter_out_html_tags_cpp2: (sv: std::string_view) //
  sv.transform(:(e) e == '<' || e == '>')
    .call(:(x) zip_transform(std::logical_or(), x, scan_left(x, true, std::not_equal_to())))
    .zip(sv)
    .filter(:(t) !t.get<0>())
    .values()
    .to<std::string>();

[JohelEGP]

IIRC, that proposal has seen push back
due to having to specify the semantics
of pipe arguments as non-first argument and
of multiple pipe arguments in the same function call.