POC for better buffer object chaining

[weefuzzy]

depends on flucoma/flucoma-core#103

As discussed a great, great many times, we would be able to reduce buffer~ boilerplate in pipelines of fluid.buf* processes substantially, if

• they did like jitter and had a default output object(s) provided when none is specified
• had a way to set the source of following processes

This is the minimal-breakage version of that idea:

• a new message buffer <symbol> sets source buffer(s). Objects with multiple sources acquire additional inlets, that will dispatch incoming buffer messages to the correct attribute to update
• objects now emit buffer <X> for each of their output buffers, followed by a bang, meaning that the source for the following process can be set

The reason this is the minimal breakage version is that all of these messages come out of the existing 0th outlet, whereas the idiomatic thing to do would actually be to introduce new extra outlets for additional output buffers (e.g. in HPSS) and to make buffer received in inlet 0 trigger processing.

As it stands, this addition is no use at all in @blocking 2, because the output buffers couldn't be resized on the fly

Also, an object like fluid.bufnmf is a weird edge case, where @bases and @activations are both input and output, depending on the configuration

Anyway, kick its tyres @tremblap and @jamesb93 and we can work out remaining design questions

[weefuzzy]

A motivating example, courtesy of James

----------begin_max5_patcher----------
994.3ocuW0zaiaCD8r8uBBgdzlfyL7ydZwdo+HJVDH6vjnE1RFRxYS6hM+16
PJmDmtN0pdUBO3PMbHey6wgCY997YEqZdH1UH9cweJlM66ymMKaJYX1gumUr
s7g0aJ6xtUrM10UdarXwvX8wG5y1ASvHARqHRfAxIw.5Pifj9fyRFARRdv.P
BGRVIBAG5DKQkLXb.4ErQkynTFgMPRilnfU3.mz38.u.fzDPqWKXSA.bJPPf
1H0TPaHgVZLZsgMx94LreJdlZB7bGjz.fbmPHP40xYz7pJVpjDZBZH0CsJuM
4t057dKuRJvf.HPoEwfEYKLTFKGhRTSp.yUIELJcxGC27D6CGV.GQnMHcVd9
Ldo9AWPo3vWRduhiOsDbT.CLgBR.CZFxkjVZUJdxdI5CDZE7R4j5fJQ.j0OI
G2NLwSsKXPdAsxfkbAmvYC7hRHwVSJqwQZqSPFN9AOxwef2k3EUCVgAXE2Sl
.q8ofSa7bnfZoyGrZT.5ry73NiXInXwvY8nPyQ1Sa+appiqa1WmyArGLVuea
y99Mw9bFCbv5tx902UUe6Usw08CYbfwKUKD7Nc5Olgu7AoR7kCSZXc5+qcwg
YTT77PUWmy6ZV80kghWPtp9IfwjseLed5mEiL6tN9MdA+oj6a1ru5Z4p82fa
p55K9eySelfTJG33llVvGFRCg3+EoyfdJhC1OLleylx99X8iml7z4IeJI931
BQPcdpWrHECMk8oNmVCLmTCf2AMnquru6RT.cVATZo+3lwwY6zjnA5Or7ftc
LuZK2zcW4t3EmMvWMHoia.mOvEtlDw.mrDh25xtz4gXq32fEhUk029z36Zic
wZNIopo9pWUZjtvRFHwWg7pFLTg7rxzIUFXxRSVsuuuo9RqDBJW91xWZ7QAL
eJI86aSqrZeJp4edOnrsbarO1dUrtb0l7DUuyEH96Mhaaa1uSPmVPvyuOyBB
9pVXgvBi63vIEj28JBLg6ihgSBWJsAG+ZmiadKein5Wf1z6MsGH7iht10h+n
MFqW94lGr7wzuUd+kJCNSVL71yS7mK88lmDvOrqCuaWW2ihO00ruccLIHW5s
AfFkvqZoaCL5I41fyJG4YkeI6+5eDJGvI6uViF36g.3oWgIdIHuN10WUmuD3
Hm32nJ.wWN41vnAROBfRuGJ4zuFR3XPROEHYFCR1o.IXLHgS.RiZaBl.f7iA
noP5FANzDfCMBb94jtgCvk61cers6fyYH3xYesoM8oeQ9yp5gOykjKZi2W8j
+4mHTT1xUn54xS6aygUwC1g6QK11bcrsde0ghGL4XHykJq4mazsqbfG4Jpy+
w7+ARrj8hA
-----------end_max5_patcher-----------

[jamesb93]

Forgive me if something is not yet implemented or you're aware. I'll also try keep my thoughts granular so theyre easier to dismiss if needed 😆

In this scenario my first hunch was that I would get the name of the buffer explicitly set for that case. Just putting that out there as an instinctual feeling. Again, it may be that you agree and it's just not the case right now for the code.

[weefuzzy]

Right, that's a bug: I forgot to consider setting the attrs in the box, so it's getting overridden in the object constructor. It sticks if you do it by message

[jamesb93]

Right, that's a bug: I forgot to consider setting the attrs in the box, so it's getting overridden in the object constructor. It sticks if you do it by message

Nice. I still get a valid buffer empty buffer with a u2831832 type name - don't know if that would go away with the attr fix.

[jamesb93]

Another scenario is this:

I find the route/list processing interaction not as nice as it could be. As you've said a more idiomatic way is to have separate outlets which would nullify the need for such parsing. I suppose that is another level of discussion over the interface though, whether or not wholesale breakage is worth it.

[weefuzzy]

Nice. I still get a valid buffer empty buffer with a u2831832 type name - don't know if that would go away with the attr fix.

Couldn't quite grasp that. This works, yes?

:
If I fix the setting-in-the-box then one can still drop back to an internal buffer by unsetting the attribute in question (sending it attrname with no arg)

[weefuzzy]

I find the route/list processing interaction not as nice as it could be. As you've said a more idiomatic way is to have separate outlets which would nullify the need for such parsing. I suppose that is another level of discussion over the interface though, whether or not wholesale breakage is worth it.

Exactly

[jamesb93]

]

I find the route/list processing interaction not as nice as it could be. As you've said a more idiomatic way is to have separate outlets which would nullify the need for such parsing. I suppose that is another level of discussion over the interface though, whether or not wholesale breakage is worth it.

Exactly

and my answer would be yes lol

[jamesb93]

I guess another thing that might surprise people is that you always get everything and that would in theory increase the work of some jobs (maybe)?

In the case of bufnmf~ is it significantly more cycles to output the activations/bases or is it a matter of copying internal data that is already been processed?

[weefuzzy]

Only buffer sizing (when its needed) and data copying, by and large/ Again, NMF is the awkward one as I think it has loads of switches in there to decide whether or not to irs output buffers, and will do resynthesis iff the resynth buffer is present.

Which is perhaps more of a case for simplifying the nmf interface than complicating anything else 😆

[jamesb93]

Only buffer sizing (when its needed) and data copying, by and large/ Again, NMF is the awkward one as I think it has loads of switches in there to decide whether or not to irs output buffers, and will do resynthesis iff the resynth buffer is present.

Which is perhaps more of a case for simplifying the nmf interface than complicating anything else 😆

Separate NMF objects for creating activations + bases and then NMF can take any data in? The mind runs wild

[weefuzzy]

Or just a message to resynthesize – we didn't have messages when the object was designed

[jamesb93]

This is wildly faster and more readable than creating all your own buffers and right now its quite nice to have access to using a singular inlet with everything tagged up as needed or using the inlets:

[tremblap]

This exciting development has a lot of potentials, and this POC will help make a list of pros and cons. I have started my own list and I reckon we should hold a meeting around it all - the breakage is quite significant in terms of interface and pedagogy which is where I mostly live. I am also curious to see how that fits in the overall idea of a buffer rectangle message-object, and the idea of removing many outlets to our objects. In the light of the workshops, there are a few ideas to bounce off here too.

Let's talk about the horizon of this on Monday's meeting and schedule a brainstorming interface session to get all interface-breaking stuff at once, at least on the public side of things.

[balintlaczko]

Hey there, @jamesb93 brought this very tasty thread in my attention. I had an idea to use names and some sense of OOP-style object dictionaries, it is probably a bad idea, and most likely I don't know what I'm talking about, but I couldn't handle myself, so here it is:

Slightly inspired by the ability to name things in jit.gl, and maybe JS? Or something OOPish, don't know. But the idea is that everything can stay the same, you should just be able to name certain objects and refer to their output buffers in case you haven't manually specified one. This exact example would be most probably bad since it should be some notation that cannot be ambiguous.
OK, I am stuffing my nose into things I shouldn't, sorry. In any case, reducing the boilerplate is a genius idea!

[jamesb93]

Hey there, @jamesb93 brought this very tasty thread in my attention. I had an idea to use names and some sense of OOP-style object dictionaries, it is probably a bad idea, and most likely I don't know what I'm talking about, but I couldn't handle myself, so here it is: Slightly inspired by the ability to name things in jit.gl, and maybe JS? Or something OOPish, don't know. But the idea is that everything can stay the same, you should just be able to name certain objects and refer to their output buffers in case you haven't manually specified one. This exact example would be most probably bad since it should be some notation that cannot be ambiguous. OK, I am stuffing my nose into things I shouldn't, sorry. In any case, reducing the boilerplate is a genius idea!

It's always good to hear many perspectives, especially people with your breadth of experience :) This is likely to be a long form evolution and change rather than something quick so its great to have things in the mix early!

[weefuzzy]

This been extended

Now depends on flucoma/flucoma-core#115

New things, to pile up the breaking changes:
– an outlet per output buffer (for things like HPSS etc)
– get rid of our progress outlet and use the dump instead, whilst also drawing progress on the box itself
– provide default instances for output containers (buffers, datasets, labelsets) in messages; also send output from left outlet for model-related messages

[balintlaczko]

Hey there! When I try to compile this for Windows with flucoma-core#115 in release mode I get a lot of two errors. These examples are from bufcompose, but it happens basically everywhere. First one:

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2664	'void *proxy_new(void *,long,long *)': cannot convert argument 3 from 'fluid::index *' to 'long *'	fluid.bufcompose_tilde	...\flucoma-max\source\include\FluidMaxWrapper.hpp	1036

The line:

mProxies.push_back(proxy_new(this, i + 1, &this->mProxyNumber));

And the other error is:

Severity	Code	Description	Project	File	Line	Suppression State
Error	C3493	'N' cannot be implicitly captured because no default capture mode has been specified	fluid.bufcompose_tilde	...\flucoma-max\source\include\FluidMaxWrapper.hpp	1054

The line:

        static const std::string param_name = lowerCase(x->params().template descriptorAt<N>().name);

Any help would be appreciated. :))

[weefuzzy]

Thanks for testing @balintlaczko – more entries to the Big Fun Book of Things That Upset MSVC But Not Other Compilers

Last push should fix it for you.

• long is a different size on x_64 windows than other, arguably more sensible, platforms
• clang / GCC seem to interpret the lambda capture rules differently from MSVC

[jamesb93]

What's nice is the fact that now CI wont horrifically break. Thanks for testing Balint!

[balintlaczko]

What's nice is the fact that now CI wont horrifically break. Thanks for testing Balint!

Thanks for showing me how to compile and live on the edge! I believe now!

[jamesb93]

@weefuzzy it seems the fancy "progress on the object box" works quite well! My only issue is that if I use the cancel message to stop a long job, the progress does not reset and the object remains unclickable for that time.

[tremblap]

ok compilation errors - yes I deleted the build folder - and 103 is merged in core so I'm at the tip of dev there and tip of pr140 on Max. I get this:

In file included from /Users/pa/Desktop/fromBU/documents@hudd/research/projects/fluid corpus navigation/research/flucoma-max/source/projects/fluid.libmanipulation/fluid.libmanipulation.cpp:23: /Users/pa/Desktop/fromBU/documents@hudd/research/projects/fluid corpus navigation/research/flucoma-max/source/projects/fluid.libmanipulation/../../include/FluidMaxWrapper.hpp:1611:29: error: 'this' argument to member function 'name' has type 'const SharedClientRef<fluid::client::dataset::DataSetClient>', but function is not marked const t_symbol* name = gensym(ds.name());

[tremblap]

tried again cleaning everything, still no luck. @jamesb93 and @balintlaczko you manage to compile?

[jamesb93]

tried again cleaning everything, still no luck. @jamesb93 and @balintlaczko you manage to compile?

latest max-sdk-base?

[tremblap]

no, I always compile against 7.3.3

[jamesb93]

no, I always compile against 7.3.3

sounds like that is the issue. Try compiling against 8.2/3

[balintlaczko]

I compiled successfully after the last change, but yes, I have latest max-sdk-base. NB. as I remember the CLASS_ATTR_LONG had issues on debug build, but no issues on release.

[tremblap]

r.e. latest sdk, I will wait to check with @weefuzzy to make sure we don't break max7 support here.

r.e. build type, I'm always in release until something breaks :)

[tremblap]

sorry it depends on flucoma/flucoma-core#115 - will try again

[balintlaczko]

r.e. build type, I'm always in release until something breaks :)

Especially true if things break because of the debug mode. 😄

[tremblap]

r.e. build type, I'm always in release until something breaks :)

Especially true if things break because of the debug mode. 😄

indeed although the inverse tend to happen - we've had bugs in release that were not there in debug, and Owen created a Test type to give us some optimisation on to find those gremlins...

[tremblap]

sorry it depends on flucoma/flucoma-core#115 - will try again

this sorted it - I'm now super excited of trying that new interface ;)

[tremblap]

ok this is very exciting, and definitely a keeper and the interface is very good and consistent I think - but definitely intermediate to advanced as the magic of all of it working without seeing buffers/datasets is frighteningly slick... the workload to get it to release is ambitious though, so here is a proposal of things in order:

• we need to fix all Max doc, help, examples - I tried umap and by the time I sorted all the problems of embedded patch and stuff to make it work I was puzzled a bit - I presume one gets faster at finding all the broken bits

• I think that we should keep the route on the first page, to make clear that what comes out of the object - so we need to find how we will present the streamlined processes. a tab? an example? a shared tab like the bufthreaddemo?

• we need to sort the bufnmf conundrum. I vote for changing the order of outlets to bases/activations/resynth and create a resynth message - this way we seed the idea that there is much more to this object than demixing

• was there any other object that needed i/o thinking?

** this would be the point where we merge this in the max dev branch **

then the list of todos for the next official release:

• redo the Max videos (especially with the number of outlet changing!)
• sort the Pd equivalent, then help/example/etc
• sort the SC equivalent, then help/example/etc

** then we can do a beta6 (or v1 if we are there) so our less adventurous users will not have conflicting documentation/help/interface **

for SC, I am tempted to do the same interface, i.e. if no buffer is provided, push one. with a default of 1024 buffers on the server that shouldn't be an issue, and we could flag that in our helper for advanced users, no? Ideally we woun't do like Max and create a magic empty buffer at instantiation, so that count would stay under control whilst beginner manually create their buffers...

I'll continue playing with it all a bit, but this is fun and makes the code much less cluttered once the workflow is clear (and when tasks like transforming vs predicting are clear in one's head)

[jamesb93]

• we need to fix all Max doc, help, examples - I tried umap and by the time I sorted all the problems of embedded patch and stuff to make it work I was puzzled a bit - I presume one gets faster at finding all the broken bits

Which means a complete freeze on documentation and clear out of outstanding PRs that deal with help files because diffing Max help files in git makes everyone sad.

#156
#155
#139

I also would suggest that we make an explicit video on how these changes work. It can be relatively rough, but at least a walkthrough of what once was and now is would be good to pair with the release.

• I think that we should keep the route on the first page, to make clear that what comes out of the object - so we need to find how we will present the streamlined processes. a tab? an example? a shared tab like the bufthreaddemo?

Not sure what the route on the first page is but I think a shared tab would be ideal to put in every buffer dealing object's help.

• we need to sort the bufnmf conundrum. I vote for changing the order of outlets to bases/activations/resynth and create a resynth message - this way we seed the idea that there is much more to this object than demixing

I think resynth is what people will do most of the time and so should be first, otherwise we get an "always using the right outlet and now my patch is messy" situation, ala fluid.dataset~.

• was there any other object that needed i/o thinking?

I can't think of any.

** this would be the point where we merge this in the max dev branch **

I agree

then the list of todos for the next official release:

• redo the Max videos (especially with the number of outlet changing!)

Yeah this definitely a complete rewrite. I didn't think about dataset~ changing outlets too. It'll be much faster to teach though it will change the dynamic of how we chain together things a little. For example:

How, and at what point do you explain why the output is a message and not a bang and why does it include the output? I would opt for number 1 in this screenshot, with the caveat of We'll explain why it does this later.

** then we can do a beta6 (or v1 if we are there) so our less adventurous users will not have conflicting documentation/help/interface **

Beta6. I sense we are not done breaking just yet.

for SC, I am tempted to do the same interface, i.e. if no buffer is provided, push one. with a default of 1024 buffers on the server that shouldn't be an issue, and we could flag that in our helper for advanced users, no? Ideally we woun't do like Max and create a magic empty buffer at instantiation, so that count would stay under control whilst beginner manually create their buffers...

I think we have had that conversation to death. What does "pushing" a buffer mean? Do you get it as a return from the instance method or is it passed via action function? @tedmoore has the most articulate reasoning against both.

[tremblap]

• we need to fix all Max doc, help, examples - I tried umap and by the time I sorted all the problems of embedded patch and stuff to make it work I was puzzled a bit - I presume one gets faster at finding all the broken bits

Which means a complete freeze on documentation and clear out of outstanding PRs that deal with help files because diffing Max help files in git makes everyone sad.

#156 #155

indeed these are next on my list

#139 has been merged but without the requested changes. So I reckon we should do that too.

I also would suggest that we make an explicit video on how these changes work. It can be relatively rough, but at least a walkthrough of what once was and now is would be good to pair with the release.

it is a good idea

• I think that we should keep the route on the first page, to make clear that what comes out of the object - so we need to find how we will present the streamlined processes. a tab? an example? a shared tab like the bufthreaddemo?

Not sure what the route on the first page is but I think a shared tab would be ideal to put in every buffer dealing object's help.

I mean that the first tab of all helps should not use the shortcuts.

• we need to sort the bufnmf conundrum. I vote for changing the order of outlets to bases/activations/resynth and create a resynth message - this way we seed the idea that there is much more to this object than demixing

I think resynth is what people will do most of the time and so should be first, otherwise we get an "always using the right outlet and now my patch is messy" situation, ala fluid.dataset~.

we'll have to pick one for the left and the idea is to entice our users to think about something else. resynth is sexy indeed but we will do a specialised message for it. the 'default' message of 'buffer' will generate bases/activations, hence me proposing to move them to the main outlet positions.

• was there any other object that needed i/o thinking?

I can't think of any.
@weefuzzy we talking about those with 2 audio inlets for instance.

• redo the Max videos (especially with the number of outlet changing!)

Yeah this definitely a complete rewrite.

We'll need to have a discussion on which approach is the most relevant. I am not convinced about the shortcut first, especially since transformers and predictors will need to be explained... too much magic at the front has bitten us in the past.

Beta6. I sense we are not done breaking just yet.

We'll need to consolidate soon, considering the time left on the project!

for SC, I am tempted to do the same interface, i.e. if no buffer is provided, push one.

I think we have had that conversation to death.

not enough it seems, since SC users in Oslo had ideas and were envious... @tedmoore said he would poke them and see, so I just made my proposal here.

[jamesb93]

#139 has been merged but without the requested changes. So I reckon we should do that too.

A separate issue has been filed to track the status of that object's documentation. I don't think it is pressing right now and it is not forgotten.

I mean that the first tab of all helps should not use the shortcuts.

Agree.

we'll have to pick one for the left and the idea is to entice our users to think about something else. resynth is sexy indeed but we will do a specialised message for it. the 'default' message of 'buffer' will generate bases/activations, hence me proposing to move them to the main outlet positions.

Is that your idea or everyone's? Maybe I missed the memo. In my mind the most useful and creatively stimulating function of bufnmf~ is unsupervised decomposition. Indeed if bang only does b/a then making those left/right is sensical. I actually think bang should resynth and we should have a message for the other things possibly... I don't know... but sidelining the resynthesis (a massive wow moment for musicians) seems a shame just gauging the response of every room it has been presented in to first timers.

We'll need to have a discussion on which approach is the most relevant. I am not convinced about the shortcut first, especially since transformers and predictors will need to be explained... too much magic at the front has bitten us in the past.

But that is not a shortcut nor what I am proposing as an ethos. The first examples simply abuses the fact that anything going into a message's left inlet triggers the message. If I add the sel fittransform you have to explain what comes out of the outlets and thus can of worms be opened... but you sort of have to explain what comes out of the object's outlets anyway.

We'll need to consolidate soon, considering the time left on the project!

Version 1.0 demands all the documentation be up to date too

not enough it seems, since SC users in Oslo had ideas and were envious... @tedmoore said he would poke them and see, so I just made my proposal here.

⏰

[jamesb93]

@weefuzzy I have a use case which might be unaccounted for or at least minorly worth making viable.

I think that frombuffer should report the dataset that is formed from that method.

[jamesb93]

@weefuzzy I have a use case which might be unaccounted for or at least minorly worth making viable.

I think that frombuffer should report the dataset that is formed from that method.

Actually I am dumb, you already know it in advance, but if you don't provide a dataset~ name perhaps it can spew it out for you?

[tremblap]

actually, you are not dumb at all - one could have set the dataset to refer to something else, and if we are to adopt this paradigm of chaining, we might as well pass on the data when there are no cost and that reinforces the consistent syntax/behaviour.

[weefuzzy]

Be that as it may, it would need to be a separate mechanism to the stuff in this PR: the dataset name is not derived from an output parameter of the function. It could be added later as a nice-to-have by making frombuffer return its dataset name, but for now one can always retrieve the name of the dataset being worked on (re-referred or not) with getname.

[tremblap]

If you think it is not a blocker, and retrieving is possible by other means, I am happy to oblige. On a related matter: is tobuffer passing the name of its destination buffer ?

[weefuzzy]

Yes, and both from / to take an output LabelSet, which would get captured by the new behaviour

[tremblap]

ok this is great. In the light of all the workshops, I am completely convinced of our (old) buffer interface decision was good, and that this is an improvement for more advanced users.