L3: 3-D (axes+pen servo) FIFO-enqueued synchronized low-level motion command

[lsemprini]

Hi,

Thank you for your awesome EBB.

We would love to use our AxiDraw for 3-D drawing where the vertical (pen) axis is as accurate as possible given the capabilities of the servo mechanism.

We have many applications for this in mind, but one of them is simply to control a soft brush pen with varying vertical positions to give expressive brush thickness effects that vary during strokes---a particularly good application because vertical resolution is not critical.

Currently it appears that the firmware only has the concept of 0/1 up/down with a programmable servo position given to the SC command, and indeed we can program using SC to achieve a variety of positions...BUT...the SC command is documented as having immediate effect. So unless I'm missing something, I don't see a way to synchronize pen servo changes with a/b or x/y changes.

Is there a way to do it now?

If not, I'd like to propose an L3 command that extends LM.

I am proposing to extend LM rather than XM/SM because it will be more powerful for users (albeit more complex) and I suspect it will be LESS work for EmbeddedMan :)

The details no doubt will depend on how the servo control works but here is a rough idea:

L3 – 3-D (axes + pen servo) Low-level Move

• Command: RateTerm1,AxisSteps1,DeltaR1,RateTerm2,AxisSteps2,DeltaR2,RateTerm3,AxisSteps3,DeltaR3
• Response: OK
• Firmware versions: XXX and newer
• Execution: Added to FIFO motion queue
• Arguments: (expand to explain how the servo can be controlled: units, step size, etc.)
• Description: (expand to explain how the servo can be controlled: units, step size, etc.)

[oskay]

Might be simpler to add an alternate form of SC that is added to the motion queue.

[oskay]

Also, here are some known approaches to doing 3D moves with the existing firmware:

• Use shorter movements. The FIFO is only one command deep... so if you care to treat it that way, it can function as a system where your 3D move consists of many short moves, each with a different Z value. (This is much like how movements in XY are composed of many linear segments.) If your SM moves are much shorter in duration than the servo movement time (say under 20 ms), you can simply alternate updating the servo position and sending new movements. You can use the QM command if necessary to verify when the FIFO queue is empty in order to ensure that things remain synchronized.
• You can send a pen lift (or lower) command directly before an XY move command. For example, the command SP,1\rSM,1000,0,3200 will begin the Z move and then (almost immediately) begin the XY move, such that the two happen simultaneously in practice. Note that this use of the SP command does not specify a delay time. If you do specify a delay time (as in SP,1,1000\rSM,1000,0,3200), then the XY move will not begin until the Z move completes. The rate of the pen lift can be specified with the SC command.
• You can directly change the definition of (for example) the pen-down Z position, by giving a command like SC,4,10000\rSP,1 that both configures the pen down position and moves to that new position. Note that the value of Z position that you send (10000 here) should be kept in the range specified in axidraw_conf.py, 7500 to 28000, in order to prevent damage to the mechanism. This type of command also be executed before an XY move, to give the same type of simultaneous motion.

[lsemprini]

Hi oskay! Thanks for your response.

A queued version of SC would also be OK (the only disadvantage being no support for an acceleration parameter, but not sure that would even be physically possible to implement with the pen servo).

As to your known approaches (thanks for the list!), we had tried some of those but kept running into different issues:

• the strategy of sending a large number of SC+SP+tiny move commands, and waiting for a response before proceeding so that all 3 commands are basically treated as unqueued/immediate, is the first thing we tried, but because there is so much latency getting from our python software process through the OS USB layers, through the hardware USB, and getting the response back again, this led to pretty crazy bad performance. Because the hardware has no motion queued up ever when using this strategy, it spends most of its time (most of its duty cycle) not moving, waiting for another motion command. That unqueued approach also lead to such jerky movement in all axes on the actual device that the picture drawn by the pen was affected and notably not smooth (maybe due to extra vibration?). It would be way, way better if we could somehow queue up movements in X, Y, and Z well ahead of time to keep the device happy and not have to worry about the USB latency. Note that in this case it is high latency (as opposed to unpredictable latency) that is the problem; contrast that with the last strategy described below.

• the strategy of firing off a long-duration set of XY move commands at about the same time as a single long pen-up/pen-down command works if we want to move the pen linearly during that period of time, but of course that depends on the content being rendered. If we want the motion in the Z direction to be "curved" (anything but a straight line over time), as we often see when stroking with a brush, it won't work. But thanks for mentioning this strategy.

• another strategy we thought of was whether we could issue the usual high-performance, very-very-queued up XY motion commands ahead of time and then try to issue the immediate pen commands at the "right" time. This has two big problems. One is that no modern OS allows our software process reliable scheduling so that we can run the code to send the pen commands at the right time. Another is that even if we had real-time scheduling in the OS, in order to know when to send the pen commands we would have to know the exact value of the latency through the whole system (OS USB software/hardware, device USB software/hardware). But that latency is unknown (to clarify, for this strategy, the latency could be very large with no problem, but the fact that its value is unknown is a big problem). One could possibly rig up a closed-loop test using the QM command to try to measure the latency at a given moment, but I'm sure sometimes the latency changes due to different congestion conditions at different points in the pipeline. It seems basically hopeless to get this approach working, even if we had proper OS scheduling, but since we don't even have real-time scheduling, it's moot.

So thanks again for sending those "known approaches." I appreciate the time you're taking to help us and others, and when I wrote this comment I just wanted to write down the issues with those approaches to help motivate why doing a queued pen-movement command would be worthwhile.

Thanks!

[EmbeddedMan]

Thinking about this some more, I like the L3 approach the best. I don't think it would be that much more work (saying that without the code in front of me), and it would give the best results. I'll find some time to play around with it and see what can be accomplished. Honestly, all of the functionality is actually in place (moving the steppers some amount per unit time, and moving the servo some amount per unit time), you just can't do both once with the current code. *Brian

…

On Mon, Jun 4, 2018 at 4:15 PM, lsemprini ***@***.***> wrote: Hi oskay! Thanks for your response. A queued version of SC would also be OK (the only disadvantage being no support for an acceleration parameter, but not sure that would even be physically possible to implement with the pen servo). As to your known approaches (thanks for the list!), we had tried some of those but kept running into different issues: - the strategy of sending a large number of SC+SP+tiny move commands, and waiting for a response before proceeding so that all 3 commands are basically treated as unqueued/immediate, is the first thing we tried, but because there is so much latency getting from our python software process through the OS USB layers, through the hardware USB, and getting the response back again, this led to pretty crazy bad performance. Because the hardware has no motion queued up ever when using this strategy, it spends most of its time (most of its duty cycle) not moving, waiting for another motion command. That unqueued approach also lead to such jerky movement in all axes on the actual device that the picture drawn by the pen was affected and notably not smooth (maybe due to extra vibration?). It would be way, way better if we could somehow queue up movements in X, Y, and Z well ahead of time to keep the device happy and not have to worry about the USB latency. Note that in this case it is high latency (as opposed to unpredictable latency) that is the problem; contrast that with the last strategy described below. - the strategy of firing off a long-duration set of XY move commands at about the same time as a single long pen-up/pen-down command works if we want to move the pen linearly during that period of time, but of course that depends on the content being rendered. If we want the motion in the Z direction to be "curved" (anything but a straight line over time), as we often see when stroking with a brush, it won't work. But thanks for mentioning this strategy. - another strategy we thought of was whether we could issue the usual high-performance, very-very-queued up XY motion commands ahead of time and then try to issue the immediate pen commands at the "right" time. This has two big problems. One is that no modern OS allows our software process reliable scheduling so that we can run the code to send the pen commands at the right time. Another is that even if we had real-time scheduling in the OS, in order to know when to send the pen commands we would have to know the exact value of the latency through the whole system (OS USB software/hardware, device USB software/hardware). But that latency is unknown (to clarify, for this strategy, the latency could be very large with no problem, but the fact that its value is unknown is a big problem). One could possibly rig up a closed-loop test using the QM command to try to measure the latency at a given moment, but I'm sure sometimes the latency changes due to different congestion conditions at different points in the pipeline. It seems basically hopeless to get this approach working, even if we had proper OS scheduling, but since we don't even have real-time scheduling, it's moot. So thanks again for sending those "known approaches." I appreciate the time you're taking to help us and others, and when I wrote this comment I just wanted to write down the issues with those approaches to help motivate why doing a queued pen-movement command would be worthwhile. Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAbeCBVK2NU3iMOF5Yw6ygEFBgtv9BCcks5t5aNVgaJpZM4UYhMm> .

[oskay]

If we do, it would be nice to add matching S3 and X3 commands as well.

[EmbeddedMan]

Completely agreed. That shouldn't be too hard either. *Brian

…

On Mon, Jun 4, 2018 at 5:29 PM, Windell Oskay ***@***.***> wrote: If we do, it would be nice to add matching S3 and X3 commands as well. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAbeCG2lhNEj14ut39X4IMHAu1tfvWtSks5t5bTkgaJpZM4UYhMm> .

[EmbeddedMan]

OK, after looking at the code for a few minutes, this isn't going to be trivial. The code that handles RC servo moves currently is broken out into two sections. One runs at the high ISR rate (25KHz) and pulls RC servo moves off of the motion queue. When the current command is an RC servo move, then it sets a bunch of values which start that RC servo moving and then it's done. The second piece of code runs at the low ISR rate (1KHz) and it looks at the values set up by the first piece of code. Each time the particular RC servo channel's slot comes around, it sees if it needs to change the position, and if so by how much. There's nothing wrong with this, but in order for the L3 command to work, we're going to have to add a flag into the second piece of code. If the currently executing command is an L3 command, then we can't pull the next command off the motion queue until both steppers are done moving AND the servo is done moving (this is the point of the new flag, which will be set by the low ISR when a servo move is complete). I need to be careful not to change the behavior of the existing SP commands at all, so when I get this code done I'll want you guys to test it out in multiple real world situations before we call it good. But I think this should not be too hard - we'll create a new kind of command that does all 3, and we will expand the structure put onto the motion queue to include RC servo information. One thing we have to figure out is the RC servo channel. Remember, we can have up to 8 RC servos going at the same time on the EBB. One of them is normally used for the pen up/down servo. Should the L3 command have an RC servo channel parameter that lets you choose any of the 8 channels to apply the RC servo move to? Or should we just hard-code the L3 command to always use the RC servo channel that the pen up/down (SP) is tied to? We could (easily) of course make this take a set of 3 parameters for all 8 of the RC servo channels, so you could have all 8 doing different things at different times, speeding up, slowing down, etc. That would in essence give you a 10 channel motion control system, where 2 of them are high resolution full rotation, and 8 of them are low resolution partial rotation. Hmmm. Other than like weird puppetry, I'm not sure I can think of a good use for that. Since we're not really trying to make the EBB the world's best RC servo controller, maybe just limiting the command to one RC servo channel is good enough. But I like the idea of allowing it to act on ANY of the 8 channels. *Brian

…

On Mon, Jun 4, 2018 at 6:50 PM, Brian Schmalz ***@***.***> wrote: Completely agreed. That shouldn't be too hard either. *Brian On Mon, Jun 4, 2018 at 5:29 PM, Windell Oskay ***@***.***> wrote: > If we do, it would be nice to add matching S3 and X3 commands as well. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#101 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAbeCG2lhNEj14ut39X4IMHAu1tfvWtSks5t5bTkgaJpZM4UYhMm> > . >

[oskay]

What about the idea of making a version of SC that goes into the FIFO?
SC, value1, value2[, addtoFIFO]<CR>

[EmbeddedMan]

That would be a much bigger change. Currently none of the various SC commands enter the FIFO, and there is nothing on the other end to 'execute' them. Right now, all SC commands are simply executed as soon as they are parsed. If I were to do this whole thing over, I'd make all of the commands (or at least most of them) be able to be executed immediately, or go into the motion control queue and executed at their proper point in time. Doing that now would, I fear, result in a huge refactoring of some parts of the codebase, but would make this thing far more flexible. Ahh, hindsight. *Brian

…

On Mon, Jun 4, 2018 at 7:35 PM, Windell Oskay ***@***.***> wrote: What about the idea of making a version of SC that goes into the FIFO? SC, *value1*, *value2*[, *addtoFIFO*]<CR> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAbeCGCefpYqQWIwxySYKnD-jc95Qu2Jks5t5dJEgaJpZM4UYhMm> .

[oskay]

As one more alternative approach, what about adding the third axis directly to the SM command? (S3?)

It could be something like:
S3,duration,AxisSteps1[,AxisSteps2][,servoFinal]

The advantage of this would be that it is time definite, so when it comes time to execute the S3 command from the FIFO, it could be modeled as doing the following existing things:

(1) Execute an immediate change of the pen-up or pen-down position (whichever we're not in), the same way that we do with SC,4/SC,5
(2) Execute an immediate change of the pen slew rate, the same way that we do with SC,10. This should be straightforward, since we know both the total time duration in ms and the difference between the pen-up and pen-down positions.
(3) Execute an immediate start of the pen vertical motion, the same way that we do with TP.
(4) Execute the remainder of the S3 command, as one would normally with an SM.

[EmbeddedMan]

Yes, this is in essense how the S3 command will need to work. But internally it will compute all of the parameters to an L3 command, and then call L3 (just like SM currently computes all of the parameters for the LM and then calls LM). *Brian

…

On Mon, Jun 4, 2018 at 8:18 PM, Windell Oskay ***@***.***> wrote: One more alternative approach, what about adding the third axis directly to the SM command? (S3?) It could be something like: S3,*duration*,*AxisSteps1*[,*AxisSteps2*][,*servoFinal*] The advantage of this would be that it is *time definite*, so when it comes time to execute that command from the FIFO, it could be modeled as doing the following existing things: (1) Execute an immediate change of the pen-up or pen-down position (whichever we're not in), the same way that we do with SC,4/SC,5 (2) Execute an immediate change of the pen slew rate, the same way that we do with SC,10. This should be straightforward, since we *know* both the total time duration in ms and the difference between the pen-up and pen-down positions. (3) Execute an immediate start of the pen vertical motion, the same way that we do with TP. (4) Execute the remainder of the SM command. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAbeCBAUAki0WGzzQAF70w3FQeDG0kUXks5t5dxWgaJpZM4UYhMm> .

[lsemprini]

Glad to hear from you both! L3 sounds great as EmbeddedMan describes.

As to the multiple servo issue, we can certainly dream up uses for the other 7 servo signals (first one that comes to mind is 2 servos for pen tilt), but for now we're only looking for xy+pen motion.

For some future-proofing, you could define the command as:

L3, termX,stepsX,deltaX, termY,stepsY,deltaY, servo_id0,term0,steps0,delta0, servo_id1,term1,steps1,delta1, ...

where the caller provides between 1 and 8 groups of servo_id,term,steps,delta for the servos that the user wants to move with the command.

If variable-numbers-of-arguments is a pain, then you can simply require us to provide all 8 with steps==0 for the servos we don't want to move, in which case no need for servo_id:

L3, termX,stepsX,deltaX, termY,stepsY,deltaY, term0,steps0,delta0, term1,steps1,delta1, term2,steps2,delta2, term3,steps3,delta3, term4,steps4,delta4, term5,steps5,delta5, term6,steps6,delta6, term7,steps7,delta7

Though in this case you can think about whether or not you're always going to have 8 servos on your hardware.

Or you could also think about separating the commands for XY and servo (retaining LM and adding another servo-only queued command "SV - Queued Servo Move" that simply takes 4 fixed args servo_id,term,steps,delta) and that lets the app decide which servos to move by which commands they send. So in our case we would queue up a stream of LM,SV,LM,SV,LM,SV to achieve synchronized 3-D motion. Not sure if that makes it easier or harder for you to weave together the command effects on the firmware, but it does solve the multi-servo issue more cleanly.

With all proposals above, you could support exactly 1 servo for now, and later you would still have room to implement multiple servos.

I assume there's a way that we can know or query which servo is being used for the pen?

I need to be careful not to change the behavior of the existing SP commands at all, so when I get this code done I'll want you guys to test it out in multiple real world situations before we call it good.

Definitely we're eager to test! We don't have a logic analyzer or anything here but we can test the behavior for sure.