`accel_brake_map_calibrator` does not want to calibrate some values

[VRichardJP]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I'm convinced that this is not my fault but a bug.

Description

I have been trying to calibrate accel/brake of my vehicle.

After many painful attempts with a joystick controller (too sensitive), I have implemented a keyboard based controller, with which I am able to send fixed accel/brake command values (from -1.0 to 1.0). Event after 2 hours of calibration on a mostly straight circuit, the calibration map is far from being finished:

In particular, I observed the calibrator tool "refuses" to map some specific values. In my case, the 0.3, -0.3, -0.5 and -0.6 rows are extremelly difficult to get data on (the cursor is almost always red). But it is also true for all the remaining blue/grey squares. On the other hands, all the pink squares were super easy to get, and would only require a few minutes for each (or even seconds for the 0.2 row).

How come some cells are nearly impossible to map, while some others are super easy? is there any parameter I should adjust? is it because of the default accel/brake map?

Unfortunatelly, I tested the calibration result in autonomous mode, and autoware is unable to accurately stop at goal position. I have tried to modify the brake_map.csv values by hand and got some improvement, but this is a less than desirable method. At least, it proved me my issue is a calibration issue.

Expected behavior

calibrator tool can map any actuation/speed value

Actual behavior

some actuation/speed values are nearly impossible to calibrate

Steps to reproduce

1. Have my vehicle? (not a Lexus)

Versions

• Ubunut 20.04
• ROS2 galactic
• main

Possible causes

No response

Additional context

No response

[TakaHoribe]

@VRichardJP

I observed the calibrator tool "refuses" to map some specific values

The current algorithm refuses the data if it is not in the stable condition, like the pedal is moving fast. You can change the thresholds to use all data, but it could cause a bad accuracy. Please see the readme in detail.
https://autowarefoundation.github.io/autoware.universe/main/vehicle/accel_brake_map_calibrator/accel_brake_map_calibrator/

Unfortunatelly, I tested the calibration result in autonomous mode, and autoware is unable to accurately stop at goal position.

Would you share the calibration result? You can create the figure from the calibration data as described in the readme. The error covariance, a number of valid data, etc are visualized there.
https://autowarefoundation.github.io/autoware.universe/main/vehicle/accel_brake_map_calibrator/accel_brake_map_calibrator/#how-to-visualize-calibration-data

[VRichardJP]

Thank you for the feedback.

During the calibration, my main pattern was like:

• speed: 0.0 -> set accel_cmd to 0.X
• wait until max speed is reached
• at max speed -> set accel_cmd to -0.X
• wait until vehicle is stopped.

I assumed that if the actuation command is maintained for a few seconds like so, the calibrator would take plenty of data. However, there are definitely cmd values for which the tool takes value and others for which it refects almost everything. Typically, during my calibration session, 0.4, 0.2, -0.2 and -0.4 went ok, but 0.3 and -0.3 would barely record anything.

I would understand there is less data to collect for big accel/brake values, but the 0.3/-0.3 row shows it is not simply that.

I don't have the calibration file right now. But one particularity I observed is that the 0.3 row values from brake_map.csv are quite wrong (maybe the same for the 0.3 row of accel_map.csv, but I have not checked). On my vehicle, the 0.3 brake is very soft. It feels barely stronger than 0.2. It can't even keep the vehicle stopped (min speed is maybe ~1km/h). On the other hand, the 0.4 is already a good/smooth brake. As a consequence, I would expect the decel value for 0.3 row to be closer to those from row 0.2, than 0.4. In the file it is the opposite. I get values like:

0.2,-0.1,-0.3,0.35,...
0.3,-0.7,-0.8,-0.85,...
0.4,-0.9,-1.1,-1.2,...

I can share the exact calibration result tomorrow.

[TakaHoribe]

@VRichardJP I noticed that no description is in the readme for the data-rejection-related parameters. I've made a PR #1918 for it. We need to care about the initial_covariance or **_threshold parameters, etc.

You can also see why the data is not used in the debug msg here (you need to change the logger level to DEBUG). This would be helpful for the analysis.
https://github.com/autowarefoundation/autoware.universe/blob/main/vehicle/accel_brake_map_calibrator/accel_brake_map_calibrator/src/accel_brake_map_calibrator_node.cpp/#L256-L269

[VRichardJP]

I went through a lot today :)

@TakaHoribe
As you suggested, I went for another calibration with the debug logger and checked why the values were rejected. Most data rejection comes from jerk. I drove 3 loops over my calibration course, one with +-0.4, one with +-0.3 and one with +-0.2 values. After the 3 loops, the calibration map looks like this:

I went on and inspected the accel/jerk values received by the calibrator, and it's indeed pretty bad (in many case, it just looks like random values between -3.0 and +3.0). So I checked the /vehicle/status/velocity_status speed report, and it is indeed quite jerky. Actually I am surprised the calibrator even accept any of that data.

The problem with my vehicle is that the /vehicle/status/velocity_status values are based on the vehicle speed feedback from the CAN network, and that speed feedback seems to have 1km/h step increment. In other words, when I accelerate, the speed report is at 0.0, then jumps to 0.277, then to 0.55 and so on. Needless to say it is impossible to compute accelation with such feedback. For some strange reason, it was when the pedal value was at +0.3/-0.3 that this jerk was the strongest (hence the weird calibration map above)

I think the speed itself is not an issue for the calibrator. The problem is that the calibrator uses that speed to compute acceleration, then jerk. So I had to find another way to get a smooth acceleration value in autoware.

Interestingly, autoware produces no good acceleration report. The closest thing I found is /localization/acceleration, but it is based on EKF output, and EKF basically copies the speed value from my inaccurate speed report. Actually EKF does some smoothing because it takes into account several messages. But when I plot the output /localization/acceleration, it just looks like a slightly smoothened step function.

Since I have a fairly good IMU, I thought it should be possible to use EKF to update the speed. So I modified the imu_corrector to add some fixed stddev covariance to the IMU longitudinal_acceleration messages, I added this IMU feedback to EKF (I followed the same approach than pose and twist), and added accelation to the EKF model, so I could update speed like so: vx{k+1} = vx{k} + a{k} * dt. After some noise/stdded tuning, I managed to get a very smooth speed estimation from EKF (in /localization/kinematic_state).

Finally, I modified the calibrator to use the kinematic state in place of my inaccurate speed report. And the calibration suddenly became quite easy! In ~45minutes, I managed to map almost everything I wanted (I skipped the -0.7/-0.8 rows because this was like emergency brake). The only part I couldn't map was the 0.0 column (<2.5km/h):

So I guess my calibration is not bad? Here are the files:
brake_map.csv
accel_map.csv

and associated plots:

Unfortunatelly, when I use this calibration to drive the vehicle autonomously, autoware fails to stop at the goal point, but always pass the goal point and eventually stop with an emergency brake. It is somewhat similar to #993, except it is a new vehicle and my other one does not have any issue anymore (:

I guess this must be related to the low speed not being calibrated properly. But how can I have good calibration at low speed if the calibrator does not want to map any data?

I also observe the planned path has a very short brake distance. Maybe it is a configuration issue from autoware's side?

[VRichardJP]

I solved the that last issue the same way I solved it in #993:

• reduced normal.min_accel and normal.min_jerk from 1.0 to 0.4~0.5
• set speed limit to 1m/s 2 meters from stop point
• shifted all the values of the first column of the brake_map.csv, essentially adding a +0.1 command offset when speed is <5km/h

Although it seems to work, the last trick is far from ideal. So the problem comes back to:

• the calibrator should not assume speed is smooth enough to compute accel and jerk. I believe some more sophisticated filter should do that job, such as EKF
• Even if speed report is accurate, the calibrator struggle to map any value at low speed. As a consequence, it is difficult to have the vehicle stop precisely at any location.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.