bad conversion between string and decimal types with trailing 0s

[jycor]

When converting between strings and decimals for HASH IN <tuple> expressions, trailing 0s cause a problems.

There's a skipped test added in this PR:
dolthub/go-mysql-server#2267

Setup:

create table t (v varchar(100));
insert into t values ('0'), ('0.0'), ('123'), ('123.0');
create table t_idx (v varchar(100));
create index idx on t_idx(v);
insert into t_idx values ('0'), ('0.0'), ('123'), ('123.0');

MySQL:

mysql> select * from t where (v in (0.0, 123));
+-------+
| v     |
+-------+
| 0     |
| 0.0   |
| 123   |
| 123.0 |
+-------+
4 rows in set (0.0343 sec)
mysql> select * from t_idx where (v in (0.0, 123));
+-------+
| v     |
+-------+
| 0     |
| 0.0   |
| 123   |
| 123.0 |
+-------+
4 rows in set (0.0069 sec)

Dolt:

tmp> select * from t where (v in (0.0, 123));
+-----+
| v   |
+-----+
| 123 |
+-----+
1 row in set (0.00 sec)

tmp> select * from t_idx where (v in (0.0, 123));
+-----+
| v   |
+-----+
| 123 |
+-----+
1 row in set (0.01 sec)

[max-hoffman]

Seems like a two-way relationship. Any string that's convertible to a decimal also hash matches any form of that decimal, and vice versa. But strings only match their literal string. So the mysql hash function is maybe adding two entries to the hash map for every decimal-convertible string? The string literal, and a simplified/normalized decimal if possible.

[jycor]

That's a good idea.

We should also add tests for multiple zeros, negative values (specificically -0.0), and scientific notation

123.0...0
-0.0...0
123e3
-123.4e-2