-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problems with type conversions #78
Comments
If it could be useful I am running MariaDB 10.0.28. |
Can you test last engineering version 4.041_1? Or version from git master? |
On Fri, Jan 20, 2017 at 04:19:32PM -0800, pali wrote:
Can you test last engineering version 4.041_1? Or version from git master?
I tried git master and the issue is still there, it seems that amavisd run correctly the query but does not undestand that the returned result is a float and it treats it as an int.
Cheers & Thanks
Giovanni
… --
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
#78 (comment)
|
Can you describe problem in DBD::mysql? Sorry, but from your report I just understand that amavisd-new does not work... |
On Sat, Jan 21, 2017 at 08:03:19AM -0800, pali wrote:
Can you describe problem in DBD::mysql? Sorry, but from your report I just understand that amavisd-new does not work...
The problem happens only with DBD::mysql > 4.037, I am investigating also on
amavisd-new side.
Giovanni
… --
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
#78 (comment)
|
Ideally if you can provide perl code which outputs that "wrong result". |
I believe this is the relevant code in amavisd-new (note I haven't been able to reproduce this myself):
This results in output like:
Where the "0" values are wrong. These aren't just rounding errors, the values in the database were very different, e.g.:
For more details see:
To me it looks like the relevant part is sprintf using "%s", I don't see how this could get the wrong value, even if it is string, float, or double. So I suspect DBD-mysql might be returning bad data. Note that that I haven't reproduced this myself, just assembling data from existing reports. |
That is strange :-( I'm still not able to reproduce this problem... I tried:
and its output is correct:
Data stored in mysql are:
So I see only option now, enabling DBI tracing in that problematic amavis code and sending stderr trace output for later inspection. Tracing can be enabled by |
Full debug output is available here: https://bugs.debian.org/cgi-bin/bugreport.cgi?att=2;bug=847311;filename=amavisd-debug.txt;msg=54 I believe these are the most relevant lines:
(side note: what result would you expect if the database has NULL values?) I believe (but not confirmed) this is the database schema used: I have requested the submitter provide the following information:
|
Ok, so DBI trace show that DBI itself got Please provide that SELECT and schemas from mysql console client to verify that there is not problem on server. I would like to see output from that SELECT by simple perl script (e.g. reuse my above example). And also please provide affected perl version (perl -V). Also I would like to see what is internally stored in perl scalar represening that
(in check which column in $ref is spam_tag_level). Dump outputs scalar structure to stderr. Also if you can it would be good to provide also TCP dump of mysql protocol communication. That could verify what is really sent from MySQL server. It should be possible e.g. via wireshark or tcpdump and setting mysql to connect via TCP on 127.0.0.1 port 3306 (beware that specifying "localhost" means to connect via UNIX filesystem socket! so IP address needs to be used). |
Some additional information you asked for (but not all of it):
|
On 02/13/17 23:27, pali wrote:
Ok, so DBI trace show that DBI itself got |0| for spam_tag_level column. Strange.
Please provide that SELECT and schemas from mysql console client to verify that there is not problem on server.
I would like to see output from that SELECT by simple perl script (e.g. reuse my above example). And also please provide affected perl version (perl -V).
Also I would like to see what is /internally/ stored in perl scalar represening that |spam_tag_level|. You can do that by calling function |Dump| from |Devel::Peek|:
|use Devel::Peek; Dump($ref->[3]); |
(in check which column in $ref is spam_tag_level). Dump outputs scalar structure to stderr.
Also if you can it would be good to provide also TCP dump of mysql protocol communication. That could verify what is really sent from MySQL server. It should be possible e.g. via wireshark or tcpdump and setting mysql to connect via TCP on 127.0.0.1 port 3306 (beware that specifying "localhost" means to connect via UNIX filesystem socket! so IP address needs to be used).
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#78 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACYK1ogJygF--sKRhNpCRsJow2sunbKfks5rcNi0gaJpZM4LjKl9>.
Some infos attached.
Cheers
Giovanni
$VAR1 = [
[
'id',
'sys_userid',
'sys_groupid',
'sys_perm_user',
'sys_perm_group',
'sys_perm_other',
'server_id',
'priority',
'policy_id',
'email',
'fullname',
'local',
'id',
'sys_userid',
'sys_groupid',
'sys_perm_user',
'sys_perm_group',
'sys_perm_other',
'policy_name',
'virus_lover',
'spam_lover',
'banned_files_lover',
'bad_header_lover',
'bypass_virus_checks',
'bypass_spam_checks',
'bypass_banned_checks',
'bypass_header_checks',
'spam_modifies_subj',
'virus_quarantine_to',
'spam_quarantine_to',
'banned_quarantine_to',
'bad_header_quarantine_to',
'clean_quarantine_to',
'other_quarantine_to',
'spam_tag_level',
'spam_tag2_level',
'spam_kill_level',
'spam_dsn_cutoff_level',
'spam_quarantine_cutoff_level',
'addr_extension_virus',
'addr_extension_spam',
'addr_extension_banned',
'addr_extension_bad_header',
'warnvirusrecip',
'warnbannedrecip',
'warnbadhrecip',
'newvirus_admin',
'virus_admin',
'banned_admin',
'bad_header_admin',
'spam_admin',
'spam_subject_tag',
'spam_subject_tag2',
'message_size_limit',
'banned_rulenames',
'id'
],
[
402,
1,
0,
'riud',
'riud',
'',
6,
10,
11,
'giovanni@paclan.it',
'giovanni@paclan.it',
'Y',
11,
1,
1,
'riud',
'riud',
'r',
'Normal-no-banned',
'N',
'N',
'Y',
'N',
'N',
undef,
'N',
'N',
'Y',
undef,
undef,
undef,
undef,
undef,
undef,
'1',
'3.5',
'4.5',
'0',
'0',
'',
'',
'',
'',
'N',
'N',
'N',
'',
'',
'',
'',
'',
'',
'***SPAM***',
0,
'',
402
],
{
'newvirus_admin' => '',
'addr_extension_bad_header' => '',
'fullname' => 'giovanni@paclan.it',
'policy_name' => 'Normal-no-banned',
'addr_extension_spam' => '',
'sys_perm_group' => 'riud',
'clean_quarantine_to' => undef,
'email' => 'giovanni@paclan.it',
'sys_userid' => 1,
'virus_admin' => '',
'spam_kill_level' => '4.5',
'spam_tag_level' => '1',
'bypass_virus_checks' => 'N',
'banned_rulenames' => '',
'bad_header_admin' => '',
'spam_tag2_level' => '3.5',
'addr_extension_banned' => '',
'spam_lover' => 'N',
'spam_subject_tag2' => '***SPAM***',
'spam_admin' => '',
'warnvirusrecip' => 'N',
'server_id' => 6,
'spam_quarantine_to' => undef,
'message_size_limit' => 0,
'warnbannedrecip' => 'N',
'banned_files_lover' => 'Y',
'banned_quarantine_to' => undef,
'bad_header_lover' => 'N',
'sys_perm_user' => 'riud',
'banned_admin' => '',
'spam_quarantine_cutoff_level' => '0',
'policy_id' => 11,
'warnbadhrecip' => 'N',
'spam_dsn_cutoff_level' => '0',
'virus_lover' => 'N',
'bypass_header_checks' => 'N',
'bypass_banned_checks' => 'N',
'priority' => 10,
'id' => 402,
'sys_perm_other' => 'r',
'bad_header_quarantine_to' => undef,
'spam_modifies_subj' => 'Y',
'other_quarantine_to' => undef,
'sys_groupid' => 1,
'addr_extension_virus' => '',
'virus_quarantine_to' => undef,
'bypass_spam_checks' => undef,
'local' => 'Y',
'spam_subject_tag' => ''
},
'id=>"402", sys_userid=>"1", sys_groupid=>"1", sys_perm_user=>"riud", sys_perm_group=>"riud", sys_perm_other=>"r", server_id=>"6", priority=>"10", policy_id=>"11", email=>"giovanni@paclan.it", fullname=>"giovanni@paclan.it", local=>"Y", id=>"402", sys_userid=>"1", sys_groupid=>"1", sys_perm_user=>"riud", sys_perm_group=>"riud", sys_perm_other=>"r", policy_name=>"Normal-no-banned", virus_lover=>"N", spam_lover=>"N", banned_files_lover=>"Y", bad_header_lover=>"N", bypass_virus_checks=>"N", bypass_spam_checks=>-, bypass_banned_checks=>"N", bypass_header_checks=>"N", spam_modifies_subj=>"Y", virus_quarantine_to=>-, spam_quarantine_to=>-, banned_quarantine_to=>-, bad_header_quarantine_to=>-, clean_quarantine_to=>-, other_quarantine_to=>-, spam_tag_level=>"1", spam_tag2_level=>"3.5", spam_kill_level=>"4.5", spam_dsn_cutoff_level=>"0", spam_quarantine_cutoff_level=>"0", addr_extension_virus=>"", addr_extension_spam=>"", addr_extension_banned=>"", addr_extension_bad_header=>"", warnvirusrecip=>"N", warnbannedrecip=>"N", warnbadhrecip=>"N", newvirus_admin=>"", virus_admin=>"", banned_admin=>"", bad_header_admin=>"", spam_admin=>"", spam_subject_tag=>"", spam_subject_tag2=>"***SPAM***", message_size_limit=>"0", banned_rulenames=>"", id=>"402"'
];
"402"
$VAR1 = [
[
'id',
'sys_userid',
'sys_groupid',
'sys_perm_user',
'sys_perm_group',
'sys_perm_other',
'server_id',
'priority',
'policy_id',
'email',
'fullname',
'local',
'id',
'sys_userid',
'sys_groupid',
'sys_perm_user',
'sys_perm_group',
'sys_perm_other',
'policy_name',
'virus_lover',
'spam_lover',
'banned_files_lover',
'bad_header_lover',
'bypass_virus_checks',
'bypass_spam_checks',
'bypass_banned_checks',
'bypass_header_checks',
'spam_modifies_subj',
'virus_quarantine_to',
'spam_quarantine_to',
'banned_quarantine_to',
'bad_header_quarantine_to',
'clean_quarantine_to',
'other_quarantine_to',
'spam_tag_level',
'spam_tag2_level',
'spam_kill_level',
'spam_dsn_cutoff_level',
'spam_quarantine_cutoff_level',
'addr_extension_virus',
'addr_extension_spam',
'addr_extension_banned',
'addr_extension_bad_header',
'warnvirusrecip',
'warnbannedrecip',
'warnbadhrecip',
'newvirus_admin',
'virus_admin',
'banned_admin',
'bad_header_admin',
'spam_admin',
'spam_subject_tag',
'spam_subject_tag2',
'message_size_limit',
'banned_rulenames',
'id'
],
[
1122,
1,
32,
'riud',
'riud',
'',
6,
5,
5,
'@paclan.it',
'@paclan.it',
'Y',
5,
1,
0,
'riud',
'riud',
'r',
'Normal',
'N',
'N',
'N',
'N',
'N',
'N',
'N',
'N',
'Y',
'',
'',
'',
'',
'',
'',
'1',
'3.5',
'4.5',
'0',
'0',
'',
'',
'',
'',
'N',
'N',
'N',
'',
'',
'',
'',
'',
'',
'***SPAM***',
0,
'',
1122
],
{
'bypass_spam_checks' => 'N',
'virus_quarantine_to' => '',
'addr_extension_virus' => '',
'sys_groupid' => 0,
'other_quarantine_to' => '',
'spam_subject_tag' => '',
'local' => 'Y',
'priority' => 5,
'bypass_banned_checks' => 'N',
'bypass_header_checks' => 'N',
'virus_lover' => 'N',
'spam_dsn_cutoff_level' => '0',
'warnbadhrecip' => 'N',
'spam_modifies_subj' => 'Y',
'sys_perm_other' => 'r',
'bad_header_quarantine_to' => '',
'id' => 1122,
'banned_admin' => '',
'bad_header_lover' => 'N',
'sys_perm_user' => 'riud',
'policy_id' => 5,
'spam_quarantine_cutoff_level' => '0',
'warnbannedrecip' => 'N',
'message_size_limit' => 0,
'banned_files_lover' => 'N',
'banned_quarantine_to' => '',
'addr_extension_banned' => '',
'spam_tag2_level' => '3.5',
'spam_quarantine_to' => '',
'warnvirusrecip' => 'N',
'server_id' => 6,
'spam_subject_tag2' => '***SPAM***',
'spam_lover' => 'N',
'spam_admin' => '',
'virus_admin' => '',
'sys_userid' => 1,
'bad_header_admin' => '',
'bypass_virus_checks' => 'N',
'banned_rulenames' => '',
'spam_tag_level' => '1',
'spam_kill_level' => '4.5',
'sys_perm_group' => 'riud',
'clean_quarantine_to' => '',
'addr_extension_spam' => '',
'email' => '@paclan.it',
'addr_extension_bad_header' => '',
'newvirus_admin' => '',
'policy_name' => 'Normal',
'fullname' => '@paclan.it'
},
'id=>"1122", sys_userid=>"1", sys_groupid=>"0", sys_perm_user=>"riud", sys_perm_group=>"riud", sys_perm_other=>"r", server_id=>"6", priority=>"5", policy_id=>"5", email=>"@paclan.it", fullname=>"@paclan.it", local=>"Y", id=>"1122", sys_userid=>"1", sys_groupid=>"0", sys_perm_user=>"riud", sys_perm_group=>"riud", sys_perm_other=>"r", policy_name=>"Normal", virus_lover=>"N", spam_lover=>"N", banned_files_lover=>"N", bad_header_lover=>"N", bypass_virus_checks=>"N", bypass_spam_checks=>"N", bypass_banned_checks=>"N", bypass_header_checks=>"N", spam_modifies_subj=>"Y", virus_quarantine_to=>"", spam_quarantine_to=>"", banned_quarantine_to=>"", bad_header_quarantine_to=>"", clean_quarantine_to=>"", other_quarantine_to=>"", spam_tag_level=>"1", spam_tag2_level=>"3.5", spam_kill_level=>"4.5", spam_dsn_cutoff_level=>"0", spam_quarantine_cutoff_level=>"0", addr_extension_virus=>"", addr_extension_spam=>"", addr_extension_banned=>"", addr_extension_bad_header=>"", warnvirusrecip=>"N", warnbannedrecip=>"N", warnbadhrecip=>"N", newvirus_admin=>"", virus_admin=>"", banned_admin=>"", bad_header_admin=>"", spam_admin=>"", spam_subject_tag=>"", spam_subject_tag2=>"***SPAM***", message_size_limit=>"0", banned_rulenames=>"", id=>"1122"'
];
"1122"
|
This is from your perl code, some more info attached. |
Ah... mysql (resp. mariadb) sees data correctly. Simple SELECT over perl DBI from amavisd-new code sees data correctly. And amavisd-new itself with DBI gets incorrect data, plus DBI sees incorrect data. Has some else got similar problem with other application as amavisd-new? Otherwise conclusion is that just amavisd-new with DBD::mysql does not work and it is only one reproducer... The last thing which can show some light into this problem is looking at Dump information from Devel::Peek. |
Some more tests: |
Please call Dump as early as possible, ideally before other functions (like that join) which read that scalar (as they can alter it). And you do not have to check if scalar is defined. Dump correctly handle also undefs. From your Dump we can just see that variable is tainted. Are you running perl in taint (-T) mode? (Note that it should have no effect for DBI...) |
Now I called Dump just after "$match = {}; @$match{@NAMEs} = @$a_ref;". |
remove "perl -T" from amavisd-new first line, amavisd gives a lot of errors because it is not running in tainted mode but the query give correct results. |
Thank you for info! Now we know that problem is related to taint mode. I'm still not able to reproduce this problem, but from your Dump it looks like perl refused to assign value from mysql to perl scalar. Can you try to apply this patch to DBD-mysql and recompile it?
It should print additional info to stderr for all fetch commands. sv_dump is same as Dump. |
Data attached. |
Thank you! From your output we can see that NV value (float) is not filled in second dump when it should be. So problem is in SvNV() call. Going to look into perl source code when and why should it happen. Which perl version are you using? And can you recheck that NV value in second dump is filled correctly? |
Will double check tomorrow morning CEST, |
I suspect there is bug (or maybe it is feature?) in perl itself. Function sv_2nv_flags() in perl (called by SvNV()) does not upgrade scalar to NV (float) in specific conditions (and float value is lost). This is just observation from reading perl source code. First I need to be able to reproduce this bug and then I can say if problem is in amavis, DBD::mysql, DBI or perl. |
Some more infos (maybe) with a new diff. |
Yes, I expected that those dumps are from problematic float columns. As I wrote yesterday I will try to reproduce that scenario when SvNV() does not upgrade scalar to NV if it is really truth. And then decide next steps... |
I opened tiket in perl bug tracker for this problem: https://rt.perl.org/Public/Bug/Display.html?id=130801 |
Thanks! |
So it is probably problem with understanding perlapi documentation and different behavior for floating point magic scalar seems to be OK... It is just (for me) strange that non-magic integer, magic integer and non-magic float is working, just magic float is problematic... Can you try to apply this patch? It explicitly set float (NV) value via sv_setnv(). And in same case it set also integer values.
|
On 02/23/17 23:50, pali wrote:
So it is probably problem with understanding perlapi documentation and different behavior for floating point magic scalar seems to be OK... It is just (for me) strange that non-magic integer, magic integer and non-magic float is working, just magic float is problematic...
Can you try to apply this patch? It explicitly set float (NV) value via sv_setnv(). And in same case it set also integer values.
|diff --git a/dbdimp.c b/dbdimp.c index 91cc1a8..d484d80 100644 --- a/dbdimp.c +++ b/dbdimp.c @@ -4584,8 +4584,7 @@ process: if (!(fields[i].flags & ZEROFILL_FLAG)) { /* Coerce to dobule and set scalar as NV */ - (void) SvNV(sv); - SvNOK_only(sv); + sv_setnv(sv, SvNV(sv)); } break; @@ -4594,15 +4593,9 @@ process: { /* Coerce to integer and set scalar as UV resp. IV */ if (fields[i].flags & UNSIGNED_FLAG) - { - (void) SvUV(sv); - SvIOK_only_UV(sv); - } + sv_setuv(sv, SvUV(sv)); else - { - (void) SvIV(sv); - SvIOK_only(sv); - } + sv_setiv(sv, SvIV(sv)); } break; |
This fixes the issue, thanks.
Giovanni
|
Calling SvNV() for magical scalar is not enough for float type conversion. It caused problem for Amavis in tainted mode -- all float values were zero. On the other hand SvIV() and SvUV() seems to work fine. To be sure that correct value of float is in scalar use sv_setnv() with explicit NV float value. Similar code is changed also for integers IV/UV. This patch should fix reported Amavis bug: perl5-dbi#78 See also reported perl bug about SvNV(): https://rt.perl.org/Public/Bug/Display.html?id=130801
Calling SvNV() for magical scalar is not enough for float type conversion. It caused problem for Amavis in tainted mode -- all float values were zero. On the other hand SvIV() and SvUV() seems to work fine. To be sure that correct value of float is in scalar use sv_setnv() with explicit NV float value. Similar code is changed also for integers IV/UV. This patch should fix reported Amavis bug: perl5-dbi#78 See also reported perl bug about SvNV(): https://rt.perl.org/Public/Bug/Display.html?id=130801
Hello. @pali Thanks for you help with this. Can I please confirm: Does this only cause problems when Perl is in tainted mode? Thanks |
@brianmay Problem with floats seems to be only if DBD::mysql is filling values into magical scalars. Tainted scalars are magical. Scalars are created by DBI (not DBD::mysql) so this probably depends on DBI version... I was not fully able to reproduce this problem even in tainted Perl mode. I just found code path in perl sources which could trigger it. So... in case DBI does not pass magical scalars to DBD::mysql then this problem could not happen. Note that tainted scalars are not the only type of magical scalars. I have no idea if DBI could not pass another magical scalars... So I cannot answer to your question. What I can say is that @bigio already tested my change and confirmed that it fixes this problem. Function sv_setnv() really should store float value into variable. I re-checked not only documentation, but also implementation perl. This is probably the most suspicious problem which I ever seen and I was not able to reproduce it. Now I do not thing that SvNV() behavior would change to guarantee filling NV slot. Maybe just documentation would be extended or fixed for better understanding. For sure current and older perl versions would not be changed, so fix in DBD::mysql is needed. |
…l/amavisd-new and other software that uses float fields and perl in tainted mode. More info on the issue fixed: perl5-dbi/DBD-mysql#78 ok sthen@
@bigio You can update OpenBSD to 4.042 now and the AmavisD problem should be fixed. |
This fix was reverted in 4.043. |
Upstream changes: 2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043) YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041 This version is the same as 4.041 with all its bugs and limitations. In version 4.042 there were some changes to Unicode handling that turned out to be causing issues with existing implementations. While it is possible to argue that the old behaviour was wrong and buggy, lots of applications and scripts were depending on this behaviour so it is NOT a good idea to change this. There were lots of commits since 4.041, we'll add those back bit by bit in a future release, excluding the ones which cause problems. 2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01) * Use Devel::CheckLib 1.09 or newer, fixes perl5-dbi/DBD-mysql#109 * Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali) * Specify bigint as test dependency. 2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042) * Full release to include development releases 4.041_2 and 4.041_1. 2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2) * Statement handle now also has mysql_sock attribute, just as database handle. (by Pali) * Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and Giovanni Bechis, fix by Pali. https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html perl5-dbi/DBD-mysql#78 * Fix UTF8-encoding of table names, field names, warnings and error messages. Reported by Tanabe Yoshinori, fix by Pali. https://rt.cpan.org/Public/Bug/Display.html?id=120141 * Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by Vladimir Marek. perl5-dbi/DBD-mysql#95 * Improve regex for removing database from dsn (pali) https://rt.cpan.org/Public/Bug/Display.html?id=118837 * Locate MySQL libs using Devel::CheckLib (pali) * Support async on Windows (pali) * Fix test suite on range of older and newer MySQL and MariaDB versions (perl5-dbi/DBD-mysql#87) * Fix compilation on MySQL 4.1 (pali) * Do not leak dangling pointer to mysql result (pali) * Fix logic when assigning to variable bind_comment_placeholders (pali) * mysql_fd() still returned file descriptor after closing connection. Reported by Larry Leszczynski, fixed by Pali Roh獺r. (https://rt.cpan.org/Public/Bug/Display.html?id=110983) * Fix parsing configure libs from mysql_config --libs output in Makefile.PL Libraries in mysql_config --libs output can be specified by library name with the -l prefix or by absolute path to library name without any prefix. Parameters must start with a hyphen, so treat all options without leading hyphen in mysql_config --libs output as libraries with full path. Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898 Fix by Pali Roh獺r. * Fix support for magic scalars (pali) (perl5-dbi/DBD-mysql#76) 2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1) * Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4, previous versions of DBD::mysql did not properly encode input statements to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the column charset. Fix by Pali Roh獺r. Reported and feedback on fix by Marc Lehmann (https://rt.cpan.org/Public/Bug/Display.html?id=87428) Also, the UTF-8 flag was not set for decoded data: (https://rt.cpan.org/Public/Bug/Display.html?id=53130) * Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Roh獺r. (https://rt.cpan.org/Public/Bug/Display.html?id=118977)
can we re open this issue plz ? 2 month and the last version of DBD-mysql is still broken |
Calling SvNV() for magical scalar is not enough for float type conversion. It caused problem for Amavis in tainted mode -- all float values were zero. On the other hand SvIV() and SvUV() seems to work fine. To be sure that correct value of float is in scalar use sv_setnv() with explicit NV float value. Similar code is changed also for integers IV/UV. This patch should fix reported Amavis bug: perl5-dbi#78 See also reported perl bug about SvNV(): https://rt.perl.org/Public/Bug/Display.html?id=130801 Bugs: perl5-dbi#78 Bugs-Debian: https://bugs.debian.org/856064
Fedora 27, for which the current shipped version is perl-DBD-MySQL-4.043-6, also still has this bug. amavisd-new SQL lookups of float fields still return 0. |
Calling SvNV() for magical scalar is not enough for float type conversion. It caused problem for Amavis in tainted mode -- all float values were zero. On the other hand SvIV() and SvUV() seems to work fine. To be sure that correct value of float is in scalar use sv_setnv() with explicit NV float value. Similar code is changed also for integers IV/UV. This patch should fix reported Amavis bug: perl5-dbi#78 See also reported perl bug about SvNV(): https://rt.perl.org/Public/Bug/Display.html?id=130801 Bugs: perl5-dbi#78 Bugs-Debian: https://bugs.debian.org/856064
Changes: 2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043) YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041 This version is the same as 4.041 with all its bugs and limitations. In version 4.042 there were some changes to Unicode handling that turned out to be causing issues with existing implementations. While it is possible to argue that the old behaviour was wrong and buggy, lots of applications and scripts were depending on this behaviour so it is NOT a good idea to change this. There were lots of commits since 4.041, we'll add those back bit by bit in a future release, excluding the ones which cause problems. 2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01) * Use Devel::CheckLib 1.09 or newer, fixes perl5-dbi/DBD-mysql#109 * Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali) * Specify bigint as test dependency. 2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042) * Full release to include development releases 4.041_2 and 4.041_1. 2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2) * Statement handle now also has mysql_sock attribute, just as database handle. (by Pali) * Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and Giovanni Bechis, fix by Pali. https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html perl5-dbi/DBD-mysql#78 * Fix UTF8-encoding of table names, field names, warnings and error messages. Reported by Tanabe Yoshinori, fix by Pali. https://rt.cpan.org/Public/Bug/Display.html?id=120141 * Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by Vladimir Marek. perl5-dbi/DBD-mysql#95 * Improve regex for removing database from dsn (pali) https://rt.cpan.org/Public/Bug/Display.html?id=118837 * Locate MySQL libs using Devel::CheckLib (pali) * Support async on Windows (pali) * Fix test suite on range of older and newer MySQL and MariaDB versions (perl5-dbi/DBD-mysql#87) * Fix compilation on MySQL 4.1 (pali) * Do not leak dangling pointer to mysql result (pali) * Fix logic when assigning to variable bind_comment_placeholders (pali) * mysql_fd() still returned file descriptor after closing connection. Reported by Larry Leszczynski, fixed by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=110983) * Fix parsing configure libs from mysql_config --libs output in Makefile.PL Libraries in mysql_config --libs output can be specified by library name with the -l prefix or by absolute path to library name without any prefix. Parameters must start with a hyphen, so treat all options without leading hyphen in mysql_config --libs output as libraries with full path. Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898 Fix by Pali Rohár. * Fix support for magic scalars (pali) (perl5-dbi/DBD-mysql#76) 2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1) * Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4, previous versions of DBD::mysql did not properly encode input statements to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the column charset. Fix by Pali Rohár. Reported and feedback on fix by Marc Lehmann (https://rt.cpan.org/Public/Bug/Display.html?id=87428) Also, the UTF-8 flag was not set for decoded data: (https://rt.cpan.org/Public/Bug/Display.html?id=53130) * Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=118977) 2016-11-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041) * Fix use-after-free for repeated fetchrow_arrayref calls when mysql_server_prepare=1 Function dbd_st_fetch() via Renew() can reallocate output buffer for mysql_stmt_fetch() call. But it does not update pointer to that buffer in imp_sth->stmt structure initialized by mysql_stmt_bind_result() function. That leads to use-after-free in any mysql function which access imp_sth->stmt structure (e.g. mysql_stmt_fetch()). This patch fix this problem and properly updates pointer in imp_sth->stmt structure after Renew() call. This is a medium level security issue to which the Debian security team assigned identifier CVE-2016-1251. Discovered and fixed by Pali Rohár. * auto_reconnect now also matches CR_SERVER_LOST, previously this only matched CR_SERVER_GONE. Fixes http://bugs.mysql.com/bug.php?id=27613 Fix suggested by Wouter de Jong. * Fix compilation fixes (Pali Rohár). 2016-11-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.040) * Since 4.038 we had problems compiling on big-endian architectures, such as MIPS, s390 and Sparc. Thanks to Salvatore Bonaccorso @ Debian project (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844538) and Vladimir Marek (https://rt.cpan.org/Public/Bug/Display.html?id=118835) for reporting the issues. Fix by Pali Rohár. Fix integer types when server side prepare statements are enabled Fixed problems: * SQL_BIGINT was incorrectly handled as 32bit MYSQL_TYPE_LONG type instead 64bit MYSQL_TYPE_LONGLONG which led to integer overflow/underflow * 32bit MYSQL_TYPE_LONG was used for perl's IV storage when IV was 64bit and 64bit MYSQL_TYPE_LONGLONG was used when IV was 32bit * All unsigned types were handled as signed, so all high positive values were treated as negative * Numeric conversions in perl which led to overflow/underflow was ignored even when mysql strict mode was enabled * Test t/41int_min_max.t was running only for normal non-prepared statements * Test t/40server_prepare.t used incorrect SQL type for big (64bit) integers 2016-11-15 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.039) * Fix for security issue Out-of-bounds read by DBD::mysql CVE-2016-1249 (pali) 2016-10-30 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038_01) * Fix compilation of embedded server (pali) (perl5-dbi/DBD-mysql#68) * Fix compilation against libmariadbclient. First version by H.Merijn Brand, improved by Bernt Johnsen @ Oracle. * For efficiency use newSVpvn() instead newSVpv() where possible (pali) * Correctly coerce fetched scalar values when mysql_server_prepare is not used (pali) * Add support for fetching columns of BIT type with mysql_server_prepare = 1 (pali) Fixes https://rt.cpan.org/Public/Bug/Display.html?id=88006 * Use correct format in printf instead of casting variable types (pali) * Include errno.h for MYSQL_ASYNC because it uses errno variable (pali) * Travis: also test on perl 5.22 and 5.24. 2016-10-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038) * Version 4.037_1 had fixes for MySQL 8.0 provided Bernt Johnsen @ Oracle that were not in the Changelogs (perl5-dbi/DBD-mysql#56) * Fixes for compiling against newer libmysqlclient on Windows (kmx) * Fix unit test for 40server_prepare_crash on Windows (pali) * Perl's IV in scalar can store 64bit integer when perl was compiled with 64 bit support (default on 64bit linux with gcc). Use this feature and stores MYSQL_TYPE_LONGLONG as integers instead of strings when possible. (pali, perl5-dbi/DBD-mysql#57) 2016-10-14 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037_01) * Newest versions of libmysqlclient and the MariaDB C connector no longer export the __WIN__ macro. If this macro is not present we would not compile in the poll.h-based async-support. Changed to use the _WIN32 macro instead. Thanks to Sergei Golubchik for suggesting the fix. * Fix from Pali Rohár for use-after-free in prepared statements, changes to bind logic, and added test 40server_prepare_crash. 2016-10-03 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037) * Security release to patch possible buffer overflow in unsafe sprintf with variable length. Reported and fixed by Pali Rohár. This vulnerability is present in all releases at least back to versions 3.0 of the driver, which were released in 2005. The CVE identifier for this vulnerability is CVE-2016-1246. Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Armin Kuster <akuster808@gmail.com>
Changes: 2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043) YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041 This version is the same as 4.041 with all its bugs and limitations. In version 4.042 there were some changes to Unicode handling that turned out to be causing issues with existing implementations. While it is possible to argue that the old behaviour was wrong and buggy, lots of applications and scripts were depending on this behaviour so it is NOT a good idea to change this. There were lots of commits since 4.041, we'll add those back bit by bit in a future release, excluding the ones which cause problems. 2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01) * Use Devel::CheckLib 1.09 or newer, fixes perl5-dbi/DBD-mysql#109 * Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali) * Specify bigint as test dependency. 2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042) * Full release to include development releases 4.041_2 and 4.041_1. 2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2) * Statement handle now also has mysql_sock attribute, just as database handle. (by Pali) * Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and Giovanni Bechis, fix by Pali. https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html perl5-dbi/DBD-mysql#78 * Fix UTF8-encoding of table names, field names, warnings and error messages. Reported by Tanabe Yoshinori, fix by Pali. https://rt.cpan.org/Public/Bug/Display.html?id=120141 * Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by Vladimir Marek. perl5-dbi/DBD-mysql#95 * Improve regex for removing database from dsn (pali) https://rt.cpan.org/Public/Bug/Display.html?id=118837 * Locate MySQL libs using Devel::CheckLib (pali) * Support async on Windows (pali) * Fix test suite on range of older and newer MySQL and MariaDB versions (perl5-dbi/DBD-mysql#87) * Fix compilation on MySQL 4.1 (pali) * Do not leak dangling pointer to mysql result (pali) * Fix logic when assigning to variable bind_comment_placeholders (pali) * mysql_fd() still returned file descriptor after closing connection. Reported by Larry Leszczynski, fixed by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=110983) * Fix parsing configure libs from mysql_config --libs output in Makefile.PL Libraries in mysql_config --libs output can be specified by library name with the -l prefix or by absolute path to library name without any prefix. Parameters must start with a hyphen, so treat all options without leading hyphen in mysql_config --libs output as libraries with full path. Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898 Fix by Pali Rohár. * Fix support for magic scalars (pali) (perl5-dbi/DBD-mysql#76) 2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1) * Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4, previous versions of DBD::mysql did not properly encode input statements to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the column charset. Fix by Pali Rohár. Reported and feedback on fix by Marc Lehmann (https://rt.cpan.org/Public/Bug/Display.html?id=87428) Also, the UTF-8 flag was not set for decoded data: (https://rt.cpan.org/Public/Bug/Display.html?id=53130) * Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=118977) 2016-11-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041) * Fix use-after-free for repeated fetchrow_arrayref calls when mysql_server_prepare=1 Function dbd_st_fetch() via Renew() can reallocate output buffer for mysql_stmt_fetch() call. But it does not update pointer to that buffer in imp_sth->stmt structure initialized by mysql_stmt_bind_result() function. That leads to use-after-free in any mysql function which access imp_sth->stmt structure (e.g. mysql_stmt_fetch()). This patch fix this problem and properly updates pointer in imp_sth->stmt structure after Renew() call. This is a medium level security issue to which the Debian security team assigned identifier CVE-2016-1251. Discovered and fixed by Pali Rohár. * auto_reconnect now also matches CR_SERVER_LOST, previously this only matched CR_SERVER_GONE. Fixes http://bugs.mysql.com/bug.php?id=27613 Fix suggested by Wouter de Jong. * Fix compilation fixes (Pali Rohár). 2016-11-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.040) * Since 4.038 we had problems compiling on big-endian architectures, such as MIPS, s390 and Sparc. Thanks to Salvatore Bonaccorso @ Debian project (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844538) and Vladimir Marek (https://rt.cpan.org/Public/Bug/Display.html?id=118835) for reporting the issues. Fix by Pali Rohár. Fix integer types when server side prepare statements are enabled Fixed problems: * SQL_BIGINT was incorrectly handled as 32bit MYSQL_TYPE_LONG type instead 64bit MYSQL_TYPE_LONGLONG which led to integer overflow/underflow * 32bit MYSQL_TYPE_LONG was used for perl's IV storage when IV was 64bit and 64bit MYSQL_TYPE_LONGLONG was used when IV was 32bit * All unsigned types were handled as signed, so all high positive values were treated as negative * Numeric conversions in perl which led to overflow/underflow was ignored even when mysql strict mode was enabled * Test t/41int_min_max.t was running only for normal non-prepared statements * Test t/40server_prepare.t used incorrect SQL type for big (64bit) integers 2016-11-15 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.039) * Fix for security issue Out-of-bounds read by DBD::mysql CVE-2016-1249 (pali) 2016-10-30 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038_01) * Fix compilation of embedded server (pali) (perl5-dbi/DBD-mysql#68) * Fix compilation against libmariadbclient. First version by H.Merijn Brand, improved by Bernt Johnsen @ Oracle. * For efficiency use newSVpvn() instead newSVpv() where possible (pali) * Correctly coerce fetched scalar values when mysql_server_prepare is not used (pali) * Add support for fetching columns of BIT type with mysql_server_prepare = 1 (pali) Fixes https://rt.cpan.org/Public/Bug/Display.html?id=88006 * Use correct format in printf instead of casting variable types (pali) * Include errno.h for MYSQL_ASYNC because it uses errno variable (pali) * Travis: also test on perl 5.22 and 5.24. 2016-10-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038) * Version 4.037_1 had fixes for MySQL 8.0 provided Bernt Johnsen @ Oracle that were not in the Changelogs (perl5-dbi/DBD-mysql#56) * Fixes for compiling against newer libmysqlclient on Windows (kmx) * Fix unit test for 40server_prepare_crash on Windows (pali) * Perl's IV in scalar can store 64bit integer when perl was compiled with 64 bit support (default on 64bit linux with gcc). Use this feature and stores MYSQL_TYPE_LONGLONG as integers instead of strings when possible. (pali, perl5-dbi/DBD-mysql#57) 2016-10-14 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037_01) * Newest versions of libmysqlclient and the MariaDB C connector no longer export the __WIN__ macro. If this macro is not present we would not compile in the poll.h-based async-support. Changed to use the _WIN32 macro instead. Thanks to Sergei Golubchik for suggesting the fix. * Fix from Pali Rohár for use-after-free in prepared statements, changes to bind logic, and added test 40server_prepare_crash. 2016-10-03 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037) * Security release to patch possible buffer overflow in unsafe sprintf with variable length. Reported and fixed by Pali Rohár. This vulnerability is present in all releases at least back to versions 3.0 of the driver, which were released in 2005. The CVE identifier for this vulnerability is CVE-2016-1246. Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Armin Kuster <akuster808@gmail.com>
I know this is an old thread, but the problem with amavis and DBD-MySQL still persist - At least on FreeBSD. Is there any conclusion ? The perl bug report is marked as solved - |
@dbielefeldt see my #78 (comment) fix was reverted due to issue #117 |
This is still a problem in 4.048, reverting to 4.037 works (4.040 does not) UNSIGNED BIGINT(20) PRIMARY KEY is not returned "correctly" to Amavis, breaking sql_storage |
On 6/13/19 8:34 AM, Tom Sommer wrote:
It still does not work with 4.050 and latest Amavisd - foreign key constraint fails because of BIGINT type-error
@bigio <https://github.com/bigio> How did you solve it?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#78?email_source=notifications&email_token=AATAVVTLVIFSPLKIGXOOI23P2HS75A5CNFSM4C4MVF62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXSVKWI#issuecomment-501568857>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATAVVXLJOHWLRNX6CRE2XDP2HS75ANCNFSM4C4MVF6Q>.
Atm I am using 4.050 + the following patch:
https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/databases/p5-DBD-mysql/patches/patch-dbdimp_c?rev=1.4&content-type=text/x-cvsweb-markup
|
@bigio Do you have an idea of what an amavis-side patch/fix would look like? |
Calling SvNV() for magical scalar is not enough for float type conversion. It caused problem for Amavis in tainted mode -- all float values were zero. On the other hand SvIV() and SvUV() seems to work fine. To be sure that correct value of float is in scalar use sv_setnv() with explicit NV float value. Similar code is changed also for integers IV/UV. Fixes perl5-dbi#78, perl5-dbi#312 Credit kentnl-gentoo@b6b8540
On 6/13/19 10:37 AM, Tom Sommer wrote:
@bigio <https://github.com/bigio> Do you have an idea of what an amavis-side patch would look like?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#78?email_source=notifications&email_token=AATAVVV772AORZKIG7QUU3DP2IBLZA5CNFSM4C4MVF62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXS6XTA#issuecomment-501607372>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATAVVVV2FK4UTUPT3CLHJDP2IBLZANCNFSM4C4MVF6Q>.
I do not think that anything could be done on amavisd side, amavisd is just able to trigger a bug in DBD::Mysql
|
Calling SvNV() for magical scalar is not enough for float type conversion. It caused problem for Amavis in tainted mode -- all float values were zero. On the other hand SvIV() and SvUV() seems to work fine. To be sure that correct value of float is in scalar use sv_setnv() with explicit NV float value. Similar code is changed also for integers IV/UV. Fixes #78, #312 Credit kentnl-gentoo@b6b8540
Changes: 2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043) YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041 This version is the same as 4.041 with all its bugs and limitations. In version 4.042 there were some changes to Unicode handling that turned out to be causing issues with existing implementations. While it is possible to argue that the old behaviour was wrong and buggy, lots of applications and scripts were depending on this behaviour so it is NOT a good idea to change this. There were lots of commits since 4.041, we'll add those back bit by bit in a future release, excluding the ones which cause problems. 2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01) * Use Devel::CheckLib 1.09 or newer, fixes perl5-dbi/DBD-mysql#109 * Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali) * Specify bigint as test dependency. 2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042) * Full release to include development releases 4.041_2 and 4.041_1. 2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2) * Statement handle now also has mysql_sock attribute, just as database handle. (by Pali) * Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and Giovanni Bechis, fix by Pali. https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html perl5-dbi/DBD-mysql#78 * Fix UTF8-encoding of table names, field names, warnings and error messages. Reported by Tanabe Yoshinori, fix by Pali. https://rt.cpan.org/Public/Bug/Display.html?id=120141 * Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by Vladimir Marek. perl5-dbi/DBD-mysql#95 * Improve regex for removing database from dsn (pali) https://rt.cpan.org/Public/Bug/Display.html?id=118837 * Locate MySQL libs using Devel::CheckLib (pali) * Support async on Windows (pali) * Fix test suite on range of older and newer MySQL and MariaDB versions (perl5-dbi/DBD-mysql#87) * Fix compilation on MySQL 4.1 (pali) * Do not leak dangling pointer to mysql result (pali) * Fix logic when assigning to variable bind_comment_placeholders (pali) * mysql_fd() still returned file descriptor after closing connection. Reported by Larry Leszczynski, fixed by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=110983) * Fix parsing configure libs from mysql_config --libs output in Makefile.PL Libraries in mysql_config --libs output can be specified by library name with the -l prefix or by absolute path to library name without any prefix. Parameters must start with a hyphen, so treat all options without leading hyphen in mysql_config --libs output as libraries with full path. Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898 Fix by Pali Rohár. * Fix support for magic scalars (pali) (perl5-dbi/DBD-mysql#76) 2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1) * Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4, previous versions of DBD::mysql did not properly encode input statements to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the column charset. Fix by Pali Rohár. Reported and feedback on fix by Marc Lehmann (https://rt.cpan.org/Public/Bug/Display.html?id=87428) Also, the UTF-8 flag was not set for decoded data: (https://rt.cpan.org/Public/Bug/Display.html?id=53130) * Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=118977) 2016-11-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041) * Fix use-after-free for repeated fetchrow_arrayref calls when mysql_server_prepare=1 Function dbd_st_fetch() via Renew() can reallocate output buffer for mysql_stmt_fetch() call. But it does not update pointer to that buffer in imp_sth->stmt structure initialized by mysql_stmt_bind_result() function. That leads to use-after-free in any mysql function which access imp_sth->stmt structure (e.g. mysql_stmt_fetch()). This patch fix this problem and properly updates pointer in imp_sth->stmt structure after Renew() call. This is a medium level security issue to which the Debian security team assigned identifier CVE-2016-1251. Discovered and fixed by Pali Rohár. * auto_reconnect now also matches CR_SERVER_LOST, previously this only matched CR_SERVER_GONE. Fixes http://bugs.mysql.com/bug.php?id=27613 Fix suggested by Wouter de Jong. * Fix compilation fixes (Pali Rohár). 2016-11-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.040) * Since 4.038 we had problems compiling on big-endian architectures, such as MIPS, s390 and Sparc. Thanks to Salvatore Bonaccorso @ Debian project (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844538) and Vladimir Marek (https://rt.cpan.org/Public/Bug/Display.html?id=118835) for reporting the issues. Fix by Pali Rohár. Fix integer types when server side prepare statements are enabled Fixed problems: * SQL_BIGINT was incorrectly handled as 32bit MYSQL_TYPE_LONG type instead 64bit MYSQL_TYPE_LONGLONG which led to integer overflow/underflow * 32bit MYSQL_TYPE_LONG was used for perl's IV storage when IV was 64bit and 64bit MYSQL_TYPE_LONGLONG was used when IV was 32bit * All unsigned types were handled as signed, so all high positive values were treated as negative * Numeric conversions in perl which led to overflow/underflow was ignored even when mysql strict mode was enabled * Test t/41int_min_max.t was running only for normal non-prepared statements * Test t/40server_prepare.t used incorrect SQL type for big (64bit) integers 2016-11-15 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.039) * Fix for security issue Out-of-bounds read by DBD::mysql CVE-2016-1249 (pali) 2016-10-30 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038_01) * Fix compilation of embedded server (pali) (perl5-dbi/DBD-mysql#68) * Fix compilation against libmariadbclient. First version by H.Merijn Brand, improved by Bernt Johnsen @ Oracle. * For efficiency use newSVpvn() instead newSVpv() where possible (pali) * Correctly coerce fetched scalar values when mysql_server_prepare is not used (pali) * Add support for fetching columns of BIT type with mysql_server_prepare = 1 (pali) Fixes https://rt.cpan.org/Public/Bug/Display.html?id=88006 * Use correct format in printf instead of casting variable types (pali) * Include errno.h for MYSQL_ASYNC because it uses errno variable (pali) * Travis: also test on perl 5.22 and 5.24. 2016-10-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038) * Version 4.037_1 had fixes for MySQL 8.0 provided Bernt Johnsen @ Oracle that were not in the Changelogs (perl5-dbi/DBD-mysql#56) * Fixes for compiling against newer libmysqlclient on Windows (kmx) * Fix unit test for 40server_prepare_crash on Windows (pali) * Perl's IV in scalar can store 64bit integer when perl was compiled with 64 bit support (default on 64bit linux with gcc). Use this feature and stores MYSQL_TYPE_LONGLONG as integers instead of strings when possible. (pali, perl5-dbi/DBD-mysql#57) 2016-10-14 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037_01) * Newest versions of libmysqlclient and the MariaDB C connector no longer export the __WIN__ macro. If this macro is not present we would not compile in the poll.h-based async-support. Changed to use the _WIN32 macro instead. Thanks to Sergei Golubchik for suggesting the fix. * Fix from Pali Rohár for use-after-free in prepared statements, changes to bind logic, and added test 40server_prepare_crash. 2016-10-03 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037) * Security release to patch possible buffer overflow in unsafe sprintf with variable length. Reported and fixed by Pali Rohár. This vulnerability is present in all releases at least back to versions 3.0 of the driver, which were released in 2005. The CVE identifier for this vulnerability is CVE-2016-1246. Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Changes: 2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043) YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041 This version is the same as 4.041 with all its bugs and limitations. In version 4.042 there were some changes to Unicode handling that turned out to be causing issues with existing implementations. While it is possible to argue that the old behaviour was wrong and buggy, lots of applications and scripts were depending on this behaviour so it is NOT a good idea to change this. There were lots of commits since 4.041, we'll add those back bit by bit in a future release, excluding the ones which cause problems. 2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01) * Use Devel::CheckLib 1.09 or newer, fixes perl5-dbi/DBD-mysql#109 * Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali) * Specify bigint as test dependency. 2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042) * Full release to include development releases 4.041_2 and 4.041_1. 2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2) * Statement handle now also has mysql_sock attribute, just as database handle. (by Pali) * Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and Giovanni Bechis, fix by Pali. https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html perl5-dbi/DBD-mysql#78 * Fix UTF8-encoding of table names, field names, warnings and error messages. Reported by Tanabe Yoshinori, fix by Pali. https://rt.cpan.org/Public/Bug/Display.html?id=120141 * Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by Vladimir Marek. perl5-dbi/DBD-mysql#95 * Improve regex for removing database from dsn (pali) https://rt.cpan.org/Public/Bug/Display.html?id=118837 * Locate MySQL libs using Devel::CheckLib (pali) * Support async on Windows (pali) * Fix test suite on range of older and newer MySQL and MariaDB versions (perl5-dbi/DBD-mysql#87) * Fix compilation on MySQL 4.1 (pali) * Do not leak dangling pointer to mysql result (pali) * Fix logic when assigning to variable bind_comment_placeholders (pali) * mysql_fd() still returned file descriptor after closing connection. Reported by Larry Leszczynski, fixed by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=110983) * Fix parsing configure libs from mysql_config --libs output in Makefile.PL Libraries in mysql_config --libs output can be specified by library name with the -l prefix or by absolute path to library name without any prefix. Parameters must start with a hyphen, so treat all options without leading hyphen in mysql_config --libs output as libraries with full path. Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898 Fix by Pali Rohár. * Fix support for magic scalars (pali) (perl5-dbi/DBD-mysql#76) 2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1) * Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4, previous versions of DBD::mysql did not properly encode input statements to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the column charset. Fix by Pali Rohár. Reported and feedback on fix by Marc Lehmann (https://rt.cpan.org/Public/Bug/Display.html?id=87428) Also, the UTF-8 flag was not set for decoded data: (https://rt.cpan.org/Public/Bug/Display.html?id=53130) * Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=118977) 2016-11-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041) * Fix use-after-free for repeated fetchrow_arrayref calls when mysql_server_prepare=1 Function dbd_st_fetch() via Renew() can reallocate output buffer for mysql_stmt_fetch() call. But it does not update pointer to that buffer in imp_sth->stmt structure initialized by mysql_stmt_bind_result() function. That leads to use-after-free in any mysql function which access imp_sth->stmt structure (e.g. mysql_stmt_fetch()). This patch fix this problem and properly updates pointer in imp_sth->stmt structure after Renew() call. This is a medium level security issue to which the Debian security team assigned identifier CVE-2016-1251. Discovered and fixed by Pali Rohár. * auto_reconnect now also matches CR_SERVER_LOST, previously this only matched CR_SERVER_GONE. Fixes http://bugs.mysql.com/bug.php?id=27613 Fix suggested by Wouter de Jong. * Fix compilation fixes (Pali Rohár). 2016-11-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.040) * Since 4.038 we had problems compiling on big-endian architectures, such as MIPS, s390 and Sparc. Thanks to Salvatore Bonaccorso @ Debian project (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844538) and Vladimir Marek (https://rt.cpan.org/Public/Bug/Display.html?id=118835) for reporting the issues. Fix by Pali Rohár. Fix integer types when server side prepare statements are enabled Fixed problems: * SQL_BIGINT was incorrectly handled as 32bit MYSQL_TYPE_LONG type instead 64bit MYSQL_TYPE_LONGLONG which led to integer overflow/underflow * 32bit MYSQL_TYPE_LONG was used for perl's IV storage when IV was 64bit and 64bit MYSQL_TYPE_LONGLONG was used when IV was 32bit * All unsigned types were handled as signed, so all high positive values were treated as negative * Numeric conversions in perl which led to overflow/underflow was ignored even when mysql strict mode was enabled * Test t/41int_min_max.t was running only for normal non-prepared statements * Test t/40server_prepare.t used incorrect SQL type for big (64bit) integers 2016-11-15 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.039) * Fix for security issue Out-of-bounds read by DBD::mysql CVE-2016-1249 (pali) 2016-10-30 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038_01) * Fix compilation of embedded server (pali) (perl5-dbi/DBD-mysql#68) * Fix compilation against libmariadbclient. First version by H.Merijn Brand, improved by Bernt Johnsen @ Oracle. * For efficiency use newSVpvn() instead newSVpv() where possible (pali) * Correctly coerce fetched scalar values when mysql_server_prepare is not used (pali) * Add support for fetching columns of BIT type with mysql_server_prepare = 1 (pali) Fixes https://rt.cpan.org/Public/Bug/Display.html?id=88006 * Use correct format in printf instead of casting variable types (pali) * Include errno.h for MYSQL_ASYNC because it uses errno variable (pali) * Travis: also test on perl 5.22 and 5.24. 2016-10-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038) * Version 4.037_1 had fixes for MySQL 8.0 provided Bernt Johnsen @ Oracle that were not in the Changelogs (perl5-dbi/DBD-mysql#56) * Fixes for compiling against newer libmysqlclient on Windows (kmx) * Fix unit test for 40server_prepare_crash on Windows (pali) * Perl's IV in scalar can store 64bit integer when perl was compiled with 64 bit support (default on 64bit linux with gcc). Use this feature and stores MYSQL_TYPE_LONGLONG as integers instead of strings when possible. (pali, perl5-dbi/DBD-mysql#57) 2016-10-14 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037_01) * Newest versions of libmysqlclient and the MariaDB C connector no longer export the __WIN__ macro. If this macro is not present we would not compile in the poll.h-based async-support. Changed to use the _WIN32 macro instead. Thanks to Sergei Golubchik for suggesting the fix. * Fix from Pali Rohár for use-after-free in prepared statements, changes to bind logic, and added test 40server_prepare_crash. 2016-10-03 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037) * Security release to patch possible buffer overflow in unsafe sprintf with variable length. Reported and fixed by Pali Rohár. This vulnerability is present in all releases at least back to versions 3.0 of the driver, which were released in 2005. The CVE identifier for this vulnerability is CVE-2016-1246. Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Armin Kuster <akuster808@gmail.com>
Changes: 2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043) YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041 This version is the same as 4.041 with all its bugs and limitations. In version 4.042 there were some changes to Unicode handling that turned out to be causing issues with existing implementations. While it is possible to argue that the old behaviour was wrong and buggy, lots of applications and scripts were depending on this behaviour so it is NOT a good idea to change this. There were lots of commits since 4.041, we'll add those back bit by bit in a future release, excluding the ones which cause problems. 2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01) * Use Devel::CheckLib 1.09 or newer, fixes perl5-dbi/DBD-mysql#109 * Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali) * Specify bigint as test dependency. 2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042) * Full release to include development releases 4.041_2 and 4.041_1. 2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2) * Statement handle now also has mysql_sock attribute, just as database handle. (by Pali) * Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and Giovanni Bechis, fix by Pali. https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html perl5-dbi/DBD-mysql#78 * Fix UTF8-encoding of table names, field names, warnings and error messages. Reported by Tanabe Yoshinori, fix by Pali. https://rt.cpan.org/Public/Bug/Display.html?id=120141 * Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by Vladimir Marek. perl5-dbi/DBD-mysql#95 * Improve regex for removing database from dsn (pali) https://rt.cpan.org/Public/Bug/Display.html?id=118837 * Locate MySQL libs using Devel::CheckLib (pali) * Support async on Windows (pali) * Fix test suite on range of older and newer MySQL and MariaDB versions (perl5-dbi/DBD-mysql#87) * Fix compilation on MySQL 4.1 (pali) * Do not leak dangling pointer to mysql result (pali) * Fix logic when assigning to variable bind_comment_placeholders (pali) * mysql_fd() still returned file descriptor after closing connection. Reported by Larry Leszczynski, fixed by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=110983) * Fix parsing configure libs from mysql_config --libs output in Makefile.PL Libraries in mysql_config --libs output can be specified by library name with the -l prefix or by absolute path to library name without any prefix. Parameters must start with a hyphen, so treat all options without leading hyphen in mysql_config --libs output as libraries with full path. Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898 Fix by Pali Rohár. * Fix support for magic scalars (pali) (perl5-dbi/DBD-mysql#76) 2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1) * Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4, previous versions of DBD::mysql did not properly encode input statements to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the column charset. Fix by Pali Rohár. Reported and feedback on fix by Marc Lehmann (https://rt.cpan.org/Public/Bug/Display.html?id=87428) Also, the UTF-8 flag was not set for decoded data: (https://rt.cpan.org/Public/Bug/Display.html?id=53130) * Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Rohár. (https://rt.cpan.org/Public/Bug/Display.html?id=118977) 2016-11-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041) * Fix use-after-free for repeated fetchrow_arrayref calls when mysql_server_prepare=1 Function dbd_st_fetch() via Renew() can reallocate output buffer for mysql_stmt_fetch() call. But it does not update pointer to that buffer in imp_sth->stmt structure initialized by mysql_stmt_bind_result() function. That leads to use-after-free in any mysql function which access imp_sth->stmt structure (e.g. mysql_stmt_fetch()). This patch fix this problem and properly updates pointer in imp_sth->stmt structure after Renew() call. This is a medium level security issue to which the Debian security team assigned identifier CVE-2016-1251. Discovered and fixed by Pali Rohár. * auto_reconnect now also matches CR_SERVER_LOST, previously this only matched CR_SERVER_GONE. Fixes http://bugs.mysql.com/bug.php?id=27613 Fix suggested by Wouter de Jong. * Fix compilation fixes (Pali Rohár). 2016-11-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.040) * Since 4.038 we had problems compiling on big-endian architectures, such as MIPS, s390 and Sparc. Thanks to Salvatore Bonaccorso @ Debian project (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844538) and Vladimir Marek (https://rt.cpan.org/Public/Bug/Display.html?id=118835) for reporting the issues. Fix by Pali Rohár. Fix integer types when server side prepare statements are enabled Fixed problems: * SQL_BIGINT was incorrectly handled as 32bit MYSQL_TYPE_LONG type instead 64bit MYSQL_TYPE_LONGLONG which led to integer overflow/underflow * 32bit MYSQL_TYPE_LONG was used for perl's IV storage when IV was 64bit and 64bit MYSQL_TYPE_LONGLONG was used when IV was 32bit * All unsigned types were handled as signed, so all high positive values were treated as negative * Numeric conversions in perl which led to overflow/underflow was ignored even when mysql strict mode was enabled * Test t/41int_min_max.t was running only for normal non-prepared statements * Test t/40server_prepare.t used incorrect SQL type for big (64bit) integers 2016-11-15 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.039) * Fix for security issue Out-of-bounds read by DBD::mysql CVE-2016-1249 (pali) 2016-10-30 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038_01) * Fix compilation of embedded server (pali) (perl5-dbi/DBD-mysql#68) * Fix compilation against libmariadbclient. First version by H.Merijn Brand, improved by Bernt Johnsen @ Oracle. * For efficiency use newSVpvn() instead newSVpv() where possible (pali) * Correctly coerce fetched scalar values when mysql_server_prepare is not used (pali) * Add support for fetching columns of BIT type with mysql_server_prepare = 1 (pali) Fixes https://rt.cpan.org/Public/Bug/Display.html?id=88006 * Use correct format in printf instead of casting variable types (pali) * Include errno.h for MYSQL_ASYNC because it uses errno variable (pali) * Travis: also test on perl 5.22 and 5.24. 2016-10-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038) * Version 4.037_1 had fixes for MySQL 8.0 provided Bernt Johnsen @ Oracle that were not in the Changelogs (perl5-dbi/DBD-mysql#56) * Fixes for compiling against newer libmysqlclient on Windows (kmx) * Fix unit test for 40server_prepare_crash on Windows (pali) * Perl's IV in scalar can store 64bit integer when perl was compiled with 64 bit support (default on 64bit linux with gcc). Use this feature and stores MYSQL_TYPE_LONGLONG as integers instead of strings when possible. (pali, perl5-dbi/DBD-mysql#57) 2016-10-14 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037_01) * Newest versions of libmysqlclient and the MariaDB C connector no longer export the __WIN__ macro. If this macro is not present we would not compile in the poll.h-based async-support. Changed to use the _WIN32 macro instead. Thanks to Sergei Golubchik for suggesting the fix. * Fix from Pali Rohár for use-after-free in prepared statements, changes to bind logic, and added test 40server_prepare_crash. 2016-10-03 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037) * Security release to patch possible buffer overflow in unsafe sprintf with variable length. Reported and fixed by Pali Rohár. This vulnerability is present in all releases at least back to versions 3.0 of the driver, which were released in 2005. The CVE identifier for this vulnerability is CVE-2016-1246. Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Armin Kuster <akuster808@gmail.com>
Hi,
after upgrading to DBD::mysql 4.041 a problem appeared in amavisd-new:
https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html
The problem could be related to this commit:
caea0b7
Cheers
Giovanni
The text was updated successfully, but these errors were encountered: