VerifyInstalledTermInfosParse fails for (upcoming) Fedora 38

[tmds]

This test fails as:

System.InvalidOperationException: System.InvalidOperationException : The terminfo database is invalid.
at System.TermInfo.DatabaseFactory.ReadDatabase(String term, String directoryPath) in /home/tester/runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs:line 114
at System.TermInfo.DatabaseFactory.ReadDatabase(String term) in /home/tester/runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs:line 59
at TermInfoTests.VerifyInstalledTermInfosParse() in /home/tester/runtime/src/libraries/System.Console/tests/TermInfo.Unix.cs:line 29
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /home/tester/runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 69

I've extended the test a bit, changes are in #82025.

Then it fails with more information:

  Error Message:
   System.AggregateException : One or more errors occurred. (Exception while verifying '/usr/share/terminfo/x/xterm-direct256') (Exception while verifying '/usr/share/terminfo/x/xterm-16color')
---- System.Exception : Exception while verifying '/usr/share/terminfo/x/xterm-direct256'
-------- System.InvalidOperationException : The terminfo database is invalid.
---- System.Exception : Exception while verifying '/usr/share/terminfo/x/xterm-16color'
-------- System.InvalidOperationException : The terminfo database is invalid.
  Stack Trace:
     at TermInfoTests.VerifyInstalledTermInfosParse() in /root/repos/runtime/src/libraries/System.Console/tests/TermInfo.Unix.cs:line 64
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /root/repos/runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 59
----- Inner Stack Trace #1 (System.Exception) -----

----- Inner Stack Trace -----
   at System.TermInfo.DatabaseFactory.ReadDatabase(String term, String directoryPath) in /root/repos/runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs:line 126
   at TermInfoTests.VerifyInstalledTermInfosParse() in /root/repos/runtime/src/libraries/System.Console/tests/TermInfo.Unix.cs:line 38
----- Inner Stack Trace #2 (System.Exception) -----

----- Inner Stack Trace -----
   at System.TermInfo.DatabaseFactory.ReadDatabase(String term, String directoryPath) in /root/repos/runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs:line 126
   at TermInfoTests.VerifyInstalledTermInfosParse() in /root/repos/runtime/src/libraries/System.Console/tests/TermInfo.Unix.cs:line 38

For bothxterm-16color and xterm-direct256, the test fails at:

runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs

Lines 110 to 115 in be84df7

	const int MaxTermInfoLength = 4096; // according to the term and tic man pages, 4096 is the terminfo file size max
	const int HeaderLength = 12;
	if (termInfoLength <= HeaderLength || termInfoLength > MaxTermInfoLength)
	{
	throw new InvalidOperationException(SR.IO_TermInfoInvalid);
	}

These files exceed the allowed length:

ls -l /usr/share/terminfo/x/xterm-direct256 /usr/share/terminfo/x/xterm-16color
-rw-r--r--. 1 root root 4185 Jan 19 00:00 /usr/share/terminfo/x/xterm-16color
-rw-r--r--. 1 root root 4126 Jan 19 00:00 /usr/share/terminfo/x/xterm-direct256

@adamsitnik can we increase the allowed length, or drop the max length check? Or do you think there is an issue with these files?

cc @omajid

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[tmds]

These are the two files ReadDatabase trips over: terminfos.tar.gz

Tagging subscribers to this area: @dotnet/area-system-console
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This test fails as:

System.InvalidOperationException: System.InvalidOperationException : The terminfo database is invalid.
at System.TermInfo.DatabaseFactory.ReadDatabase(String term, String directoryPath) in /home/tester/runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs:line 114
at System.TermInfo.DatabaseFactory.ReadDatabase(String term) in /home/tester/runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs:line 59
at TermInfoTests.VerifyInstalledTermInfosParse() in /home/tester/runtime/src/libraries/System.Console/tests/TermInfo.Unix.cs:line 29
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /home/tester/runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 69

I've extended the test a bit, changes are in #82025.

Then it fails with more information:

  Error Message:
   System.AggregateException : One or more errors occurred. (Exception while verifying '/usr/share/terminfo/x/xterm-direct256') (Exception while verifying '/usr/share/terminfo/x/xterm-16color')
---- System.Exception : Exception while verifying '/usr/share/terminfo/x/xterm-direct256'
-------- System.InvalidOperationException : The terminfo database is invalid.
---- System.Exception : Exception while verifying '/usr/share/terminfo/x/xterm-16color'
-------- System.InvalidOperationException : The terminfo database is invalid.
  Stack Trace:
     at TermInfoTests.VerifyInstalledTermInfosParse() in /root/repos/runtime/src/libraries/System.Console/tests/TermInfo.Unix.cs:line 64
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /root/repos/runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 59
----- Inner Stack Trace #1 (System.Exception) -----

----- Inner Stack Trace -----
   at System.TermInfo.DatabaseFactory.ReadDatabase(String term, String directoryPath) in /root/repos/runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs:line 126
   at TermInfoTests.VerifyInstalledTermInfosParse() in /root/repos/runtime/src/libraries/System.Console/tests/TermInfo.Unix.cs:line 38
----- Inner Stack Trace #2 (System.Exception) -----

----- Inner Stack Trace -----
   at System.TermInfo.DatabaseFactory.ReadDatabase(String term, String directoryPath) in /root/repos/runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs:line 126
   at TermInfoTests.VerifyInstalledTermInfosParse() in /root/repos/runtime/src/libraries/System.Console/tests/TermInfo.Unix.cs:line 38

For bothxterm-16color and xterm-direct256, the test fails at:

runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs

Lines 110 to 115 in be84df7

	const int MaxTermInfoLength = 4096; // according to the term and tic man pages, 4096 is the terminfo file size max
	const int HeaderLength = 12;
	if (termInfoLength <= HeaderLength || termInfoLength > MaxTermInfoLength)
	{
	throw new InvalidOperationException(SR.IO_TermInfoInvalid);
	}

These files exceed the allowed length:

ls -l /usr/share/terminfo/x/xterm-direct256 /usr/share/terminfo/x/xterm-16color
-rw-r--r--. 1 root root 4185 Jan 19 00:00 /usr/share/terminfo/x/xterm-16color
-rw-r--r--. 1 root root 4126 Jan 19 00:00 /usr/share/terminfo/x/xterm-direct256

@adamsitnik can we increase the allowed length, or drop the max length check? Or do you think there is an issue with these files?

cc @omajid

Author:	tmds
Assignees:	-
Labels:	area-System.Console, untriaged
Milestone:	-

[adamsitnik]

can we increase the allowed length, or drop the max length check? Or do you think there is an issue with these files?

Am I reading this correctly that we would need to increase the max size by roughly 100 bytes (from 4096 to 4185)?

I would prefer to increase the size for now (easier to estimate risk when backporting the fix).

@tmds thank you for pro-actively running the tests on new Fedora! It would be a pity if even a simple .NET Console app could not run on it..

[tmds]

Am I reading this correctly that we would need to increase the max size by roughly 100 bytes (from 4096 to 4185)?

Yes, I guess this limit is meant as a safeguard do avoid taking in overly large files?
Maybe we can double it, 8kiB?

I would prefer to increase the size for now (easier to estimate risk when backporting the fix).

Do you want me include the increase in #82025 (assuming you are interested in these changes), or make a separate PR?

[tmds]

runtime/src/libraries/System.Console/src/System/TermInfo.DatabaseFactory.cs

Line 110 in be84df7

const int MaxTermInfoLength = 4096; // according to the term and tic man pages, 4096 is the terminfo file size max

After taking a look at the man pages, and the ncurses sources, I think 4KiB is the max size for a compiled entry, and not for the complete file.

[adamsitnik]

Maybe we can double it, 8kiB?

Good idea 👍

Do you want me include the increase in #82025 (assuming you are interested in these changes), or make a separate PR?

A separate PR please.