gc:arc - Illegal storage access

[SFR0815]

Hi!

First of all I have to apologize that I can not actually provide a simple code example causing the issue - the code I am running is a bit too large to be documented here. I have not yet been able to identify where this below mentioned behavior actually originates form.

What I observe is that "nim c -r some_file.nim" does not cause any issue

while "nim c -r --gc:arc some_file.nim" fails with

..\nim-1.4.0\lib\system\alloc.nim(955) alloc ..\nim-1.4.0\lib\system\alloc.nim(787) rawAlloc SIGSEGV: Illegal storage access. (Attempt to read from nil?)
with line of my code mentioned to fail just does some trivial stuff (like: result = result & "x")

[Araq]

with line of my code mentioned to fail just does some trivial stuff (like: result = result & "x")

Valid bug of course but you should really use result.add "x" instead, much faster and more idiomatic code.

[SFR0815]

Thank's for the adivce - but any idea what the valid bug could be related to?

[Araq]

To this very statement (result = result & "x").

[timotheecour]

@SFR0815 there are many things you can do to improve your bug report:

• add echo type(result) and report it
• show full stacktrace
• does it happen without --gc:arc
• check whether same root cause as --gc:arc SIGSEGV with ptr aliasing #13137
• if result is cstring, echo result == nil
• echo cast[int](result[0].addr) => this will help determine if address is from ROM or heap or stack
• also run on devel
• with above info in mind, minimize

without any minimum reproducible example, not much one can do about it

[SFR0815]

I will go item by item. Order will be somewhat different:

No. 2: show full stacktrace

Traceback (most recent call last)
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_requests\list\unit_test\ut_uiapi_list.nim(23) ut_uiapi_list
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_requests\list\uiapi_list.nim(87) getApiBody_SubEntitySelectionList
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_requests\list\uiapi_list.nim(42) getApiBody_NameList
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_requests\list\name_list_api_body.nim(33) getApiBodyWithDbTableData
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\table_data\interfaces\db_data\specific_procs\reporting_date\reporting_period_selector.nim(430) getBusinessYear_OfDate
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\table_data\interfaces\db_data\specific_procs\reporting_date\reporting_period_selector.nim(344) getBusinessYears_OfClient
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\data_item_interface\i_data_item.nim(308) value=
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\date_time_with_tz_item.nim(44) setValue_DateTime_Impl
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\utils\iso8601.nim(398) normalizeDateTime
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\utils\iso8601.nim(394) getDateTime
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\utils\iso8601.nim(382) normalizeTimestamp
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\utils\iso8601.nim(369) getTimestamp
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\utils\iso8601.nim(287) getDate
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\utils\iso8601.nim(261) getDateOfDateString
C:\Users\steph\.choosenim\toolchains\nim-#devel\lib\system\alloc.nim(958) alloc0
C:\Users\steph\.choosenim\toolchains\nim-#devel\lib\system\alloc.nim(955) alloc
C:\Users\steph\.choosenim\toolchains\nim-#devel\lib\system\alloc.nim(787) rawAlloc
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

[SFR0815]

No. 3: does it happen without --gc:arc

No. Without --gc:arc everything is fine.

[timotheecour]

from the stacktrace + code in alloc.nim please also re-run your code with -d:useSysAssert and show stacktrace+errors

[SFR0815]

nim c -r --gc:arc -d:useSysAssert ut_uiapi_list.nim

gives:

[SYSASSERT] rawAlloc: requested size too small

[timotheecour]

even with --stacktrace:on ?

please put yourself in position of someone looking at your bug report; it's really hard to do anything about it without any context or minimized reproducible example.

what is the code of getDateOfDateString? or ideally, what's the content of iso8601.nim (eg put in some gist https://gist.github.com/)

[SFR0815]

I fully understand that without a code example it's a mess. No clue on how I can help on this.

Please note the following:
This iso8601 module converts a variety of iso8601-type DateTime formats I am receiving form other sources into some standard that I am using throughout my code - it's procs are called in tons of occasions. I does under near-to-all circumstances work smoothly - even with --gc:arc. Thus, I have no clue on where to start isolating the issue ...

No. 1: add echo type(result) and report it

Just in case it is of help here comes the code of my proc that produces the issue:

Line 261 is " day = day & $character"
type of 'day' is string, see below

proc getDateOfDateString(date_string: string, spec_string: string): DateSpec =
  let spec_string = c_date_spec_string
  var year: string
  var month: string 
  var day: string 
  for char_position in countup(date_string.low, date_string.high):
    let character = date_string[char_position]
    let position_spec_characater = spec_string[char_position]
    case position_spec_characater
    of c_year_pointer:
      if not character.isDigit(): 
        raise newException(Iso8601Error,
          "Some part of year specification is not numeric. \n" &
          "   - non-numeric part   : " & $character & "\n" & 
          "   - on position        : " & $char_position & "\n" & 
          "   - in time zone string: " & date_string)
      year = year & $character
    of c_month_pointer:
      if not character.isDigit(): 
        raise newException(Iso8601Error,
          "Some part of month specification is not numeric. \n" &
          "   - non-numeric part   : " & $character & "\n" & 
          "   - on position        : " & $char_position & "\n" & 
          "   - in time zone string: " & date_string)
      month = month & $character
    of c_day_pointer:
      if not character.isDigit(): 
        raise newException(Iso8601Error,
          "Some part of day specification is not numeric. \n" &
          "   - non-numeric part   : " & $character & "\n" & 
          "   - on position        : " & $char_position & "\n" & 
          "   - in time zone string: " & date_string)
      #echo "day: " & day & " - character: " & $character
      day = day & $character
    of c_date_divisor: discard
    else: 
      raise newException(Iso8601Error, 
        "Irregular content specification on date specification: \n" &
        "   - character                 : " & $position_spec_characater & "\n" & 
        "   - on position               : " & $char_position & "\n" & 
        "   - in time zone specification: " & spec_string)
  if year.len() == 0: year = "1950"
  if month.len() == 0: month = "1"
  if day.len() == 0: day = "1"
  return DateSpec(
            year: year.parseInt().int16,
            month: month.parseInt().int16,
            day: day.parseInt().int16,
            date_string: date_string,
            spec_string: spec_string)
proc getDefaultDate(): DateSpec =
  return DateSpec(year: 1950, month: 1, day: 1)
proc getDate(date_string: string): DateSpec =
  let spec_string = c_date_spec_string
  if spec_string.len() != date_string.len():
    raise newException(Iso8601Error,
      "Date string provided is not compliant with ISO8601. \n" & 
      "     date string provided: " & date_string & " - of length " & $date_string.len() & "\n" & 
      "     spec string provided: " & spec_string & " - of length " & $spec_string.len() & "\n")
  return date_string.getDateOfDateString(spec_string)

I just introduce " echo "typedesc of day: " & $typedesc(day)" before line 261
... and run "nim c -r --gc:arc ut_uiapi_list.nim" again
... it prints tons of "typedesc of day: string"

... here comes the funny peace:
... usually it breaks at the same place BUT
... I saw it breaking somewhere else as well (in rare cases as it seems - seemingly somewhat difficult to reproduce):

Traceback (most recent call last)
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_requests\list\unit_test\ut_uiapi_list.nim(33) ut_uiapi_list
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_requests\list\uiapi_list.nim(87) getApiBody_SubEntitySelectionList
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_requests\list\uiapi_list.nim(42) getApiBody_NameList
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_requests\list\name_list_api_body.nim(21) getApiBodyWithDbTableData
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_data_standards_2\request\source_db_data.nim(38) getDbData
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\api\uiapi_data_standards_2\request\source_db_data.nim(24) sourceByParentId
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\table_data\interfaces\db_data\base_type.nim(357) loadData_ByParentId
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\table_data\interfaces\db_data\base_type.nim(346) loadData_ByEntityId
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\table_data\interfaces\db_data\base_type.nim(317) loadData
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\table_data\interfaces\table_data\base_type.nim(274) loadData
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\db_read.nim(26) loadDataSets
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_statements\main.nim(86) getQueryJsonSql
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_statements\main.nim(50) getQuerySqlNode
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_statements\main.nim(29) getPreSelectFromNode
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_statements\main.nim(55) getQuerySqlNode
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_conditions\interfaces\column_condition\base_type.nim(400) getColumnConditions_SqlNode
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_conditions\interfaces\column_condition\base_type.nim(286) getLevelConditionBlock
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_conditions\interfaces\column_condition\base_type.nim(261) getLevelConditionPartitions
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_conditions\interfaces\column_condition\base_type.nim(73) getCopy
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_conditions\interfaces\column_condition\base_type.nim(70) getCopy
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_conditions\interfaces\column_condition\base_type.nim(65) getCopy
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_conditions\interfaces\column_condition\column_condition_value.nim(53) getCopy
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_io\read\sql_conditions\interfaces\column_condition\column_condition_value.nim(49) getCopy
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\data_item.nim(108) getCopy
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\data_item_interface\i_data_item.nim(171) getValue_String
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\date_time_with_tz_item.nim(70) getValue_String_Impl
D:\Documents\Programs\Projects\InvReporting\Prototype II\Progamming\invrep\src\db_model_2\db_table\data_set\item\implementations\utils\iso8601.nim(387) getTimestamp
C:\Users\steph\.choosenim\toolchains\nim-#devel\lib\pure\times.nim(1578) initTimeFormat
C:\Users\steph\.choosenim\toolchains\nim-#devel\lib\system\strmantle.nim(283) nimCharToStr
C:\Users\steph\.choosenim\toolchains\nim-#devel\lib\system\alloc.nim(958) alloc0
C:\Users\steph\.choosenim\toolchains\nim-#devel\lib\system\alloc.nim(955) alloc
C:\Users\steph\.choosenim\toolchains\nim-#devel\lib\system\alloc.nim(787) rawAlloc
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

Will try my luck on the gist

[SFR0815]

https://gist.github.com/SFR0815/eedd5b5e1f88daf6d5bebc75e37b4d3f

[SFR0815]

--stacktrace:on makes no difference

[SFR0815]

No. 4: check whether same root cause as #13137
Could be. Quite a number of ref objects flying around.
BUT: (see comments above) Somewhat hard to believe that it is the code mentioned in the stacktrace ... (mentioned code usually works fine ...)

[SFR0815]

No. 5: if result is cstring, echo result == nil
Variable is of string type.