Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Error: internal error: genRecordFieldAux" - in the "version-1-4" branch #16069

Closed
stefantalpalaru opened this issue Nov 20, 2020 · 8 comments
Closed

Comments

@stefantalpalaru
Copy link
Contributor

setup

https://github.com/status-im/nimbus-eth2/tree/1d7fb2ed0c78debc3f9c49ae962e627d115d55fc

https://github.com/nim-lang/Nim/tree/577b4f795cfb5dcc387dbb5a3751b4c1eec42a66

[amd] 15 Fri Nov 20 14:02:12 |/mnt/sda3/storage/CODE/status/nim-beacon-chain-clean|
stefan$ rm -rf nimcache; PATH="/src/77_DLD/CODE/00_github/Nim/bin:$PATH" make USE_SYSTEM_NIM=1

problem

/mnt/sde1/storage/nim-beacon-chain-clean/vendor/nim-stew/stew/byteutils.nim(86, 6) Error: internal error: genRecordFieldAux
Traceback (most recent call last)
/mnt/sda3/storage/CODE/00_github/Nim/compiler/nim.nim(118) nim
/mnt/sda3/storage/CODE/00_github/Nim/compiler/nim.nim(83) handleCmdLine
/mnt/sda3/storage/CODE/00_github/Nim/compiler/main.nim(245) mainCommand
/mnt/sda3/storage/CODE/00_github/Nim/compiler/main.nim(210) compileToBackend
/mnt/sda3/storage/CODE/00_github/Nim/compiler/main.nim(86) commandCompileToC
/mnt/sda3/storage/CODE/00_github/Nim/compiler/modules.nim(160) compileProject
/mnt/sda3/storage/CODE/00_github/Nim/compiler/modules.nim(96) compileModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/passes.nim(214) processModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/passes.nim(86) processTopLevelStmt
/mnt/sda3/storage/CODE/00_github/Nim/compiler/sem.nim(601) myProcess
/mnt/sda3/storage/CODE/00_github/Nim/compiler/sem.nim(569) semStmtAndGenerateGenerics
/mnt/sda3/storage/CODE/00_github/Nim/compiler/semstmts.nim(2294) semStmt
/mnt/sda3/storage/CODE/00_github/Nim/compiler/semexprs.nim(1026) semExprNoType
/mnt/sda3/storage/CODE/00_github/Nim/compiler/semexprs.nim(2872) semExpr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/importer.nim(217) evalImport
/mnt/sda3/storage/CODE/00_github/Nim/compiler/importer.nim(188) impMod
/mnt/sda3/storage/CODE/00_github/Nim/compiler/importer.nim(158) myImportModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/modules.nim(110) importModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/modules.nim(96) compileModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/passes.nim(208) processModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/passes.nim(86) processTopLevelStmt
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(1939) myProcess
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(984) genProcBody
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2771) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2555) genStmtList
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2771) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2555) genStmtList
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2735) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(781) genCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(778) genAsgnCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(377) genPrefixCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(611) initLocExpr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2665) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(1235) genProc
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(1177) genProcNoForward
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(1043) genProcAux
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(984) genProcBody
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2771) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2544) genStmtList
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2772) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(458) genIf
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2771) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2544) genStmtList
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2772) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(451) genIf
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2800) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1574) genAsgn
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(129) loadInto
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(778) genAsgnCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(383) genPrefixCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(364) genParams
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(288) genArg
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(624) initLocExprSingleUse
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2768) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(898) genCheckedRecordField
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(836) genRecordField
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(799) genRecordFieldAux
/mnt/sda3/storage/CODE/00_github/Nim/compiler/msgs.nim(583) internalErrorImpl
/mnt/sda3/storage/CODE/00_github/Nim/compiler/msgs.nim(539) liMessage
/mnt/sda3/storage/CODE/00_github/Nim/compiler/msgs.nim(406) handleError
/mnt/sda3/storage/CODE/00_github/Nim/compiler/msgs.nim(395) quit
@timotheecour
Copy link
Member

timotheecour commented Nov 20, 2020

@stefantalpalaru please minimize, ideally should not contain any imports since it's a compiler bug.

(and try on latest devel + show its hash)

it's more work for you but less work for everyone else reading this bug :-)

@stefantalpalaru
Copy link
Contributor Author

I don't know if it can be minimised. The problem might appear only when some internal compiler buffer is filled.

This, for example, does not exhibit the bug:

[amd] 17 Fri Nov 20 21:21:49 |/mnt/sda3/storage/CODE/status/nim-beacon-chain-clean/vendor/nim-stew|
stefan$ PATH="/src/77_DLD/CODE/00_github/Nim/bin:$PATH" nim --skipParentCfg:on c -r -f tests/test_byteutils.nim

@stefantalpalaru
Copy link
Contributor Author

Here's Nim devel:

[amd] 15 Fri Nov 20 22:03:51 |/mnt/sda3/storage/CODE/status/nim-beacon-chain-clean|
stefan$ PATH="/src/77_DLD/CODE/00_github/Nim/bin:$PATH" nim --version
Nim Compiler Version 1.5.1 [Linux: amd64]
Compiled at 2020-11-20
Copyright (c) 2006-2020 by Andreas Rumpf

git hash: d9038ed792b923cfbb9593ab82825fdb48728adc
active boot switches:
[amd] 15 Fri Nov 20 22:06:26 |/mnt/sda3/storage/CODE/status/nim-beacon-chain-clean|
stefan$ rm -rf nimcache; PATH="/src/77_DLD/CODE/00_github/Nim/bin:$PATH" make USE_SYSTEM_NIM=1
/mnt/sde1/storage/nim-beacon-chain-clean/vendor/nim-stew/stew/byteutils.nim(86, 6) Error: internal error: genRecordFieldAux
Traceback (most recent call last)
/mnt/sda3/storage/CODE/00_github/Nim/compiler/nim.nim(119) nim
/mnt/sda3/storage/CODE/00_github/Nim/compiler/nim.nim(84) handleCmdLine
/mnt/sda3/storage/CODE/00_github/Nim/compiler/main.nim(240) mainCommand
/mnt/sda3/storage/CODE/00_github/Nim/compiler/main.nim(205) compileToBackend
/mnt/sda3/storage/CODE/00_github/Nim/compiler/main.nim(86) commandCompileToC
/mnt/sda3/storage/CODE/00_github/Nim/compiler/modules.nim(163) compileProject
/mnt/sda3/storage/CODE/00_github/Nim/compiler/modules.nim(91) compileModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/passes.nim(202) processModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/passes.nim(73) processTopLevelStmt
/mnt/sda3/storage/CODE/00_github/Nim/compiler/sem.nim(601) myProcess
/mnt/sda3/storage/CODE/00_github/Nim/compiler/sem.nim(569) semStmtAndGenerateGenerics
/mnt/sda3/storage/CODE/00_github/Nim/compiler/semstmts.nim(2295) semStmt
/mnt/sda3/storage/CODE/00_github/Nim/compiler/semexprs.nim(1028) semExprNoType
/mnt/sda3/storage/CODE/00_github/Nim/compiler/semexprs.nim(2876) semExpr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/importer.nim(217) evalImport
/mnt/sda3/storage/CODE/00_github/Nim/compiler/importer.nim(188) impMod
/mnt/sda3/storage/CODE/00_github/Nim/compiler/importer.nim(158) myImportModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/modules.nim(113) importModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/modules.nim(91) compileModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/passes.nim(196) processModule
/mnt/sda3/storage/CODE/00_github/Nim/compiler/passes.nim(73) processTopLevelStmt
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(1931) myProcess
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(984) genProcBody
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2771) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2555) genStmtList
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2771) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2555) genStmtList
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2735) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(781) genCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(778) genAsgnCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(377) genPrefixCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(611) initLocExpr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2665) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(1235) genProc
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(1177) genProcNoForward
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(1043) genProcAux
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(984) genProcBody
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2771) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2544) genStmtList
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2772) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(458) genIf
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2771) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2544) genStmtList
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1581) genStmts
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2772) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(451) genIf
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2800) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(1574) genAsgn
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgstmts.nim(129) loadInto
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(778) genAsgnCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(383) genPrefixCall
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(364) genParams
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgcalls.nim(288) genArg
/mnt/sda3/storage/CODE/00_github/Nim/compiler/cgen.nim(624) initLocExprSingleUse
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(2768) expr
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(898) genCheckedRecordField
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(836) genRecordField
/mnt/sda3/storage/CODE/00_github/Nim/compiler/ccgexprs.nim(799) genRecordFieldAux
/mnt/sda3/storage/CODE/00_github/Nim/compiler/msgs.nim(584) internalErrorImpl
/mnt/sda3/storage/CODE/00_github/Nim/compiler/msgs.nim(540) liMessage
/mnt/sda3/storage/CODE/00_github/Nim/compiler/msgs.nim(406) handleError
/mnt/sda3/storage/CODE/00_github/Nim/compiler/msgs.nim(395) quit

@timotheecour
Copy link
Member

timotheecour commented Nov 20, 2020

I don't know if it can be minimised.

of course it can, this is how most bugs are approached; but it sometimes takes a bit of effort, whether from you or for the N people who will look at it, but you known your code better than the person looking at it.

Think about it from perspective of the person looking at your bug report, wondering whether running some large test from a repo is gonna potentially create havoc on his system, overwrite some files etc.

Your repo has monster dependencies, and your bug report comes with almost no install/repro instructions.

git submodule update --init --recursive from your repo installs a ton of dependencies

rm -rf nimcache; PATH="/src/77_DLD/CODE/00_github/Nim/bin:$PATH" make USE_SYSTEM_NIM=1

=> at very least this should be a simiplified to a nim command

The problem might appear only when some internal compiler buffer is filled.

I don't think this is the problem:

proc genRecordFieldAux(p: BProc, e: PNode, d, a: var TLoc) =
  initLocExpr(p, e[0], a)
  if e[1].kind != nkSym: internalError(p.config, e.info, "genRecordFieldAux")
  d.inheritLocation(a)
  discard getTypeDesc(p.module, a.t) # fill the record's fields.loc

here are ways to improve this bug report:

import astalgo
...
debug(e)
echo e[1].kind
  • EDIT
    you could've at least written down this:
func fromHex*[N](A: type array[N, byte], hexStr: string): A
                {.raises: [ValueError, Defect], noInit, inline.}=
  ## Read an hex string and store it in a byte array. No "endianness" reordering is done.
  hexToByteArray(hexStr, result)

@stefantalpalaru
Copy link
Contributor Author

Think about it from perspective of the person looking at your bug report, wondering whether running some large test from a repo is gonna potentially create havoc on his system, overwrite some files etc.

Not on my watch. Everything stays inside the project directory.

Your repo has monster dependencies, and your bug report comes with almost no install/repro instructions.

Is the largest Nim codebase such a mystery to you?

git submodule update --init --recursive from your repo installs a ton of dependencies

All confined to the project's "vendor" directory. And do yourself a favour by running make update instead.

you could've at least written down this:

Maybe you missed the part where running the relevant test suite ("tests/test_byteutils.nim") in the "nim-stew" repo does not exhibit the bug? Why would anything smaller than that be buggy?

Anyway, the bug also appears in the "version-1-2" branch, where I had to do the ugliest manual bisection ever, because most commits were failing somewhere else and needed 7bba0de reverted to get past that. Here's the culprit: 9599d95ca

Can you guys manage from here?

@narimiran
Copy link
Member

setup: https://github.com/status-im/nimbus-eth2/tree/1d7fb2ed0c78debc3f9c49ae962e627d115d55fc

I already have nimbus on my machine, so I did the following steps in nimbus directory:

git pull origin master
make update
make USE_SYSTEM_NIM=1 test

And with both Nim 1.4.1 and devel all the test suites and tests run successfully, without any error reported.

@stefantalpalaru
Copy link
Contributor Author

all the test suites and tests

Now run make USE_SYSTEM_NIM=1 nimbus_beacon_chain

@narimiran
Copy link
Member

Upon further investigation, the original https://github.com/status-im/nimbus (which is set up as remote on my machine) now points to https://github.com/status-im/nimbus-eth1, but you have tested with https://github.com/status-im/nimbus-eth2 (notice the eth1 vs eth2 difference).

@Araq Araq closed this as completed in d306a04 Nov 24, 2020
narimiran pushed a commit that referenced this issue Nov 24, 2020
* fixes #16069; refs nim-lang/RFCs#257 [backport:1.2] [backport:1.4]

* make tests green again

(cherry picked from commit d306a04)
ringabout pushed a commit to ringabout/Nim that referenced this issue Nov 25, 2020
* fixes nim-lang#16069; refs nim-lang/RFCs#257 [backport:1.2] [backport:1.4]

* make tests green again
narimiran pushed a commit that referenced this issue Nov 26, 2020
* fixes #16069; refs nim-lang/RFCs#257 [backport:1.2] [backport:1.4]

* make tests green again

(cherry picked from commit d306a04)
mildred pushed a commit to mildred/Nim that referenced this issue Jan 11, 2021
* fixes nim-lang#16069; refs nim-lang/RFCs#257 [backport:1.2] [backport:1.4]

* make tests green again
ardek66 pushed a commit to ardek66/Nim that referenced this issue Mar 26, 2021
* fixes nim-lang#16069; refs nim-lang/RFCs#257 [backport:1.2] [backport:1.4]

* make tests green again
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants