Skip to content

segfault related to TemplateHaskell #36

Open
Open
segfault related to TemplateHaskell#36

Description

@gelisam
gelisam
opened on Apr 29, 2019

Building the following code (e.g. with stack build) and then running haskell-code-explorer results in a segfault (yes, a segfault in Haskell, yikes!):

module Lib where

import Prelude

import qualified Data.Yaml as Yaml
import Language.Haskell.TH (Exp, Q, runIO)
import Language.Haskell.TH.Syntax (addDependentFile)

migrations :: ()
migrations = $(do
  _ <- runIO (Yaml.decodeFileEither "file.txt" :: IO (Either Yaml.ParseException Yaml.Value))
  [|()|])

The above snippet is of course silly, but I encountered the bug in a real project and the above is the result of deleting as much as possible from that project in order to create a minimal repro case. I have put the code in a git repository for your convenience: https://github.com/gelisam/haskell-code-explorer-bug

Note that since this is TemplateHaskell code, the Yaml.decodeFileEither "file.txt" is executed at compile time. Building the project succeeds, so it doesn't seem to be a bug in the yaml package. However, replacing Yaml.decodeFileEither "file.txt" with the simpler readFile "file.txt" does not trigger the bug, so there must be some weird interaction between the yaml library and haskell-code-explorer. Since the symptom is a segfault, I suspect it has to do with the libyaml C library it uses under the hood.

Activity

  1. alexwl commented on Apr 29, 2019

    @alexwl
    alexwl
    on Apr 29, 2019
    Owner

    Interesting,

    There was a bug in yaml library that caused a segfault (snoyberg/yaml#5), but that bug has been fixed since version 0.4.1.2.

    I successfully indexed the package from https://github.com/gelisam/haskell-code-explorer-bug using haskell-code-indexer built with ghc-8.6.3.

    I also tried:
    -haskell-code-indexer built with ghc-8.6.4 (changed resolver in stack.yaml to lts-13.12): no errors
    -haskell-code-indexer built with ghc-8.4.4 (changed resolver in stack.yaml to lts-12.16): no errors
    -haskell-code-indexer built with ghc-8.2.2 (changed resolver in stack.yaml to lts-11.3): got "unhandled ELF relocation(RelA) type 42" (it's a known GHC bug that has already been fixed).

    It doesn't matter whether file.txt exists or not, still no segfault.

    At the moment it is not clear how to reproduce the bug.

    Can you post the full output of haskell-code-indexer -v?

  2. gelisam commented on Apr 29, 2019

    @gelisam
    gelisam
    on Apr 29, 2019
    Author

    Oh! I forgot to mention something important: we tried it on two other macOS machines, and the segfault only occurs on Mojave.

  3. alexwl commented on Apr 29, 2019

    @alexwl
    alexwl
    on Apr 29, 2019
    Owner

    It may be related to https://gitlab.haskell.org/ghc/ghc/issues/16117 (Random segfaults at runtime on macOS Mojave caused by GHC 8.6.3 linker misaligning sections)

  4. gelisam commented on Apr 30, 2019

    @gelisam
    gelisam
    on Apr 30, 2019
    Author

    Huh, that looks pretty nasty. Since you cannot reproduce it, I guess I'll just have to wait until that bug is fixed and then check if the segfault still occurs. Thanks!

  5. gelisam commented on Apr 30, 2019

    @gelisam
    gelisam
    on Apr 30, 2019
    Author

    Still, here is the full output of haskell-code-indexer -v:

    $ stack install
haskell-code-explorer-bug-0.0.0: configure (lib)
Configuring haskell-code-explorer-bug-0.0.0...
haskell-code-explorer-bug-0.0.0: build (lib)
Preprocessing library for haskell-code-explorer-bug-0.0.0..
Building library for haskell-code-explorer-bug-0.0.0..
[1 of 2] Compiling Lib              ( src/Lib.hs, .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/Lib.o )
[2 of 2] Compiling Paths_haskell_code_explorer_bug ( .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/Paths_haskell_code_explorer_bug.hs, .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/Paths_haskell_code_explorer_bug.o )
haskell-code-explorer-bug-0.0.0: copy/register
Installing library in /Users/gelisam/working/haskell/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/lib/x86_64-osx-ghc-8.6.3/haskell-code-explorer-bug-0.0.0-LClv1IjW8llBci7bWj9Oxx
Registering library for haskell-code-explorer-bug-0.0.0..
$ haskell-code-indexer -v
2019-04-30 10:26:51.834241 EDT : [info] haskell-code-indexer version 0.1.0.0, GHC version 8.6.3.0
2019-04-30 10:26:51.83692 EDT : [debug] IndexerConfig {configPackageDirectoryPath = ".", configPackageDistDirRelativePath = Nothing, configOutputDirectoryName = Nothing, configLog = StdOut, configMinLogLevel = LevelDebug, configSourceCodePreprocessing = AfterPreprocessing, configCompression = Gzip, configGhcOptions = [], configIgnoreDirectories = []}
2019-04-30 10:26:51.839883 EDT : [info] Found stack.yaml. Executing "stack path --dist-dir" to get dist directory.
2019-04-30 10:26:52.050873 EDT : [info] Stack dist directory : .stack-work/dist/x86_64-osx/Cabal-2.4.0.1
2019-04-30 10:26:53.328096 EDT : [info] Indexing haskell-code-explorer-bug-0.0.0
2019-04-30 10:26:53.335399 EDT : [debug] Component id : setup
2019-04-30 10:26:53.335457 EDT : [debug] Modules : []
2019-04-30 10:26:53.33548 EDT : [debug] GHC command line options : 
2019-04-30 10:26:53.684626 EDT : [debug] Component id : lib
2019-04-30 10:26:53.684683 EDT : [debug] Modules : ["Lib","Paths_haskell_code_explorer_bug"]
2019-04-30 10:26:53.684704 EDT : [debug] GHC command line options : -fbuilding-cabal-package -O -outputdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -odir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -hidir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -stubdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -i -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -isrc -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -optP-include -optP.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/cabal_macros.h -this-unit-id haskell-code-explorer-bug-0.0.0-LClv1IjW8llBci7bWj9Oxx -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /Users/gelisam/.stack/snapshots/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-db /Users/gelisam/working/haskell/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-id base-4.12.0.0 -package-id template-haskell-2.14.0.0 -package-id yaml-0.11.0.0-GAWf3vpOhGMK7o9d6zMyJu -XHaskell2010 -XTemplateHaskell Lib Paths_haskell_code_explorer_bug
Segmentation fault: 11

  6. alexwl commented on Apr 30, 2019

    @alexwl
    alexwl
    on Apr 30, 2019
    Owner

    Thanks!

    I tried indexing on macOS Mojave (10.14) and also got "Segmentation fault: 11" with both GHG 8.6.3 and GHC 8.6.4.

    Now I can investigate this issue.

    There is a bug in persistent library that might have the same cause: yesodweb/persistent#830 (Segmentation fault: 11 on macOS).

    I'll try to reproduce the segfault without the involvement of haskell-code-indexer.

  7. alexwl commented on May 3, 2019

    @alexwl
    alexwl
    on May 3, 2019
    Owner

    I've managed to reproduce the segfault on macOS Mojave using only GHC API.

    This code segfaults with "Segmentation fault: 11" message:

    import Control.Monad.IO.Class
import qualified Data.List as L
import GHC
import Packages
import DynFlags
import GHC.Paths (libdir)
import Digraph (flattenSCCs)

ghcFlags = "-fbuilding-cabal-package -O -outputdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -odir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -hidir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -stubdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -i -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -isrc -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -optP-include -optP.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/cabal_macros.h -this-unit-id haskell-code-explorer-bug-0.0.0-34p2k9iR4eu61IAyrOu1BO -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /Users/alexwl/.stack/snapshots/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-db /Users/alexwl/Documents/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-id base-4.12.0.0 -package-id yaml-0.11.0.0-GAWf3vpOhGMK7o9d6zMyJu -package-id template-haskell-2.14.0.0 -XHaskell2010 -XTemplateHaskell"

main = do
  runGhcT (Just libdir) $ do
    flags <- getSessionDynFlags
    (flags', _, _) <-
      parseDynamicFlagsCmdLine flags (map noLoc $ words ghcFlags)
    (flags'', _) <- liftIO $ initPackages flags'
    setSessionDynFlags $
      L.foldl'
        gopt_set
        (flags''
           {hscTarget = HscAsm, ghcLink = LinkInMemory, ghcMode = CompManager})
        [Opt_Haddock]
    let modules = ["Lib"] -- file from  https://github.com/gelisam/haskell-code-explorer-bug
    targets <- mapM (`guessTarget` Nothing) modules
    setTargets targets
    _ <- load LoadAllTargets
    modGraph <- getModuleGraph
    let topSortMods = flattenSCCs (topSortModuleGraph False modGraph Nothing)
        modSum = head topSortMods
    parsedModule <- parseModule modSum
    typecheckedModule <- typecheckModule parsedModule
    liftIO $ print "ok"

  8. gelisam commented on May 3, 2019

    @gelisam
    gelisam
    on May 3, 2019
    Author

    The LinkInMemory bit reminds me of the https://gitlab.haskell.org/ghc/ghc/issues/16117 bug you linked to earlier, is there an alternative value ghcLink can be given?

  9. alexwl commented on May 3, 2019

    @alexwl
    alexwl
    on May 3, 2019
    Owner

    is there an alternative value ghcLink can be given?

    There are 5 possible values (https://hackage.haskell.org/package/ghc-8.6.4/docs/DynFlags.html#t:GhcLink):
    NoLink Don't link at all
    LinkBinary Link object code into a binary
    LinkInMemory Use the in-memory dynamic linker (works for both bytecode and object code).
    LinkDynLib Link objects into a dynamic lib (DLL on Windows, DSO on ELF platforms)
    LinkStaticLib Link objects into a static lib

    I tried them all but still got the segfault.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions