Spyder 6.0 hangs at startup on Linux (proc.communicate never times out)

[Silvio-Re]

Issue Report Checklist

• Searched the issues page for similar reports
  • maybe related to :
    • Above 5.4.3, Spyder launch stuck when loading Pythonpath manager #21172
    • PR: Use timeout when getting user environment variables #21237
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

6.0.0 not launching due to proc.communicate never timing out

What steps reproduce the problem?

1. launch spyder in a situation that proc.communicate shoud timeout to proceed

I'm not sure why for my system (ubuntu VM) stdout, stderr = proc.communicate(... never return

What is the expected output? What do you see instead?

Spyder should launch

Versions

• Spyder version: 6.0.0
• Python version: 3.10.8
• Qt version: 5.15.8
• PyQt version: 5.15.7
• Operating System name/version: kubuntu 22.04

[Silvio-Re]

Dirty fix

In the file /home/silvio/anaconda3/envs/spyder_env_3/lib/python3.10/site-packages/spyder/utils/environ.py, coment out the section l.103 elif os.name == 'posix': ...

Spyder 6.0.0 starts as expected.

[ccordoba12]

Hey @Silvio-Re, thanks for reporting. Could you post the contents of your ~/.bashrc or ~/.zhrc file so that we can try to reproduce this problem on our side?

Otherwise it won't be possible for us to fix it.

[Silvio-Re]

Hi @ccordoba12, thank you as well for maintaining this amazing tool.

Below is the content of the ~/.bashrc file. I have also attempted to start Spyder with an empty .bashrc, but the error persists.

# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth

# append to the history file, don't overwrite it
shopt -s histappend

# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=1000
HISTFILESIZE=2000

# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize

# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
#shopt -s globstar

# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then
    debian_chroot=$(cat /etc/debian_chroot)
fi

# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
    xterm-color|*-256color) color_prompt=yes;;
esac

# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes

if [ -n "$force_color_prompt" ]; then
    if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
	# We have color support; assume it's compliant with Ecma-48
	# (ISO/IEC-6429). (Lack of such support is extremely rare, and such
	# a case would tend to support setf rather than setaf.)
	color_prompt=yes
    else
	color_prompt=
    fi
fi

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
    ;;
*)
    ;;
esac

# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
    test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
    alias ls='ls --color=auto'
    #alias dir='dir --color=auto'
    #alias vdir='vdir --color=auto'

    alias grep='grep --color=auto'
    alias fgrep='fgrep --color=auto'
    alias egrep='egrep --color=auto'
fi

# colored GCC warnings and errors
#export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'

# some more ls aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'

# Add an "alert" alias for long running commands.  Use like so:
#   sleep 10; alert
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'

# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.

if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if ! shopt -oq posix; then
  if [ -f /usr/share/bash-completion/bash_completion ]; then
    . /usr/share/bash-completion/bash_completion
  elif [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
  fi
fi

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/silvio/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/silvio/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/home/silvio/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/home/silvio/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

alias smv='/home/silvio/Documents/resource/fds/bin/smvbin/smokeview'
alias fds='/home/silvio/Documents/resource/fds/bin/bin/fds'

FDSBINDIR=/home/silvio/Documents/resource/fds/bin/bin
export PATH=$FDSBINDIR:$PATH
export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
#  set OMP_NUM_THREADS to max of 4 and "Total Number of Cores" 
#  obtained from running:
#  grep -c processor /proc/cpuinfo
export OMP_NUM_THREADS=3

# Intel runtime environment

impihome=/home/silvio/Documents/resource/fds/bin/bin/INTEL
export FI_PROVIDER_PATH=$impihome/prov
export LD_LIBRARY_PATH=$impihome/lib:$LD_LIBRARY_PATH
export PATH=$impihome/bin:$PATH
export GOPATH=${HOME}/go
export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin

alias gmsafir='conda activate spyder_env_3 && python /media/sf_Documents/CERIB/resources/safir/gmsafir/gmsafir/gmsafir.py'
alias safir='wine /media/sf_Documents/CERIB/resources/safir/SAFIR.exe'
alias safir_2022b2='wine /media/sf_Documents/CERIB/resources/safir/SAFIRCruise.exe'
alias safir_dec2020_cerib='wine /media/sf_Documents/CERIB/resources/safir/SAFIR_dec2020_cerib.exe'

[ccordoba12]

Below is the content of the ~/.bashrc file.

Thanks for sharing your bashrc @Silvio-Re! I don't see anything suspicious in it.

I have also attempted to start Spyder with an empty .bashrc, but the error persists.

This is worrisome. Do you use zsh by any chance?

[Silvio-Re]

This is worrisome. Do you use zsh by any chance?

No, zsh is not installed on my system.

I tried:

    elif os.name == 'posix':
        try:
            proc = run_shell_command('sleep 0.8', env={}, text=True)
            print("start proc communicate")
            stdout, stderr = proc.communicate(
                timeout=3 if running_in_ci() else 0.5
            )
            print("end  proc communicate")
            print(stdout, stderr)
        except Exception as exc:
            print(exc)
            logger.info(exc)

and the output is:

start proc communicate
Command 'sleep 0.8' timed out after 0.5 seconds
start proc communicate
Command 'sleep 0.8' timed out after 0.5 seconds
start proc communicate
Command 'sleep 0.8' timed out after 0.5 seconds
start proc communicate
Command 'sleep 0.8' timed out after 0.5 seconds
start proc communicate
Command 'sleep 0.8' timed out after 0.5 seconds
start proc communicate
Command 'sleep 0.8' timed out after 0.5 seconds

I'm not certain why this function is called six times at startup, but the timeout appears to function as intended with this simple command.

In fact it was simply that the run_shell_command was caled with /home/silvio/.config/spyder-py3/user-env.sh as cmdstr argument.
I belive the corect cmdstr is ./home/silvio/.config/spyder-py3/user-env.sh

I think this does solve the issue.

[Silvio-Re]

Investingating further, there was two problems :

1. I got used to launch spyder with spyder & to free the console. It appears to have the side effect of disrupting the behavior of proc.communicate in get_user_environment_variables and never return in certain conditions (it do return with sleep 8 but not always when executing user-env.sh)
2. When spyder is launched with spyder keeping the console, proc.communicate behave as expected and sometimes 0.5 s is too short to retrieve the env on my machine : Command '/home/silvio/.config/spyder-py3/user-env.sh' timed out after 0.5 seconds, the main window appear but gets stuck like this :

If the timeout in increased to 1, spyder start as expected when launched with spyder but not spyder & where the problen is still the same, proc.communicate never return ans the spashscreen gets stuck at Loading Code Analysis....

What is surprising is

1. With the dirty fix, spyder manage to start, with spyder & and spyder wheras the env is obvously not retrived.
2. When proc.communicate throw timeout, spyder fail to start.

[ccordoba12]

Thanks for the extra info @Silvio-Re! We'll try to address this in 6.0.1 because it's a very serious bug.

[mrclary]

This is a result of #!/usr/local/bin/bash -i in user-env.sh. The interactive -i does not seem to play nice with sending a parent process to the background (&). I can reproduce this on macOS as well.

Ultimately, we should find a less hackish way to get current user environment variables on posix systems. 🤔

[ccordoba12]

I can reproduce this on macOS as well.

What do you mean? Can you reproduce it with 6.0.0? Because this should be fixed in 6.0.1.

Ultimately, we should find a less hackish way to get current user environment variables on posix systems. 🤔

I don't think there is one (at least VSCode seems to be doing the same).

[mrclary]

I can reproduce this on macOS as well.

What do you mean? Can you reproduce it with 6.0.0? Because this should be fixed in 6.0.1.

I mean that I can reproduce it on macOS without #22504. I just reverted that PR locally to see if I could reproduce it.

Ultimately, we should find a less hackish way to get current user environment variables on posix systems. 🤔

I don't think there is one (at least VSCode seems to be doing the same).

I don't know that there is one either; but it's good to know that we're in good company if VSCode is doing the same. Do you have a link to their implementation?

[ccordoba12]

Do you have a link to their implementation?

Nop, but I saw that VSCode leaves an entry in the shell history when getting env vars (just as we did in one of our 6.0 alphas).

[mrclary]

Nop, but I saw that VSCode leaves an entry in the shell history when getting env vars (just as we did in one of our 6.0 alphas).

Does that shell history entry reference a script that can be located and viewed? I'm just curious whether they are doing anything different (better?) than we are.

[ccordoba12]

Does that shell history entry reference a script that can be located and viewed?

I think so. You need to install/start VSCode and then go to a terminal to inspect that command.