investigate shell integration perf

[meganrogge]

reported #142004, we'll want to investigate zsh perf with shell integration enabled using zsh bench

[meganrogge]

Initial test shows no significant difference

[romkatv]

zsh-bench measures performance in a subshell. If you execute zsh in a shell with integration, the newly created shell won't have integration in it. This is why zsh-bench shows the same performance in your tests -- it's literally the same thing that gets measured in both cases.

You can measure the performance of a shell with integration by creating ~/.zshrc with the following content:

# ---------------------------------------------------------------------------------------------
#   Copyright (c) Microsoft Corporation. All rights reserved.
#   Licensed under the MIT License. See License.txt in the project root for license information.
# ---------------------------------------------------------------------------------------------
autoload -Uz add-zsh-hook

#### commented out ####
#
# if [ -f ~/.zshenv ]; then
# 	. ~/.zshenv
# fi
# if [[ -o "login" &&  -f ~/.zprofile ]]; then
# 	. ~/.zprofile
# fi
# if [ -f ~/.zshrc ]; then
# 	. ~/.zshrc
# fi
# unset ZDOTDIR # ensure ~/.zlogout runs as expected
#
#### commented out ####

IN_COMMAND_EXECUTION="1"
LAST_HISTORY_ID=0

prompt_start() {
	printf "\033]633;A\007"
}

prompt_end() {
	printf "\033]633;B\007"
}

update_cwd() {
	printf "\033]633;P;Cwd=%s\007" "$PWD"
}

command_output_start() {
	printf "\033]633;C\007"
}

continuation_start() {
	printf "\033]633;F\007"
}

continuation_end() {
	printf "\033]633;G\007"
}

right_prompt_start() {
	printf "\033]633;H\007"
}

right_prompt_end() {
	printf "\033]633;I\007"
}

command_complete() {
	local HISTORY_ID=$(history | tail -n1 | awk '{print $1;}')
	if [[ "$HISTORY_ID" == "$LAST_HISTORY_ID" ]]; then
		printf "\033]633;D\007"
	else
		printf "\033]633;D;%s\007" "$STATUS"
		LAST_HISTORY_ID=$HISTORY_ID
	fi
	update_cwd
}

update_prompt() {
	PRIOR_PROMPT="$PS1"
	IN_COMMAND_EXECUTION=""
	PS1="%{$(prompt_start)%}$PREFIX$PS1%{$(prompt_end)%}"
	PS2="%{$(continuation_start)%}$PS2%{$(continuation_end)%}"
	if [ -n "$RPROMPT" ]; then
		PRIOR_RPROMPT="$RPROMPT"
		RPROMPT="%{$(right_prompt_start)%}$RPROMPT%{$(right_prompt_end)%}"
	fi
}

precmd() {
	local STATUS="$?"
	if [ -z "${IN_COMMAND_EXECUTION-}" ]; then
		# not in command execution
		command_output_start
	fi

	command_complete "$STATUS"

	# in command execution
	if [ -n "$IN_COMMAND_EXECUTION" ]; then
		# non null
		update_prompt
	fi
}

preexec() {
	PS1="$PRIOR_PROMPT"
	if [ -n "$RPROMPT" ]; then
		RPROMPT="$PRIOR_RPROMPT"
	fi
	IN_COMMAND_EXECUTION="1"
	command_output_start
}
add-zsh-hook precmd precmd
add-zsh-hook preexec preexec

# Show the welcome message
if [ -z "${VSCODE_SHELL_HIDE_WELCOME-}" ]; then
	echo "\033[1;32mShell integration activated\033[0m"
else
	VSCODE_SHELL_HIDE_WELCOME=""
fi

This is https://github.com/microsoft/vscode/blob/f37acd3b99f701caae29ae2f9eece1dd5e1b4c68/src/vs/workbench/contrib/terminal/browser/media/shellIntegration.zsh with one block of code commented out.

Then run zsh-bench as before.

You'll see that shell integration increases a bunch of metrics. I suggest focusing on prompt lag. This is the time it takes between your pressing ENTER and the next prompt appearing on the screen. You should be able to feel the difference without the benchmark. Try pressing ENTER a few times in a normal shell and in one with shell integration. You can press ENTER without any command or with a simple command like true, echo hello, or pwd.

As far as shell integration goes, 1ms is a reasonable target for prompt lag. This is about 10% of the total budget that users are working with (you start noticing prompt lag when it reaches 10ms).

You might want to take a look at zsh integration that I wrote for kitty: https://github.com/kovidgoyal/kitty/blob/master/shell-integration/zsh/kitty-integration. There are several design decisions in there I disagree with but you can still find many useful techniques there. This implements OSC 133: https://iterm2.com/documentation-escape-codes.html (search for "FinalTerm").

[meganrogge]

@romkatv I'm only seeing a 2.4ms difference in prompt lag for ITerm without shell integration (and with norcs) vs VS Code with shell integration when I have the shell integration script within my .zshrc file as you suggest above and run all rc files.

[romkatv]

Maybe it changed. I'm afraid I don't have the time to re-run the benchmark. I would consider 2.4ms slow as it's 24% of the whole latency budget that users operate with.

[meganrogge]

Thanks for the investigation into this issue. I spent time looking at the file you indicated above and do not see how we can improve upon / speed up what we have currently for the script, so am closing this.