zsh-bench measures performance in a subshell. If you execute zsh in a shell with integration, the newly created shell won't have integration in it. This is why zsh-bench shows the same performance in your tests -- it's literally the same thing that gets measured in both cases.

You can measure the performance of a shell with integration by creating ~/.zshrc with the following content:

# ---------------------------------------------------------------------------------------------
#   Copyright (c) Microsoft Corporation. All rights reserved.
#   Licensed under the MIT License. See License.txt in the project root for license information.
# ---------------------------------------------------------------------------------------------
autoload -Uz add-zsh-hook

#### commented out ####
#
# if [ -f ~/.zshenv ]; then
# 	. ~/.zshenv
# fi
# if [[ -o "login" &&  -f ~/.zprofile ]]; then
# 	. ~/.zprofile
# fi
# if [ -f ~/.zshrc ]; then
# 	. ~/.zshrc
# fi
# unset ZDOTDIR # ensure ~/.zlogout runs as expected
#
#### commented out ####

IN_COMMAND_EXECUTION="1"
LAST_HISTORY_ID=0

prompt_start() {
	printf "\033]633;A\007"
}

prompt_end() {
	printf "\033]633;B\007"
}

update_cwd() {
	printf "\033]633;P;Cwd=%s\007" "$PWD"
}

command_output_start() {
	printf "\033]633;C\007"
}

continuation_start() {
	printf "\033]633;F\007"
}

continuation_end() {
	printf "\033]633;G\007"
}

right_prompt_start() {
	printf "\033]633;H\007"
}

right_prompt_end() {
	printf "\033]633;I\007"
}

command_complete() {
	local HISTORY_ID=$(history | tail -n1 | awk '{print $1;}')
	if [[ "$HISTORY_ID" == "$LAST_HISTORY_ID" ]]; then
		printf "\033]633;D\007"
	else
		printf "\033]633;D;%s\007" "$STATUS"
		LAST_HISTORY_ID=$HISTORY_ID
	fi
	update_cwd
}

update_prompt() {
	PRIOR_PROMPT="$PS1"
	IN_COMMAND_EXECUTION=""
	PS1="%{$(prompt_start)%}$PREFIX$PS1%{$(prompt_end)%}"
	PS2="%{$(continuation_start)%}$PS2%{$(continuation_end)%}"
	if [ -n "$RPROMPT" ]; then
		PRIOR_RPROMPT="$RPROMPT"
		RPROMPT="%{$(right_prompt_start)%}$RPROMPT%{$(right_prompt_end)%}"
	fi
}

precmd() {
	local STATUS="$?"
	if [ -z "${IN_COMMAND_EXECUTION-}" ]; then
		# not in command execution
		command_output_start
	fi

	command_complete "$STATUS"

	# in command execution
	if [ -n "$IN_COMMAND_EXECUTION" ]; then
		# non null
		update_prompt
	fi
}

preexec() {
	PS1="$PRIOR_PROMPT"
	if [ -n "$RPROMPT" ]; then
		RPROMPT="$PRIOR_RPROMPT"
	fi
	IN_COMMAND_EXECUTION="1"
	command_output_start
}
add-zsh-hook precmd precmd
add-zsh-hook preexec preexec

# Show the welcome message
if [ -z "${VSCODE_SHELL_HIDE_WELCOME-}" ]; then
	echo "\033[1;32mShell integration activated\033[0m"
else
	VSCODE_SHELL_HIDE_WELCOME=""
fi

This is https://github.com/microsoft/vscode/blob/f37acd3b99f701caae29ae2f9eece1dd5e1b4c68/src/vs/workbench/contrib/terminal/browser/media/shellIntegration.zsh with one block of code commented out.

Then run zsh-bench as before.

You'll see that shell integration increases a bunch of metrics. I suggest focusing on prompt lag. This is the time it takes between your pressing ENTER and the next prompt appearing on the screen. You should be able to feel the difference without the benchmark. Try pressing ENTER a few times in a normal shell and in one with shell integration. You can press ENTER without any command or with a simple command like true, echo hello, or pwd.

As far as shell integration goes, 1ms is a reasonable target for prompt lag. This is about 10% of the total budget that users are working with (you start noticing prompt lag when it reaches 10ms).

You might want to take a look at zsh integration that I wrote for kitty: https://github.com/kovidgoyal/kitty/blob/master/shell-integration/zsh/kitty-integration. There are several design decisions in there I disagree with but you can still find many useful techniques there. This implements OSC 133: https://iterm2.com/documentation-escape-codes.html (search for "FinalTerm").

investigate shell integration perf #145576

Description

meganroggeopenedon Mar 21, 2022

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

meganrogge
openedon Mar 21, 2022