Token counting logic

[RodolfoCastanheira]

copilot-api/src/lib/tokenizer.ts

Lines 5 to 18 in 3b673be

	export const getTokenCount = (messages: Array<Message>) => {
	const input = messages.filter(
	(m) => m.role !== "assistant" && typeof m.content === "string",
	)
	const output = messages.filter((m) => m.role === "assistant")
	const inputTokens = countTokens(input)
	const outputTokens = countTokens(output)
	return {
	input: inputTokens,
	output: outputTokens,
	}
	}

I may be misreading, but are you counting assistant messages as output tokens and other roles as input tokens, is that it?

If so, the correct should be:.

• Everything sent to the api is input
• Everything sent by the api is output

If I'm misunderstanding the code, please just ignore. And thanks for this project!

[ericc-ch]

You're welcome! Yeah I just realized you're maybe correct here, because we can send assistant message to the API

So after a completion, only the last assistant message is counted as output token. Am I understanding this correctly?

[RodolfoCastanheira]

yes, exactly!

I'll try to make a PR, if I can find some time in the next few days.