Question about splitting doc into multiple docs

[Vagab]

As I understand it, passing multiple documents into KeyBERT is faster than iterating over documents while passing only one of them because each word gets embedded only once in the former approach. Does that mean that one should also see some performance benefits from splitting one huge document into multiple documents?
To provide an example, my intuition tells me the second option here should be faster. But from my tests it isn't. Shouldn't it be?

# first
kw_model = KeyBERT()
kw_model.extract_keywords(doc)

# second
kw_model = KeyBERT()

docs = split_the_doc_somehow(doc)
kw_model.extract(docs)

[MaartenGr]

Why do you think it would result in a performance boost? You mention something about embedding once compared to iteration, but how does that relate to splitting documents?

The embedding get's done over the entire vocabulary of the input, so it shouldn't matter whether something is split up into sentences/paragraphs or not.

There is also another process that take some time/compute and that is actually basic cosine similarity step:

KeyBERT/keybert/_model.py

Lines 213 to 255 in 55a9d1e

	# Find keywords
	all_keywords = []
	for index, _ in enumerate(docs):
	try:
	# Select embeddings
	candidate_indices = df[index].nonzero()[1]
	candidates = [words[index] for index in candidate_indices]
	candidate_embeddings = word_embeddings[candidate_indices]
	doc_embedding = doc_embeddings[index].reshape(1, -1)
	# Maximal Marginal Relevance (MMR)
	if use_mmr:
	keywords = mmr(
	doc_embedding,
	candidate_embeddings,
	candidates,
	top_n,
	diversity,
	)
	# Max Sum Distance
	elif use_maxsum:
	keywords = max_sum_distance(
	doc_embedding,
	candidate_embeddings,
	candidates,
	top_n,
	nr_candidates,
	)
	# Cosine-based keyword extraction
	else:
	distances = cosine_similarity(doc_embedding, candidate_embeddings)
	keywords = [
	(candidates[index], round(float(distances[0][index]), 4))
	for index in distances.argsort()[0][-top_n:]
	][::-1]
	all_keywords.append(keywords)
	# Capturing empty keywords
	except ValueError:
	all_keywords.append([])

That process also takes some time and having more documents might actually slow down inference (albeit only a little bit compared to creating embeddings). This is actually only noticable I believe when using something like Model2Vec.