Slow for runtime usage

If we want to run the code at runtime, the brute force code like this should not be used:

while n != index:
            new_string = api_generate_sentence(
                result_str, cut=char_per_index, **api_args
            )
            hash = hashlib.sha256(new_string.encode(encoding="utf-8")).hexdigest()
            hash_int = int(hash, 16)
            n = hash_int % base

because the loop is like the "proof of work" mechanism in crypto mining which takes a lot of time (and power).

[HighDoping]

I understand that this code is computationally expensive. A workaround might be to use a smaller base to increase the chance of collisions, or to use a simpler hash function, but this does not solve the underlying problem. Can you suggest a better algorithm?

使用 hash 的话，或许可以预计算生成查找表，但是不定长要求（char_per_index）会导致查找表大小指数上升。

如果 char_per_index 是定死的，且值比较小，则查找表还是可以一试的。

我的新想法是，不用hash这种“硬要求”，而采用其它简单的要求。

这个问题是让ai生成符合定量要求的字符。这个量可以任取。

比如可以 让 ai接一个 或长或短的句子(以标点分隔)，长表示0，短表示1 这种，具体几个字算长可以用户自己设。

更难的，可以让ai生成某种“情绪”的句子，然后解码用 sentiment classification.

其它方案还在想

[HighDoping]

We can have fixed char_per_index, but a LUT would made the text easily detectable. Kinda like BIP39.
The LLMs usually can't count, as shown in strawberry test.
The ratio of punctuation to characters is usually less than 10% in CJK and English, the information density isn't great.
Sentiment classification seems like a good idea, but that might need a matching model for decoding.

well, as long as you are using a local fine-tuned llm, nobody will know what's your LUT look like. The LUT or llm model doesn't have to be public availiable.

We do not need to let LLM count like "how many r in berry", no need to be that specific. We can just let LLM generate "relatively long", or "relatively short".

Well, you have to choose "information density" or "speed". Want it all might be hasty, in steganography.

By the way, in steganography, "information density" is never the thing we pursue. It's usually below 3%.
If it's too high, we will have Steganalysis methods to detect whether there's secrets in the text.

另外，还有一点我觉得值的研究的是，是否对于某种固定的 二进制输入，为了得到指定的 hash, ai 生成的 字符总是 一致的？如果存在，那么就可以破解了。

举个例子。本项目目前算法是基于 llm 生成内容的随机性。即 前提是：对于 一个 hash, llm 必须能生成多种 字符组合来表示。

那么假设我想生成 hash 为 1 的 字符，在 prompt 为 “说来话长xxxx”的情况下，如果 llm每次都生成 "abcdefgh" 才能满足 hash 为1 （假设），那这就可以成为特征。

如果能生成的 变体不是1 个，但是仅为有限多个，且相对于 char_per_index 少几个数量级（如 char_per_index =8 ，即 x的8次方, x 为 字符集大小，如果仅能生成 x种，x的1次方），那么破解依然不难

也就是说，这可能是一个需要验证的问题

为何值得想一下？因为正常情况下，“说来话长”（或任何前缀），符合语境的 前 8 个字 不可能是无限多个（此乃废话，因为任何语言的字符集都是有限的），要符合某 hash 的 组合就会更加被限制。

这甚至是可以现在就去试试验证一下的。给定时间，暴力让llm 生成 hash 为 指定数值的 8个字符，看看它能生成多少种可能。

只要关注最开始的前8个字符就可以。

开了另一个issue 以讨论安全性问题。此issue 用于讨论速度问题

[HighDoping]

LLM outputs possibilities of tokens within the vocab_size (151936 for Qwen2.5-0.5B), then use top_k or top_p to filter the list, and choose a token to output. It's very likely that with a large top_k value, there is enough tokens to choose from that matches the hash. Though the output will likely to be gibberish.
You may check the implementation of fast_generate_text in 'nativellm.py' in develop branch.

LLM-Steganography/llm_steganography/nativellm.py

Lines 146 to 213 in 55c5143

	def fast_generate_text(
	prompt: str,
	index_list: list[int],
	model="Qwen/Qwen2.5-0.5B",
	top_k=500,
	temperature=1.0,
	base=16,
	retry_limit=100,
	**kwargs,
	) -> str:
	"""
	Fast text generation, only for 1 character per index.
	Does not support char_per_index parameter.
	"""
	device = select_device()
	tokenizer = AutoTokenizer.from_pretrained(model)
	model = AutoModelForCausalLM.from_pretrained(model)
	model.to(device)
	model.eval()
	generated_text = prompt
	original_top_k = top_k
	for index in tqdm(index_list, desc="Native Fast Encoding", leave=False):
	def process_token(token_index):
	token_text = tokenizer.decode(token_index, skip_special_tokens=True)[:1]
	hash_int = int(hashlib.sha256(token_text.encode()).hexdigest(), 16)
	return token_text if hash_int % base == index else None
	for retry in range(retry_limit):
	clear_cache()
	# Generate multiple token sequences
	input_ids = tokenizer.encode(generated_text, return_tensors="pt").to(device)
	with torch.no_grad():
	# Perform forward pass
	outputs = model(input_ids=input_ids)
	logits = outputs.logits[:, -1, :] # Extract logits of the last token
	logits = logits / temperature # Apply temperature scaling
	probs = torch.softmax(logits, dim=-1) # Convert logits to probabilities
	# Perform top-k filtering
	top_k_probs, top_k_indices = torch.topk(probs, top_k, dim=-1)
	available_tokens = []
	# Use map to process all tokens and filter out None values
	available_tokens = list(filter(None, map(
	process_token,
	top_k_indices.tolist()[0]
	)))
	if len(available_tokens) == 0:
	if retry == retry_limit - 1:
	raise ValueError(
	f"No valid token found after {retry_limit} retries"
	)
	top_k = top_k * 2
	logging.debug(f"No valid token found, retrying with top_k={top_k}")
	else:
	# randomly select token
	logging.debug(f"Available tokens: {available_tokens}")
	selected_text = random.choice(available_tokens)
	# selected_text = available_tokens[0]
	top_k = original_top_k
	break
	generated_text += selected_text
	logging.info(f"Generated: {generated_text}")
	return generated_text

And test_decode_extreme_256() in 'test_decode.py' for encoded text.

LLM-Steganography/tests/test_decode.py

Lines 32 to 49 in 55c5143

	def test_decode_extreme_256():
	encoded = """说来话长，遇而特丽去武将初事杂下到谢父亲曹超却分到话干的卞祖泉前也喜惧其音前或疑喜还亟令再从谢爹讨前是末节为授刺父吏却道都授内傅谢劝归复都去里处吏特不都弟东讨白遇非说贼董超
	门至扬离武曰尝东里到叔出营忆往晋孝忌白舍燕且往是赏言才曹勃始与尝处过过班班邀功"""
	plaintext = "This is a secret."
	password = "123456"
	decoded = __main__.llm_decode(
	encoded, password=password, base=256, char_per_index=1
	)
	assert decoded == plaintext
	def test_decode_extreme_256_2():
	encoded = """说来话长，特乎不呢到门,其德尤特扬坦巨恢省邑不感岁寒叶没酸渐甚虚自慨白江川分授院院至自特亲旷内看花说 无代虚帝至喜讲通泰笑颜谓寿泰慨自迟光临恭愿允说愿临曰白言慰至舍屋御惟至嘉仪劝橘橘会请"""
	plaintext = "Secret"
	password = "password"
	decoded = __main__.llm_decode(
	encoded, password=password, base=256, char_per_index=1
	)
	assert decoded == plaintext

说来话长，特乎不呢到门,其德尤特扬坦巨恢省邑不感岁寒叶没

这个任何人看就知道有问题吧，这都快赶上 熊曰论道了

这样就不用llm隐写了，用熊曰佛曰得了

说来话长，特乎 这就已经不对了。正常人说话 "特“后面就不能跟着 ”乎“。 （除非你读不懂中文，那样我就 sorry 了）

（我猜你懂中文，所以我才一直用中文和您交流，因为你代码中Prompt 用的 ”说来话长“这四个汉字，再加上您引用的 论坛也是中文的）

[HighDoping]

It's gibberish under extreme settings, each character encodes a byte, so word choice is limited.
I understand Chinese, I use English for better reach.

所以输出是 gibberish 就不行。实际8字符的可能性 依然被大大限制了。不知道您拿 gibberish 举例有什么用途。

[HighDoping]

It's to show the possibility of token choice. In fact, every example in this repo is kind of gibberish and nonsense.

These kind of "massive" amount of token choice is no use when talking about "Natual language steganography".