[Feature request] Bing OCR

[bropines]

Bing has an analog of GOOGLE LENS that works well in Latin languages. I wrote a simple parser (but it could be outdated) that works with Bing. I wrote it together with Gemini, so there's a lot of neurocode. I think it will be more beautiful and efficient in your paws. It describes approximate methods of working with their api, which I found

https://gist.githubusercontent.com/bropines/4c303a8232a6f12f54c57147ccab83b1/raw/b746af24d90e35217c67559ba512492087115b55/bing.py

[bropines]

And this is how I found it. (sorry yandex disk, file wouldn't load)

https://disk.yandex.ru/d/rzcTmELCwk1zpg

[AuroraWright]

Thanks implemented in 6dfde19 !

[bropines]

Bing has updated something and stopped working. I have a problem with the request, I suppose that your method of making requests returns 200 instead of 302.

[AuroraWright]

Bing has updated something and stopped working. I have a problem with the request, I suppose that your method of making requests returns 200 instead of 302.

It still works here, ig gradual rollout? Note I changed it a bit after the LunaTranslator dev pointed out a fix for China, so maybe they just ported over the China changes worldwide?

[bropines]

Bing has updated something and stopped working. I have a problem with the request, I suppose that your method of making requests returns 200 instead of 302.

It still works here, ig gradual rollout? Note I changed it a bit after the LunaTranslator dev pointed out a fix for China, so maybe they just ported over the China changes worldwide?

Enter the path to the image file (or just Enter to exit): d:\bropi\Documents\ShareX\Screenshots\2025-04\msedge_HkFe2WjLwu.png
2025-04-07 18:32:24 - __main__ - INFO - 
Starting the recognition process for: d:\bropi\Documents\ShareX\Screenshots\2025-04\msedge_HkFe2WjLwu.png
2025-04-07 18:32:24 - __main__ - INFO - Initializing Bing Image OCR session...
2025-04-07 18:32:24 - __main__ - INFO - Bing Image OCR ready.
2025-04-07 18:32:24 - __main__ - INFO - Image info collected: 218x87, 2905 bytes, format 'png', base64 length 3876
2025-04-07 18:32:24 - __main__ - INFO - Uploading image data to Bing...
2025-04-07 18:33:07 - __main__ - INFO - Upload request finished in 42.34 seconds. Status: 302
2025-04-07 18:33:07 - __main__ - INFO - Redirect URL obtained: https://www.bing.com/images/search?view=detailV2&insightsToken=bcid_RGYSYIbbDE4IqxcxoNWLuD9SqbotqVTd...
2025-04-07 18:33:07 - __main__ - INFO - Extracted insightsToken: bcid_RGYSYIbbDE4Iqxc... API Host: www.bing.com
2025-04-07 18:33:07 - __main__ - INFO - Requesting OCR data from Knowledge API: https://www.bing.com/images/api/custom/knowledge
2025-04-07 18:33:08 - __main__ - INFO - OCR API request finished in 0.61 seconds.Status: 200
2025-04-07 18:33:08 - __main__ - INFO - Extracted 2 lines of text
2025-04-07 18:33:08 - __main__ - INFO - OCR process completed successfully.
2025-04-07 18:33:08 - __main__ - INFO - --- Process completed ---

[bropines]

42 seconds per request... And that's what you have, what I have in the script that I sent as an example. In Russia, your script returns 200 response (most likely with the download of some file)

[AuroraWright]

42 seconds per request... And that's what you have, what I have in the script that I sent as an example. In Russia, your script returns 200 response (most likely with the download of some file)

That sucks. I wonder if Edge uses the same api or a separate one like chromium does for the screenshot feature

[bropines]

42 seconds per request... And that's what you have, what I have in the script that I sent as an example. In Russia, your script returns 200 response (most likely with the download of some file)

That sucks. I wonder if Edge uses the same api or a separate one like chromium does for the screenshot feature

surprisingly, the situation there is similar to that with Google. That is, it is done instantly from the browser, but not from Python.

[bropines]

42 seconds per request... And that's what you have, what I have in the script that I sent as an example. In Russia, your script returns 200 response (most likely with the download of some file)

That sucks. I wonder if Edge uses the same api or a separate one like chromium does for the screenshot feature

https://github.com/lexiforest/curl-impersonate

A friend told me that the BING and GOOGLE APIs detect handshakes. Can you see? You can use this tool to fake a request on behalf of the browser

[AuroraWright]

42 seconds per request... And that's what you have, what I have in the script that I sent as an example. In Russia, your script returns 200 response (most likely with the download of some file)

That sucks. I wonder if Edge uses the same api or a separate one like chromium does for the screenshot feature

https://github.com/lexiforest/curl-impersonate

A friend told me that the BING and GOOGLE APIs detect handshakes. Can you see? You can use this tool to fake a request on behalf of the browser

That's interesting, I tried with google and I can't honestly get it to work. eg this still causes a 10s slow response:

import curl_cffi
requests_session = curl_cffi.Session()

url = 'https://lens.google.com/v3/upload'
mp = curl_cffi.CurlMime.from_list(
    [
        {
            "name": "encoded_image",
            "content_type": "image/png",
            "filename": "image.png",
            "data": self._preprocess(img)
        }
    ]
)
cookies = {'SOCS': 'CAESEwgDEgk0ODE3Nzk3MjQaAmVuIAEaBgiA_LyaBg'}

try:
    res = requests_session.post(url, multipart=mp, cookies=cookies, timeout=20, allow_redirects=False, impersonate="chrome")
except requests.exceptions.Timeout:
    return (False, 'Request timeout!')
except requests.exceptions.ConnectionError:
    return (False, 'Connection error!')

if res.status_code != 303:
    return (False, 'Unknown error!')

redirect_url = res.headers.get('Location')
if not redirect_url:
    return (False, 'Error getting redirect URL!')

parsed_url = urlparse(redirect_url)
query_params = parse_qs(parsed_url.query)

if ('vsrid' not in query_params) or ('gsessionid' not in query_params):
    return (False, 'Unknown error!')

try:
    res = requests_session.get(f"https://lens.google.com/qfmetadata?vsrid={query_params['vsrid'][0]}&gsessionid={query_params['gsessionid'][0]}", timeout=20, impersonate="chrome")
except requests.exceptions.Timeout:
    return (False, 'Request timeout!')
except requests.exceptions.ConnectionError:
    return (False, 'Connection error!')

if (len(res.text.splitlines()) != 3):
    return (False, 'Unknown error!')

Adding referer/origin/etc doesn't help either:

headers = {
    'Host': 'lens.google.com',
    'Referer': 'https://www.google.com/',
    'Origin': 'https://www.google.com',
    'Alt-Used': 'lens.google.com',
}
cookies = {'SOCS': 'CAESEwgDEgk0ODE3Nzk3MjQaAmVuIAEaBgiA_LyaBg'}

try:
    res = requests_session.post(url, multipart=mp, headers=headers, cookies=cookies, timeout=20, allow_redirects=False, impersonate="chrome")

[bropines]

Hmmmm. Interesting. By the way, I suggest opening this ISSUE until I can fix bing properly. At least we will share ideas here...

[bropines]

It's just too weird that browsers work fine, curl also worked fine for me, but for some reason, python returns 200 code...

What if the requests themselves sign the handshake and I can't make the correct request because of this... Stupid but an idea

UPD
And now it works -_-... Maybe Microsoft's servers just broke down then (I reinstalled Windows, so only now I was able to check again)

[AuroraWright]

It's just too weird that browsers work fine, curl also worked fine for me, but for some reason, python returns 200 code...

What if the requests themselves sign the handshake and I can't make the correct request because of this... Stupid but an idea

UPD And now it works -_-... Maybe Microsoft's servers just broke down then (I reinstalled Windows, so only now I was able to check again)

Bing always worked fine for me and fast too, i also checked the requests made by the screenshot feature in Edge and sadly it uses the same api (it sends some additional headers but i don’t think it makes a difference). Even with a paid vpn connecting through Russia i can’t reproduce your issues so i’m a bit at a loss.
Maybe your IP was rate limited or something?

[bropines]

Bing always worked fine for me and fast too, i also checked the requests made by the screenshot feature in Edge and sadly it uses the same api (it sends some additional headers but i don’t think it makes a difference). Even with a paid vpn connecting through Russia i can’t reproduce your issues so i’m a bit at a loss.
Maybe your IP was rate limited or something?

I assume that something was wrong with the network. At the very least, I took down the computer to clean up all the garbage that had accumulated over 4 years, and since we have those problems with blockages in Russia, my network was passing through a bunch of garbage... I'll observe for a couple of days, if everything is normal, I'll integrate it into BallonTranslator...

I'm also looking for new OCR methods for you... As an idea, there is such a thing

dmMaze/BallonsTranslator#685 (comment)