adding support for asyncio in deep-translator

[jiaulislam]

Earlier I was thinking about adding asyncio support for deep-translator. But Is there going to be any performance boost or do the community need asyncio for this package?

I think i can contribute in asyncio part or atleast show a proof of concept of performance of doing asyncio.

[jiaulislam]

So far, I'm getting improvement while making batch translation process after using just asyncio. Around 40-50% performance boost can be found.

Test Async Code (with httpx package for network call as Request don't suport asyncio):

    async def async_translate_batch(self, batch: List[str]):

        _batch_to_translate = [
            word
            for word in batch
            if is_input_valid(word)
        ]


        self._url_params["tl"] = self._target
        self._url_params["sl"] = self._source

        async with httpx.AsyncClient(
            proxies=self.proxies
        ) as client:
            _translated_word = []
            for word in _batch_to_translate:
                response = await client.get(
                    url=self._base_url,
                    params=self._url_params,
                )

                if self.payload_key:
                    self._url_params[self.payload_key] = word

                if response.status_code == 429:
                    raise TooManyRequests()

                if response.status_code != 200:
                    raise RequestError()

                soup = BeautifulSoup(response.text, "html.parser")

                element = soup.find(self._element_tag, self._element_query)

                if not element:
                    element = soup.find(
                        self._element_tag, self._alt_element_query
                    )
                    if not element:
                        raise TranslationNotFound(word)
                if element.get_text(strip=True) == word.strip():
                    to_translate_alpha = "".join(
                        ch for ch in word.strip() if ch.isalnum()
                    )
                    translated_alpha = "".join(
                        ch
                        for ch in element.get_text(strip=True)
                        if ch.isalnum()
                    )
                    if (
                        to_translate_alpha
                        and translated_alpha
                        and to_translate_alpha == translated_alpha
                    ):
                        self._url_params["tl"] = self._target
                        if "hl" not in self._url_params:
                            return word.strip()
                        del self._url_params["hl"]
                        return self.translate(word)

                else:
                    _translated_word.append(element.get_text(strip=True)) 
            
            return _translated_word

Perf-Counter Results

Synchronus Took 7.471747110000251 Seconds
Asynchronus Took 3.064987882999958 Seconds
Performance Ratio 41.0210334728507

[NorthOC]

I'll try this on a real file case. Where should I place this code in?

[jiaulislam]

I'll try this on a real file case. Where should I place this code in?

Well this was just a test to check, I tried that example in google.py file for batch_translate where the function was is taking list of strings. You can think of something to read files and then provide the words in that function as parameter.