tag:github.com,2008:https://github.com/scrapy/w3lib/releasesRelease notes from w3lib2025-01-27T14:20:46Ztag:github.com,2008:Repository/1624565/v2.3.12025-01-27T14:23:06Zv2.3.1<ul>
<li>No code changes from v2.3.0.</li>
</ul>wRARtag:github.com,2008:Repository/1624565/v2.3.02025-01-27T14:12:57Zv2.3.0<ul>
<li>
<p>Dropped Python 3.8 support (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2581483579" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/232" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/232/hovercard" href="https://github.com/scrapy/w3lib/pull/232">#232</a>).</p>
</li>
<li>
<p>Removed the following functions, deprecated in 2.0.0:</p>
<ul>
<li><code>w3lib.util.str_to_unicode</code></li>
<li><code>w3lib.util.to_native_str</code></li>
<li><code>w3lib.util.unicode_to_str</code></li>
</ul>
<p>(<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2811098439" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/235" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/235/hovercard" href="https://github.com/scrapy/w3lib/pull/235">#235</a>).</p>
</li>
<li>
<p>Added Python 3.13 support (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2581483579" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/232" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/232/hovercard" href="https://github.com/scrapy/w3lib/pull/232">#232</a>).</p>
</li>
<li>
<p>Fixed running tests with newer point releases of Python 3.10 and 3.11 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2801371707" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/233" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/233/hovercard" href="https://github.com/scrapy/w3lib/pull/233">#233</a>).</p>
</li>
<li>
<p>Cleanup and CI improvements (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2581483579" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/232" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/232/hovercard" href="https://github.com/scrapy/w3lib/pull/232">#232</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2810304697" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/234" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/234/hovercard" href="https://github.com/scrapy/w3lib/pull/234">#234</a>).</p>
</li>
</ul>wRARtag:github.com,2008:Repository/1624565/v2.2.12025-01-27T14:11:58Zv2.2.1<ul>
<li><code>canonicalize_url()</code> no longer applies lowercase to the userinfo URL component. (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2347130173" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/229" data-hovercard-type="issue" data-hovercard-url="/scrapy/w3lib/issues/229/hovercard" href="https://github.com/scrapy/w3lib/issues/229">#229</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2347866379" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/230" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/230/hovercard" href="https://github.com/scrapy/w3lib/pull/230">#230</a>)</li>
</ul>wRARtag:github.com,2008:Repository/1624565/v2.2.02024-06-05T07:24:45Zv2.2.0<ul>
<li>Dropped Python 3.7 support (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1789995794" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/214" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/214/hovercard" href="https://github.com/scrapy/w3lib/pull/214">#214</a>).</li>
<li>Added Python 3.12 and PyPy 3.10 support (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1928911940" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/218" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/218/hovercard" href="https://github.com/scrapy/w3lib/pull/218">#218</a>).</li>
<li>Added the description to the package metadata (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2330680570" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/227" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/227/hovercard" href="https://github.com/scrapy/w3lib/pull/227">#227</a>).</li>
<li>Improved type hints (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2328342619" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/226" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/226/hovercard" href="https://github.com/scrapy/w3lib/pull/226">#226</a>).</li>
<li>Added <code>.readthedocs.yml</code> (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2048805734" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/219" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/219/hovercard" href="https://github.com/scrapy/w3lib/pull/219">#219</a>).</li>
<li>Updated the intersphinx URLs (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2255180754" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/224" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/224/hovercard" href="https://github.com/scrapy/w3lib/pull/224">#224</a>).</li>
<li>Added the <code>pre-commit</code> configuration, code reformatted with <code>black</code> (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2121388745" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/220" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/220/hovercard" href="https://github.com/scrapy/w3lib/pull/220">#220</a>).</li>
<li>Updated CI configuration (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1923968974" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/217" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/217/hovercard" href="https://github.com/scrapy/w3lib/pull/217">#217</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2330680570" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/227" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/227/hovercard" href="https://github.com/scrapy/w3lib/pull/227">#227</a>).</li>
</ul>wRARtag:github.com,2008:Repository/1624565/v2.1.22023-08-03T08:49:53Zv2.1.2<ul>
<li>Fix test failures on Python 3.11.4+ (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1759551979" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/212" data-hovercard-type="issue" data-hovercard-url="/scrapy/w3lib/issues/212/hovercard" href="https://github.com/scrapy/w3lib/issues/212">#212</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1767414487" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/213" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/213/hovercard" href="https://github.com/scrapy/w3lib/pull/213">#213</a>).</li>
<li>Fix an incorrect type hint (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1580952364" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/211" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/211/hovercard" href="https://github.com/scrapy/w3lib/pull/211">#211</a>).</li>
<li>Add project URLs to setup.py (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1806745374" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/215" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/215/hovercard" href="https://github.com/scrapy/w3lib/pull/215">#215</a>).</li>
</ul>wRARtag:github.com,2008:Repository/1624565/v2.1.12022-12-09T11:11:47Zv2.1.1<h2>What's Changed</h2>
<ul>
<li>safe_url_string, canonicalize_url: apply stripping from the URL living standard by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Gallaecio/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Gallaecio">@Gallaecio</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1466761097" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/207" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/207/hovercard" href="https://github.com/scrapy/w3lib/pull/207">#207</a></li>
<li>Changelog by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/kmike/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/kmike">@kmike</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1482015777" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/208" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/208/hovercard" href="https://github.com/scrapy/w3lib/pull/208">#208</a></li>
<li>fix tox 4 compatibility by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/kmike/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/kmike">@kmike</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1486492747" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/209" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/209/hovercard" href="https://github.com/scrapy/w3lib/pull/209">#209</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/scrapy/w3lib/compare/v2.1.0...v2.1.1"><tt>v2.1.0...v2.1.1</tt></a></p>kmiketag:github.com,2008:Repository/1624565/v2.1.02022-12-09T11:11:07Z2.1.0<h2>What's Changed</h2>
<ul>
<li>update type annotation of auto_detect_fun param in html_to_unicode() by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/BurnzZ/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/BurnzZ">@BurnzZ</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1383300626" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/190" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/190/hovercard" href="https://github.com/scrapy/w3lib/pull/190">#190</a></li>
<li>update html_to_unicode() so that the BOM is used first to check the e… by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/BurnzZ/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/BurnzZ">@BurnzZ</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1383408620" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/191" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/191/hovercard" href="https://github.com/scrapy/w3lib/pull/191">#191</a></li>
<li>changing basic_auth_header flavor to b64encode instead of urlsafe_b64encode by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gsweiz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gsweiz">@gsweiz</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1401086469" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/192" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/192/hovercard" href="https://github.com/scrapy/w3lib/pull/192">#192</a></li>
<li>Add support for Python 3.11. by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/wRAR/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/wRAR">@wRAR</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1417947894" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/195" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/195/hovercard" href="https://github.com/scrapy/w3lib/pull/195">#195</a></li>
<li>Use latest version of Ubuntu by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Laerte/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Laerte">@Laerte</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1419009122" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/197" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/197/hovercard" href="https://github.com/scrapy/w3lib/pull/197">#197</a></li>
<li>187 safe url string already encoded user pass by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/felipeboffnunes/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/felipeboffnunes">@felipeboffnunes</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1418875290" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/196" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/196/hovercard" href="https://github.com/scrapy/w3lib/pull/196">#196</a></li>
<li>Strip spaces in canonicalize_url by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Gallaecio/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Gallaecio">@Gallaecio</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="494599647" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/136" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/136/hovercard" href="https://github.com/scrapy/w3lib/pull/136">#136</a></li>
<li>unit test Issue 91 is fixed by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/felipeboffnunes/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/felipeboffnunes">@felipeboffnunes</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1427580267" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/198" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/198/hovercard" href="https://github.com/scrapy/w3lib/pull/198">#198</a></li>
<li>Drop Python 3.6 support by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Laerte/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Laerte">@Laerte</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1436593322" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/200" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/200/hovercard" href="https://github.com/scrapy/w3lib/pull/200">#200</a></li>
<li>[MRG+1] Added: Removing comments before extracting base URLs. Not a solution to <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="170725455" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/70" data-hovercard-type="issue" data-hovercard-url="/scrapy/w3lib/issues/70/hovercard" href="https://github.com/scrapy/w3lib/issues/70">#70</a>, but does help in some cases. by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/starrify/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/starrify">@starrify</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="185030793" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/77" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/77/hovercard" href="https://github.com/scrapy/w3lib/pull/77">#77</a></li>
<li>Handle <code>OverflowError</code> exception on <code>convert_entity</code> by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Laerte/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Laerte">@Laerte</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1442363937" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/202" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/202/hovercard" href="https://github.com/scrapy/w3lib/pull/202">#202</a></li>
<li>safe_url_string: escape additional characters by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Gallaecio/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Gallaecio">@Gallaecio</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1461655584" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/203" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/203/hovercard" href="https://github.com/scrapy/w3lib/pull/203">#203</a></li>
<li>I have added contributing.md file and code_of_conduct.md file along with some very minor changes in the Readme.rst file by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/VanshajPoonia/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/VanshajPoonia">@VanshajPoonia</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1405071438" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/194" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/194/hovercard" href="https://github.com/scrapy/w3lib/pull/194">#194</a></li>
<li>Full typing by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/wRAR/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/wRAR">@wRAR</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1465482633" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/206" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/206/hovercard" href="https://github.com/scrapy/w3lib/pull/206">#206</a></li>
<li>Add release notes for version 2.1.0 by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Gallaecio/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Gallaecio">@Gallaecio</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1465424827" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/205" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/205/hovercard" href="https://github.com/scrapy/w3lib/pull/205">#205</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/BurnzZ/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/BurnzZ">@BurnzZ</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1383300626" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/190" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/190/hovercard" href="https://github.com/scrapy/w3lib/pull/190">#190</a></li>
<li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gsweiz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gsweiz">@gsweiz</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1401086469" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/192" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/192/hovercard" href="https://github.com/scrapy/w3lib/pull/192">#192</a></li>
<li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/felipeboffnunes/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/felipeboffnunes">@felipeboffnunes</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1418875290" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/196" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/196/hovercard" href="https://github.com/scrapy/w3lib/pull/196">#196</a></li>
<li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/starrify/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/starrify">@starrify</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="185030793" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/77" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/77/hovercard" href="https://github.com/scrapy/w3lib/pull/77">#77</a></li>
<li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/VanshajPoonia/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/VanshajPoonia">@VanshajPoonia</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1405071438" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/194" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/194/hovercard" href="https://github.com/scrapy/w3lib/pull/194">#194</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/scrapy/w3lib/compare/v2.0.1...v2.1.0"><tt>v2.0.1...v2.1.0</tt></a></p>kmiketag:github.com,2008:Repository/1624565/v2.0.12022-10-21T08:31:10Z2.0.1<p>Backwards incompatible changes:</p>
<ul>
<li>Python 2 is no longer supported; Python 3.6+ is required now (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="836789185" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/168" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/168/hovercard" href="https://github.com/scrapy/w3lib/pull/168">#168</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1048671223" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/175" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/175/hovercard" href="https://github.com/scrapy/w3lib/pull/175">#175</a>).</li>
<li><code>w3lib.url.safe_url_string</code> and <code>w3lib.url.canonicalize_url</code><br>
no longer convert "%23" to "#" when it appears in the URL path. This is a bug<br>
fix. It's listed as a backward-incomatible change because in some cases the<br>
output of <code>w3lib.url.canonicalize_url</code> is going to change, and so, if<br>
this output is used to generate URL fingerprints, new fingerprints might be<br>
incompatible with those created with the previous w3lib versions<br>
(<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="509030275" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/141" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/141/hovercard" href="https://github.com/scrapy/w3lib/pull/141">#141</a>).</li>
</ul>
<p>Deprecation removals (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="839610965" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/169" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/169/hovercard" href="https://github.com/scrapy/w3lib/pull/169">#169</a>):</p>
<ul>
<li>The <code>w3lib.form</code> module is removed.</li>
<li>The <code>w3lib.html.remove_entities</code> function is removed.</li>
<li>The <code>w3lib.url.urljoin_rfc</code> function is removed.</li>
</ul>
<p>The following functions are deprecated, and will be removed in future releases<br>
(<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="841925593" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/170" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/170/hovercard" href="https://github.com/scrapy/w3lib/pull/170">#170</a>):</p>
<ul>
<li><code>w3lib.util.str_to_unicode</code></li>
<li><code>w3lib.util.unicode_to_str</code></li>
<li><code>w3lib.util.to_native_str</code></li>
</ul>
<p>Other improvements and bug fixes:</p>
<ul>
<li>Type annotations are added (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="916106931" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/172" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/172/hovercard" href="https://github.com/scrapy/w3lib/pull/172">#172</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1332136460" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/184" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/184/hovercard" href="https://github.com/scrapy/w3lib/pull/184">#184</a>).</li>
<li>Added support for Python 3.9 and 3.10 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="836789185" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/168" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/168/hovercard" href="https://github.com/scrapy/w3lib/pull/168">#168</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1048677531" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/176" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/176/hovercard" href="https://github.com/scrapy/w3lib/pull/176">#176</a>).</li>
<li>Fixed <code>w3lib.html.get_meta_refresh</code> for <code><meta></code> tags where<br>
<code>http-equiv</code> is written after <code>content</code> (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1271383374" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/179" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/179/hovercard" href="https://github.com/scrapy/w3lib/pull/179">#179</a>).</li>
<li>Fixed <code>w3lib.url.safe_url_string</code> for IDNA domains with ports (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1014428956" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/174" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/174/hovercard" href="https://github.com/scrapy/w3lib/pull/174">#174</a>).</li>
<li><code>w3lib.url.url_query_cleaner</code> no longer adds an unneeded <code>#</code> when<br>
<code>keep_fragments=True</code> is passed, and the URL doesn't have a fragment<br>
(<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="734349659" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/159" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/159/hovercard" href="https://github.com/scrapy/w3lib/pull/159">#159</a>).</li>
<li>Removed a workaround for an ancient pathname2url bug (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="510760001" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/142" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/142/hovercard" href="https://github.com/scrapy/w3lib/pull/142">#142</a>)</li>
<li>CI is migrated to GitHub Actions (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="827902952" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/166" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/166/hovercard" href="https://github.com/scrapy/w3lib/pull/166">#166</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1048682869" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/177" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/177/hovercard" href="https://github.com/scrapy/w3lib/pull/177">#177</a>); other CI improvements (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="736438445" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/160" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/160/hovercard" href="https://github.com/scrapy/w3lib/pull/160">#160</a>,<br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1330834917" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/182" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/182/hovercard" href="https://github.com/scrapy/w3lib/pull/182">#182</a>).</li>
<li>The code is formatted using black (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="958270377" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/173" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/173/hovercard" href="https://github.com/scrapy/w3lib/pull/173">#173</a>).</li>
</ul>kmiketag:github.com,2008:Repository/1624565/v2.0.02022-08-11T13:44:48Zv2.0.0<p>Bump version: 1.22.0 → 2.0.0</p>kmiketag:github.com,2008:Repository/1624565/v1.22.02020-05-13T19:36:00Zv1.22.0<ul>
<li>Python 3.4 is no longer supported (issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="606852755" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/156" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/156/hovercard" href="https://github.com/scrapy/w3lib/pull/156">#156</a>)</li>
<li><code>w3lib.url.safe_url_string</code> now supports an optional <code>quote_path</code><br>
parameter to disable the percent-encoding of the URL path (issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="393512815" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/119" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/119/hovercard" href="https://github.com/scrapy/w3lib/pull/119">#119</a>)</li>
<li><code>w3lib.url.add_or_replace_parameter</code> and<br>
<code>w3lib.url.add_or_replace_parameters</code> no longer remove duplicate<br>
parameters from the original query string that are not being added or<br>
replaced (issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="425735757" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/126" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/126/hovercard" href="https://github.com/scrapy/w3lib/pull/126">#126</a>)</li>
<li><code>w3lib.html.remove_tags</code> now raises a <code>ValueError</code> exception<br>
instead of <code>AssertionError</code> when using both the <code>which_ones</code> and the<br>
<code>keep</code> parameters (issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="528657381" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/154" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/154/hovercard" href="https://github.com/scrapy/w3lib/pull/154">#154</a>)</li>
<li>Test improvements (issues <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="511886713" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/143" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/143/hovercard" href="https://github.com/scrapy/w3lib/pull/143">#143</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="522055586" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/146" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/146/hovercard" href="https://github.com/scrapy/w3lib/pull/146">#146</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="522100013" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/148" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/148/hovercard" href="https://github.com/scrapy/w3lib/pull/148">#148</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="522794464" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/149" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/149/hovercard" href="https://github.com/scrapy/w3lib/pull/149">#149</a>)</li>
<li>Documentation improvements (issues <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="507977250" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/140" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/140/hovercard" href="https://github.com/scrapy/w3lib/pull/140">#140</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="521396768" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/144" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/144/hovercard" href="https://github.com/scrapy/w3lib/pull/144">#144</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="522039963" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/145" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/145/hovercard" href="https://github.com/scrapy/w3lib/pull/145">#145</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="524210269" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/151" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/151/hovercard" href="https://github.com/scrapy/w3lib/pull/151">#151</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="524423667" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/152" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/152/hovercard" href="https://github.com/scrapy/w3lib/pull/152">#152</a>, <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="524432444" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/153" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/153/hovercard" href="https://github.com/scrapy/w3lib/pull/153">#153</a>)</li>
<li>Code cleanup (issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="505249728" data-permission-text="Title is private" data-url="https://github.com/scrapy/w3lib/issues/139" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/w3lib/pull/139/hovercard" href="https://github.com/scrapy/w3lib/pull/139">#139</a>)</li>
</ul>Gallaecio