[From Scrapy]: Incorrect XPath processing

[chemiron]

Xpath //h1 can't extract data correctly from https://www.imdb.com/title/tt6757474/

[BurnzZ]

The website might change, so here's a quick reproducible example from the current state of the page:

>>> import parsel
>>> html = '<h1 class="">LA Galaxy <@ San Jose Earthquakes&nbsp;            </h1>'
>>> parsel.Selector(text=html).xpath('//h1').get()  
'<h1 class="">LA Galaxy </h1>'

Though IMHO, this was an invalid HTML to begin with, should the webpage have encoded the < char as <, it should work correctly:

import parsel
html = '<h1 class="">LA Galaxy &lt;@ San Jose Earthquakes&nbsp;            </h1>'
parsel.Selector(text=html).xpath('//h1').get()  
'<h1 class="">LA Galaxy &lt;@ San Jose Earthquakes\xa0            </h1>'

[Gallaecio]

@chemiron May we close this issue in favor of #126?

[chemiron]

closed in favor of #126