This is due to the derived Debug on ElementRef. ego_tree::NodeRef prints the entire tree in its derived Debug impl, but there are already outstanding (soundness?) bugs in ego_tree and I don't think they'll be fixed anytime soon.

To reproduce, you need to have an active log::Log logger running enabled at log::Level::Debug

Snipped of produced output

Matching complex selector body .entry-content* p for ElementRef { node: NodeRef { id: NodeId(173), tree: Tree { vec: [Node { parent: None, prev_sibling: None, next_sibling: None, children: Some((NodeId(2), NodeId(1001))), value: Document }, Node { parent: Some(NodeId(1)), prev_sibling: None, next_sibling: Some(NodeId(3)), children: None, value: Doctype(<!DOCTYPE html PUBLIC "" "">) }, Node { parent: Some(NodeId(1)), prev_sibling: Some(NodeId(2)), next_sibling: Some(NodeId(999)), children: Some((NodeId(4), NodeId(245))), value: Element(<html lang="en-US" data-wp-dark-mode-preset="2" data-wp-dark-mode-active="true" data-wp-dark-mode-loading="true">) }, Node { parent: Some(NodeId(3)), prev_sibling: None, next_sibling: Some(NodeId(244)), children: Some((NodeId(5), NodeId(243))), value: Element(<head>) }, Node { parent: Some(NodeId(4)), prev_sibling: None, next_sibling: Some(NodeId(6)), children: None, value: Text("\n") }, Node { parent: Some(NodeId(4)), prev_sibling: Some(NodeId(5)), next_sibling: Some(NodeId(7)), children: None, value: Element(<meta charset="UTF-8">) }, Node { parent: Some(NodeId(4)), prev_sibling: Some(NodeId(6)), next_sibling: Some(NodeId(8)), children: None, value: Text("\n") }, Node { parent: Some(NodeId(4)), prev_sibling: Some(NodeId(7)), next_sibling: Some(NodeId(9)), children: None, value: Element(<meta name="viewport" content="width=device-width, initial-scale=1">) }, Node { parent: Some(NodeId(4)), prev_sibling: Some(NodeId(8)), next_sibling: Some(NodeId(10)), children: None, value: Text("\n") }, Node { parent: Some(NodeId(4)), prev_sibling: Some(NodeId(9)), next_sibling: Some(NodeId(11)), children: None, value: Element(<link rel="profile" href="https://gmpg.org/xfn/11">) }, Node { parent: Some(NodeId(4)), prev_sibling: Some(NodeId(10)), next_sibling: Some(NodeId(12)), children: None, value: Text("\n") }, Node { parent: Some(NodeId(4)), prev_sibling: Some(NodeId(11)), next_sibling: Some(NodeId(13)), children: No

Running my program produced 4 GB in 40 seconds before I stopped it.

I will submit a PR for this.

Scraper logs 1gb #186

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions