feat: add HTML parser initial implementation #1

JuroOravec · 2025-01-23T22:29:35Z

Here is the Rust implementation of the HTML parser 🎉

JuroOravec · 2025-01-23T22:49:50Z

.github/workflows/publish.yml

@@ -0,0 +1,179 @@
+# This file is autogenerated by maturin v1.8.1


This is the CI job for publishing the package, as provided by maturin. I have yet to see if this will work. It requires PYPI_API_TOKEN to be set as secret.

@EmilStenstrom Regarding the PYPI_API_TOKEN, the question remains how to manage this package on Pypi - e.g. release it under my or under your account?

Either way, this package should have a separate PYPI_API_TOKEN from the one we use for django-components, for extra security.

JuroOravec · 2025-01-23T22:53:22Z

CODE_OF_CONDUCT.md

+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at emil@emilstenstrom.se. All


I copied the project meta files like the code of conduct. So this one contains your contact info @EmilStenstrom. Let me know if it's ok or if I should put mine or other or remove.

JuroOravec · 2025-01-24T09:35:51Z

pyproject.toml

@@ -0,0 +1,96 @@
+[build-system]
+requires = ["maturin>=1.8,<2.0"]
+build-backend = "maturin"


This is a regular pyproject.toml (based on the one for django-components.

One key difference, though, is that build system is set to maturin.

JuroOravec · 2025-01-24T09:36:24Z

pyproject.toml

+keywords = ["django", "components", "html"]
+readme = "README.md"
+authors = [
+    {name = "Juro Oravec", email = "juraj.oravec.josefson@gmail.com"},


I've put me and my email as the author

JuroOravec · 2025-01-24T09:41:31Z

src/lib.rs

+/// A Python module implemented in Rust for high-performance HTML transformation.
+#[pymodule]
+fn djc_core_html_parser(_py: Python, m: &PyModule) -> PyResult<()> {
+    m.add_function(wrap_pyfunction!(html_parser::transform_html, m)?)?;


This is how, using pyo3, we register a function within a Python module.

So this means that when one imports the package in Python, they can access set_html_attributes under:

from djc_core_html_parser import set_html_attributes

JuroOravec · 2025-01-24T09:44:27Z

__init__.pyi

@@ -0,0 +1,34 @@
+from typing import List, Dict, Optional
+
+def set_html_attributes(


Don't know if this __init__.pyi is necessary also for the published package, but at least locally, python language server was not picking up the generated python module in the test file. So I added this stub so test file is correctly typed.

JuroOravec · 2025-01-24T09:46:23Z

tests/test_html_parser.py

@@ -0,0 +1,148 @@
+# This same set of tests is also found in django-components, to ensure that


This package includes both Rust and Python tests - Rust tests are for development + making sure things work as expected. And these Python tests are to ensure that the generated Python module works as expected too.

JuroOravec · 2025-01-24T09:48:12Z

src/html_parser.rs

+#[pyo3(
+    text_signature = "(html, root_attributes, all_attributes, *, check_end_names=False, watch_on_attribute=None)"
+)]
+pub fn set_html_attributes(


This is the entrypoint. I renamed it to set_html_attributes, so it shares the same name as the entrypoint for the pure python implementation.

JuroOravec · 2025-01-24T09:50:19Z

src/html_parser.rs

+}
+
+/// Add attributes to a HTML start tag (e.g. `<div>`) based on the configuration
+fn add_attributes(


This function is the equivalent to the on_tag callback defined in the pure python impl,

JuroOravec · 2025-01-24T09:51:43Z

src/html_parser.rs

+    let mut depth: i32 = 0;
+
+    // Read the HTML event by event
+    loop {


And here is where we process the HTML as a stream of tokens ("events")

JuroOravec · 2025-01-24T09:53:49Z

src/html_parser.rs

+}
+
+#[cfg(test)]
+mod tests {


And in Rust it's standard to write the tests in the same file as the implementation. The #[cfg(test)] is like a compiler flag that tells Rust compiler that the contents of tests should 1. Be excluded from the build, and 2. should be run when running cargo test.

EmilStenstrom

Looks good to me! Impressed by this whole thing, well done! :)

EmilStenstrom · 2025-01-24T21:13:54Z

CODE_OF_CONDUCT.md

+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at emil@emilstenstrom.se. All


JuroOravec · 2025-01-24T22:35:11Z

@EmilStenstrom ✨ It lives! ✨

Lol but it's crazy, maturin builds 103 different platform releases. See here.

And here's the pipeline if interested.

Btw I've used my PyPI account to upload it, and I've set the PYPI_API_TOKEN secret for this repo.

EmilStenstrom · 2025-01-24T23:12:05Z

I’ve requested a PyPi org that we can share access to, will get back when we get accepted. Let’s move both projects there

JuroOravec added 4 commits January 23, 2025 23:29

feat: add HTML parser initial implementation

221b71b

ci: create virtual env

3b7e435

refactor: source venv

d723339

refactor: ues language-specific command to load Python env

a10f496

JuroOravec commented Jan 23, 2025

View reviewed changes

refactor: revert the feature of expanding self-closing tags

0740166

JuroOravec commented Jan 24, 2025

View reviewed changes

refactor: rename transform_html to set_html_attributes

1499d58

JuroOravec commented Jan 24, 2025

View reviewed changes

JuroOravec marked this pull request as ready for review January 24, 2025 09:54

JuroOravec requested a review from EmilStenstrom January 24, 2025 09:55

EmilStenstrom approved these changes Jan 24, 2025

View reviewed changes

JuroOravec merged commit 42e41c5 into main Jan 24, 2025
12 checks passed

JuroOravec deleted the jo-feat-html-parser branch January 24, 2025 22:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add HTML parser initial implementation #1

feat: add HTML parser initial implementation #1

JuroOravec commented Jan 23, 2025 •

edited

Loading

JuroOravec Jan 23, 2025 •

edited

Loading

JuroOravec Jan 24, 2025

JuroOravec Jan 23, 2025

EmilStenstrom Jan 24, 2025

JuroOravec Jan 24, 2025

JuroOravec Jan 24, 2025

JuroOravec Jan 24, 2025

JuroOravec Jan 24, 2025

JuroOravec Jan 24, 2025

JuroOravec Jan 24, 2025

JuroOravec Jan 24, 2025

JuroOravec Jan 24, 2025

JuroOravec Jan 24, 2025

EmilStenstrom left a comment

EmilStenstrom Jan 24, 2025

JuroOravec commented Jan 24, 2025

EmilStenstrom commented Jan 24, 2025

		@@ -0,0 +1,179 @@
		# This file is autogenerated by maturin v1.8.1

		@@ -0,0 +1,34 @@
		from typing import List, Dict, Optional

		def set_html_attributes(

		@@ -0,0 +1,148 @@
		# This same set of tests is also found in django-components, to ensure that

feat: add HTML parser initial implementation #1

feat: add HTML parser initial implementation #1

Conversation

JuroOravec commented Jan 23, 2025 • edited Loading

JuroOravec Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EmilStenstrom left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JuroOravec commented Jan 24, 2025

EmilStenstrom commented Jan 24, 2025

JuroOravec commented Jan 23, 2025 •

edited

Loading

JuroOravec Jan 23, 2025 •

edited

Loading