Huggingface

[percevalw]

Description

Added

• Add multi-modal transformers (huggingface-embedding) with windowing options
• Add render_page option to pdfminer extractor, for multi-modal PDF features

Changed

• Updated API to follow EDS-NLP's refactoring
• Updated confit to 0.4.2 (better errors) and foldedtensor to 0.3.0 (better multiprocess support)
• Better test coverage

Checklist

• If this PR is a bug fix, the bug is documented in the test suite.
• Changes were documented in the changelog (pending section).
• If necessary, changes were made to the documentation.

[codecov]

Codecov Report

Patch coverage is 97.56% of modified lines.

Files Changed	Coverage
edspdf/pipes/embeddings/box_layout_preprocessor.py	ø
edspdf/structures.py	ø
edspdf/utils/torch.py	ø
edspdf/registry.py	92.85%
edspdf/layers/relative_attention.py	100.00%
edspdf/pipeline.py	100.00%
edspdf/pipes/embeddings/huggingface_embedding.py	100.00%
edspdf/pipes/embeddings/simple_text_embedding.py	100.00%
edspdf/pipes/extractors/pdfminer.py	100.00%
edspdf/trainable_pipe.py	100.00%
... and 1 more

📢 Thoughts on this report? Let us know!.