Oct dataset #8
Merged
Oct dataset #8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add create_sample_dataset.py: Create complete sampled datasets with all related files - Add sample_papers_by_category.py: Stratified sampling by category distribution - Add outputs_oct/: 100-paper sample dataset with citations, related works, and LaTeX sources - Sampling method: Stratified random sampling maintaining category proportions - Categories: 29 CV, 17 LG, 9 SE, 8 HC, 6 RO, and 13 other CS categories - Includes: papers.csv, paper_content.csv, citations/, related_works/, latex_source/ - All papers meet quality criteria: related works present, 5+ citations, 200-10K chars - Random seed 42 for reproducibility
…nce citation counts - Updated generate_nuggets_from_reports.py to work with data pipeline output structure - Enhanced get_important_citations.py with flexible column mapping - Added comprehensive documentation to README.md - Created PULL_REQUEST.md documenting all new features - Added nuggets generation for ground truth reports - Added important citation filtering for ground truth reports - Integrated reference citation counts using OpenAlex API
…ence citation counts - Updated generate_nuggets_from_reports.py to work with data pipeline output structure - Enhanced get_important_citations.py with flexible column mapping - Added comprehensive documentation to README.md - Created PULL_REQUEST.md documenting all new features - Added nuggets generation for ground truth reports - Added important citation filtering for ground truth reports - Integrated reference citation counts using OpenAlex API
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.