Search results

Config files for dolma 1.7

Status: Open.

#255 In allenai/dolma;

· RobinQrtz opened on Mar 31, 2025

Analyzer not processing multiple "attributes"

Status: Open.

#254 In allenai/dolma;

· zxnie opened on Mar 12, 2025

Resume continue the pipeline

Status: Open.

#246 In allenai/dolma;

· wannaphong opened on Feb 26, 2025

Generation of own dataset with Dolma Tokenizer CLI

Status: Open.

#225 In allenai/dolma;

· WenJett opened on Jan 26, 2025

7 comments

FastText training data

Status: Open.

#209 In allenai/dolma;

· msaebi1993 opened on Sep 25, 2024

Mixer progress tracking

Status: Open.

#199 In allenai/dolma;

· aetting opened on Sep 12, 2024

Is there explicitly instruction-following data in the version of Dolma used to train Olmo v1?

Status: Open.

#177 In allenai/dolma;

· john-hewitt opened on Jul 15, 2024

3 comments

Need help in customizing python/dolma/taggers/c4.py

Status: Open.

#173 In allenai/dolma;

· mihara-bot opened on Jun 19, 2024

Need clarification of Gopher in Step 2

Status: Open.

#172 In allenai/dolma;

· mihara-bot opened on Jun 18, 2024

PyPI release

Status: Open.

#167 In allenai/dolma;

· baberabb opened on Jun 1, 2024

dedupe.documents.attribute_name does not work

Status: Open.

#166 In allenai/dolma;

· mathCrazyy opened on May 29, 2024

1 comment

Duplicate ids in Dolma v1.7

Status: Open.

#157 In allenai/dolma;

· Vedaad-Shakib opened on May 3, 2024

Inquiry about Web Pipeline Availability

Status: Open.

#151 In allenai/dolma;

· codefly13 opened on Apr 22, 2024

2 comments

Running paragraph level deduplication on c4

Status: Open.

#150 In allenai/dolma;

· andrewhojel opened on Apr 20, 2024

2 comments

S3 mixer doesn't start

Status: Open.

#143 In allenai/dolma;

· marcopasqua opened on Mar 26, 2024

2 comments

Data out of bounds when using ‘dolma tokens --dtype uint32’

Status: Open.

#142 In allenai/dolma;

· Jackwaterveg opened on Mar 25, 2024

1 comment

Some race condition in url taggers

Status: Open.

#138 In allenai/dolma;

· peterbjorgensen opened on Mar 21, 2024

1 comment

Is there a way to intergratge Dolma toolkit to Spark?

Status: Open.

#136 In allenai/dolma;

· DangoWang opened on Mar 19, 2024

1 comment

Support providing streams into mixer via CLI

enhancementNew feature or requestNew feature or request

Status: Open.

#130 In allenai/dolma;

· soldni opened on Feb 29, 2024



not_alphanum_paragraph_v1 tagger takes forever to run on certain inputs.

Status: Open.

#123 In allenai/dolma;

· peterbjorgensen opened on Feb 15, 2024

1 comment



make_wikipedia.py: long running time

Status: Open.

#121 In allenai/dolma;

· chschroeder opened on Feb 13, 2024

3 comments

make_wikipedia.py fails on linux

Status: Open.

#58 In allenai/dolma;

· peterbjorgensen opened on Oct 17, 2023

10 comments

Simplify how rules in the mixer are provided

enhancementNew feature or requestNew feature or request

Status: Open.

#50 In allenai/dolma;

· soldni opened on Sep 25, 2023

 ·  

0.10.0

4 comments



Potential race condition

Status: Open.

#33 In allenai/dolma;

· chris-ha458 opened on Aug 25, 2023

5 comments

Add Style Guide and Pre-commit.

Status: Open.

#30 In allenai/dolma;

· soldni opened on Aug 22, 2023

2 comments