Commits

Branch selector

 master

User selector

adarob

Datepicker

All time

Commit History

Commits on Jul 11, 2022

• Update CommonCrawl download URL for C4.

  Show description for f725ff0
  

  adarob

  authored and

  The TensorFlow Datasets Authors

  committed
  f725ff0

  Copy full SHA for f725ff0

Commits on Mar 14, 2022

• Download files from "www.statmt.org" sequentially to avoid 503 due to rate-limiting.

  Show description for bc7882e
  

  adarob

  authored and

  The TensorFlow Datasets Authors

  committed
  bc7882e

  Copy full SHA for bc7882e

Commits on Jun 1, 2021

• Add 1.0.x GLUE versions as supported.

  Show description for bf04ec1
  

  adarob

  authored and

  copybara-github

  committed
  bf04ec1

  Copy full SHA for bf04ec1

Commits on Dec 29, 2020

• Further increase parallelism of c4 download stage.

  Show description for 9b5e431
  

  adarob

  authored and

  copybara-github

  committed
  9b5e431

  Copy full SHA for 9b5e431

• Remove bottleneck in C4 pipeline.

  Show description for 3e372b7
  

  adarob

  authored and

  copybara-github

  committed
  3e372b7

  Copy full SHA for 3e372b7

Commits on Nov 19, 2020

• [C4] Automatically download all wet files.

  Show description for 0fd5b90
  

  adarob

  authored and

  copybara-github

  committed
  0fd5b90

  Copy full SHA for 0fd5b90

Commits on Nov 5, 2020

• Avoid pickling full SplitBuilder class when running on beam.

  Show description for d274e31
  

  adarob

  authored and

  copybara-github

  committed
  d274e31

  Copy full SHA for d274e31

Commits on Oct 22, 2020

• [C4] Only access the manual_dir when required to avoid throwing an error unnecessarily. Update the instructions to cover all configs.

  Show description for ff3076c
  

  adarob

  authored and

  copybara-github

  committed
  ff3076c

  Copy full SHA for ff3076c

Commits on Oct 15, 2020

• Open source multilingual C4 (mC4).

  Show description for 783136d
  

  adarob

  authored and

  copybara-github

  committed
  783136d

  Copy full SHA for 783136d

Commits on Oct 14, 2020

• Fix missing formatting arg in exception message.

  Show description for f8c6b1f
  

  adarob

  authored and

  copybara-github

  committed
  f8c6b1f

  Copy full SHA for f8c6b1f

Commits on Sep 19, 2020

• Add "Natural Questions Open" open domain QA dataset.

  Show description for 9c5b96d
  

  adarob

  authored and

  copybara-github

  committed
  9c5b96d

  Copy full SHA for 9c5b96d

Commits on Aug 27, 2020

• Add salient span wikipedia dataset from REALM.

  Show description for 971ec90
  

  adarob

  authored and

  copybara-github

  committed
  971ec90

  Copy full SHA for 971ec90

Commits on Aug 18, 2020

• Fix bug where xquad and tydi qa kept the same context for all questions.

  Show description for 1a35ca6
  

  adarob

  authored and

  copybara-github

  committed
  1a35ca6

  Copy full SHA for 1a35ca6

Commits on Aug 10, 2020

• Add TyDiQA GoldP task. Create qa_utils to reduce code duplication in tfds.question_answering.

  Show description for f71a057
  

  adarob

  authored and

  copybara-github

  committed
  f71a057

  Copy full SHA for f71a057

Commits on Aug 7, 2020

• Add XQuAD with translate-train and translate-dev sets from XTREME.

  Show description for 2e7240b
  

  adarob

  authored and

  copybara-github

  committed
  2e7240b

  Copy full SHA for 2e7240b

Commits on Jul 22, 2020

• C4: filter empty pages after deduplication regardless of the min_num_sentences arg.

  Show description for 028efb9
  

  adarob

  authored and

  copybara-github

  committed
  028efb9

  Copy full SHA for 028efb9

Commits on Jul 20, 2020

• Add new version of NSynth with CREPE fixes for F0 estimation. Fixes magenta/ddsp#54

  Show description for 2b0cf6b
  

  adarob

  authored and

  copybara-github

  committed
  2b0cf6b

  Copy full SHA for 2b0cf6b

Commits on Jul 17, 2020

• Refactor C4 deduplication to reduce memory usage. Instead of emitting lines to remove, emit lines to keep, which avoids needing to materialize the full list of URLs for a duplicated line in memory.

  Show description for f16eabb
  

  adarob

  authored and

  copybara-github

  committed
  f16eabb

  Copy full SHA for f16eabb

Commits on Jul 10, 2020

• Fix typo in c4 comment.

  Show description for 58798a3
  

  adarob

  authored and

  copybara-github

  committed
  58798a3

  Copy full SHA for 58798a3

Commits on Jul 6, 2020

• Add back support for '-' in split names.

  Show description for e54b6b2
  

  adarob

  authored and

  copybara-github

  committed
  e54b6b2

  Copy full SHA for e54b6b2

Commits on Apr 8, 2020

• Specify correct dataset dependency for beam workers, which excludes the dataset config.

  Show description for f2f1267
  

  adarob

  authored and

  copybara-github

  committed
  f2f1267

  Copy full SHA for f2f1267

Commits on Apr 4, 2020

• Remove stray debug prints.

  Show description for 7be5bf6
  

  adarob

  authored and

  copybara-github

  committed
  7be5bf6

  Copy full SHA for 7be5bf6

Commits on Apr 3, 2020

• Add WebQuestions dataset.

  Show description for 1161561
  

  adarob

  authored and

  copybara-github

  committed
  1161561

  Copy full SHA for 1161561

Commits on Apr 1, 2020

• C4: download raw CC files in beam workers.

  Show description for bc1d0cb
  

  adarob

  authored and

  copybara-github

  committed
  bc1d0cb

  Copy full SHA for bc1d0cb

Commits on Mar 31, 2020

• Make DownloadManager pickle-able.

  Show description for 06049c4
  

  adarob

  authored and

  copybara-github

  committed
  06049c4

  Copy full SHA for 06049c4

Commits on Mar 27, 2020

• Bump CREPE version w/ fix for NSynth and bump NSynth version.

  Show description for e248a7d
  

  adarob

  authored and

  copybara-github

  committed
  e248a7d

  Copy full SHA for e248a7d

Commits on Feb 5, 2020

• Add token features to Natural Questions.

  Show description for af829e6
  

  adarob

  authored and

  copybara-github

  committed
  af829e6

  Copy full SHA for af829e6

Commits on Jan 31, 2020

• Expose API for reading individual TFRecord files.

  Show description for 34ae6dd
  

  adarob

  authored and

  copybara-github

  committed
  34ae6dd

  Copy full SHA for 34ae6dd

Commits on Jan 29, 2020

• Make size_in_bytes reflect final dataset size instead of downloaded size.

  Show description for 077788c
  

  adarob

  authored and

  copybara-github

  committed
  077788c

  Copy full SHA for 077788c

• Flip True/False label indices in eraser_multi_rc to make False 0 and True 1.

  Show description for fca831c
  

  adarob

  authored and

  copybara-github

  committed
  fca831c

  Copy full SHA for fca831c

Commits on Jan 27, 2020

• Add Natural Questions dataset.

  Show description for 399036f
  

  adarob

  authored and

  copybara-github

  committed
  399036f

  Copy full SHA for 399036f

Commits on Jan 22, 2020

• Add C4 validation set.

  Show description for dc484fd
  

  adarob

  authored and

  copybara-github

  committed
  dc484fd

  Copy full SHA for dc484fd

• Add support for using beam in _split_generators()

  Show description for 851eddb
  

  adarob

  authored and

  copybara-github

  committed
  851eddb

  Copy full SHA for 851eddb

Commits on Dec 19, 2019

• Run calc_loudness op in a session when not executing eagerly.

  Show description for 5bff65b
  

  adarob

  authored and

  copybara-github

  committed
  5bff65b

  Copy full SHA for 5bff65b

Commits on Dec 17, 2019

• Update NSynth f0 and loudness computations for consistency with ddsp lib.

  Show description for 29f97df
  

  adarob

  authored and

  copybara-github

  committed
  29f97df

  Copy full SHA for 29f97df

Pagination

Previous

Next