Commits

Branch selector

 zijun/optimized-jetstream

User selector

All users

Datepicker

All time

Commit History

Commits on Jul 26, 2024

• Performance optimized interleaved mode JetStream server

  JoeZijunZhou

  committed
  20ba07c

  Copy full SHA for 20ba07c

Commits on Jul 23, 2024

• Free engine resource for the slot after finished one request decoding (#119)

  Show description for af1b918
  FanhaiLu1

  authored
  af1b918

  Copy full SHA for af1b918

• Add http server to JetStream (#115)

  Show description for 1830342
  JoeZijunZhou

  authored
  1830342

  Copy full SHA for 1830342

Commits on Jul 22, 2024

• Add `jetstream_server_startup_latency` metric (#118)

  Show description for bd6d013
  Bslabe123

  authored
  bd6d013

  Copy full SHA for bd6d013

Commits on Jul 19, 2024

• fix (#117)

  lsy323

  authored
  6ec67e4

  Copy full SHA for 6ec67e4

Commits on Jul 18, 2024

• del prefill_result & update dev image (#116)

  Show description for c88d14d
  morgandu

  authored
  c88d14d

  Copy full SHA for c88d14d

Commits on Jul 17, 2024

• fix (#114)

  morgandu

  authored
  2b712af

  Copy full SHA for 2b712af

Commits on Jul 16, 2024

• update images (#113)

  morgandu

  authored
  8060d05

  Copy full SHA for 8060d05

• Cleanup orchestrator proto (#112)

  Show description for 46c152f
  JoeZijunZhou

  authored
  46c152f

  Copy full SHA for 46c152f

Commits on Jul 11, 2024

• Model warmup support with AOT and endpoint for JetStream (#92)

  Show description for 196beda
  vivianrwu

  authored
  196beda

  Copy full SHA for 196beda

Commits on Jul 9, 2024

• Bump zipp from 3.17.0 to 3.19.1 in the pip group (#111)

  Show description for 166dcd1
  dependabot[bot]

  authored
  166dcd1

  Copy full SHA for 166dcd1

Commits on Jul 8, 2024

• Bump certifi from 2024.2.2 to 2024.7.4 in the pip group (#110)

  Show description for 0db39c2
  dependabot[bot]

  authored
  0db39c2

  Copy full SHA for 0db39c2

Commits on Jul 2, 2024

• Add loadgen in dev image (#109)

  Show description for 69ce8a2
  morgandu

  authored
  69ce8a2

  Copy full SHA for 69ce8a2

Commits on Jun 28, 2024

• change the detokenization thread to return the actual eos token. (#108)

  Show description for 46e444c
  jwyang-google

  authored
  46e444c

  Copy full SHA for 46e444c

• Prefill return first token (#105)

  Show description for c3fe3ce
  jwyang-google

  authored
  c3fe3ce

  Copy full SHA for c3fe3ce

Commits on Jun 27, 2024

• Update docs with metrics observation instructions (#107)

  Show description for cd6eb2d
  

  Bslabe123

  and

  JoeZijunZhou

  authored
  cd6eb2d

  Copy full SHA for cd6eb2d

Commits on Jun 26, 2024

• Update docs for benchmark warmup mode (#106)

  Show description for 3ddc26f
  

  JoeZijunZhou

  and

  yeandy

  authored
  3ddc26f

  Copy full SHA for 3ddc26f

Commits on Jun 24, 2024

• Added `jetstream_transfer_backlog_size` and `jetstream_generate_backlog_size` metrics (#103)

  Show description for 7c1686e
  Bslabe123

  authored
  7c1686e

  Copy full SHA for 7c1686e

Commits on Jun 17, 2024

• Bump urllib3 from 2.2.0 to 2.2.2 in the pip group across 1 directory (#104)

  Show description for f83e042
  dependabot[bot]

  authored
  f83e042

  Copy full SHA for f83e042

Commits on Jun 12, 2024

• Change `jetstream_slots_available_percentage` to `jetstream_slots_used_percentage` (#102)

  Show description for 26872c3
  Bslabe123

  authored
  26872c3

  Copy full SHA for 26872c3

Commits on Jun 10, 2024

• Add profiling server for proxy backend (#101)

  Show description for 8a1e313
  

  zhihaoshan-google

  and

  Zhihao Shan

  authored
  8a1e313

  Copy full SHA for 8a1e313

• Add inference sampling utils in JetStream (#100)

  Show description for 120c68b
  JoeZijunZhou

  authored
  120c68b

  Copy full SHA for 120c68b

Commits on Jun 7, 2024

• add ssh support for profile (#99)

  FanhaiLu1

  authored
  cc19103

  Copy full SHA for cc19103

Commits on Jun 3, 2024

• fix typo (#98)

  morgandu

  authored
  e73ba53

  Copy full SHA for e73ba53

• Update benchmark config for xlml automation (#96)

  Show description for b1a1f6a
  morgandu

  authored
  b1a1f6a

  Copy full SHA for b1a1f6a

• Add tensorboard plugin dep for remote access (#97)

  JoeZijunZhou

  authored
  06eb8fc

  Copy full SHA for 06eb8fc

Commits on May 31, 2024

• Release v0.2.2 (#95)

  JoeZijunZhou

  authored
  ec26ec2

  Copy full SHA for ec26ec2

Commits on May 29, 2024

• Add JetStream E2E test CI (#89)

  Show description for dd896e7
  JoeZijunZhou

  authored
  dd896e7

  Copy full SHA for dd896e7

• Add healthcheck support for JetStream (#90)

  Show description for 0c56aac
  vivianrwu

  authored
  0c56aac

  Copy full SHA for 0c56aac

Commits on May 28, 2024

• Ensure server warmup before benchmark (#91)

  Show description for a223df9
  JoeZijunZhou

  authored
  a223df9

  Copy full SHA for a223df9

Commits on May 23, 2024

• Add ray disaggregated serving support (#87)

  Show description for e19a790
  FanhaiLu1

  authored
  e19a790

  Copy full SHA for e19a790

Commits on May 21, 2024

• Add profiling support and update docs (#85)

  Show description for eaf0d6e
  JoeZijunZhou

  authored
  eaf0d6e

  Copy full SHA for eaf0d6e

• --- (#86)

  Show description for 5d1e317
  dependabot[bot]

  authored
  5d1e317

  Copy full SHA for 5d1e317

Commits on May 20, 2024

• Decode Batch Percentage Metrics/Improved Scraping (#82)

  Show description for 44686b0
  Bslabe123

  authored
  44686b0

  Copy full SHA for 44686b0

Commits on May 17, 2024

• Allow tokenizer to customize stop_tokens (#84)

  qihqi

  authored
  8128c8a

  Copy full SHA for 8128c8a

Pagination

Previous

Next