Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Logs] Streamlining log-tailing sequence #44658

Merged
merged 5 commits into from
Apr 12, 2024
Merged

[Logs] Streamlining log-tailing sequence #44658

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
51bf04c
Revisited log-tailing sequence to
alexeykudinkin Apr 11, 2024
abd563f
`lint`
alexeykudinkin Apr 11, 2024
7717615
Fixed NPE
alexeykudinkin Apr 11, 2024
81f8b6c
Fixed tests
alexeykudinkin Apr 11, 2024
7a473c9
`lint`
alexeykudinkin Apr 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixed NPE 
Signed-off-by: Alexey Kudinkin <ak@anyscale.com>
  • Loading branch information
@alexeykudinkin
alexeykudinkin committed Apr 11, 2024
commit 77176155ddd4770cfc382733a053812501c494be
11 changes: 3 additions & 8 deletions dashboard/modules/job/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,6 @@ def file_tail_iterator(path: str) -> Iterator[Optional[List[str]]]:
curr_line = None

while True:
if curr_line is None:
# Only read the next line in the file
# if there's no remaining "curr_line" to process
curr_line = f.readline()

# We want to flush current chunk in following cases:
# - We accumulated 10 lines
# - We accumulated at least MAX_CHUNK_CHAR_LENGTH total chars
Expand All @@ -99,7 +94,9 @@ def file_tail_iterator(path: str) -> Iterator[Optional[List[str]]]:

lines = []
chunk_char_count = 0
curr_line = None

# Read next line
curr_line = f.readline()

# `readline` will return
# - '' for EOF
Expand All @@ -108,11 +105,9 @@ def file_tail_iterator(path: str) -> Iterator[Optional[List[str]]]:
# Add line to current chunk
lines.append(curr_line)
chunk_char_count += len(curr_line)
curr_line = None
else:
# If EOF is reached sleep for 1s before continuing
time.sleep(1)
curr_line = None


async def parse_and_validate_request(
Expand Down