Skip to content

Conversation

Ghass-M
Copy link
Collaborator

@Ghass-M Ghass-M commented Aug 27, 2025

Summary

Introduces a centralized slicing queue to enforce FIFO ordering and bounded concurrency across uploads, and resets adaptive upload parameters on session changes.

Key changes

  • New UploadSlicingManager: Adds IUploadSlicingManager and UploadSlicingManager using a Channel<Func<Task>>, FIFO processing, and max concurrency of 2; ignores stale work after session resets.
  • Session-aware adaptive reset: AdaptiveUploadController subscribes to ISessionService.SessionObservable and fully resets chunk size, parallelism, and window.
  • DI and factory wiring: Registers AdaptiveUploadController and UploadSlicingManager in SingletonsModule; resolves and injects IUploadSlicingManager in FileUploadProcessorFactory.
  • Processor integration: FileUploadProcessor enqueues slicing via IUploadSlicingManager instead of starting it directly to ensure ordered execution.
  • Tests updated: Adjusts unit tests to use SliceAndEncryptAdaptiveAsync, injects and stubs IUploadSlicingManager, and wires ISessionService mock for the controller.

Rationale

  • Problem: Parallel slicers could interleave across files, causing contention and non-deterministic ordering; adaptive state leaked across sessions.
  • Solution: Centralize slicing with FIFO + bounded concurrency and reset adaptive state on session change.

Notes

  • Backward compatibility: No public API changes; behavior is more predictable under load.
  • Configurability: MAX_CONCURRENT_SLICES is set to 2 and can be tuned later.

@Ghass-M Ghass-M requested a review from paul-fresquet August 28, 2025 15:30
@paul-fresquet paul-fresquet requested a review from Copilot August 29, 2025 07:29
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces centralized upload slicing management to enforce FIFO ordering and bounded concurrency, plus session-aware adaptive parameter reset functionality. The implementation aims to prevent parallel slicer contention and ensure deterministic upload behavior under load.

  • Adds UploadSlicingManager with FIFO queue processing and configurable concurrency limits
  • Implements session-aware reset functionality in AdaptiveUploadController to clear state on session changes
  • Updates dependency injection and factory patterns to wire the new components

Reviewed Changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/ByteSync.Client/Services/Communications/Transfers/Uploading/UploadSlicingManager.cs New centralized slicing manager with Channel-based FIFO queue and bounded worker concurrency
src/ByteSync.Client/Services/Communications/Transfers/Uploading/AdaptiveUploadController.cs Adds session service subscription for state reset and improved downscaling logic
src/ByteSync.Client/Services/Communications/Transfers/Uploading/FileUploadProcessor.cs Integrates slicing manager to enqueue slicing operations instead of direct execution
src/ByteSync.Client/Services/Communications/Transfers/Uploading/FileSlicer.cs Adds backpressure control based on pending slice count vs upload parallelism
src/ByteSync.Client/DependencyInjection/Modules/SingletonsModule.cs Registers new components as singletons in DI container
src/ByteSync.Client/Factories/FileUploadProcessorFactory.cs Updates factory to resolve and inject slicing manager
tests/ByteSync.Client.Tests/Services/Communications/Transfers/Uploading/*Tests.cs Updates unit tests for new slicing manager integration and adaptive method usage

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines 60 to 73
_workers.Add(Task.Run(async () =>
{
await foreach (var work in _queue.Reader.ReadAllAsync())
{
try
{
await work();
}
catch (Exception ex)
{
_logger.LogError(ex, "Slicing worker error");
}
}
}));
Copy link
Preview

Copilot AI Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worker tasks are started with Task.Run but never awaited or tracked for completion. If workers fail or the channel completes, they won't be restarted. Consider monitoring worker health and restarting failed workers.

Copilot uses AI. Check for mistakes.

Copy link
Contributor

@paul-fresquet paul-fresquet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job 👍 Here are a few comments


public interface IUploadSlicingManager
{
Task Enqueue(Func<Task> startSlicingAsync);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: the signature might be: Task<UploadProgressState> Enqueue(SharedFileDefinition sharedFileDefinition).

UploadSlicingManager would then instanciate the FileSlicer, the UploadProgressState and return the UploadProgressState.

Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants