A typo in a flag passed to azcopy by the Rust agent causes azcopy to display an error and the job to fail

[puhley]

Information

• Onefuzz version: 8.0.0
• OS: Linux

Provide detailed reproduction steps (if any)

1. When performing a libfuzzer merge, the job will fail with the error message provided below. The error message states that at this line of the Rust agent code https://github.com/microsoft/onefuzz/blob/main/src/agent/onefuzz/src/az_copy.rs#L193 , the flag --delete_destination is being specified. According to the Microsoft documentation, the correct flag is --delete-destination (https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-blobs-synchronize). That flag requires a parameter of true, false, or prompt to also be provided. Since an undefined flag is provided to azcopy, the command fails and azcopy dumps its help output.

For completeness, I checked for other occurrences in the code and the correct parameter and flag is used in the Python deploy script: https://github.com/microsoft/onefuzz/blob/main/src/deployment/deploy.py#L1016

Changing the flag in the /src/agent/onefuzz/src/az_copy.rs file from --delete_destination to --delete-destination should solve one part of the issue. The second part of this issue is that it requires a flag that is set to true, false, or prompt. In the Python code noted above, the true parameter is passed which I assume is what is required here.

This is an example error message. This failure happens a few times because the command occurs in a retry loop. Only the first error is shown below.

[2023-04-24 2:03:24.467591271 UTC] INFO: agent ready, dispatching task
[2023-04-24 2:03:24.467633972 UTC] task_start: event_type libfuzzer_merge
[2023-04-24 2:03:39.682223962 UTC] INFO: Merging corpus
[2023-04-25 3:36:10.001018097 UTC] INFO: Added #### new files to the corpus
[2023-04-25 3:36:10.471877815 UTC] INFO: azcopy attempt failed after 4.608774582s: azcopy sync attempt 1 failed.  (failure 1)

Caused by:
    azcopy sync failed src:"task_uniqueinputs_0/" dst:"https://{fuzzcontainer}.blob.core.windows.net/corpus-merge-test?...&sig=REDACTED"
stdout:"unknown flag: --delete_destination"
stderr:"Error: unknown flag: --delete_destination
Usage:
  azcopy sync [flags]

Aliases:
  sync, sc, s

Examples:

-- This section of the error message is just the remaining output of the azcopy sync -h command line. I am removing it for brevity. The error message then continues showing the crash in the OneFuzz azcopy sync functionality:

" log:"unable to read azcopy log file from: no log file in path: /tmp/.tmpVJNpVl

Stack backtrace:
   0: <backoff::future::Retry<S,B,N,Fn,Fut> as core::future::future::Future>::poll
   1: onefuzz::az_copy::retry_az_impl::{{closure}}
   2: onefuzz::syncdir::SyncedDir::sync::{{closure}}
   3: onefuzz_task::tasks::config::Config::run::{{closure}}
   4: <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
   5: onefuzz_task::managed::cmd::run::{{closure}}
   6: onefuzz_task::run::{{closure}}
   7: tokio::runtime::park::CachedParkThread::block_on
   8: tokio::runtime::runtime::Runtime::block_on
   9: onefuzz_task::main
  10: std::sys_common::backtrace::__rust_begin_short_backtrace
  11: std::rt::lang_start::{{closure}}
  12: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/ops/function.rs:287:13
      std::panicking::try::do_call
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:483:40
      std::panicking::try
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panic.rs:140:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:483:40
      std::panicking::try
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panic.rs:140:14
      std::rt::lang_start_internal
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/rt.rs:148:20
  13: main
  14: __libc_start_main
             at /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
  15: _start"

Stack backtrace:
   0: <backoff::future::Retry<S,B,N,Fn,Fut> as core::future::future::Future>::poll
   1: onefuzz::az_copy::retry_az_impl::{{closure}}
   2: onefuzz::syncdir::SyncedDir::sync::{{closure}}
   3: onefuzz_task::tasks::config::Config::run::{{closure}}
   4: <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
   5: onefuzz_task::managed::cmd::run::{{closure}}
   6: onefuzz_task::run::{{closure}}
   7: tokio::runtime::park::CachedParkThread::block_on
   8: tokio::runtime::runtime::Runtime::block_on
   9: onefuzz_task::main
  10: std::sys_common::backtrace::__rust_begin_short_backtrace
  11: std::rt::lang_start::{{closure}}
  12: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/ops/function.rs:287:13
      std::panicking::try::do_call
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:483:40
      std::panicking::try
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panic.rs:140:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:483:40
      std::panicking::try
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panic.rs:140:14
      std::rt::lang_start_internal
             at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/rt.rs:148:20
  13: main
  14: __libc_start_main
             at /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
  15: _start

Expected result

The proper flag with the proper setting should be provided to the azcopy sync command allowing the job to succeed.

Actual result

A typo results in an invalid flag being provided to the azcopy sync command which causes the job to fail.

AB#140868