How to use tokio::fs::File in sync way?

[linrongbin16]

tokio::fs::File provide many APIs that let user operate on a file. And they are all async version of std::fs::File.

My application starts tokio::Runtime::new().block_on(...) right after the main function entry of the program, so the whole application uses tokio async runtime. Most of the time they are working great.

But in such async runtime, sometimes I have special needs that I need to call tokio APIs in a sync function. I known this is may block the thread, but it is a must for my program. So how should I call tokio::fs::File async APIs within a sync function, running inside a tokio runtime?

I actually have several possible solutions (but still not sure which one is the best/recommended):

TryIntoStd/FromStd

tokio::fs::File provides two API that can convert the File structure into a raw file fd:

For Windows, it is the file handle, which should be a "pointer", but here in this discussion, let's ignore Windows to make the discussion easier.

• try_into_std
• from_std

With these 2 APIs, I could first convert tokio::fs::File into std::fs::File, std Files are pure sync, so I can read/write files in a sync caller. After everything done, I can convert std::fs::File back to tokio::fs::File.

One more question is: I found there are some buffers and locks inside tokio::fs::File:

tokio/tokio/src/fs/file.rs

Lines 90 to 94 in 5dacc2e

	pub struct File {
	std: Arc<StdFile>,
	inner: Mutex<Inner>,
	max_buf_size: usize,
	}

tokio/tokio/src/fs/file.rs

Lines 489 to 521 in 5dacc2e

	pub async fn into_std(mut self) -> StdFile {
	self.inner.get_mut().complete_inflight().await;
	Arc::try_unwrap(self.std).expect("Arc::try_unwrap failed")
	}
	/// Tries to immediately destructure `File` into a [`std::fs::File`].
	///
	/// # Errors
	///
	/// This function will return an error containing the file if some
	/// operation is in-flight.
	///
	/// # Examples
	///
	/// ```no_run
	/// use tokio::fs::File;
	///
	/// # async fn dox() -> std::io::Result<()> {
	/// let tokio_file = File::open("foo.txt").await?;
	/// let std_file = tokio_file.try_into_std().unwrap();
	/// # Ok(())
	/// # }
	/// ```
	#[allow(clippy::result_large_err)]
	pub fn try_into_std(mut self) -> Result<StdFile, Self> {
	match Arc::try_unwrap(self.std) {
	Ok(file) => Ok(file),
	Err(std_file_arc) => {
	self.std = std_file_arc;
	Err(self)
	}
	}
	}

Will tokio release the inner buffer inside?

Does Tokio Runtime provide some helper functions?

Does tokio runtime provide some helper functions, that can run a blocking_spawn or something?

Or I can simply use below code?

fn my_sync_function(filename: &Path) -> tokio::fs::File {
  tokio::runtime::Handle::current().block_on(async {
    tokio::fs::File::open(filename).await
  })
}

Reference

Here is how I send a message to tokio::sync::mpsc::Receiver in a sync function:

  tokio::runtime::Handle::current().spawn_blocking(move || {
    master_tx.blocking_send(message).unwrap();
  })

1. With tokio::runtime::handle::current(), I can get the "global" runtime my application is running inside.
2. With tokio::sync::mpsc::Sender::blocking_send, I can send the message in a blocking/sync way, thus I can use this code inside a pure sync function.

[Darksonn]

You can do something along these lines:

let std_file = Handle::current().block_on(tokio_file.into_std());
std_file.write(...);
let tokio_file = File::from_std(std_file);

[linrongbin16]

Hi, @Darksonn thanks for reply.

Just to confirm, so "Runtime.block_on" api can be used multiple times and nested, correct? I mean, I can write code like this:

fn sync_logic(filepath: &Path) -> std::fs::File {
  let filepath = filepath.to_path_buf();
  let handle = tokio::runtime::Handle::current();

  // This is the 2nd time calling the "block_on" api.
  // I can use multiple "block_on" nested, correct?

  handle.block_on(async move {
    let file = tokio::fs::File::open(&filepath).await;
    file.into_std().await
  })
}

fn main() {
 let runtime = tokio::Runtime::new();

 // This is the 1st time to call "block_on" api.
 // It is the entry of whole application.

 runtime.block_on(async {
  let filename = "README.me";
  let filename = Path::new(filename);

  // Image we have a lot of logic here, even in this example, it seems stupid.

  let text = sync_logic(&filename);

  // Continue, image here we have a lot of logic...
 });
}

[Darksonn]

You need to do something special to nest it. For example, you might do this:

let text = block_in_place(|| sync_logic(&filename));

[linrongbin16]

Ok, after check this doc: https://docs.rs/tokio/latest/tokio/task/fn.block_in_place.html

Let me write this example again:

fn sync_logic(filepath: &Path) -> std::fs::File {
  let filepath = filepath.to_path_buf();
  let handle = tokio::runtime::Handle::current();

  // This is the 2nd time calling the "block_on" api.
  // Wrap it with a "block_in_place"

  tokio::task::block_in_place(move || {
    handle.block_on(async move {
      let file = tokio::fs::File::open(&filepath).await;
      file.into_std().await
    })
  })
}

fn main() {
 let runtime = tokio::Runtime::new();

 // This is the 1st time to call "block_on" api.
 // It is the entry of whole application.

 runtime.block_on(async {
  let filename = "README.me";
  let filename = Path::new(filename);

  // Image we have a lot of logic here, even in this example, it seems stupid.

  let file = sync_logic(&filename);

  // Continue, image here we have a lot of logic...
 });
}