Recreate current rolling file immediately to log on file deleting

[asset-2]

It is not a new issue. It partially interacts with #36, #87, #96.

Service uses an hour as rolling interval. Current log file has been deleted in-between. Logger waits till chosen interval ends and then only creates new file to log things. All events and messages are ignored during remained time. Expected behavior for me is to recreate file immediately and write to it, as soon as we have something to log.

Do you guys plan to implement this thing? What possible workarounds do you suggest?

[cocowalla]

Do you mean you are manually deleting the log file during the period that Serilog would expect to be able to write to it?

[asset-2]

Yes, it's possible when you run your application in Docker container for example.

[cocowalla]

Hmm, I'm not getting how running in a container deletes log files during writing - would appreciate if you could describe a scenario when this happens?

[asset-2]

No, 'running in container' does not delete log files during writing.
Application is running in Docker container. It performs logging into file. It's possible to get access to that log file and delete it manually. I was expecting that file will be recreated immediately, as I wrote before. But no. It creates a new one when rolling interval is over. In this scenario we lose a bunch of messages.

[cocowalla]

It's possible to get access to that log file and delete it manually

OK, but why would you do that? 😄

I'm trying to figure out if this is a realistic, common thing to happen, or an edge case where you have to actively be trying to cause a problem for it to be an issue.

Also, as an aside, if you configure the sink using shared: false, it will hold an exclusive lock on the log file during writing, preventing you from deleting it.

[asset-2]

Also, as an aside, if you configure the sink using shared: false, it will hold an exclusive lock on the log file during writing, preventing you from deleting it.

I will give a try. But something tells me that this approach does not work.

OK, but why would you do that? 😄

Someone accidentally removes log file on the production... The data has been lost. At that point we have embarked investigation.
As you see, it is pretty realistic occasion. From my perspective it is pretty evident to logger to proceed without interruption. The client loses its upcomming data for remained rolling period when human/system deletes log file unintentionally or not . Well, I really do not know whether that thing fits well with the strategy that you guys keep.

[cocowalla]

But something tells me that this approach does not work

I don't know about the intricacies of the Linux file system, but it absolutely works on Windows.

Well, I really do not know whether that thing fits well with the strategy that you guys keep

Just a little piece of unsolicited advice: being rude to people will not get you what you want from them, especially if they are maintaining OSS in their spare time, for free.

If I'm being honest, I'm not really seeing how this could be s a realistic or common problem to be dealt with by a logging framework. That said, it would be nice if we didn't lose messages in the rare event that somebody explicitly deletes a file that wasn't locked.

The most obvious ways to do this (e.g. checking if the file exists before every write) are likely to hurt performance badly, so we need to be careful about that.

Do you have any suggestions for an approach?

[nblumhardt]

Hi all! @asset-2 thanks for raising this as a separate issue; I see it's related to #96 but not covered by it, guess we track these two things separately. Thanks for digging in @cocowalla!

Well, I really do not know whether that thing fits well with the strategy that you guys keep

Just a little piece of unsolicited advice: being rude to people will not get you what you want from them, especially if they are maintaining OSS in their spare time, for free.

I think you might be misunderstanding each other, I can see both interpretations of that statement and think that From my perspective is intended to put the later statement in context - does this fit with the goals of the project? (I'm not sure what the answer is, at this point :-))

#96 won't be an issue on Unix, as file "locks" are only advice on that platform, unlike Windows where a lock is enforced. Because of that, this ticket is something of an inverse to that problem.

I think we could attempt to implement this, but because the file will still "exist" from the app's perspective on Unix (it will just be unlinked from its name in the directory hierarchy, until the process closes the file handle), we can't do it in response to an exception and instead, this would necessitate some kind of monitor process, AFAIK (need to grab my copy of Kerrisk's book to confirm!).

I'm not sure it will be worth that amount of effort, but keeping an open mind if someone is interested in exploring it. To answer your original questions, @asset-2 👍

Do you guys plan to implement this thing?

Not at this point, sorry.

What possible workarounds do you suggest?

At this point it's probably an operational problem to solve (avoid accidentally deleting log files by using automated runbooks/scripts to manage production servers).

[asset-2]

@cocowalla @nblumhardt Thank you guys for the questions consideration.

[qiqo]

I need this feature and my use case is that I move logs to another storage on a regular schedule for a bank application, so we roll logs daily, but audit happens every 2 hrs, and we have to remove the processed logs out without restarting the application. For perspective, log4net and NLog does this, but we chose serilog because it just works - except for recreating deleted logs that it has handle to. I'm wondering if adding a conditional to check if a file exist would fix it.

[rspears74]

Hi, just wanted to chime in and say that I'm getting around this issue by calling Serilog.Log.CloseAndFlush(), deleting the file with System.IO.File.Delete(file) (after a System.IO.File.Exists(file) check), then recreating if I need to by calling my Log.Logger = new LoggerConfiguration()... code again. We upload log files with user-submitted error reports, and then immediately delete the file. We usually do this as the application is closing, but recreating seems to work just fine in my testing.