fix logrotate empty file hole issue

[wnpllrzodiac]

refer to #1554

[winlinvip]

trunk/etc/logrotate.d/srs

Can you only submit the modifications to the log code? It would be better to place this file in the wiki or issue, as everyone's logrotate configuration may be different.

TRANS_BY_GPT3

[wnpllrzodiac]

logrotate file removed from the repo.

/etc/logrotate.d./srs

/data/log/srs/*.log {

weekly

size 200M

copytruncate

rotate 5

dateext

noolddir

}

[winlinvip]

I need to update the document. I will take some time to update the document first before merging this PR. Thanks again 👍

TRANS_BY_GPT3

[winlinvip]

I think this logrotate issue causing file fragmentation should be resolved. It's great!

After thinking about this, the previous logic was to create a file if it doesn't exist when opening it. I wonder if it would be better to merge it into a single "open" operation? Perhaps this would be a more appropriate way to make the change.

fd = ::open(filename.c_str(), O_RDWR | O_APPEND);
if(fd == -1 && errno == ENOENT) {
  fd = open(filename.c_str(),  O_RDWR | O_CREAT | O_TRUNC, ...

Could you consider this improvement?

TRANS_BY_GPT3

[winlinvip]

I took a look, and these two sentences can be combined as: O_RDWR | O_CREAT | O_APPEND:

1. If it doesn't exist, open it; if it does, append without truncating.
2. logrotate requires APPEND, refer to: https://unix.stackexchange.com/questions/475524/how-copytruncate-actually-works

Note: It is not recommended to use the copytruncate method. It is advised to upgrade to SRS3 and use the SIGUSR1 method to reopen the log files. Please note that copytruncate has an inherent race condition. It is possible that the writer may append a line to the log file just after logrotate has finished copying and before it has issued the truncate operation. This race condition can result in the loss of those lines of log permanently. That is why rotating logs using copytruncate is usually not recommended unless it is the only possible way to do it.

Below is an example program for reference:

#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
int main(int argc, char** argv) {
    printf("O_RDONLY=0x%02x, O_WRONLY=0x%02x, O_RDWR=0x%02x, O_APPEND=0x%04x, O_CREAT=0x%02x, O_TRUNC=0x%04x\n",
        O_RDONLY, O_WRONLY, O_RDWR, O_APPEND, O_CREAT, O_TRUNC);
    printf("S_IRUSR=0x%04x, S_IWUSR=0x%02x, S_IRGRP=0x%02x, S_IWGRP=0x%02x, S_IROTH=0x%02x\n",
        S_IRUSR, S_IWUSR, S_IRGRP, S_IWGRP, S_IROTH);

    char* filename = argv[1];
    int flags = O_RDWR | O_CREAT | O_APPEND;
    int options = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH;
    int fd = ::open(filename, flags, options);
    printf("open %s with flags=0x%04x, options=0x%04x, errno=%d, fd=%d\n", filename, flags, options, errno, fd);
    write(fd, "Hello, world!\n", 14);
    ::close(fd);
    return 0;
}

First write:

[root@b190805829e6 trunk]# g++ test.cpp -o test && ./test t.log
O_RDONLY=0x00, O_WRONLY=0x01, O_RDWR=0x02, O_APPEND=0x0400, O_CREAT=0x40, O_TRUNC=0x0200
S_IRUSR=0x0100, S_IWUSR=0x80, S_IRGRP=0x20, S_IWGRP=0x10, S_IROTH=0x04
open t.log with flags=0x0442, options=0x01b4, errno=0, fd=3

[root@b190805829e6 trunk]# cat t.log 
Hello, world!

Continuing to write if the file exists for the second time:

[root@b190805829e6 trunk]# ll t.log 
-rw-r--r-- 1 root root 14 Jan  8 09:31 t.log
[root@b190805829e6 trunk]# g++ test.cpp -o test && ./test t.log
O_RDONLY=0x00, O_WRONLY=0x01, O_RDWR=0x02, O_APPEND=0x0400, O_CREAT=0x40, O_TRUNC=0x0200
S_IRUSR=0x0100, S_IWUSR=0x80, S_IRGRP=0x20, S_IWGRP=0x10, S_IROTH=0x04
open t.log with flags=0x0442, options=0x01b4, errno=0, fd=3

[root@b190805829e6 trunk]# cat t.log 
Hello, world!
Hello, world!

TRANS_BY_GPT3

[winlinvip]

Wiki:

1. https://github.com/ossrs/srs/wiki/v3_CN_LogRotate#copytruncate
2. https://github.com/ossrs/srs/wiki/v3_EN_LogRotate#copytruncate

I will first merge this PR and then make one more code modification. Thanks @wnpllrzodiac

TRANS_BY_GPT3

[winlinvip]

Improvement PR: 731e878
Merged two opens.

TRANS_BY_GPT3

[wnpllrzodiac]

I remember discussing this issue with the author in the issue thread. I have submitted a pull request, you can search for it. You mentioned 'one of them,' but I'm not quite clear, are there no issues with the others? Get BlueMail for Android

On July 2, 2020, at 09:55, Qiao Lin ***@***.***> wrote: I deployed version 3.0 of the k8s cluster, and on one of the host machines, I used the copytruncate method to rotate the logs. However, I encountered the phenomenon of sparse files. Is it possible that version 3.0 release 0 does not support the copytruncate method yet, and currently only supports version 2.0? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1561 (comment)

`TRANS_BY_GPT3`