Skip to content

Commit 0f96e0a

Browse files
authored
Transport refactor and winston 3.x compatibility (#107)
* refactoring transport to use third-party file rotation component * adding winston 3.x compatibility
1 parent 0e5ca89 commit 0f96e0a

File tree

12 files changed

+1901
-1705
lines changed

12 files changed

+1901
-1705
lines changed

.eslintrc.js

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
module.exports = exports = {
2+
extends: 'xo',
3+
env: {
4+
node: true,
5+
mocha: true
6+
},
7+
rules: {
8+
indent: ['error', 4],
9+
camelcase: ['error', {properties: 'never'}]
10+
}
11+
};

.gitignore

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
# Logs
2-
logs
3-
*.log
4-
*.log.*
2+
*log*
3+
test.js
54

65
# Runtime data
76
pids

.travis.yml

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,14 @@
22
sudo: false
33
language: node_js
44
node_js:
5-
- "0.10"
6-
- "0.12"
7-
- "4.1"
5+
- "4"
6+
- "6"
7+
- "8"
8+
9+
env:
10+
- WINSTON_VER=winston@^2
11+
- WINSTON_VER=winston@next
12+
13+
before_install:
14+
- npm install $WINSTON_VER
15+

README.md

Lines changed: 29 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,41 @@
22

33
[![NPM version][npm-image]][npm-url] [![Build Status][travis-image]][travis-url] [![Dependency Status][daviddm-image]][daviddm-url]
44

5-
> A transport for winston which logs to a rotating file each day.
5+
A transport for [winston](https://github.com/winstonjs/winston) which logs to a rotating file. Logs can be rotated based on a date, size limit, and old logs can be removed based on count or elapsed days.
66

7-
## Usage
7+
Starting with version 2.0.0, the transport has been refactored to leverage the the [file-stream-rotator](https://github.com/rogerc/file-stream-rotator/) module. _Some of the options in the 1.x versions of the transport have changed._ Please review the options below to identify any changes needed.
8+
9+
## Install
10+
```
11+
npm install winston-daily-rotate-file
12+
```
813

14+
## Options
15+
The DailyRotateFile transport can rotate files by minute, hour, day, month, year or weekday. In addition to the options accepted by the logger, `winston-daily-rotate-file` also accepts the following options:
16+
17+
* **datePattern:** A string representing the [moment.js date format](http://momentjs.com/docs/#/displaying/format/) to be used for rotating. The meta characters used in this string will dictate the frequency of the file rotation. For example, if your datePattern is simply 'HH' you will end up with 24 log files that are picked up and appended to every day. (default 'YYYY-MM-DD')
18+
* **zippedArchive:** A boolean to define whether or not to gzip archived log files. (default 'false')
19+
* **filename:** Filename to be used to log to. This filename can include the `%DATE%` placeholder which will include the formatted datePattern at that point in the filename. (default: 'winston.log.%DATE%)
20+
* **dirname:** The directory name to save log files to. (default: '.')
21+
* **stream:** Write directly to a custom stream and bypass the rotation capabilities. (default: null)
22+
* **maxSize:** Maximum size of the file after which it will rotate. This can be a number of bytes, or units of kb, mb, and gb. If using the units, add 'k', 'm', or 'g' as the suffix. The units need to directly follow the number. (default: null)
23+
* **maxFiles:** Maximum number of logs to keep. If not set, no logs will be removed. This can be a number of files or number of days. If using days, add 'd' as the suffix. (default: null)
24+
25+
## Usage
926
``` js
1027
var winston = require('winston');
1128
require('winston-daily-rotate-file');
1229

1330
var transport = new (winston.transports.DailyRotateFile)({
14-
filename: './log',
15-
datePattern: 'yyyy-MM-dd.',
16-
prepend: true,
17-
level: process.env.ENV === 'development' ? 'debug' : 'info'
31+
filename: 'application-%DATE%.log',
32+
datePattern: 'YYYY-MM-DD-HH',
33+
zippedArchive: true,
34+
maxSize: '20m',
35+
maxFiles: '14d'
36+
});
37+
38+
transport.on('rotate', function(oldFilename, newFilename) {
39+
// do something fun
1840
});
1941

2042
var logger = new (winston.Logger)({
@@ -26,30 +48,7 @@
2648
logger.info('Hello World!');
2749
```
2850

29-
The DailyRotateFile transport can rotate files by minute, hour, day, month, year or weekday. In addition to the options accepted by the File transport, the Daily Rotate File Transport also accepts the following options:
30-
31-
* __datePattern:__ A string representing the pattern to be used when appending the date to the filename (default 'yyyy-MM-dd'). The meta characters used in this string will dictate the frequency of the file rotation. For example, if your datePattern is simply 'HH' you will end up with 24 log files that are picked up and appended to every day.
32-
* __prepend:__ Defines if the rolling time of the log file should be prepended at the beginning of the filename (default 'false').
33-
* __localTime:__ A boolean to define whether time stamps should be local (default 'false' means that UTC time will be used).
34-
* __zippedArchive:__ A boolean to define whether or not to gzip archived log files (default 'false').
35-
* __maxDays:__ A number representing the maximum number of days a log file will be saved. Any log file older than this specified number of days will be removed. If not value or a 0, no log files will be removed (default 0).
36-
* __createTree:__ When combined with a `datePattern` that includes path delimiters, the transport will create the entire folder tree to the log file. Example: `datePattern: '/yyyy/MM/dd.log', createTree: true` will create the entire path to the log file prior to writing an entry.
37-
38-
Valid meta characters in the datePattern are:
39-
40-
* __yy:__ Last two digits of the year.
41-
* __yyyy:__ Full year.
42-
* __M:__ The month.
43-
* __MM:__ The zero padded month.
44-
* __d:__ The day.
45-
* __dd:__ The zero padded day.
46-
* __H:__ The hour.
47-
* __HH:__ The zero padded hour.
48-
* __m:__ The minute.
49-
* __mm:__ The zero padded minute.
50-
* __ddd:__ The weekday (Mon, Tue, ..., Sun).
51-
52-
*Metadata:* Logged via util.inspect(meta);
51+
You can listen for the *rotate* custom event. The rotate event will pass two parameters to the callback (*oldFilename*, *newFilename*).
5352

5453
## LICENSE
5554
MIT

daily-rotate-file.js

Lines changed: 260 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
'use strict';
2+
3+
var fs = require('fs');
4+
var os = require('os');
5+
var path = require('path');
6+
var util = require('util');
7+
var semver = require('semver');
8+
var zlib = require('zlib');
9+
var winston = require('winston');
10+
var compat = require('winston-compat');
11+
var MESSAGE = require('triple-beam').MESSAGE;
12+
var PassThrough = require('stream').PassThrough;
13+
var Transport = semver.major(winston.version) === 2 ? compat.Transport : require('winston-transport');
14+
15+
var loggerDefaults = {
16+
json: false,
17+
colorize: false,
18+
eol: os.EOL,
19+
logstash: null,
20+
prettyPrint: false,
21+
label: null,
22+
stringify: false,
23+
depth: null,
24+
showLevel: true,
25+
timestamp: function () {
26+
return new Date().toISOString();
27+
}
28+
};
29+
30+
var DailyRotateFile = function (options) {
31+
options = options || {};
32+
Transport.call(this, options);
33+
34+
function throwIf(target /* , illegal... */) {
35+
Array.prototype.slice.call(arguments, 1).forEach(function (name) {
36+
if (options[name]) {
37+
throw new Error('Cannot set ' + name + ' and ' + target + ' together');
38+
}
39+
});
40+
}
41+
42+
function getMaxSize(size) {
43+
if (size && typeof size === 'string') {
44+
var _s = size.toLowerCase().match(/^((?:0\.)?\d+)([k|m|g])$/);
45+
if (_s) {
46+
return size;
47+
}
48+
} else if (size && Number.isInteger(size)) {
49+
var sizeK = Math.round(size / 1024);
50+
return sizeK === 0 ? '1k' : sizeK + 'k';
51+
}
52+
return null;
53+
}
54+
55+
this.options = Object.assign({}, loggerDefaults, options);
56+
57+
if (options.stream) {
58+
throwIf('stream', 'filename', 'maxsize');
59+
this.logStream = new PassThrough();
60+
this.logStream.pipe(options.stream);
61+
} else {
62+
this.filename = options.filename ? path.basename(options.filename) : 'winston.log';
63+
this.dirname = options.dirname || path.dirname(options.filename);
64+
65+
var self = this;
66+
67+
this.logStream = require('file-stream-rotator').getStream({
68+
filename: path.join(this.dirname, this.filename),
69+
frequency: 'custom',
70+
date_format: options.datePattern ? options.datePattern : 'YYYY-MM-DD',
71+
verbose: false,
72+
size: getMaxSize(options.maxSize),
73+
max_logs: options.maxFiles
74+
});
75+
76+
this.logStream.on('rotate', function (oldFile, newFile) {
77+
self.emit('rotate', oldFile, newFile);
78+
});
79+
80+
if (options.zippedArchive) {
81+
this.logStream.on('rotate', function (oldFile) {
82+
var gzip = zlib.createGzip();
83+
var inp = fs.createReadStream(oldFile);
84+
var out = fs.createWriteStream(oldFile + '.gz');
85+
inp.pipe(gzip).pipe(out).on('finish', function () {
86+
fs.unlinkSync(oldFile);
87+
});
88+
});
89+
}
90+
}
91+
};
92+
93+
module.exports = DailyRotateFile;
94+
95+
util.inherits(DailyRotateFile, Transport);
96+
97+
DailyRotateFile.prototype.name = 'dailyRotateFile';
98+
99+
var noop = function () {};
100+
if (semver.major(winston.version) === 2) {
101+
DailyRotateFile.prototype.log = function (level, msg, meta, callback) {
102+
callback = callback || noop;
103+
var options = Object.assign({}, this.options, {
104+
level: level,
105+
message: msg,
106+
meta: meta
107+
});
108+
109+
var output = compat.log(options) + options.eol;
110+
this.logStream.write(output);
111+
callback(null, true);
112+
};
113+
} else {
114+
DailyRotateFile.prototype.normalizeQuery = compat.Transport.prototype.normalizeQuery;
115+
DailyRotateFile.prototype.log = function (info, callback) {
116+
callback = callback || noop;
117+
118+
this.logStream.write(info[MESSAGE] + this.options.eol);
119+
this.emit('logged', info);
120+
callback(null, true);
121+
};
122+
}
123+
124+
DailyRotateFile.prototype.close = function () {
125+
var self = this;
126+
if (this.logStream) {
127+
this.logStream.end(function () {
128+
self.emit('finish');
129+
});
130+
}
131+
};
132+
133+
DailyRotateFile.prototype.query = function (options, callback) {
134+
if (typeof options === 'function') {
135+
callback = options;
136+
options = {};
137+
}
138+
139+
if (!this.filename) {
140+
throw new Error('query() may not be used when initializing with a stream');
141+
}
142+
143+
var self = this;
144+
var results = [];
145+
var row = 0;
146+
options = self.normalizeQuery(options);
147+
148+
var logFiles = (function () {
149+
var fileRegex = new RegExp(self.filename.replace('%DATE%', '.*'), 'i');
150+
return fs.readdirSync(self.dirname).filter(function (file) {
151+
return path.basename(file).match(fileRegex);
152+
});
153+
})();
154+
155+
if (logFiles.length === 0 && callback) {
156+
callback(null, results);
157+
}
158+
159+
(function processLogFile(file) {
160+
if (!file) {
161+
return;
162+
}
163+
var logFile = path.join(self.dirname, file);
164+
var buff = '';
165+
166+
var stream = fs.createReadStream(logFile, {
167+
encoding: 'utf8'
168+
});
169+
170+
stream.on('error', function (err) {
171+
if (stream.readable) {
172+
stream.destroy();
173+
}
174+
if (!callback) {
175+
return;
176+
}
177+
return err.code === 'ENOENT' ? callback(null, results) : callback(err);
178+
});
179+
180+
stream.on('data', function (data) {
181+
data = (buff + data).split(/\n+/);
182+
var l = data.length - 1;
183+
184+
for (var i = 0; i < l; i++) {
185+
if (!options.start || row >= options.start) {
186+
add(data[i]);
187+
}
188+
row++;
189+
}
190+
191+
buff = data[l];
192+
});
193+
194+
stream.on('close', function () {
195+
if (buff) {
196+
add(buff, true);
197+
}
198+
199+
if (options.order === 'desc') {
200+
results = results.reverse();
201+
}
202+
203+
if (logFiles.length) {
204+
processLogFile(logFiles.shift());
205+
} else if (callback) {
206+
callback(null, results);
207+
}
208+
});
209+
210+
function add(buff, attempt) {
211+
try {
212+
var log = JSON.parse(buff);
213+
if (check(log)) {
214+
push(log);
215+
}
216+
} catch (e) {
217+
if (!attempt) {
218+
stream.emit('error', e);
219+
}
220+
}
221+
}
222+
223+
function check(log) {
224+
if (!log || typeof log !== 'object') {
225+
return;
226+
}
227+
228+
var time = new Date(log.timestamp);
229+
if ((options.from && time < options.from) || (options.until && time > options.until)) {
230+
return;
231+
}
232+
233+
return true;
234+
}
235+
236+
function push(log) {
237+
if (options.rows && results.length >= options.rows && options.order !== 'desc') {
238+
if (stream.readable) {
239+
stream.destroy();
240+
}
241+
return;
242+
}
243+
244+
if (options.fields) {
245+
var obj = {};
246+
options.fields.forEach(function (key) {
247+
obj[key] = log[key];
248+
});
249+
log = obj;
250+
}
251+
252+
if (options.order === 'desc') {
253+
if (results.length >= options.rows) {
254+
results.shift();
255+
}
256+
}
257+
results.push(log);
258+
}
259+
})(logFiles.shift());
260+
};

0 commit comments

Comments
 (0)