LocalFileSink

Behavior

LocalFileSink writes messages into local file system. The name of each written file is formatted as <datetime when the file is created><hostname><random UUID>

datetime is formatted as PyyyyMMddTHHmmss. LocalFileSink is an asynchronous sink, and employs an internal queue for buffering messages. We can specify which type of message queue to use, maximum time to buffer data, and the batch size of queued messages.

Properties

LocalFileSink is QueuedSink and has the following common properties:

• queue4Sink
• batchSize
• batchTimeout

If the default value is not specified, it is required.

Properties	Description	type	Default
outputDir	directory path where files are stored	String
writer	Its type can be 'text' or 'sequence'. 'text' means text file and 'sequence' means Hadoop sequence file. User can specify Hadoop codec class.	FileWriter	TextFileWriter without any compression
maxFileSize	when the file size reaches to this value, local file sink will rotate the file: closing current file, creating a new file, and starting to write messages to the new file.	long	200MB
rotationPeriod	local file sink will rotate files with this period. It can be an ISO-formatted period string such as PT10m, or milliseconds value.	String	PT2m
minPercentFreeDisk	The minimum percentage of free disk space. When the free disk space drops below this threshold, local file sink will abort the operation, and notifies Suro server, which in turn will stop taking traffic.	int	15
notice	It describes Notice type for how to notify its progress.	Notice	QueueNotice