博客

Logstash -> Input配置如下：

file {
    type => "test_aaa"
    path => ["/usr1/app/logs/aaa"]    
    sincedb_path => "/home/jfy/soft/logstash-2.0.0/sincedb_aaa.access"
    start_position => "beginning"
    codec => multiline {
        max_bytes => "10MiB"
        max_lines => 500
        charset => "GBK"
        pattern => "^(?!.*?=== >>>>>>>> ===).*$"
        what => "previous"
    }         
}

multiline合并，默认最大合并500行，最大大小10M

昨天由于程序出错导致一个请求出现了50000多行Err Log，由于Logstash multiline合并导致出现了一个大文档，在kibana显示时显示不出来，可以通过适当调节这两个参数来限制合并日志的行数与大小