Handling files which begin with non-format-conforming logs

[MorganDavidInf]

Thanks for the work on this tool it's really useful!

I am having an issue with some files because the start of the file has lots of non-conforming logs, but the file later contains 100s of logs in a standard format that I know lnav formats properly in log mode when in a file on their own.

Use-case

I'm piping the output of my terminal from tmux into lnav, and using that to view live logs (using tmux pipe-pane into a .log file and then opening that with lnav).

The start of the terminal instance is a bunch of automated bash scripts (not important to read, but clearly not logs as lnav understands them):

  nvm use;
  source /Users/morgan/development/editor/scripts/dev-env.sh;
  export TF_VAR_USE_LOCAL_K8S="false";
## etc. 
## then a bunch of terraform logs that also don't conform, around 1k lines.

Problem

I believe lnav decides whether to open a file in log mode or not based on whether the first few lines of the file conform to a format?

This is causing all my logs to be opened in text mode instead of log mode that I want. My logs do conform to a format, further down the page (lnav formats them beautifully if I remove all the stuff at the start).

Solutions

Is a way to deal with this? Maybe a greedy mode that checks the whole file and takes any conforming files? Or maybe a 'fallback' format that accepts any format and just prints it as raw text? I tried to create this custom "fallback" format myself, the issue is that formats require a timestamp, which obviously none of bash scripts at the start of the file have. Is there a way I can put in a dummy timestamp?

Thanks for any help!

[FaffeF]

Can you predict the amount of preamble, and maybe pipe it through tail -f -n 1000?

[MorganDavidInf]

Unfortunately no, but I could write a script that searches for the first real log line and then removes everything above it. that was going to be my workaround if there's no way to do this inside lnav.

[tstack]

What version of lnav are you using?

I think this should work right now, but there is a limit on the number of unrecognized lines. Currently it is 1,000, but you should be able to increase it by changing the max-unrecognized-lines value. Try running the following command to increase it to 10,000:

:config /tuning/logfile/max-unrecognized-lines 10000

[MorganDavidInf]

Thanks! This solves the problem