log file policies

[David Schwartz 413]

I'm curious if anybody has any particular policies they follow when it comes to the use of log files.

 

I've actually never run into any such policy anywhere I've ever worked. But we've run into a problem that is leaving us open to someone asking, quite legitimately, "What sort of policies do you guys follow when it comes to log files and data logging?"

 

 

[Arnaud Bouchez 380]

Here is what we usually do:

1. Store logs in a dedicated folder, with proper read policy.
   -> especially under Unix, all background daemons run with a dedicated user, and the log files have chmod 640 to restrain read access.
2. Always run the logs in Verbose mode, even on production, unless the execution environment is really constrained.
   (you never know what will happen, and detailed logs is a need for proper investigation, especially on production)
   -> you need a logging solution which is fast, and can log dozen MB/s with no performance penalty (e.g. our SynLog.pas unit writes in a background thread)
3. Rotate logs to keep history as long as possible
   -> SynLog.pas has built-in SynLZ-compression of the logs for very fast and efficient rotation
4. Never log passwords, or such sensitive information.
   -> password are always stored and persisted in salted + hashed form - but when transmitted as parameters, or as settings, they should be encrypted - even a naive encryption may be enough like with a TSynPersistentWithPassword class from SynCommons.pas
   -> more generally, sensitive information should be handled in memory with strong cryptography, e.g. via our cross-platform CryptDataForCurrentUser() from SynCrypto.pas
5. Have a way to hide unexpected content, when log is automated.
   -> in mORMot.pas, you can register some sensitive types which parameters will be logged as "***" during SOA automated logging of calls.
6. Ensure all those policies are always setup by default, and on all supported platforms
   -> today, we should ensure that Linux execution should be not weaker than Windows as a target (and it is easy to make executable safer under Linux)
7. Optionally use a third-party SaaS solution to store your logs in the long term, with proper encryption and search abilities
   -> like https://livemon.com - server-side written in object pascal 🙂

 

Most of it is available out-of-the-box in http://mormot.net

Edited December 29, 2018 by Arnaud Bouchez

[David Schwartz 413]

Thanks Arnaud, this is one "level" of detail I'm curious about.

 

Another thing I'd like to see is ... what kinds of stuff gets logged?

 

There are some devs who set up log files to only capture errors. Some capture more. Context is often a big clue to tracking down programming errors.

 

Do you have a policy to make an effort to recover from exceptions, or just log them and cancel whatever was happening and move on?

 

When you log errors, do you log just the exception messages, or do you log contextual data as well? (I like to use MadExcept because it grabs a whole bunch of useful contextual info that's sometimes quite helpful.)

 

I realize everybody is different, but what I'm asking is as a general practice, what kinds of policies have you guys seen?

 

(If you get sued and someone alleges your program was throwing errors left and right for months and months, but the log files show no unexpected activities that can be proven DID in fact occur ... what's your defense?)

[Guest]

Well the verbose mode should be very narrative because thats why it is called verbose.

 

If you do not need a verbose log the change the config to debug or info and you will have a reduced log file.

[Arnaud Bouchez 380]

On server side, we log a lot of stuff... almost everything possible information. It is needed for post-mortem or post-bug forensic.
Of course, our mORMot framework logs automatically all SOA calls, and all ORM and/or SQL process.
And it also logs all exceptions.
See https://synopse.info/files/html/Synopse mORMot Framework SAD 1.18.html#TITL_73

 

This was for low-level text logs.
At higher level, we also log all high-level SOA calls, in form of local Sqlite3 databases.
This is very efficient to make any kind of statistics, via some simple SQL SELECT.
https://synopse.info/files/html/Synopse mORMot Framework SAD 1.18.html#TITLE_445