Here is what we usually do:
Store logs in a dedicated folder, with proper read policy.
-> especially under Unix, all background daemons run with a dedicated user, and the log files have chmod 640 to restrain read access.
Always run the logs in Verbose mode, even on production, unless the execution environment is really constrained.
(you never know what will happen, and detailed logs is a need for proper investigation, especially on production)
-> you need a logging solution which is fast, and can log dozen MB/s with no performance penalty (e.g. our SynLog.pas unit writes in a background thread)
Rotate logs to keep history as long as possible
-> SynLog.pas has built-in SynLZ-compression of the logs for very fast and efficient rotation
Never log passwords, or such sensitive information.
-> password are always stored and persisted in salted + hashed form - but when transmitted as parameters, or as settings, they should be encrypted - even a naive encryption may be enough like with a TSynPersistentWithPassword class from SynCommons.pas
-> more generally, sensitive information should be handled in memory with strong cryptography, e.g. via our cross-platform CryptDataForCurrentUser() from SynCrypto.pas
Have a way to hide unexpected content, when log is automated.
-> in mORMot.pas, you can register some sensitive types which parameters will be logged as "***" during SOA automated logging of calls.
Ensure all those policies are always setup by default, and on all supported platforms
-> today, we should ensure that Linux execution should be not weaker than Windows as a target (and it is easy to make executable safer under Linux)
Optionally use a third-party SaaS solution to store your logs in the long term, with proper encryption and search abilities
-> like https://livemon.com - server-side written in object pascal 🙂
Most of it is available out-of-the-box in http://mormot.net