.. _logrotation-check:
Logs and Data Protection checks
===============================
On Wire.com, we keep logs for a maximum of 72 hours as described in the `privacy whitepaper `_
We recommend you do the same and limit the amount of logs kept on your servers.
How can I see how far in the past access logs are still available on my servers?
--------------------------------------------------------------------------------
Look at the timestamps of your earliest nginz logs:
.. code:: sh
export NAMESPACE=default # this may be 'default' or 'wire'
kubectl -n "$NAMESPACE" get pods | grep nginz
# choose one of the resulting names, it might be named e.g. nginz-6d75755c5c-h9fwn
kubectl -n "$NAMESPACE" logs -c nginz | head -10
If the timestamp is more than 3 days in the past, your logs are kept for unnecessary long amount of time and you should configure log rotation.
I used your ansible scripts and prefer to have the default 72 hour maximum log availability configured automatically.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can use `the kubernetes_logging.yml ansible playbook `_
I am not using ansible and like to SSH into hosts and configure things manually
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SSH into one of your kubernetes worker machines.
If you installed as per the instructions on docs.wire.com, then the default logging strategy is ``json-file`` with ``--log-opt max-size=50m --log-opt max-file=5`` storing logs in files under ``/var/lib/docker/containers//.log``. You can check this with these commands:
.. code:: sh
docker info --format '{{.LoggingDriver}}'
ps aux | grep log-opt
(Options configured in ``/etc/systemd/system/docker.service.d/docker-options.conf``)
The default will thus keep your logs around until reaching 250 MB per pod, which is far longer than three days. Since docker logs don't allow a time-based log rotation, we can instead make use of `logrotate `__ to rotate logs for us.
Create the file ``/etc/logrotate.d/podlogs`` with the following contents:
..
NOTE: in case you change these docs, also make sure to update the actual code
under https://github.com/wireapp/wire-server-deploy/blob/develop/ansible/kubernetes_logging.yml
.. code::
"/var/lib/docker/containers/*/*.log"
{
daily
missingok
rotate 2
maxage 1
copytruncate
nocreate
nocompress
}
Repeat the same for all the other kubernetes worker machines, the file needs to exist on all of them.
There should already be a cron job for logrotate for other parts of the system, so this should be sufficent, you can stop here.
You can check for the cron job with::
ls /etc/cron.daily/logrotate
And you can manually run a log rotation using::
/usr/sbin/logrotate -v /etc/logrotate.conf
If you want to clear out old logs entirely now, you can force log rotation three times (again, on all kubernetes machines)::
/usr/sbin/logrotate -v -f /etc/logrotate.conf
/usr/sbin/logrotate -v -f /etc/logrotate.conf
/usr/sbin/logrotate -v -f /etc/logrotate.conf