Logging?

[bjaglin]

The instructions are explicit about using 127.0.0.1 for logging, but AFAIK, the rsyslogd daemon is not started within the container (if it's even installed). Am I missing something?

I am now leaning towards --linking this container towards a rsyslogd one, to use it as log target within the configuration. Anyone else had a similar approach?

Thanks!

[acdameli]

This is the only solution that came to mind for me as well. @bjaglin was this your final thought on the matter or did you find an alternative?

[nicot]

Unless I'm mistaken, this container currently doesn't log anything.

I prefer to have each container log to stdout, then I can choose to send that on to a logging container if I want to. The (hackish) way I ended up doing it is here: https://gist.github.com/nicot/6c680c626156f842444f

[btalbot]

Just start the container and mount /dev/log from the host into the container. With an haproxy config that logs to /dev/log it works fine except that systemd-journald doesn't associate it with the haproxy unit (if you're using systemd units to control it).

docker run -p 80:80 -v /dev/log:/dev/log haproxy

haproxy.cfg:
global
log /dev/log local2
...

[iby]

Any chance we'll see this implemented?

Docker already has its own docker logs facility, and administrators expect things to show up there. Therefore, /dev/log seems like a poor replacement. Any chance proper logging to stdout could be added?

I tried a complicated workaround with rsyslogd in the container, but while it seems to work more or less (I'm getting regular log messages, at least some of them) I'm running into this message quite often:

[ALERT] 087/151909 (47) : sendmsg logger #1 failed: Resource temporarily unavailable (errno=11)
[ALERT] 087/151909 (47) : sendmsg logger #2 failed: Resource temporarily unavailable (errno=11)
[ALERT] 087/151909 (47) : sendmsg logger #1 failed: Resource temporarily unavailable (errno=11)
[ALERT] 087/151909 (47) : sendmsg logger #2 failed: Resource temporarily unavailable (errno=11)

The suggested solution here http://comments.gmane.org/gmane.comp.web.haproxy/4716 isn't really useful because it suggests changing a kernel option on the host which shouldn't be necessary to run a docker container properly.

Therefore it would be really really helpful if proper stdout logging could be added to haproxy itself..

[jmkgreen]

See also moby/moby#13726 for minor enlightenment.

One option is to supply remote syslog endpoints per container (marking up the messages with source host as appropriate) - logstash with multiple syslog inputs may be useful. This might even be automated.

It seems reasonable to assume anyone using this container will need to supply their own haproxy.cfg file. Perhaps the best solution is to mark up the shipping config file with comments explaining that either the end user rebuilds this container with a syslog forwarding daemon or change the IP/port to a network-point such as above suggestion.

Do I understand this correctly that we should again manually add some daemon for logging things in the container/image? Proper logging isn't some sort of "optional" feature, therefore I think documentation isn't the solution here and the container should really already have this integrated.

As a side note, just redirecting everything else from docker to the systemd logs as well seems weird, since while then everything would be in one place it's kind of an odd solution because just one single container isn't really configured to use the docker logging infrastructure properly. It seems like the wrong end to address this..

However, maybe it might be preferable/easiest to convince the haproxy folks to simply add stdout logging instead of attempting all those logging daemon workarounds..

[megastef]

+1 for stdout/stderr logging - so tools based on Docker API like logspout or sematext-docker-agent could get the logs from the Docker API, and it works one with syslog or any other log driver. The user could specify the log driver settings and forward logs to dedicated logging services.

[jmkgreen]

@Jonast and @megastef you need to argue this with the haproxy authors. This container is merely a wrapper around what they ship.

Their product logs to syslog and this container builds with that addressed as 127.0.0.1 hence it is effectively lost without intervention. Replacing haproxy.cfg needs to happen for each user anyway so that's when the change to locally available syslog resources happens naturally.

The container could ship with a syslog daemon, this might be a useful improvement. No doubt some will argue this would be bloat too...

[andrewhamon]

+1 for shipping with a syslog daemon, if that means that docker logs would work as expected.

In all honesty I know basically nothing about syslog, and the thought of managing my logs out-of-band with all my other docker services makes me a bit uncomfortable. I can imagine there are many who are in my same boat. Maybe there could be tags with and without an embedded syslog?

[felixsanz]

any update on this? how to do logging properly?

[jmkgreen]

Run rsyslog on your host. Most distributions install this by default with a configuration providing /dev/log and recording to /var/log/syslog. Launch your containers (such as haproxy) bind-mounting /dev/log in, and allow your application (haproxy) to write to that. This adheres to decades-old Unlx standards which are unlikely to change soon.

Once you move beyond managing a single host, rsyslog on each host can simply copy the logs it receives from the applications within those containers to a remote aggregation point (logstash is an example). We have logstash parsing the messages into Elastic search indices - works well.

Remember many applications log limited messages to STDOUT and dramatic errors to STDERR which are the only outputs captured by docker. These too can be sent to syslog but are very different to those logs an application author wants to send to the system's standard logging facility (syslog).

Those wondering how to disambiguate multiple identical applications on the same host, what you really want is for that application to include a symbolic name (from an environment variable, for instance) in it's messages. This has nothing to do with Docker. haproxy I believe can include it's hostname in it's messages which should suffice.

[PriceChild]

http://cbonte.github.io/haproxy-dconv/1.6/configuration.html#log-send-hostname also lets you specify the hostname to be sent.