Issue

When you set up FreeNAS as an iscsi target and use Proxmox VE to connect to the storage, you will notice some strange errors in the log. The errors eventually look like the example below. You will see this messages every 10 seconds. The way to set up iscsi in FreeNAS and Proxmox is documented quite well and not part of this little post.

May  8 22:41:53 freenas ctld[31684]: 192.168.X.Y: read: connection lost
May  8 22:41:53 freenas ctld[9280]: child process 31684 terminated with exit status 1

Cause

The cause is the storage management in Proxmox. This checks if the iscsi responds by just opening a connection to the iscsi port and immediately drop it again. The issue is known for a long time but it is not that easy to find the working solution. So here are the two forum threads that indicate a solution for the issue:

If you do not have any other issue, these messages are completely fine as the behavior is part of the way Proxmox checks the storage.

Solution

As these messages are not nice but completely fine for this setup, I like to ignore them. To test the changes, you just edit /etc/local/syslog-ng.conf and to make it permanent and survive a reboot, you have to edit /conf/base/etc/local/syslog-ng.conf.freenas.

Do not forget to run service syslog-ng reload after editing the file /etc/local/syslog-ng.conf!

First we need to define the filters. I added them above the message filters entries of the file. Beware that you have to build a more generic filter for the part with the ip address in it if you have more than just one proxmox host connecting. I have two so I just added a third filter to my local changes.

#
# Proxmox Filters
#
filter f_cut_ctld01 { not (program("ctld") and message("192.168.X.Y: read: connection lost")); };
filter f_cut_ctld02 { not (program("ctld") and message("child process") and message("terminated with exit status 1")); };

#
# message filters
#

Next locate the entries for /dev/console and comment the current entry. Just copy it to extend it with the newly added filters.

#
# *.err;kern.warning;auth.notice;mail.crit              /dev/console
#
#log { source(src); filter(f_err); filter(f_not_mdnsresponder); filter(f_not_nginx); destination(console); };
log { source(src); filter(f_err); filter(f_not_mdnsresponder); filter(f_not_nginx); filter(f_cut_ctld01); filter(f_cut_ctld02); destination(console); };

Finally we do the same thing again for the part with /var/log/messages to remove the spam from there as well.

#
# *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/messages
#
#log { source(src); filter(f_notice); filter(f_not_authpriv); filter(f_not_mdnsresponder); filter(f_not_nginx); destination(messages); };
log { source(src); filter(f_notice); filter(f_not_authpriv); filter(f_not_mdnsresponder); filter(f_not_nginx); filter(f_cut_ctld01); filter(f_cut_ctld02); destination(messages); };

This should remove the messages from the log but still let errors show up if something is wrong.