We've recently installed a Splunk syslog server and are pointing our devices to it. I noticed that when we stop/start the server (or even service) the logs from all my ESXi 5 hosts stop coming in.
It seems to be a known issue
Following step 5 does restart the log flowing. But there is no way that I want to log on to the console and run that every time someone does something in Splunk that needs a restart or that Windows box is rebooted for patches.
I started writing a bash script (below) that I could CRON on the hosts, but found out that ESXi doesn't really have a cron area like ESX4 (not i) had. If I brute force create it on the hosts, it will get removed with patches.
# this checks to see if the syslog server is writting to the Third party syslog server "SPLUNK"
# it will restart the syslog service if it sees that it has stopped
const='failed to write log'
if [ -e /var/log/.vmsyslogd.err ]; then
tail /var/log/.vmsyslogd.err -n 1 |grep "failed to write log"
if [ $? = 0 ]; then
echo "$const ; Found at the LAST line, restarting syslog server."
fi
fi
I was going to cron this to run every 15 minutes and if it saw the last line in the log was that it stopped "failed to write log" then I would add a " esxcli system syslog reload " inplace of the echo line.
I have vCenter on a Windows box and would like to have it run a scheduled task against all my hosts (maybe a csv file) and then issue "esxcli system syslog reload " if that is found. I can't figure out how to do this, can anymore help me out?
I'd like to use what I have, I don't have a vMA or splunks VM either.