amazon web services - rsyslog stale file handler with catalina.out -

amazon web services - rsyslog stale file handler with catalina.out -

problem: after deploying microservices war via aws ebs tomcat 7 container...noticed log rotation occurs @ utc day boundary leaves stale inode file.

the log rotation more of re-create n truncate, causes stale file handler rsyslog, listening changes catalina.out. what's best way prevent stale inode descriptors? should specify rollover policy in logback.xml or logrotate or ...?

output of sudo lsof /var/log/tomcat7/catalina.out (and sudo stat study latest inode)

rsyslogd 18970 root 2r reg 202,1 1250 134754 /var/log/tomcat7/catalina.out

but doesn't match log output of rsyslog in debug mode.

4638.114765354:7fc839b8c700: stream checking file alter on '/var/log/tomcat7/catalina.out', inode 135952/135952file 7 read 0 bytes

workaround stop tomcat, remove catalina.out, restart tomcat. allowed rsyslog go on streaming of new records.

however, after few hours, rsyslog fails stream newer log records rsyslog destination server. debug log of rsyslog contains same inode output of stat , lsof. if run

sudo stat /var/log/tomcat7/catalina.out

rsyslog starts streaming again.

have noticed rsyslog stop streaming intermittently outside of log rollover utilize case?

why sudo stat /var/log/tomcat7/catalina.out cause rsyslog stream again?

i have problem rsyslog not sending new records logrotated rotation on catalina.out. issuing stat doesn't quite cutting me problem tomcat stops writing catalina.out (!) ...after perusing various forums , blogs, able address through next steps:

make sure have $workdirectory defined in rsyslog configuration; allows rsyslog write "state file" catalina.out (or other log file(s) watches, matter) as noted here on loggly's blog, need stop rsyslog, delete state file, restart rsyslog on postrotate entry.

my logrotate setting catalina.out (state files in /var/lib/rsyslog):

/opt/tomcat/logs/catalina.out { rotate 7 size 50m notifempty missingok postrotate service rsyslog stop rm /var/lib/rsyslog/* service rsyslog start endscript }

amazon-web-services tomcat7 elastic-beanstalk rsyslog