Log rotation / retention in Alfresco

Alfresco creates daily log files in ALFRESCO_HOME folder with alfresco.log.{date} and another log file location is ALFRESCO_HOME/tomcat/logs/catalina.{date}.log.

These files are created everyday, but bydefault there is no configuration for removing old log files which are not needed. So I have jotted down steps to configure Alfresco Log rotation and retention.

By default Alfresco uses Log4j library for logging. While using Log4j, by default it uses "org.apache.log4j.DailyRollingFileAppender" class. Which helps to create log files daily by appending date at end of log file in ALFRESCO_HOME folder. But this class does not provide facility to remove old files.

So we can use "org.apache.log4j.RollingFileAppender" class from log4j, which provides facility of rotating log files and removing old log files. But there is a con, that it does rotates log files based on Size we provide, not daily bases. We can give value of "MaxFileSize" in log4j.properties file, so whenever it cross the size limit it will rename alfresco log file with alfresco.log.1 then alfresco.log.2 etc.
Here you can give limit, how many numbers of such log files you want to retain. That can be done through "MaxBackupIndex" parameter. Lets say if we keep "MaxBackupIndex" to 10. Then Alfresco will create file from alfresco.log.1 to alfresco.log.10. When it reaches to alfresco.log.10 and again if alfresco.log file exceeds size of "MaxFileSize" then it will rename alfresco.log file to alfresco.log.1 and alfresco.log.1 file will be renamed to alfresco.log.2...and so on. The older alfresco.log.10 file will be deleted in that case, as we have set "MaxBackupIndex" to 10.

So before we start lets remove tomcat's default java.util.logging configuration and add log4j.

You will have to download log4j jar file from here. But if you want, you can get it from tomcat/webapps/alfresco/WEB-INF/lib/log4j-1.2.15.jar location too. Apart from log4j jar file you need to get tomcat-juli.jar and tomcat-juli-adapters.jar files from this location.

So after downloading log4j jar, tomcat-juli.jar and tomcat-juli-adapters.jar files follow below steps.
1. Copy tomcat-juli.jar and tomcat-juli-adapters.jar files into tomcat/lib folder.
2. Create log4j.properties file in tomcat/lib folder with below content.

3. Now  copy downloaded tomcat-juli.jar into tomcat/bin directory. You need to replace existing tomcat-juli.jar file.

4. Then remove tomcat/conf/logging.properties file.

Now lets make changes at Alfresco side.There are two location where Alfresco has configured log4j.properties file for Alfresco and Share.

tomcat/webapps/alfresco/WEB-INF/classes/log4j.properties
tomcat/webapps/share/WEB-INF/classes/log4j.properties

You can add below mentioned snippet in log4j.properties files at both location to apply log rotation and retention. Here we have used "File" appender for alfresco.log file and "CONSOLE" appender for catalina.out file.

note: you can add below two appenders in log4j.properties file, but you will also have to remove words/line which are strikethrough.

# Set root logger level to error
log4j.rootLogger=error, Console, File, CONSOLE

###### Console appender definition #######

# All outputs currently set to be a ConsoleAppender.
log4j.appender.Console=org.apache.log4j.ConsoleAppender
log4j.appender.Console.layout=org.apache.log4j.PatternLayout
log4j.appender.Console.layout.ConversionPattern=%d{ISO8601} [%x] [%p] [%c{3}] [%t] [%r] %m%n
###### File appender definition #######
log4j.appender.File=org.apache.log4j.RollingFileAppender
log4j.appender.File.File=alfresco.log
log4j.appender.File.Append=true
log4j.appender.File.MaxFileSize=10MB
log4j.appender.File.MaxBackupIndex=10
log4j.appender.File.layout=org.apache.log4j.PatternLayout
log4j.appender.File.layout.ConversionPattern=%d{ABSOLUTE} %-5p [%c] %m%n

log4j.appender.CONSOLE=org.apache.log4j.RollingFileAppender
log4j.appender.CONSOLE.File=${catalina.base}/logs/catalina.out
log4j.appender.CONSOLE.MaxFileSize=10MB
log4j.appender.CONSOLE.MaxBackupIndex=10
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d{DATE} %x %-5p [%c{3}] %m%n

Once above changes done you can restart Alfresco and check the logs.

Hope this post helps to rotate log files in Alfresco.

org.apache.solr.common.SolrException: Internal Server Error

I have done integration of Solr with Liferay. May colleague informed me that he was getting below error frequently. So I was looking for a solution of below error since last few days. But could not find the solution. I tried tuning parameters of Solr configuration, ran load test. But as usual could not reproduce error. But at last after 2 weeks, I found the clue.

Solution:
Of course,   I saw log file at Solr too when above error occurred in Liferay. But I did not notice the time zone difference, which both applications were running. Anyway, now I checked for the error at Solr side and found below error at Solr.

Finally I got some clue. It may happened that Solr was busy running any commit operation and at the same time another commit operation has been fired by Liferay. Bydefault "writeLockTimeout" value is 1 second. So already running transaction took more than 1 sec and hence Liferay generated error saying "Internal Server Error".
So now I have set "writeLockTimeout" value to 10 sec. I hope none of the commit transaction will take atleast 10 secs.

Apache Solr Logging through Log4j

Bydefault Solr uses slf4j for logging. Follow below mentioned steps to configure Solr with log4j.

1. First delete slf4j-jdk14-1.5.5.jar file from $CATALINA_HOME/webapps/solr/WEB-INF/lib/slf4j-jdk14-1.5.5.jar location.
2. Now download slf4j source from http://www.slf4j.org/dist/slf4j-1.5.5.tar.gz and extract it.
3. Now copy slf4j-log4j12-1.5.5.jar from extracted location to $CATALINA_HOME/webapps/solr/WEB-INF/lib folder.
4. Now download http://mirrors.dcarsat.com.ar/apache/logging/log4j/1.2.17/log4j-1.2.17.tar.gz and extract it.
5. Now copy log4j-1.2.17.jar from extracted location to $CATALINA_HOME/webapps/solr/WEB-INF/lib folder.
6. Now create $CATALINA_HOME/webapps/solr/WEB-INF/classes directory and add file “log4j.properties” with below content.

log4j.rootLogger=ERROR, CONSOLE

log4j.logger.org.apache.solr=INFO

log4j.appender.CONSOLE=org.apache.log4j.RollingFileAppender

log4j.appender.CONSOLE.File=${catalina.base}/logs/catalina.out

log4j.appender.CONSOLE.MaxFileSize=200MB

log4j.appender.CONSOLE.MaxBackupIndex=10

log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout

log4j.appender.CONSOLE.layout.ConversionPattern=%d{DATE} %x %-5p [%c{3}] %m%n

7. Now rename $CATALINA_HOME/conf/logging.properties file to $CATALINA_HOME/conf/logging.properties.bak.
8. Now restart tomcat.

Above configuration will keep writing logs in catalina.out file until it reaches 200MB size. Once it cross 200MB it will be renamed with catalina.out.1 and new after that new logs will be written to catalina.out file. MaxBackupIndex parameter will retain those many number of catalina.out files. Once it cross MaxBackupIndex limit, the old file (ex: catalina.out.10) will be removed.