How to configure log4j to only keep log files for the last seven days

Question

I have the following logging problem with several Java applications using log4j for logging   I want log files to be rotated daily  like  log 2010-09-10 log 2010-09-09 log 2010-09-08 log 2010-09-07 log 2010-09-06 log 2010-09-05 log 2010-09-04   But for data security reasons we are not allowed to keep log files for longer than seven days at my company  So the generation of the next next log file log 2010-09-11 should trigger the deletion of log 2010-09-04  Is it possible to configure such a behaviour with log4j  If not  do you know another elegant solution for this kind of logging problem

User · Answer

There s also a DailyRollingFileAppender Edit  after reading this worrying statement   DailyRollingFileAppender has been observed to exhibit synchronization issues and data loss  The log4j extras companion includes alternatives which should be considered for new deployments and which are discussed in the documentation for org apache log4j rolling RollingFileAppender   from the above URL  which I never realized before   then the log4j-extras looks to be a better option

User · Answer

I had set    log4j appender R org apache log4j DailyRollingFileAppender log4j appender R DatePattern    yyyy-MM-dd   Archive log files  Keep one year of daily files  log4j appender R MaxBackupIndex 367    Like others before me  the DEBUG option showed me the error      log4j WARN No such property  maxBackupIndex  in   org apache log4j DailyRollingFileAppender    Here is an idea I have not tried yet  suppose I set the DatePattern such that the files overwrite each other after the required time period  To retain a year s worth I could try setting    log4j appender R DatePattern    MM-dd    Would it work or would it cause an error   Like that it will take a year to find out  I could try    log4j appender R DatePattern    dd    but it will still take a month to find out

User · Answer

Use the setting log4j appender FILE RollingPolicy FileNamePattern  e g  log4j appender FILE RollingPolicy FileNamePattern F  logs filename log  d dd  gz for keeping logs one month before rolling over   I didn t wait for one month to check but I tried with mm  i e  minutes  and confirmed it overwrites  so I am assuming it will work for all patterns

User · Answer

I came across this appender here that does what you want  it can be configured to keep a specific number of files that have been rolled over by date   Download  http   www simonsite org uk download htm  Example  groovy    new TimeAndSizeRollingAppender name   timeAndSizeRollingAppender      file   logs app log   datePattern    yyyy-MM-dd      maxRollFileCount  7  compressionAlgorithm   GZ      compressionMinQueueSize  2     layout  pattern conversionPattern    d   t   -5p  c 2   x -  m n

User · Answer

Inspite of starting a chrone job  for the task  we can use log4j2 properties file in config folder of logstash  Have a look at the link below  this will be helpful   https   github com elastic logstash issues 7482

User · Answer

I assume you re using RollingFileAppender  In which case  it has a property called MaxBackupIndex which you can set to limit the number of files  For example   log4j appender R org apache log4j RollingFileAppender log4j appender R File example log log4j appender R MaxFileSize 100KB log4j appender R MaxBackupIndex 7 log4j appender R layout org apache log4j PatternLayout log4j appender R layout ConversionPattern  p  t  c -  m n

User · Answer

You can perform your housekeeping in a separate script which can be cronned to run daily  Something like this   find  path to logs -type f -mtime  7 -exec rm -f

User · Answer

According to the following post  you can t do this with log4j  Use MaxBackupIndex in DailyRollingFileAppender -log4j     As far as I know  this functionality was supposed to make it into log4j 2 0 but that effort got sidetracked  According to the logback website  logback is the intended successor to log4j so you might consider using that   There s an API called SLF4J which provides a common API to logging  It will load up the actual logging implementation at runtime so depending on the configuration that you have provided  it might use java util log or log4j or logback or any other library capable of providing logging facilities  There ll be a bit of up-front work to go from using log4j directly to using SLF4J but they provide some tools to automate this process  Once you ve converted your code to use SLF4J  switching logging backends should simply be a case of changing the config file

User · Answer

The class DailyRollingFileAppender uses the DatePattern option to specify the rolling schedule  This pattern should follow the SimpleDateFormat conventions from Std  Ed  v1 4 2  So  we have to use E option  Day in week   For example    lt param name  DatePattern  value     EEE   gt    See more about DailyRollingFileAppender class from log4j javadoc here  Unfortunately the Java 1 4 2 documentation is no longer online  but you can download a copy here

User · Answer

I create this Methode and call it by closing the application     public void deleteFiles         File f   new File  log        File   fileArray   f listFiles        double timenow   System currentTimeMillis         double olderTenDays   timenow - 864000000    MS for ten days      for  int i   0  i  lt  fileArray length  i               if fileArray i  lastModified   lt  olderTenDays              fileArray i  delete

User · Answer

There is another option DailyRollingFileAppender  but it lacks the auto delete  keep 7 days log  feature you looking for  sample  log4j appender DRF org apache log4j DailyRollingFileAppender log4j appender DRF File example log log4j appender DRF DatePattern    yyyy-MM-dd   I do come across something call org apache log4j CompositeRollingAppender  which is combine both the features of the RollingFileAppender  maxSizeRollBackups  no  of backup file  and DailyRollingFileAppender  roll by day     But have not tried that out  seems is not the standard 1 2 branch log4j feature

User · Answer

log2j now has support to delete old logs  Take a look at DefaultRolloverStrategy tag and at a snippet below  It  creates up to 10 archives on the same day   will parse the   baseDir  directory that you define under the Properties tag at max depth of 2 with log filename matching  quot app-  log gz quot   delete logs older than 7 days but keep the most recent 5 logs if your most recent 5 logs are older than 7 days   lt DefaultRolloverStrategy max  quot 10 quot  gt     lt Delete basePath  quot   baseDir  quot  maxDepth  quot 2 quot  gt       lt IfFileName glob  quot   app-  log gz quot  gt         lt IfLastModified age  quot 7d quot  gt           lt IfAny gt             lt IfAccumulatedFileCount exceeds  quot 5 quot    gt           lt  IfAny gt         lt  IfLastModified gt       lt  IfFileName gt     lt  Delete gt   lt  DefaultRolloverStrategy gt     A good debug option is if you set   lt Configuration status  quot trace quot  gt   and use testMode Option like this           lt DefaultRolloverStrategy gt             lt Delete basePath  quot   baseDir  quot  testMode  quot true quot  gt               lt IfFileName glob  quot   log quot    gt               lt IfLastModified age  quot 7d quot    gt             lt  Delete gt           lt  DefaultRolloverStrategy gt            You can see in console log what files would get deleted without deleting the files right away

User · Answer

If you are using Linux  you can configure a cron job using tmpwatch   Most Linux systems have a tmpwatch cron job that cleans up the  tmp directory   You can add another that monitors your logging directory and deletes files over 7 days old   If you are using a different system  there are probably equivalent utilities

User · Answer

My script based on  dogbane s answer  etc cron daily hbase    bin sh find  var log hbase -type f -name  quot phoenix-hbase-server log  0-9  0-9  0-9  0-9 - 0-9  0-9 - 0-9  0-9  quot  -exec bzip2     quot   quot  find  var log hbase -type f -regex  quot    out  0-9  0-9   quot  -exec bzip2     quot   quot  find  var log hbase -type f -mtime  7 -name  quot   bz2 quot  -exec rm -f     quot   quot    etc cron daily tomcat    bin sh find  opt tomcat log  -type f -mtime  1 -name  quot    0-9  0-9  0-9  0-9 - 0-9  0-9 - 0-9  0-9   log quot  -exec bzip2     quot   quot  find  opt tomcat log  -type f -mtime  1 -name  quot    0-9  0-9  0-9  0-9 - 0-9  0-9 - 0-9  0-9  txt quot  -exec bzip2     quot   quot  find  opt tomcat log  -type f -mtime  7 -name  quot   bz2 quot  -exec rm -f     quot   quot   because Tomcat rotate needs one day delay

[java] How to configure log4j to only keep log files for the last seven days?

Examples related to java

Examples related to logging

Examples related to log4j