WebFaction
Community site: login faq

I see from https://docs.webfaction.com/software/general.html#logs that our raw Apache logs are rotated once a day and deleted after 7 days.

But how about the data collected every few hours/every day by the pre-made WebFaction AWStats and Webalizer applications?

Does historical stat data stick around forever or does it also have a time after which old data gets deleted?

I didn't find the answer on the documentation site for stats ( https://docs.webfaction.com/software/webstats.html ).

Thanks.

asked 21 May '16, 05:06

cpirazzi
299214
accept rate: 5%

AWStats and Webalizer should be storing the data they collect within their own application directories, eg ~/webapps/<app_name>, and are not rotated by us at all. Though you can set that up manually, this is documented on the application websites. Like: http://www.webalizer.org/faq.html

(21 May '16, 05:18) NickR ♦♦

Good to know that WebFaction will not delete any old data.

As to the issue of Webalizer or AWStats deleting old data, I had previously looked at the dox of both, including the FAQ you pointed to, and I couldn't find anything that either confirms or denies that old analyzed results are rotated out! The FAQ doesn't mention the issue. The Webalizer README at ftp://ftp.mrunix.net/pub/webalizer/README mentions rotating Apache logs, but makes no mention of rotating or archiving the OUTPUT of Webalizer. Similar for AWStats.

So I figured ask the experts to confirm for sure that there is no deletion of old data built into either Webalizer or AWStats.

(21 May '16, 05:39) cpirazzi

I should refine my response. The webstat tracking apps should be reading your existing Apache logs and keeping a record of them while those Apache logs get rotated.

(21 May '16, 05:58) NickR ♦♦

Hi Nick,

When you say "record of them" you mean that AWStats and Webalizer keep a record of their analyzed, cooked output, not the original raw Apache logs, right? I couldn't see any evidence of AWStats keeping a copy of the original raw Apache logs. Haven't checked Webalizer yet.

My original question here is about the analyzed, cooked output files (for example, in the case of AWStats, I am talking about the files called /home/<username>/webapps/<awstats_application_name>/cgi-bin/awstatsMMYYYY.txt which AWStats uses to accumulate cooked, analyzed results for a particular month).

I am trying to confirm that AWStats and Webalizer will not delete any of its analyzed, cooked output files once they become "too old" by some definition of "too old." I could not find anything to confirm or deny this in any of the AWStats or Webalizer dox. It would be nice to be 100% sure by finding a direct statement about this in the dox.

(21 May '16, 07:53) cpirazzi

By the way, for those of you like me who are trying to understand how AWStats works, the official dox are here:

http://www.awstats.org/docs/ http://www.awstats.org/docs/awstats_faq.html

BUT they are are pretty terrible, actually downright useless for understanding the big picture of how awstats works.

Fortunately this guide:

http://www.onlamp.com/pub/a/onlamp/2005/12/01/awstats.html?page=2

is great for understanding how AWStats is set up in general.

Then specifically for the WebFaction context, many missing links are filled in by looking inside our account in the pre-configured files for an AWStats WebFaction application, at:

/home/<username>/webapps/<awstats_application_name>/update_awstats.sh

/home/<username>/webapps/<awstats_application_name>/cgi-bin/awstats.conf

The update_awstats.sh file is presumably called daily from a cron job. You can see that script merges together the current raw Apache logs with the the ".1" (one day old) raw Apache logs and runs AWStats analysis on the merged data. The results of the anlysis get merged into:

/home/<username>/webapps/<awstats_application_name>/cgi-bin/awstatsMMYYYY.txt

The comments in the awstats.conf file are actually WAY BETTER documentation of AWStats than AWStats' official documentation. See the options there for major hints on what is going on.

Sadly, not even those files address my question: whether AWStats will delete old awstatsMMYYYY.txt data files when they become "too old."

There is an "ArchiveLogRecords" option in awstats.conf, but that option pertains to the raw Apache logs, NOT the analyzed, cooked output (and it is off anyway because it requires the PurgeLogFile option to be on too, and the log file that would be archived and purged would be the merged temporary log that update_awstats.sh passes to awstats, which combines several original logs---making this a very unsuitable way to even archive raw Apache logs).

Anybody have a definitive answer?

For example if someone has been running AWStats for years and can confirm that they have years of accumulated awstatsMMYYYY.txt, that would be a satisfactory answer I'd say.

(21 May '16, 07:57) cpirazzi

I can simply confirm, by default, as long as these applications are active on the site, they will continually log data. The limit would be the disk allocation of your hosting plan, which for logs would be more than sufficient for years on end.

(21 May '16, 21:37) NickR ♦♦
showing 5 of 6 show 1 more comments

Although I never found a definitive answer in any AWStats or Webalizer documentation, and although I have not had time to go read the source code of those tools to be sure, from the following data points it seems to be the case that AWStats and Webalizer never discard any of their previous analyzed output, nor does WebFaction discard any of it either:

  • NickR's confirmation above that the tools will just keep accumulating data until your account runs out of space
  • A sizeable number of posts I found on the internet of various people wondering how to delete old AWStats and Webalizer data, whose solution was always to manually delete files rather than use or wait for some built-in feature of AWStats/Webalizer
permanent link

answered 24 May '16, 09:19

cpirazzi
299214
accept rate: 5%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×18
×15
×5
×4
×2

question asked: 21 May '16, 05:06

question was seen: 7,257 times

last updated: 24 May '16, 09:19

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2019 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM