I see from https://docs.webfaction.com/software/general.html#logs that our raw Apache logs are rotated once a day and deleted after 7 days. But how about the data collected every few hours/every day by the pre-made WebFaction AWStats and Webalizer applications? Does historical stat data stick around forever or does it also have a time after which old data gets deleted? I didn't find the answer on the documentation site for stats ( https://docs.webfaction.com/software/webstats.html ). Thanks. asked 21 May '16, 05:06 cpirazzi
showing 5 of 6
show 1 more comments
|
Although I never found a definitive answer in any AWStats or Webalizer documentation, and although I have not had time to go read the source code of those tools to be sure, from the following data points it seems to be the case that AWStats and Webalizer never discard any of their previous analyzed output, nor does WebFaction discard any of it either:
answered 24 May '16, 09:19 cpirazzi |
AWStats and Webalizer should be storing the data they collect within their own application directories, eg ~/webapps/<app_name>, and are not rotated by us at all. Though you can set that up manually, this is documented on the application websites. Like: http://www.webalizer.org/faq.html
Good to know that WebFaction will not delete any old data.
As to the issue of Webalizer or AWStats deleting old data, I had previously looked at the dox of both, including the FAQ you pointed to, and I couldn't find anything that either confirms or denies that old analyzed results are rotated out! The FAQ doesn't mention the issue. The Webalizer README at ftp://ftp.mrunix.net/pub/webalizer/README mentions rotating Apache logs, but makes no mention of rotating or archiving the OUTPUT of Webalizer. Similar for AWStats.
So I figured ask the experts to confirm for sure that there is no deletion of old data built into either Webalizer or AWStats.
I should refine my response. The webstat tracking apps should be reading your existing Apache logs and keeping a record of them while those Apache logs get rotated.
Hi Nick,
When you say "record of them" you mean that AWStats and Webalizer keep a record of their analyzed, cooked output, not the original raw Apache logs, right? I couldn't see any evidence of AWStats keeping a copy of the original raw Apache logs. Haven't checked Webalizer yet.
My original question here is about the analyzed, cooked output files (for example, in the case of AWStats, I am talking about the files called /home/<username>/webapps/<awstats_application_name>/cgi-bin/awstatsMMYYYY.txt which AWStats uses to accumulate cooked, analyzed results for a particular month).
I am trying to confirm that AWStats and Webalizer will not delete any of its analyzed, cooked output files once they become "too old" by some definition of "too old." I could not find anything to confirm or deny this in any of the AWStats or Webalizer dox. It would be nice to be 100% sure by finding a direct statement about this in the dox.
By the way, for those of you like me who are trying to understand how AWStats works, the official dox are here:
http://www.awstats.org/docs/ http://www.awstats.org/docs/awstats_faq.html
BUT they are are pretty terrible, actually downright useless for understanding the big picture of how awstats works.
Fortunately this guide:
http://www.onlamp.com/pub/a/onlamp/2005/12/01/awstats.html?page=2
is great for understanding how AWStats is set up in general.
Then specifically for the WebFaction context, many missing links are filled in by looking inside our account in the pre-configured files for an AWStats WebFaction application, at:
/home/<username>/webapps/<awstats_application_name>/update_awstats.sh
/home/<username>/webapps/<awstats_application_name>/cgi-bin/awstats.conf
The update_awstats.sh file is presumably called daily from a cron job. You can see that script merges together the current raw Apache logs with the the ".1" (one day old) raw Apache logs and runs AWStats analysis on the merged data. The results of the anlysis get merged into:
/home/<username>/webapps/<awstats_application_name>/cgi-bin/awstatsMMYYYY.txt
The comments in the awstats.conf file are actually WAY BETTER documentation of AWStats than AWStats' official documentation. See the options there for major hints on what is going on.
Sadly, not even those files address my question: whether AWStats will delete old awstatsMMYYYY.txt data files when they become "too old."
There is an "ArchiveLogRecords" option in awstats.conf, but that option pertains to the raw Apache logs, NOT the analyzed, cooked output (and it is off anyway because it requires the PurgeLogFile option to be on too, and the log file that would be archived and purged would be the merged temporary log that update_awstats.sh passes to awstats, which combines several original logs---making this a very unsuitable way to even archive raw Apache logs).
Anybody have a definitive answer?
For example if someone has been running AWStats for years and can confirm that they have years of accumulated awstatsMMYYYY.txt, that would be a satisfactory answer I'd say.
I can simply confirm, by default, as long as these applications are active on the site, they will continually log data. The limit would be the disk allocation of your hosting plan, which for logs would be more than sufficient for years on end.