WebFaction
Community site: login faq

Hi

I want to analyze the bandwidth usage of one of my sites. The reason is simple: it only has a few visits but is consuming a lot of bandwidth and I want to know which files are these that are provoking that scenario (I suspect that the contents editors upload very large files).

I've installed the AWstats script, but I only can prove that because of the jpg file type, I'm consuming the 88% of the all bandwidth usage.

I see that exists a script called GoAccess that also analyzes the Apache log and offers a view of the most requested static files. If I can order this list by MB consumed, I think that I'll finally catch the problem origin.

Is it possible to install it in the WebFaction server? (requires root in order to install it via yum or do a make in order to compile it)

asked 31 Jan '14, 08:08

JavierCane
1538
accept rate: 0%


Hello,

Currently, we would not be able to install an application that requires root access without thorough testing.

A feature request has been raised for bandwidth monitoring via API:

https://community.webfaction.com/questions/11699/api-for-usage

You currently are using the optimal method for this type of analysis. You'd want to use AWStats yo analyze the logs here: ~/logs/frontend/access_<site-name-in-cp>.log

You also might get ideas of what is causing usage by looking at RAM and CPU usage:

https://community.webfaction.com/questions/2207/monitoring-usage-and-status

Let us know if there is anything else.

permanent link

answered 31 Jan '14, 09:10

NickR ♦♦
4665
accept rate: 20%

Hi Nick, thanks for your reply,

I'm very happy to see that you're currently working on a feature to analyze this directly through the CP :) But since I'm having this "problem" right now and your college says that there isn't any ETA for this feature, I would like to go deeper with the resources that we already have.

I don't know how to correlate or identify specific static files causing high bandwidth usage with the RAM or CPU total usage, could you explain this point please?

I have installed the last version of goaccess in a personal computer and run it with the logs in the path that you have specified but it couldn't read the bandwidth usage because of these logs doesn't include this value in the requests (I guess). Is it any way to edit the logs format in order to include it?

Thanks!

(01 Feb '14, 13:41) JavierCane

There is no direct way to see with RAM and CPU usage, you can get a general idea if usage is high, but that is about it.

There is no way to alter the logs.

Your option is to manually read the access logs looking for uploads or downloads, than find the file on disk and see how big it is.

You may also edit your code which does the uploading and have it record the file-sizes in a database or text file.

(01 Feb '14, 23:19) johns

I've taken mesures in order to do not allow uploads of heavy files from now, but I'm still having a lot of bandwidth consumption due to previously uploaded files (I guess) and I'm not able to manually read the entire access logs searching for accesses to large files (it seems like a very pretty tedious task).

¿Is it really impossible to get the Apache mod_log_config installed and include the response size in the logs? http://httpd.apache.org/docs/2.2/mod/mod_log_config.html

(05 Feb '14, 15:22) JavierCane

We'd prefer not to change the format of the logs -- that could break a lot of existing parsers, and demand for this is very light.

However, you should be able to quite easily add this module yourself using a private instance of the shared apache. This also has a lot of other benefits and provides a lot more control. Hope that helps!

(06 Feb '14, 01:37) ryans ♦♦

Hi Nick, thanks for your reply,

I'm very happy to see that you're currently working on a feature to analyze this directly through the CP :)
But since I'm having this "problem" right now and your college says that there isn't any ETA for this feature, I would like to go deeper with the resources that we already have.

I don't know how to correlate or identify specific static files causing high bandwidth usage with the RAM or CPU total usage, could you explain this point please?

I have installed the last version of goaccess in a personal computer and run it with the logs in the path that you have specified but it couldn't read the bandwidth usage because of these logs doesn't include this value in the requests (I guess). Is it any way to edit the logs format in order to include it?

Thanks!

permanent link

answered 01 Feb '14, 13:40

JavierCane
1538
accept rate: 0%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×18
×11
×1

question asked: 31 Jan '14, 08:08

question was seen: 4,961 times

last updated: 06 Feb '14, 01:37

                              
WEBFACTION
REACH US
SUPPORT
LEGAL
© COPYRIGHT 2003-2021 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM