[Piwik-hackers] Piwik very slow for whatever reasons

Mon Apr 7 22:36:37 CEST 2008

Actually I agree it's not a long term option, but not for the reason you 
raise.

The archiving is done ONCE per finished day, once per finished week, 
once per finished month, etc. Optimally it should be done with a crontab 
at night at around 1AM. Doing so would make sure that during all the 
day, piwik only reads the "already processed" archives. For example the 
5 widgets would simply "select" data from the database and it would be 
ultra fast.

We still have to do high load testing & optimizations, it's on the 
roadmap but we are talking to companies so they help us in this task by 
providing development ressources (if you want to help with development 
thats great).

Pelle Boese wrote:
> Raising the max_execution_time is imho not an option. Even "Week"
> results in timeouts. Let's say for "Week" it would take 60 seconds AKA
> 1 minute. That would make like 4 minutes for "Month" and around 48
> minutes for "Year". And that for each active widget. On my Dashboard
> there are currently 5 widgets, thus 5 apache childs / threads /
> whatever would be busy for 48 minutes. And that's for a site with
> quite low traffic and only one user on piwik. I don't really want to
> imagine, what would happen with many piwik users and a lot of high
> traffic sites. Maybe it's a problem with my setup, I just don't know.
> Is anyone else running piwik with more than 1.000 visits per day?
>
> 2008/4/7, Matthieu Aubry <matthieu.aubry at gmail.com>:
>   
>>  Have you tried the command line in SSH
>>  /usr/bin/php index.php --
>> "module=API&method=VisitsSummary.getVisits&idSite=1&period=year&date=today&format=original"
>>
>>  For the timeout, just inscrease the limit in your php.ini.
>>
>>  The objective is to put the line above in a crontab so it executes every
>> night at 1AM.
>>
>>
>>  Pelle Boese wrote:
>>  The page itself is working fine, but the widgets time out. List of
>> Keywords, Visit Length, Visitor Browsers etc... for "Day" it's working
>> just fine, but "Week", "Month" and "Year" result in timeouts. If you
>> want access to see yourself, just drop me a line
>>
>> Cheers, Pelle.
>>
>> 2008/4/7, Matthieu Aubry <matthieu.aubry at gmail.com>:
>>
>>
>>  Ok, download piwik 0.1.4 and try in the command line:
>>  /usr/bin/php index.php --
>> "module=API&method=VisitsSummary.getVisits&idSite=1&period=year&date=today&format=original"
>>
>>  If you still have the 30s error, then increase the max_execution_time in
>> your php.ini
>>
>>
>>
>>  Pelle Boese wrote:
>>  Yeah, I already set the indexes from bug #150, that helped a bit, but
>> the massive load comes through apache, not mysql, so it must be
>> something else... but i really have no clue
>>
>> 2008/4/7, Matthieu Aubry <matthieu.aubry at gmail.com>:
>>
>>
>>  Hi
>> Have you installed piwik from a fresh 0.1.3 or did you update from a old
>> version?
>>
>> Please check that your mysql table log_link_visit_action has an index
>> index_idvisit
>> Also the log_visit table should have index_visit_server_date index.
>>
>> If it doesn't, execute:
>> CREATE INDEX index_idvisit ON log_link_visit_action (idvisit);
>> CREATE INDEX index_idaction ON log_action (idaction);
>> CREATE INDEX index_idsite ON log_visit (idsite);
>>
>> CREATE INDEX index_visit_server_date ON log_visit (visit_server_date);
>> (see http://dev.piwik.org/trac/ticket/150)
>>
>>
>>
>> On Mon, Apr 7, 2008 at 4:17 PM, Pelle Boese <pelle at concept08.com> wrote:
>>
>>
>>
>>  Cheers,
>>
>> I'm using piwik for a bit now and for sites with more than like 5.000
>> unique visitors per day it's almost impossible to get the statistics
>> as it takes too much time to load them. I'm running the latest beta
>> (0.1.3) on a quite bored quad xeon with 4gb ram and a scsi raid, so
>> the hardware is definitely not the bottleneck. As soon as I open the
>> piwik dashboard for this month (stats for one day work), my 'top'
>> looks like that:
>>
>> 17713 www-data 25 0 214m 72m 9572 R 100 1.9 1:00.00 apache2
>> 1343 mysql 15 0 128m 52m 5868 S 2 1.4 420:21.34 mysqld
>>
>> after 30 seconds I get one of the following errors for all widgets
>> while the flash loads forever:
>>
>> Fatal error: Maximum execution time of 30 seconds exceeded in
>> /var/www/stats/piwik/modules/DataTable.php on line 861
>>
>> Fatal error: Maximum execution time of 30 seconds exceeded in
>> /var/www/stats/piwik/modules/DataTable/Row.php on line
>>
>>  153
>>
>>
>>  As the apache2 child uses 100% CPU, it's probably not a database
>> problem, but something else going completely wrong in the code.
>>
>> Any suggestions on that problem?
>>
>> I'm btw running the following setup:
>>
>> Debian Sarge
>> Apache 2.2.3 with mpm_prefork
>> PHP 5.2.0 with suhosin
>> MySQL 5.0.32
>>
>> Cheers, Pelle.
>>
>> --
>> Pelle Boese
>> Online Marketing
>>
>> Concept08 GmbH
>> Grüngürtelstrasse 77
>> 50996 Köln
>>
>> Tel: 0172 373.94.56
>> E-Mail: pelle at concept08.com
>> Web: http://www.concept08.com/
>> _______________________________________________
>> Piwik-hackers mailing list
>> Piwik-hackers at piwik.org
>>
>>
>> http://lists.piwik.org/cgi-bin/mailman/listinfo/piwik-hackers
>>
>>  _______________________________________________
>>  Piwik-hackers mailing list
>>  Piwik-hackers at piwik.org
>> http://lists.piwik.org/cgi-bin/mailman/listinfo/piwik-hackers
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>>  Piwik-hackers mailing list
>>  Piwik-hackers at piwik.org
>> http://lists.piwik.org/cgi-bin/mailman/listinfo/piwik-hackers
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>>  Piwik-hackers mailing list
>>  Piwik-hackers at piwik.org
>> http://lists.piwik.org/cgi-bin/mailman/listinfo/piwik-hackers
>>
>>
>>     
>
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.piwik.org/pipermail/piwik-hackers/attachments/20080407/d3ee391d/attachment-0001.htm 