[Piwik-hackers] API: querying data for last X

[Anton Andrievskij]

Hi, Matt.

I researched on this problem little more and I found real bug in the logic.
Follow instructions below and you will see where the problem is:

1. Setup always_archive_data = false

2. Create new html page, add it as new site to Piwik and put JavaScript code
to the page. DO NOT query for statistics for this site (actually it is only
1 page)

3. Make some visits from different browsers and/or machines.

4. Query for last day, then last week, month and for last year. You will see
the same numbers for visits, actions etc. Do you think it is correct now?
You just told me in the previous letter that week, month and year statistics
do not include today's statistics. But we can see all visits here.

Actually Piwik can't find archive built for last week, and it build it
including today's visits. So Piwik does not follow the same logic. See
next...

5. Now you have all archives built. Try to make some more visits and query
for last day, week, month and year again. You will see new visits in the
last day, but you will not see them in last month and year, but you will see
it for last week! So it is totally confusing. Now Piwik does not include new
visits from today (just like you said in your previous message). But it does
it if no archives built in the past.

P.S. If you set always_archive_data = true and ask Piwik for visits for last
day, week, month and year again - it will be ok including all today's
visits.

So Piwik even does not follow your logic described in the previous letter.
It seems that the logic is that Piwik checks whether archive is built
already and simply does not rebuild it. But then it makes problems - you can
see it in example above.

So what I propose is that you must go over the logic again and fix it. It
must NEVER include today's statistics... or ALWAYS include today's
statistics. Or even it is more better to have special configuration like you
said: include_today_statistics_in_period_archives.

Please make it high priority bug / task because it is error in the Piwik's
counting logic and I'm compelled to set always_archive_data = true to avoid
this bug. But it slows Piwik calculation very much. I checked it and I get
~4x performance when I set it back to false.

Please we need this update urgently.

Thanks you again,
Anton Andriyevskyy

On Wed, Jun 4, 2008 at 10:42 AM, Anton Andrievskij <x.meglio at gmail.com>
wrote:

> Hi, Matt.
>
> You wrote:
> >> It would be incorrect as today's stats are never up to date.
>
> Actually when you query visits by weeks, months or years you will always
> get uncompleted data.
>
> Example. If you query for last 3 weeks today, the latest week will be
> 2008-06-02 to 2008-06-08. What you are doing in Piwik is you do not include
> today's visits, but in this case the latest week still remains incomplete
> because there are also future days which are not included in the sum.
>
> So when I query for last week - I actually want to know the real value
> including today's visits - then it makes sense. I think you definitely have
> to add this option in configuration:
> include_today_statistics_in_period_archives. Or else querying for last week,
> month or year have no sense?
>
> The real example where it is interpreted as error is when I want to show to
> users the visits of their pages for last week. They tell me: I asked my
> friend to open this page and then I'm going to see statistics, but there is
> still 0 visits. Why? Every people think that it is error and statistics does
> not work. So I think the idea is to have Piwik working as user expected. It
> will be correct from user's point of view, not from developer's point of
> view. What do you think about it?
>
> Actually I'm forced to configure Piwik to rebuild archives every time - and
> then I get correct statistics and testers are happy then. But then I'm
> worried about speed. Does this configuration really slows Piwik with
> thousands of users/pages?
>
> Thanks for discussion,
> Anton Andriyevskyy
>
> On Wed, Jun 4, 2008 at 4:03 AM, Matthieu Aubry <matthieu.aubry at gmail.com>
> wrote:
>
>>  (In the future please post a new message for each distinct question)
>>
>> This is the behaviour for piwik archives: period statistics (week / month
>> / year statistics) do *not* include today's statistics.
>> It would be incorrect as today's stats are never up to date. This is also
>> for performance reasons (we have to compute these reports only once a day).
>>
>> There could be an option "include_today_statistics_in_period_archives" but
>> I'm not sure about the value here.
>>
>> Anton Andrievskij wrote:
>>
>> When I query for last 2 weeks (or last 2 days) - I get today's visits:
>>
>> http://mysite.com/piwik/?module=API&method=VisitsSummary.get&idSite=2&period=
>> *week*&date=last2&format=xml
>>
>> But if I get for last 2 months (or last 2 years) - I do not get today's
>> visits:
>>
>> http://mysite.com/piwik/?module=API&method=VisitsSummary.get&idSite=2&period=
>> *month*&date=last2&format=xml
>>
>>
>> _______________________________________________
>> Piwik-hackers mailing list
>> Piwik-hackers at piwik.org
>> http://lists.piwik.org/cgi-bin/mailman/listinfo/piwik-hackers
>>
>>
>
>
> --
> Best regards,
> Anton Andriyevskyy
>
> Please think about the environment
> and do not print this e-mail
> unless you really need to.




-- 
Best regards,
Anton Andriyevskyy

Please think about the environment
and do not print this e-mail
unless you really need to.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.piwik.org/pipermail/piwik-hackers/attachments/20080604/d2e3bcbf/attachment-0001.htm