[Spce-user] Fraud calculation takes hours

[Alejandro Grijalba]

Hi.

I've got an instance with Sipwise version mr4.5.7 installed, and from 
time to time i get this query hogging the MySql server.
The server was upgraded from 2.8 (through 3.8) and we didn't notice the 
problem before.

There are 800k entries in cdr table (about 6 months wide), but all of 
them are older than 1 month (if that matters) because this is an upgrade 
test on a snapshot.

I have read that fraud calculation queries were already optimized in 
previous release, so what is wrong?
Thank you.


This is the query ( i guess it is generated from 
/usr/share/perl5/NGCP/Panel/Controller/API/CustomerFraudPreferences.pm ):

/* QUERY */

SELECT bpinfo.id, bpinfo.reseller_id, bpinfo.interval, bpinfo.type,
        IF (bpinfo.fraud_use_reseller_rates > 0, 
SUM(cdr.source_reseller_cost),
SUM(cdr.source_customer_cost)) as interval_cost,
        bpinfo.interval_limit,
        bpinfo.interval_lock,
        bpinfo.interval_notify,
        bpinfo.fraud_use_reseller_rates as use_reseller_rates
FROM (
   SELECT c.id, n.reseller_id, bp.fraud_use_reseller_rates, i.interval,
     IF (i.interval = 'month',
         IF (cfp.fraud_interval_limit > 0,
             'account_limit', 'profile_limit'),
         IF (cfp.fraud_daily_limit > 0,
             'account_limit', 'profile_limit')
        ) AS type,
     IF (i.interval = 'month',
         IF (cfp.fraud_interval_limit > 0,
             cfp.fraud_interval_limit, bp.fraud_interval_limit),
         IF (cfp.fraud_daily_limit > 0,
             cfp.fraud_daily_limit, bp.fraud_daily_limit)
        ) AS interval_limit,
     IF (i.interval = 'month',
         IF (cfp.fraud_interval_limit > 0,
             cfp.fraud_interval_lock, bp.fraud_interval_lock),
         IF (cfp.fraud_daily_limit > 0,
             cfp.fraud_daily_lock, bp.fraud_daily_lock)
        ) AS interval_lock,
     IF (i.interval = 'month',
         IF (cfp.fraud_interval_limit > 0,
             cfp.fraud_interval_notify, bp.fraud_interval_notify),
         IF (cfp.fraud_daily_limit > 0,
             cfp.fraud_daily_notify, bp.fraud_daily_notify)
        ) AS interval_notify
   FROM (SELECT IF('day' = 'month','month','day') AS 'interval') i,
        billing.contracts c
   INNER JOIN (
        SELECT bm1.contract_id,MAX(bm1.id) AS id
        FROM billing.billing_mappings bm1
        INNER JOIN (
               SELECT bm2.contract_id, MAX(bm2.start_date) AS start_date
               FROM billing.billing_mappings bm2
               WHERE (bm2.end_date >= NOW() OR bm2.end_date IS NULL)
                      AND (bm2.start_date <= NOW() OR bm2.start_date IS 
NULL)
               GROUP BY bm2.contract_id
        ) AS max ON bm1.contract_id = max.contract_id AND bm1.start_date 
<=> max.start_date
        GROUP BY bm1.contract_id
   ) AS bm_actual ON bm_actual.contract_id = c.id
   JOIN billing.billing_mappings bm ON bm.id = bm_actual.id
   JOIN billing.billing_profiles bp ON bp.id = bm.billing_profile_id
   JOIN billing.contacts n ON n.id = c.contact_id
   JOIN billing.resellers r ON r.id = n.reseller_id
   LEFT JOIN billing.contract_fraud_preferences cfp ON cfp.contract_id = 
c.id
   WHERE c.status = 'active'
   HAVING interval_limit > 0
) AS bpinfo
JOIN accounting.cdr ON cdr.source_account_id = bpinfo.id
WHERE CASE WHEN bpinfo.interval = 'month'
   THEN cdr.start_time
     BETWEEN UNIX_TIMESTAMP(DATE_FORMAT(NOW(), '%Y-%m-01 00:00:00'))
         AND UNIX_TIMESTAMP(DATE_FORMAT(NOW() + INTERVAL 1 MONTH, 
'%Y-%m-01 00:00:00'))-1
   ELSE cdr.start_time
     BETWEEN UNIX_TIMESTAMP(DATE_FORMAT(NOW(), '%Y-%m-%d 00:00:00'))
         AND UNIX_TIMESTAMP(DATE_FORMAT(NOW() + INTERVAL 1 DAY, 
'%Y-%m-%d 00:00:00'))-1
   END
GROUP BY bpinfo.id
HAVING interval_cost >= interval_limit
) `me`