[Spce-user] URGENT: mysql database ist growing until disk is full (mr6.5.4.1)

Fri May 10 06:55:24 EDT 2019

We found the fowllowing:

root at spce:~# ls -lah /var/lib/mysql/kamailio/
total 62G
drwx------  2 mysql mysql 4.0K May  5 10:43 .
drwxr-xr-x 17 mysql mysql 4.0K May 10 12:35 ..
-rw-rw----  1 mysql mysql  66K May  5 10:43 acc.frm
-rw-rw----  1 mysql mysql 112K May  5 10:43 acc.ibd
-rw-rw----  1 mysql mysql  66K May  5 10:43 acc_backup.frm
-rw-rw----  1 mysql mysql  44M May 10 12:41 acc_backup.ibd
-rw-rw----  1 mysql mysql 1.5K Sep 18  2018 acc_cdrs.frm
-rw-rw----  1 mysql mysql 112K Sep 18  2018 acc_cdrs.ibd
-rw-rw----  1 mysql mysql  66K May  5 10:43 acc_trash.frm
-rw-rw----  1 mysql mysql  62G May 10 12:41 acc_trash.ibd

It seems that kamilio is eating all of our disk-space.
Why do we get as much  of "trash"? How can we stop that and delete the "trash"?

Regards
Thomas

Von: Thomas Georg
Gesendet: Freitag, 10. Mai 2019 12:38
An: spce-user at lists.sipwise.com
Betreff: URGENT: mysql database ist growing until disk is full (mr6.5.4.1)

Hi list,

we currently have a usage of 77%, 74% after removing the logs.
We won't get the service running over the weekend.

Is there anybody out there who can help us with this problem?

Regards
Thomas

Von: Thomas Georg
Gesendet: Freitag, 10. Mai 2019 07:28
An: spce-user at lists.sipwise.com<mailto:spce-user at lists.sipwise.com>
Betreff: mysql database ist growing until disk is full (mr6.5.4.1)

Good morning list,

since update from 6.4.1 to 6.5.4.1 our mysql database is growing and growing (about 4 Mbyte/s).
Yesterday the HDD was full and the system stopped.
After removing the log-bin.xxxx files ("reset master" in mysql) the disk usage has been at 59%. 24 hours later the disk usage is 71% (after removing the logs).
It seems that something writes constantly to the database.

In redis.log we get the following some hundred times per second!!!

May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid 'ae43124ae164853b' using key 'acc:entry::ae43124ae164853b:1556890399.299945'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '3440-417-43201983341-SS7_FRA1_1_1-0-93.189.171.91' using key 'acc:entry::3440-417-43201983341-SS7_FRA1_1_1-0-93.189.171.91:1556872562.288035'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '3440-417-43201983341-SS7_FRA1_1_1-0-93.189.171.91' using key 'acc:entry::3440-417-43201983341-SS7_FRA1_1_1-0-93.189.171.91:1556872429.108932'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '724b23507efbb3bc' using key 'acc:entry::724b23507efbb3bc:1556886822.814075'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '724b23507efbb3bc' using key 'acc:entry::724b23507efbb3bc:1556887162.785141'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '2a14180c13a0b043' using key 'acc:entry::2a14180c13a0b043:1556872760.720813'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '2a14180c13a0b043' using key 'acc:entry::2a14180c13a0b043:1556872656.970070'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid 'RQmmUzkV9xHMx6DRPYu9_Q..' using key 'acc:entry::RQmmUzkV9xHMx6DRPYu9_Q..:1556866810.751553'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid 'd2be9b99-e834-1237-319b-49ee50de5a93' using key 'acc:entry::d2be9b99-e834-1237-319b-49ee50de5a93:1556880990.788435'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid 'd2be9b99-e834-1237-319b-49ee50de5a93' using key 'acc:entry::d2be9b99-e834-1237-319b-49ee50de5a93:1556880945.826894'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '12c7fd66-e813-1237-56b9-ebb650d1e42d' using key 'acc:entry::12c7fd66-e813-1237-56b9-ebb650d1e42d:1556866455.055911'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '12c7fd66-e813-1237-56b9-ebb650d1e42d' using key 'acc:entry::12c7fd66-e813-1237-56b9-ebb650d1e42d:1556866540.224351'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid 'IMlyhHBuqMyUcQrIzzevIA..' using key 'acc:entry::IMlyhHBuqMyUcQrIzzevIA..:1556891812.437265'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid 'IMlyhHBuqMyUcQrIzzevIA..' using key 'acc:entry::IMlyhHBuqMyUcQrIzzevIA..:1556891836.441038'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '3218c40fc8dc84f2' using key 'acc:entry::3218c40fc8dc84f2:1556875485.599718'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '3218c40fc8dc84f2' using key 'acc:entry::3218c40fc8dc84f2:1556875481.255913'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '447e-424-43201912145-SS7_FRA1_1_1-0-93.189.171.91' using key 'acc:entry::447e-424-43201912145-SS7_FRA1_1_1-0-93.189.171.91:1556884906.228929'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '7b91-511-43201912557-SS7_FRA1_1_1-0-93.189.171.91' using key 'acc:entry::7b91-511-43201912557-SS7_FRA1_1_1-0-93.189.171.91:1556888191.170942'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '7b91-511-43201912557-SS7_FRA1_1_1-0-93.189.171.91' using key 'acc:entry::7b91-511-43201912557-SS7_FRA1_1_1-0-93.189.171.91:1556888114.489201'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '22be-44c-442019103619-SS7_FRA1_1_1-0-93.189.171.91' using key 'acc:entry::22be-44c-442019103619-SS7_FRA1_1_1-0-93.189.171.91:1556966271.279869'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid '22be-44c-442019103619-SS7_FRA1_1_1-0-93.189.171.91' using key 'acc:entry::22be-44c-442019103619-SS7_FRA1_1_1-0-93.189.171.91:1556966188.221314'
May 10 07:24:04 spce (warning) mediator[5130]: medredis.c:303 [medredis_check_reply_string]: Received Redis reply type 4 instead of 1 (string) for branch_id field of cid 'FEC6A23B117F676F at 185.156.157.66' using key 'acc:entry::FEC6A23B117F676F at 185.156.157.66:1556886650.827638'

We urgently need help before the weekend :/

Regards Thomas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sipwise.com/pipermail/spce-user_lists.sipwise.com/attachments/20190510/0b90f312/attachment-0001.html>