[Spce-user] CE MR 3.2.1 - API response times when calling "add_voip_account_subscriber" function.

Andreas Granig agranig at sipwise.com
Sat Apr 12 14:49:09 EDT 2014


Hi Joel,

Limiting the amount of subscribers per account is indeed the way to go,
and there shouldn't be any issues managing couple 100k accounts (aka
customers) with only a few subscribers each.

Andreas

On 04/11/2014 08:11 PM, Joel Smith | VOZELIA wrote:
> Hi Gavin, 
> 
> Well I've done the following...
> 
> When I add a subscriber to the ID #5 account (has already +17k subscribers)I see:
> 
>   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                                                                                      
> 41607 root      20   0  158m  65m 4532 S    62  0.2   0:52.63 ngcp-create_sub                                                                                              
>  3753 mysql     20   0 17.8g 1.3g 8304 S    40  4.3 711:56.63 mysqld      
> 
> 
> If I create a new account though speed is normal:
> 
> root at spce1:~# time ngcp-create_subscriber -u joel1 -d test.example.com - joeljoel
> Created VoIP account 8 with one subscriber.
> 
> real        0m0.626s
> user        0m0.588s
> sys        0m0.028s
> root at spce1:~# time ngcp-create_subscriber -u joel2 -d test.example.com -p joeljoel
> Created VoIP account 10 with one subscriber.
> 
> real        0m0.761s
> user        0m0.696s
> sys        0m0.048s
> root at spce1:~# time ngcp-create_subscriber -u joel3 -d test.example.com -p joeljoel
> Created VoIP account 11 with one subscriber.
> 
> real        0m0.666s
> user        0m0.624s
> sys        0m0.036s
> root at spce1:~# 
> 
> 
> If I specify again account ID #5 it takes for ever:
> 
> root at spce1:~# time ngcp-create_subscriber -v 5 -u joel4 -d test.example.com -p joeljoel
> Added subscriber to VoIP account 5.
> 
> real        2m26.386s
> user        1m19.641s
> sys        0m10.465s
> root at spce1:~# time ngcp-create_subscriber -v 5 -u joel5 -d test.example.com -p joeljoel
> Added subscriber to VoIP account 5.
> 
> real        2m22.935s
> user        1m18.077s
> sys        0m9.853s
> root at spce1:~# 
> 
> I have mysql slow query logging enabled:
> 
> root at spce1:/var/log/mysql# grep -e long_query -e slow /etc/mysql/my.cnf
> log_slow_queries                  = /var/log/mysql/slow-queries.log
> long_query_time                   = 5
> root at spce1:/var/log/mysql# 
> 
> 
> But nothing there either:
> 
> root at spce1:/var/log/mysql# tail slow-queries.log 
> Time                 Id Command    Argument
> /usr/sbin/mysqld, Version: 5.5.35-0+wheezy1-log ((Debian)). started with:
> Tcp port: 0  Unix socket: /var/run/mysqld/mysqld.sock
> Time                 Id Command    Argument
> /usr/sbin/mysqld, Version: 5.5.35-0+wheezy1-log ((Debian)). started with:
> Tcp port: 0  Unix socket: /var/run/mysqld/mysqld.sock
> Time                 Id Command    Argument
> /usr/sbin/mysqld, Version: 5.5.35-0+wheezy1-log ((Debian)). started with:
> Tcp port: 0  Unix socket: /var/run/mysqld/mysqld.sock
> Time                 Id Command    Argument
> root at spce1:/var/log/mysql# 
> 
> 
> I don't know perl, but I've tried to follow the code anyway and It seems that there are a couple of "for" loops in the add_voip_account_subscriber function that go through all the subscribers belonging to the account for some checks, so with a +17k subscriber account it takes long. This is only what I think can be the problem because with a new account it takes <0secs.
> 
> Also, in the Admin Panel, if I try to click on the Customers "detail" option, I get a 504 Gateway Time-out from NGINX. If I do the same with an account with 1 subscriber, the next menu appears correctly. I'm pretty sure that having a huge number of subscribers in the same account is the reason because in the Customers Detail page you can see the subscribers and their registration status...
> 
> If I can't get this fixed my workaround would be to reset the database and instead of creating 1 account with 20k subscribers, I would create 20k different accounts with 1 subscriber each (I don't know if that will have other issues but at least I think I'll get over this particular one).
> 
> Said all of this... any ideas?
> 
> 
> Thanks!
> 
> Best regards, 
> Joel.
> 
> ----- Original Message -----
>> From: "Gavin Sweet" <gavin.sweet at skyracktelecom.com>
>> To: "Joel Smith | VOZELIA" <joel at vozelia.com>
>> Sent: Friday, April 11, 2014 6:27:18 PM
>> Subject: RE: [Spce-user] CE MR 3.2.1 - API response times when calling "add_voip_account_subscriber" function.
>>
>> Hi Joel -
>> have you looked in the logs?
>> Do you have performance issues with any other aspects of the server? Have you
>> checked if there is some general resource problem ... top, mysql slow query
>> etc
>>
>> Cheers
>> Gavin
>>
>>> -----Original Message-----
>>> From: spce-user-bounces at lists.sipwise.com [mailto:spce-user-
>>> bounces at lists.sipwise.com] On Behalf Of Joel Smith | VOZELIA
>>> Sent: 11 April 2014 16:50
>>> To: Spce-user
>>> Subject: Re: [Spce-user] CE MR 3.2.1 - API response times when calling
>>> "add_voip_account_subscriber" function.
>>>
>>> Hi again,
>>>
>>> Could the problem be having 17k subscribers under the same account?
>>>
>>> Maybe the correct way should be to have a separate account for each
>>> subscriber. (Although that doesn't make too much sense)
>>>
>>> The users are created with minimum info:
>>>
>>> SIP User
>>> SIP Pass
>>> External #ID
>>>
>>>
>>> That is it, no numbers defined, no alias, etc.
>>>
>>>
>>> I really don't understand, any ideas are more than welcome!!
>>>
>>>
>>> Thank you in advanced.
>>>
>>> Best regards,
>>> Joel.
>>>
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Joel Smith | VOZELIA" <joel at vozelia.com>
>>>> To: "Spce-user" <spce-user at lists.sipwise.com>
>>>> Sent: Friday, April 11, 2014 4:50:41 PM
>>>> Subject: Re: CE MR 3.2.1 - API response times when calling
>>> "add_voip_account_subscriber" function.
>>>>
>>>> Hi,
>>>>
>>>> This is getting worse. I've tried the NGCP's own script to create a
>>>> subscriber:
>>>>
>>>>
>>>> root at spce1:/usr/bin# time ./ngcp-create_subscriber -v 5 -u joel -d
>>>> sip.testdomain.com -p joeljoel
>>>> Added subscriber to VoIP account 5.
>>>>
>>>> real        2m15.588s
>>>> user        1m14.449s
>>>> sys        0m8.893s
>>>> root at spce1:/usr/bin#
>>>>
>>>>
>>>> Can anybody guide me how to debug problems with the
>>>> "add_voip_account_subscriber" function?
>>>>
>>>> Any other request takes not even a second (example: update password,
>>> set
>>>> adm_block_mode_out, etc.)
>>>>
>>>>
>>>> Any idea on what can be happening?
>>>>
>>>> Best regards,
>>>> Joel.
>>>>
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Joel Smith | VOZELIA" <joel at vozelia.com>
>>>>> To: "Spce-user" <spce-user at lists.sipwise.com>
>>>>> Sent: Thursday, April 10, 2014 2:57:05 PM
>>>>> Subject: CE MR 3.2.1 - API response times when calling
>>>>> "add_voip_account_subscriber" function.
>>>>>
>>>>> Hi,
>>>>>
>>>>> We are experiencing a huge delay on the time the API takes to
>>> create a new
>>>>> subscriber.
>>>>>
>>>>> We have to create around 20k subscribers. At the beggining the
>>> creation was
>>>>> very fast, as the number of existing subscribers grows the create
>>> execution
>>>>> slows down.
>>>>>
>>>>> The server is not in production yet so there should be no load
>>> issues.
>>>>>
>>>>>
>>>>> At the moment it takes around 8-9seconds to create a subscriber:
>>>>>
>>>>> root at spce1:~# time ./test_adduser.php user1
>>>>> OK
>>>>>
>>>>> real        0m8.194s
>>>>> user        0m0.016s
>>>>> sys        0m0.008s
>>>>> root at spce1:~#
>>>>>
>>>>>
>>>>> root at spce1:~# time ./test_adduser.php user2
>>>>> OK
>>>>>
>>>>> real        0m9.156s
>>>>> user        0m0.024s
>>>>> sys        0m0.004s
>>>>> root at spce1:~#
>>>>>
>>>>>
>>>>>
>>>>> The adduser code is really simple:
>>>>>
>>>>> root at spce1:~# cat test_adduser.php
>>>>> #!/usr/bin/php
>>>>> <?php
>>>>>         $soap_url =
>>> "https://127.0.0.1:2443/SOAP/Provisioning.wsdl?plain";
>>>>>         $auth = array('username' => 'testuser',
>>>>>                                 'password' => 'testpass',
>>>>>                                 'type' => 'admin');
>>>>>
>>>>>         $client_options = array('login' => 'ngcpsoap',
>>>>>                                                         'password'
>>> =>
>>>>>
>>> 'ngcptestpass');
>>>>>
>>>>>         $params1 = array('id' => 5,
>>>>>                                         'subscriber' => array(
>>>>>
>>> "username"
>>>>>                                                                 =>
>>>>>
>>> $argv[1],
>>>>>
>>> "domain" =>
>>>>>
>>> "sip.testdomain.com",
>>>>>
>>> "password"
>>>>>                                                                 =>
>>>>>
>>> "hjklpojH"
>>>>>                                                         )
>>>>>                                         );
>>>>>
>>>>>         try{
>>>>>                 $client = new SoapClient($soap_url,
>>> $client_options);
>>>>>                 $ret1 = $client->add_voip_account_subscriber($auth,
>>>>>                 $params1);
>>>>>                 echo "OK \n";
>>>>>         } catch (SoapFault $exception) {
>>>>>                 echo $exception."\n";
>>>>>         }
>>>>> ?>
>>>>> root at spce1:~#
>>>>>
>>>>>
>>>>> Is there anything I can check to see why it takes soooooooooooo
>>> long to
>>>>> create?
>>>>>
>>>>> If I create a subscriber through the GUI, it takes 0seconds.
>>>>>
>>>>>
>>>>> Another thing to say, for each subscriber we create, we set the
>>>>> adm_block_out_mode to TRUE, this also takes 0sec (through the API)
>>> so the
>>>>> problem is directly related to add_voip_account_subscriber.
>>>>>
>>>>>
>>>>> Any idea on this? Please let me know if there is more information
>>> required
>>>>> to
>>>>> search for the root cause of this problem.
>>>>>
>>>>>
>>>>> Thanks in advanced.
>>>>>
>>>>> Best regards,
>>>>> Joel.
>>>>>
>>>>>
>>>
>>> _______________________________________________
>>> Spce-user mailing list
>>> Spce-user at lists.sipwise.com
>>> http://lists.sipwise.com/listinfo/spce-user
>>>
>>> -----
>>> No virus found in this message.
>>> Checked by AVG - www.avg.com
>>> Version: 2014.0.4355 / Virus Database: 3882/7321 - Release Date:
>>> 04/09/14
>>
>>
> 
> _______________________________________________
> Spce-user mailing list
> Spce-user at lists.sipwise.com
> http://lists.sipwise.com/listinfo/spce-user
> 




More information about the Spce-user mailing list