[Spce-user] CE MR 3.2.1 - API response times when calling "add_voip_account_subscriber" function.

Joel Smith | VOZELIA joel at vozelia.com
Sun Apr 13 20:27:50 EDT 2014


Hi Andreas, 

Thats what I've done. I have to mention: I tried to use the ngcp-reset-db, but that didn't delete all the data, so I ended up manually running "drop database" for each db, and then run again ngcp-reset-db for it to recreate everything. (To start from 0 again.)

Now I've created the 17k accounts (with 1 subscriber each) in 45 minutes and response time is correctly under 0secs. :)


¿Any idea why ngcp-reset-db did not "clean" the db?

Best regards, 
Joel.


----- Original Message -----
> From: "Andreas Granig" <agranig at sipwise.com>
> To: spce-user at lists.sipwise.com
> Sent: Saturday, April 12, 2014 8:49:09 PM
> Subject: Re: [Spce-user] CE MR 3.2.1 - API response times when calling "add_voip_account_subscriber" function.
> 
> Hi Joel,
> 
> Limiting the amount of subscribers per account is indeed the way to go,
> and there shouldn't be any issues managing couple 100k accounts (aka
> customers) with only a few subscribers each.
> 
> Andreas
> 
> On 04/11/2014 08:11 PM, Joel Smith | VOZELIA wrote:
> > Hi Gavin,
> > 
> > Well I've done the following...
> > 
> > When I add a subscriber to the ID #5 account (has already +17k
> > subscribers)I see:
> > 
> >   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
> > 41607 root      20   0  158m  65m 4532 S    62  0.2   0:52.63
> > ngcp-create_sub
> >  3753 mysql     20   0 17.8g 1.3g 8304 S    40  4.3 711:56.63 mysqld
> > 
> > 
> > If I create a new account though speed is normal:
> > 
> > root at spce1:~# time ngcp-create_subscriber -u joel1 -d test.example.com -
> > joeljoel
> > Created VoIP account 8 with one subscriber.
> > 
> > real        0m0.626s
> > user        0m0.588s
> > sys        0m0.028s
> > root at spce1:~# time ngcp-create_subscriber -u joel2 -d test.example.com -p
> > joeljoel
> > Created VoIP account 10 with one subscriber.
> > 
> > real        0m0.761s
> > user        0m0.696s
> > sys        0m0.048s
> > root at spce1:~# time ngcp-create_subscriber -u joel3 -d test.example.com -p
> > joeljoel
> > Created VoIP account 11 with one subscriber.
> > 
> > real        0m0.666s
> > user        0m0.624s
> > sys        0m0.036s
> > root at spce1:~#
> > 
> > 
> > If I specify again account ID #5 it takes for ever:
> > 
> > root at spce1:~# time ngcp-create_subscriber -v 5 -u joel4 -d test.example.com
> > -p joeljoel
> > Added subscriber to VoIP account 5.
> > 
> > real        2m26.386s
> > user        1m19.641s
> > sys        0m10.465s
> > root at spce1:~# time ngcp-create_subscriber -v 5 -u joel5 -d test.example.com
> > -p joeljoel
> > Added subscriber to VoIP account 5.
> > 
> > real        2m22.935s
> > user        1m18.077s
> > sys        0m9.853s
> > root at spce1:~#
> > 
> > I have mysql slow query logging enabled:
> > 
> > root at spce1:/var/log/mysql# grep -e long_query -e slow /etc/mysql/my.cnf
> > log_slow_queries                  = /var/log/mysql/slow-queries.log
> > long_query_time                   = 5
> > root at spce1:/var/log/mysql#
> > 
> > 
> > But nothing there either:
> > 
> > root at spce1:/var/log/mysql# tail slow-queries.log
> > Time                 Id Command    Argument
> > /usr/sbin/mysqld, Version: 5.5.35-0+wheezy1-log ((Debian)). started with:
> > Tcp port: 0  Unix socket: /var/run/mysqld/mysqld.sock
> > Time                 Id Command    Argument
> > /usr/sbin/mysqld, Version: 5.5.35-0+wheezy1-log ((Debian)). started with:
> > Tcp port: 0  Unix socket: /var/run/mysqld/mysqld.sock
> > Time                 Id Command    Argument
> > /usr/sbin/mysqld, Version: 5.5.35-0+wheezy1-log ((Debian)). started with:
> > Tcp port: 0  Unix socket: /var/run/mysqld/mysqld.sock
> > Time                 Id Command    Argument
> > root at spce1:/var/log/mysql#
> > 
> > 
> > I don't know perl, but I've tried to follow the code anyway and It seems
> > that there are a couple of "for" loops in the add_voip_account_subscriber
> > function that go through all the subscribers belonging to the account for
> > some checks, so with a +17k subscriber account it takes long. This is only
> > what I think can be the problem because with a new account it takes
> > <0secs.
> > 
> > Also, in the Admin Panel, if I try to click on the Customers "detail"
> > option, I get a 504 Gateway Time-out from NGINX. If I do the same with an
> > account with 1 subscriber, the next menu appears correctly. I'm pretty
> > sure that having a huge number of subscribers in the same account is the
> > reason because in the Customers Detail page you can see the subscribers
> > and their registration status...
> > 
> > If I can't get this fixed my workaround would be to reset the database and
> > instead of creating 1 account with 20k subscribers, I would create 20k
> > different accounts with 1 subscriber each (I don't know if that will have
> > other issues but at least I think I'll get over this particular one).
> > 
> > Said all of this... any ideas?
> > 
> > 
> > Thanks!
> > 
> > Best regards,
> > Joel.
> > 
> > ----- Original Message -----
> >> From: "Gavin Sweet" <gavin.sweet at skyracktelecom.com>
> >> To: "Joel Smith | VOZELIA" <joel at vozelia.com>
> >> Sent: Friday, April 11, 2014 6:27:18 PM
> >> Subject: RE: [Spce-user] CE MR 3.2.1 - API response times when calling
> >> "add_voip_account_subscriber" function.
> >>
> >> Hi Joel -
> >> have you looked in the logs?
> >> Do you have performance issues with any other aspects of the server? Have
> >> you
> >> checked if there is some general resource problem ... top, mysql slow
> >> query
> >> etc
> >>
> >> Cheers
> >> Gavin
> >>
> >>> -----Original Message-----
> >>> From: spce-user-bounces at lists.sipwise.com [mailto:spce-user-
> >>> bounces at lists.sipwise.com] On Behalf Of Joel Smith | VOZELIA
> >>> Sent: 11 April 2014 16:50
> >>> To: Spce-user
> >>> Subject: Re: [Spce-user] CE MR 3.2.1 - API response times when calling
> >>> "add_voip_account_subscriber" function.
> >>>
> >>> Hi again,
> >>>
> >>> Could the problem be having 17k subscribers under the same account?
> >>>
> >>> Maybe the correct way should be to have a separate account for each
> >>> subscriber. (Although that doesn't make too much sense)
> >>>
> >>> The users are created with minimum info:
> >>>
> >>> SIP User
> >>> SIP Pass
> >>> External #ID
> >>>
> >>>
> >>> That is it, no numbers defined, no alias, etc.
> >>>
> >>>
> >>> I really don't understand, any ideas are more than welcome!!
> >>>
> >>>
> >>> Thank you in advanced.
> >>>
> >>> Best regards,
> >>> Joel.
> >>>
> >>>
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Joel Smith | VOZELIA" <joel at vozelia.com>
> >>>> To: "Spce-user" <spce-user at lists.sipwise.com>
> >>>> Sent: Friday, April 11, 2014 4:50:41 PM
> >>>> Subject: Re: CE MR 3.2.1 - API response times when calling
> >>> "add_voip_account_subscriber" function.
> >>>>
> >>>> Hi,
> >>>>
> >>>> This is getting worse. I've tried the NGCP's own script to create a
> >>>> subscriber:
> >>>>
> >>>>
> >>>> root at spce1:/usr/bin# time ./ngcp-create_subscriber -v 5 -u joel -d
> >>>> sip.testdomain.com -p joeljoel
> >>>> Added subscriber to VoIP account 5.
> >>>>
> >>>> real        2m15.588s
> >>>> user        1m14.449s
> >>>> sys        0m8.893s
> >>>> root at spce1:/usr/bin#
> >>>>
> >>>>
> >>>> Can anybody guide me how to debug problems with the
> >>>> "add_voip_account_subscriber" function?
> >>>>
> >>>> Any other request takes not even a second (example: update password,
> >>> set
> >>>> adm_block_mode_out, etc.)
> >>>>
> >>>>
> >>>> Any idea on what can be happening?
> >>>>
> >>>> Best regards,
> >>>> Joel.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Joel Smith | VOZELIA" <joel at vozelia.com>
> >>>>> To: "Spce-user" <spce-user at lists.sipwise.com>
> >>>>> Sent: Thursday, April 10, 2014 2:57:05 PM
> >>>>> Subject: CE MR 3.2.1 - API response times when calling
> >>>>> "add_voip_account_subscriber" function.
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> We are experiencing a huge delay on the time the API takes to
> >>> create a new
> >>>>> subscriber.
> >>>>>
> >>>>> We have to create around 20k subscribers. At the beggining the
> >>> creation was
> >>>>> very fast, as the number of existing subscribers grows the create
> >>> execution
> >>>>> slows down.
> >>>>>
> >>>>> The server is not in production yet so there should be no load
> >>> issues.
> >>>>>
> >>>>>
> >>>>> At the moment it takes around 8-9seconds to create a subscriber:
> >>>>>
> >>>>> root at spce1:~# time ./test_adduser.php user1
> >>>>> OK
> >>>>>
> >>>>> real        0m8.194s
> >>>>> user        0m0.016s
> >>>>> sys        0m0.008s
> >>>>> root at spce1:~#
> >>>>>
> >>>>>
> >>>>> root at spce1:~# time ./test_adduser.php user2
> >>>>> OK
> >>>>>
> >>>>> real        0m9.156s
> >>>>> user        0m0.024s
> >>>>> sys        0m0.004s
> >>>>> root at spce1:~#
> >>>>>
> >>>>>
> >>>>>
> >>>>> The adduser code is really simple:
> >>>>>
> >>>>> root at spce1:~# cat test_adduser.php
> >>>>> #!/usr/bin/php
> >>>>> <?php
> >>>>>         $soap_url =
> >>> "https://127.0.0.1:2443/SOAP/Provisioning.wsdl?plain";
> >>>>>         $auth = array('username' => 'testuser',
> >>>>>                                 'password' => 'testpass',
> >>>>>                                 'type' => 'admin');
> >>>>>
> >>>>>         $client_options = array('login' => 'ngcpsoap',
> >>>>>                                                         'password'
> >>> =>
> >>>>>
> >>> 'ngcptestpass');
> >>>>>
> >>>>>         $params1 = array('id' => 5,
> >>>>>                                         'subscriber' => array(
> >>>>>
> >>> "username"
> >>>>>                                                                 =>
> >>>>>
> >>> $argv[1],
> >>>>>
> >>> "domain" =>
> >>>>>
> >>> "sip.testdomain.com",
> >>>>>
> >>> "password"
> >>>>>                                                                 =>
> >>>>>
> >>> "hjklpojH"
> >>>>>                                                         )
> >>>>>                                         );
> >>>>>
> >>>>>         try{
> >>>>>                 $client = new SoapClient($soap_url,
> >>> $client_options);
> >>>>>                 $ret1 = $client->add_voip_account_subscriber($auth,
> >>>>>                 $params1);
> >>>>>                 echo "OK \n";
> >>>>>         } catch (SoapFault $exception) {
> >>>>>                 echo $exception."\n";
> >>>>>         }
> >>>>> ?>
> >>>>> root at spce1:~#
> >>>>>
> >>>>>
> >>>>> Is there anything I can check to see why it takes soooooooooooo
> >>> long to
> >>>>> create?
> >>>>>
> >>>>> If I create a subscriber through the GUI, it takes 0seconds.
> >>>>>
> >>>>>
> >>>>> Another thing to say, for each subscriber we create, we set the
> >>>>> adm_block_out_mode to TRUE, this also takes 0sec (through the API)
> >>> so the
> >>>>> problem is directly related to add_voip_account_subscriber.
> >>>>>
> >>>>>
> >>>>> Any idea on this? Please let me know if there is more information
> >>> required
> >>>>> to
> >>>>> search for the root cause of this problem.
> >>>>>
> >>>>>
> >>>>> Thanks in advanced.
> >>>>>
> >>>>> Best regards,
> >>>>> Joel.
> >>>>>
> >>>>>
> >>>
> >>> _______________________________________________
> >>> Spce-user mailing list
> >>> Spce-user at lists.sipwise.com
> >>> http://lists.sipwise.com/listinfo/spce-user
> >>>
> >>> -----
> >>> No virus found in this message.
> >>> Checked by AVG - www.avg.com
> >>> Version: 2014.0.4355 / Virus Database: 3882/7321 - Release Date:
> >>> 04/09/14
> >>
> >>
> > 
> > _______________________________________________
> > Spce-user mailing list
> > Spce-user at lists.sipwise.com
> > http://lists.sipwise.com/listinfo/spce-user
> > 
> 
> _______________________________________________
> Spce-user mailing list
> Spce-user at lists.sipwise.com
> http://lists.sipwise.com/listinfo/spce-user
> 




More information about the Spce-user mailing list