[Spce-user] sip registration stuck 3.5.16
Theo
axessofficetheo at gmail.com
Fri Nov 21 10:25:55 EST 2014
ngcp-sercmd proxy ul.dump
error: 500 - Internal error creating aor struct
Any ideas on that?
On Fri, Nov 21, 2014 at 5:10 PM, Andreas Granig <agranig at sipwise.com> wrote:
> Just to add my 2 cents:
>
> The proxy (kamailio) uses in-memory caching for registrations and writes
> them back to DB. The panel displays the registrations from the DB.
>
> Now if you delete something from the DB, it's not reflected in the
> proxy, because it still will be in the cache. So what we're doing is to
> delete a registration from the cache (by telling the proxy to purge it),
> and the proxy will propagate it down to the DB.
>
> The same happens if the proxy detects that a registration has expired.
> It will delete it from the cache and propagate the deletion down to the DB.
>
> Now here is where it's getting interesting: if the registration is no
> longer in the cache (for whatever reason) and it still is in the DB (for
> whatever else reason), you see it in the panel and can click on the
> delete button, but as the deletion request is sent to the proxy, it will
> just do nothing because it doesn't have it in the cache, so it will
> still persist in the DB.
>
> This scenario should never happen (famous last words), so to
> troubleshoot it, you'd need to dump the registrations from the cache
> using "ngcp-sercmd proxy ul.dump" and check the count (see
> "Stats"->"Records") and compare it to the db by doing "mysql kamailio -e
> "select id, username, domain, expires from location".
>
> Take into account that the cache count can/will potentially be lower
> than the db count because the proxy batches deletion in a separate
> process, which is executed every 60 sec. So to fetch the "real" expired
> registrations, do this: "mysql kamailio -e "select id, username, domain,
> expires from location where expires < date_sub(now(), interval 90 second)"
>
> So this will give you a representation of your current state. Note that
> if a registration is not in the cache but only in the DB, it's not a
> real operational issue, because the proxy will not know about it.
>
> It still doesn't answer the question how this can even happen, so you
> need to provide a full trace ("tcpdump -i any -s 0 -w /tmp/reg.pcap port
> 5060") of a complete cycle (fresh registration and subsequent
> registrations, which will then cause the issue you described), along
> with the full kamailio-proxy.log, so we can check for errors etc. Also
> provide the output of "ngcp-sercmd proxy ul.dump" and "mysql kamailio -e
> "select id, username, domain, expires from location" along with the
> timestamps when you executed those commands.
>
> Andreas
>
> On 11/21/2014 03:41 PM, Raúl Alexis Betancor Santana wrote:
> >
> > Helo Aziz ... let's talk about ...
> >
> > 1- Doesn't matter ... if you have more than one register for the same
> > user, the only thing that happens it's that on an outbound request to
> > that subscriber, SPCE will launch 2 branches ... one will fail and the
> > other one will sucess. No big deal at all.
> > 2- That should not happens ... as expired registration should be
> > auto-erased. Have you installed the system from 0 ? .. or did you
> > upgrade from older version?
> > 3- That sounds as an upgrades SPCE version that didn't complete the
> upgrade.
> >
> >
> > ------------------------------------------------------------------------
> > *De: *"Aziz Palmer" <Aziz.Palmer at AirFIBER.com>
> > *Para: *"Raúl Alexis Betancor Santana" <rabs at dimension-virtual.com>,
> > spce-user at lists.sipwise.com
> > *Enviados: *Viernes, 21 de Noviembre 2014 14:31:25
> > *Asunto: *RE: sip registration stuck 3.5.16
> >
> > Thank you Raul for your input and suggestions. Not to get into a long
> > discussion but I at first didn't think it was a big deal until the
> > following accours
> > 1. the network environment at subscriber changes (change of IP) -
> > asterisk registers and now you have 2 active registrations in the db.
> > 2. Customers logging into the user portal and see the expired time stamp
> > 3. Deleting the registration in the UI will not completely delete it in
> > the db but add a second and third and so on registration.
> >
> > all my test was with public IPs i've even tested it while both asterisk
> > and sipwise was connected to the same switch both in a virtual and hard
> > environment
> >
> > Thank you again Raul but before placing into production it's absolutely
> > imperative that the basic functionality work properly - register and
> > register.
> >
> > Warmest Regards,
> > Aziz
> >
> >
> > ------------------------------------------------------------------------
> > *From:* Spce-user [spce-user-bounces at lists.sipwise.com] on behalf of
> > Raúl Alexis Betancor Santana [rabs at dimension-virtual.com]
> > *Sent:* Friday, November 21, 2014 6:06 AM
> > *To:* spce-user at lists.sipwise.com
> > *Subject:* Re: [Spce-user] sip registration stuck 3.5.16
> >
> > Just to add ... that we are using SPCE since it's first versions ... and
> > don't have that issue. With lot of asterisk customers with versions from
> > Asterisk 1.2 to Asterisk 12 .
> >
> > As it seems that the register doesn't get auto-expired (I don't
> > undestand why it hurts you that they are on the location table ... but
> > that another thing), if you are experiencing 'registration lost' with
> > Asterisk and SPCE ... 99.9% of the times it a fault on Asterisk or in a
> > middle router that 'modify' the SIP request.
> >
> > Best and faster thing you could do to track the issue ... it to capture
> > the traffic JUST on the Asterisk ... and on the SPCE and compare them,
> > if you see any difference ... then you probably have a sip-alg-activated
> > device on the middle of the path.
> >
> > If you don't have access to the Asterisk boxes ... just install one
> > yourself for testing, it's easy and fast. ... Faster than trying that
> > someneone give you the solution on a comunity list
> >
> > ------------------------------------------------------------------------
> > *De: *"Daniel Grotti" <dgrotti at sipwise.com>
> > *Para: *"Aziz Palmer" <Aziz.Palmer at AirFIBER.com>, "Theo"
> > <axessofficetheo at gmail.com>
> > *CC: *spce-user at lists.sipwise.com
> > *Enviados: *Viernes, 21 de Noviembre 2014 14:08:31
> > *Asunto: *Re: [Spce-user] sip registration stuck 3.5.16
> >
> > Hi,
> > why are you so sure ?
> > Kamailio version is the same, so if there is an issue it's also in PRO.
> > I opened the internal ticket # 0010303.
> >
> > We will update you soon.
> >
> > Daniel
> >
> >
> >
> > On 11/21/2014 02:54 PM, Aziz Palmer wrote:
> >
> > Thank you Daniel!
> >
> > quick question - i'm sure the customers that are using the pro
> > version aren't having this issue. Why is it happening on the CE
> > version? (I've spent so much time on it and i'm trying to more
> > understand it)
> >
> > Thank you,
> > Aziz
> >
> ------------------------------------------------------------------------
> > *From:* Daniel Grotti [dgrotti at sipwise.com]
> > *Sent:* Friday, November 21, 2014 1:17 AM
> > *To:* Aziz Palmer; Theo
> > *Cc:* spce-user at lists.sipwise.com
> > *Subject:* Re: [Spce-user] sip registration stuck 3.5.16
> >
> > Aziz,
> > I was able to have a look at the issue, and I opened an internal
> > ticket about that.
> > We will have a look at this and let you know.
> >
> > Daniel
> >
> >
> >
> > On 11/19/2014 12:41 AM, Aziz Palmer wrote:
> >
> > Theo,
> > can you send what you put in your sip trunk settings when you
> > connect to sipwise? (this part seems pretty straight forward but
> > would like to try someone else's settings.)
> >
> > Thank you
> >
> ------------------------------------------------------------------------
> > *From:* Theo [axessofficetheo at gmail.com]
> > *Sent:* Thursday, November 13, 2014 5:40 AM
> > *To:* Aziz Palmer
> > *Cc:* Andreas Granig; spce-user at lists.sipwise.com
> > *Subject:* Re: [Spce-user] sip registration stuck 3.5.16
> >
> > Hi
> >
> > The problem is on the asterisk side so they are the ones to fix
> > it (assuming that is indeed your problem). The "fix" on the spce
> > is merely a work around to not ban the user and accept the
> > registration.
> >
> > On Thu, Nov 13, 2014 at 3:38 PM, Aziz Palmer
> > <Aziz.Palmer at airfiber.com <mailto:Aziz.Palmer at airfiber.com>>
> wrote:
> >
> > Thank you Theo!
> >
> > but what i don't understand if this is the case why wasn't
> > fixed ?
> >
> > what should i be doing instead when having users register to
> > avoid this issue?
> >
> > Is this happening to you also? if so what did you do to fix
> > it? (follow the steps below?)
> >
> > Thank you!
> >
> >
> >
> ------------------------------------------------------------------------
> > *From:* Theo [axessofficetheo at gmail.com
> > <mailto:axessofficetheo at gmail.com>]
> > *Sent:* Thursday, November 13, 2014 5:30 AM
> > *To:* Aziz Palmer
> > *Cc:* Andreas Granig; spce-user at lists.sipwise.com
> > <mailto:spce-user at lists.sipwise.com>
> >
> > *Subject:* Re: [Spce-user] sip registration stuck 3.5.16
> >
> > Sorry I accidentally replied just to you rather than to the
> > list.:
> >
> > Hi
> >
> > This looks like the stale nonce issue which is an asterisk
> > problem. The workaround has been posted a few times - search
> > for stale nonce and I am sure you'll find it easily
> >
> > Good luck
> >
> >
> >
> >
> >
> >
> > to Aziz
> >
> > It would be this one:
> >
> >
> http://lists.sipwise.com/pipermail/spce-user/2014-January/005712.html
> >
> > On Thu, Nov 13, 2014 at 3:00 PM, Aziz Palmer
> > <Aziz.Palmer at airfiber.com <mailto:Aziz.Palmer at airfiber.com>>
> > wrote:
> >
> > kamailio.location
> > what command do I need to execute to provide this
> > information?
> >
> >
> > kamailio-proxy.log
> >
> > Nov 13 06:49:36 tx-portal-2 proxy[22474]: NOTICE:
> > <script>: Authentication failed, stale nonce -
> > R=sip:sipwise.xyz..com
> > ID=7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10
> > <mailto:7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10>
> > Nov 13 06:49:36 tx-portal-2 proxy[22473]: NOTICE:
> > <script>: New request on proxy - M=REGISTER
> > R=sip:sipwise.xyz..com
> > F=sip:master.sample.edge at sipwise.xyz..com
> > T=sip:master.sample.edge at sipwise.xyz..com
> > IP=104.34.42.10:5060 <http://104.34.42.10:5060>
> > (127.0.0.1:5060 <http://127.0.0.1:5060>)
> > ID=7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10
> > <mailto:7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10>
> > Nov 13 06:49:36 tx-portal-2 proxy[22473]: WARNING:
> > app_lua [app_lua_sr.c:100]: lua_sr_log(): lost database
> > connection. Reconnecting
> > Nov 13 06:49:36 tx-portal-2 proxy[22473]: NOTICE:
> > <script>: Contacts successfully updated, expires in 120s
> > - R=sip:sipwise.xyz..com
> > ID=7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10
> > <mailto:7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10>
> > Nov 13 06:51:21 tx-portal-2 proxy[22480]: NOTICE:
> > <script>: New request on proxy - M=REGISTER
> > R=sip:sipwise.xyz..com
> > F=sip:master.sample.edge at sipwise.xyz..com
> > T=sip:master.sample.edge at sipwise.xyz..com
> > IP=104.34.42.10:5060 <http://104.34.42.10:5060>
> > (127.0.0.1:5060 <http://127.0.0.1:5060>)
> > ID=7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10
> > <mailto:7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10>
> > Nov 13 06:51:21 tx-portal-2 proxy[22480]: WARNING:
> > app_lua [app_lua_sr.c:100]: lua_sr_log(): lost database
> > connection. Reconnecting
> > Nov 13 06:51:21 tx-portal-2 proxy[22480]: NOTICE:
> > <script>: Contacts successfully updated, expires in 120s
> > - R=sip:sipwise.xyz..com
> > ID=7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10
> > <mailto:7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10>
> > Nov 13 06:53:06 tx-portal-2 proxy[22475]: NOTICE:
> > <script>: New request on proxy - M=REGISTER
> > R=sip:sipwise.xyz..com
> > F=sip:master.sample.edge at sipwise.xyz..com
> > T=sip:master.sample.edge at sipwise.xyz..com
> > IP=104.34.42.10:5060 <http://104.34.42.10:5060>
> > (127.0.0.1:5060 <http://127.0.0.1:5060>)
> > ID=7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10
> > <mailto:7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10>
> > Nov 13 06:53:06 tx-portal-2 proxy[22475]: WARNING:
> > app_lua [app_lua_sr.c:100]: lua_sr_log(): lost database
> > connection. Reconnecting
> > Nov 13 06:53:06 tx-portal-2 proxy[22475]: NOTICE:
> > <script>: Contacts successfully updated, expires in 120s
> > - R=sip:sipwise.xyz..com
> > ID=7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10
> > <mailto:7233034d51f47c1a7ca56eb2353030ae at 104.34.42.10>
> >
> >
> >
> >
> > what does this mean "WARNING: app_lua
> > [app_lua_sr.c:100]: lua_sr_log(): lost database
> > connection. Reconnecting"?
> >
> >
> >
> > ________________________________________
> > From: Spce-user [spce-user-bounces at lists.sipwise.com
> > <mailto:spce-user-bounces at lists.sipwise.com>] on behalf
> > of Andreas Granig [agranig at sipwise.com
> > <mailto:agranig at sipwise.com>]
> > Sent: Thursday, November 13, 2014 1:26 AM
> > To: spce-user at lists.sipwise.com
> > <mailto:spce-user at lists.sipwise.com>
> > Subject: Re: [Spce-user] sip registration stuck 3.5.16
> >
> > Hi,
> >
> > It's most likely not a bug of the system, rather than a
> > problem with
> > your clients registering on said system. I risk to say
> > that because the
> > registrar is based on Kamailio, which is serving dozens
> > of millions of
> > SIP devices out there, so it would be a huge surprise
> > for it not to work
> > correctly.
> >
> > The only way to find out what the REAL issue is for you
> > to send an
> > example entry in the kamailio.location table of the DB,
> > along with a
> > full network trace of this client trying to register on
> > the system, and
> > also the kamailio-proxy.log file.
> >
> > Andreas
> >
> > On 11/13/2014 06:47 AM, Aziz Palmer wrote:
> > > thats not the issue - the problem is when you
> > RE-register a sip trunk it
> > > will stop updating sipwise. in order to fix that you
> > need to manually go
> > > into the subscriber and delete the registration. it
> > will not delete but
> > > instead a new registration will appear.
> > >
> > >
> > > SPCE - are you aware of this bug (its a serous issue -
> > since it gives
> > > the admins a lot of manual work/tech support calls) I
> > would like to put
> > > this this into partial production but can't - please
> help?
> > >
> > > Thanks!
> > >
> >
> ------------------------------------------------------------------------
> > > *From:* Raúl Alexis Betancor Santana
> > [rabs at dimension-virtual.com
> > <mailto:rabs at dimension-virtual.com>]
> > > *Sent:* Wednesday, November 12, 2014 1:54 PM
> > > *To:* Aziz Palmer
> > > *Subject:* Re: sip registration stuck 3.5.16
> > >
> > > 3600 secs ... 1 h ... it's default time on the
> config.xml
> > > kamailio.proxy.default_expire option ... you could
> > change it ... or set
> > > it on the web interface at domain-level or user-level
> > if you prefer, and
> > > as far as you don't touched the .cfg's ... they expire.
> > >
> > >
> >
> ------------------------------------------------------------------------
> > > *De: *"Aziz Palmer" <Aziz.Palmer at AirFIBER.com>
> > > *Para: *"Raúl Alexis Betancor Santana"
> > <rabs at dimension-virtual.com
> > <mailto:rabs at dimension-virtual.com>>,
> > > spce-user at lists.sipwise.com
> > <mailto:spce-user at lists.sipwise.com>
> > > *Enviados: *Miércoles, 12 de Noviembre 2014 21:17:53
> > > *Asunto: *RE: sip registration stuck 3.5.16
> > >
> > > they are not auto-expiring (I wish they were)
> > >
> > > if they are - taking a extremely long time
> > >
> >
> ------------------------------------------------------------------------
> > > *From:* Spce-user [spce-user-bounces at lists.sipwise.com
> > <mailto:spce-user-bounces at lists.sipwise.com>] on behalf
> of
> > > Raúl Alexis Betancor Santana
> > [rabs at dimension-virtual.com
> > <mailto:rabs at dimension-virtual.com>]
> > > *Sent:* Wednesday, November 12, 2014 1:04 PM
> > > *To:* spce-user at lists.sipwise.com
> > <mailto:spce-user at lists.sipwise.com>
> > > *Subject:* Re: [Spce-user] sip registration stuck
> 3.5.16
> > >
> > >
> > > Why did you whant to remove the old register? ...
> > location entries
> > > auto-expire, so you don't have to worry about them.
> > >
> > >
> >
> ------------------------------------------------------------------------
> > > *De: *"Aziz Palmer" <Aziz.Palmer at AirFIBER.com>
> > > *Para: *spce-user at lists.sipwise.com
> > <mailto:spce-user at lists.sipwise.com>
> > > *Enviados: *Miércoles, 12 de Noviembre 2014 20:47:13
> > > *Asunto: *[Spce-user] sip registration stuck 3.5.16
> > >
> > > I hate to ask again but this is turning into a real
> > issue - the
> > > "registered devices" are not automatically being
> > removed and when trying
> > > to manually delete them, nothing happens.
> > >
> > > Can i set a timer on how long these orphan registered
> > devices stay part
> > > of sipwise?
> > >
> > > how can i delete these devices without having to issue
> > the below command
> > > - I would like to delete specific devices and not all
> > of them
> > >
> > > mysql kamailio -e "delete from location"
> > > /etc/init.d/kamailio-proxy restart
> > >
> > >
> > > Thanks again,
> > > Aziz
> > >
> > >
> >
> ------------------------------------------------------------------------
> > > *From:* Spce-user [spce-user-bounces at lists.sipwise.com
> > <mailto:spce-user-bounces at lists.sipwise.com>] on behalf
> of
> > > Aziz Palmer [Aziz.Palmer at AirFIBER.com]
> > > *Sent:* Tuesday, November 11, 2014 12:15 PM
> > > *To:* spce-user at lists.sipwise.com
> > <mailto:spce-user at lists.sipwise.com>
> > > *Subject:* [Spce-user] sip registration stuck 5.3.16
> > >
> > > Hello SPCE Team,
> > > here is what appears to be a bug
> > >
> > > I enter sip registration settings into asterisk,
> > registers fine with
> > > sipwise. Then I make a change within the sip trunk
> > settings of asterisk
> > > and apply the new settings. Then previous sip
> > registration gets stuck;
> > > meaning within sipwise 2 registrations appear (old &
> > new). When i try to
> > > delete the old one, it stays stuck. The only way i
> > discovered to be able
> > > to remove the old registration is by issuing this
> > command, then i can
> > > delete the registration.
> > >
> > > /etc/init.d/kamailio-proxy restart
> > >
> > > what additional information can i provide to have your
> > team troubleshoot
> > > this issue?
> > >
> > > Thank you,
> > > Anees
> > >
> > > _______________________________________________
> > > Spce-user mailing list
> > > Spce-user at lists.sipwise.com
> > <mailto:Spce-user at lists.sipwise.com>
> > > https://lists.sipwise.com/listinfo/spce-user
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Spce-user mailing list
> > > Spce-user at lists.sipwise.com
> > <mailto:Spce-user at lists.sipwise.com>
> > > https://lists.sipwise.com/listinfo/spce-user
> > >
> > _______________________________________________
> > Spce-user mailing list
> > Spce-user at lists.sipwise.com
> > <mailto:Spce-user at lists.sipwise.com>
> > https://lists.sipwise.com/listinfo/spce-user
> > _______________________________________________
> > Spce-user mailing list
> > Spce-user at lists.sipwise.com
> > <mailto:Spce-user at lists.sipwise.com>
> > https://lists.sipwise.com/listinfo/spce-user
> >
> >
> >
> >
> >
> > _______________________________________________
> > Spce-user mailing list
> > Spce-user at lists.sipwise.com
> > https://lists.sipwise.com/listinfo/spce-user
> >
> >
> >
> >
> > _______________________________________________
> > Spce-user mailing list
> > Spce-user at lists.sipwise.com
> > https://lists.sipwise.com/listinfo/spce-user
> >
> >
> >
> >
> > _______________________________________________
> > Spce-user mailing list
> > Spce-user at lists.sipwise.com
> > https://lists.sipwise.com/listinfo/spce-user
> >
> _______________________________________________
> Spce-user mailing list
> Spce-user at lists.sipwise.com
> https://lists.sipwise.com/listinfo/spce-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sipwise.com/pipermail/spce-user_lists.sipwise.com/attachments/20141121/0241f22e/attachment-0001.html>
More information about the Spce-user
mailing list