[Spce-user] mediator fails

[Ferds Calma]

Hi

I have the same issue with qabane. i have manually deleted the entries
multiple times, but mediator keeps stopping with the same error "Failed to
convert redis reply to entry "

I'm on 6.3.2


On Mon, Sep 3, 2018 at 5:27 PM qabane me <qabaneitsolutions at gmail.com>
wrote:

> Hi
>
> I ran the cleanup commands but that didn't work - the results were as
> follows:
>
> root at spce:~# redis-cli -n 21 srem acc:meth::INVITE
> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
> (integer) 1
> root at spce:~# redis-cli -n 21 srem acc:meth::INVITE
> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
> (integer) 0
> root at spce:~# redis-cli -n 21 srem acc:meth::BYE
> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
> (integer) 0
> root at spce:~# redis-cli -n 21 srem acc:meth::BYE
> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
> (integer) 1
> root at spce:~# redis-cli -n 21 del
> acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
> (integer) 1
>
> Mediator still fails with the same error though?
>
> On Mon, Sep 3, 2018 at 8:25 PM qabane me <qabaneitsolutions at gmail.com>
> wrote:
>
>> Hi Andreas
>>
>> Thanks for the response.
>>
>> To my knowledge I did nothing but apply hot-fixes to fix the other issue
>> with mediator (the one I provided backtraces for). The only other thing
>> that stands out from this is that I cloned this from another VM - changed
>> the MAC of the NIC and changed the IP address. I have no idea if that could
>> be related or not. Other than that I cannot think of anything.
>>
>> I will give the cleanup a go and let you know how it goes.
>>
>>
>> On Mon, Sep 3, 2018 at 4:39 PM Andreas Granig <agranig at sipwise.com>
>> wrote:
>>
>>> Hi,
>>>
>>> We’ve pushed a code change to master for mediator to automatically trash
>>> and skip records with a mapping entry in acc:cid::$callid and
>>> acc:meth::$method, so starting at least with mr6.5 mediator won’t bail out
>>> on this anymore.
>>>
>>> For now, you can manually clean this up and let mediator continue by
>>> executing
>>>
>>> $ redis-cli -n 21 srem acc:meth::INVITE
>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>>> $ redis-cli -n 21 srem acc:meth::INVITE
>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>>> $ redis-cli -n 21 srem acc:meth::BYE
>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>>> $ redis-cli -n 21 srem acc:meth::BYE
>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>>> $ redis-cli -n 21 del acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>>
>>> However, this and the scheduled mediator fix are actually just removing
>>> the sympthom by masking the root of the problem, namely how you ended up in
>>> the situation of having the acc:cid and acc:meth mappings pointing to
>>> non-existent hash keys in redis. So any insights into what happened to the
>>> system before that would be beneficial. In our case, there was one instance
>>> of that situation where it happened in the labs, namely when kamailio and
>>> sems crashed on some experimental code.
>>>
>>> If this actual problem persists and happens for others as well, we’ll
>>> consider back-porting the mediator trashing feature to mr6.3 and mr6.4,
>>> otherwise the advise is to remove the offending keys manually for now and
>>> report it on the list.
>>>
>>> Andreas
>>>
>>> > On 30.08.2018, at 13:26, qabane me <qabaneitsolutions at gmail.com>
>>> wrote:
>>> >
>>> > root at spce:~# redis-cli -n 21 hgetall
>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>>> > (empty list or set)
>>> > root at spce:~# redis-cli -n 21 hgetall
>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>>> > (empty list or set)
>>> >
>>> >
>>> > On Thu, Aug 30, 2018 at 1:26 PM qabane me <qabaneitsolutions at gmail.com>
>>> wrote:
>>> > Hi
>>> >
>>> >
>>> > 1) "acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455"
>>> > 2) "acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037"
>>> >
>>> > root at spce:~# redis-cli -n 21 hgetall
>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>>> > (empty list or set)
>>> > root at spce:~# redis-cli -n 21 hgetall
>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>>> >
>>> >
>>> > On Thu, Aug 30, 2018 at 1:09 PM Andreas Granig <agranig at sipwise.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > Ah, right.
>>> >
>>> > So, please do
>>> >
>>> > redis-cli -n 21 smembers acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>> >
>>> > and for each of the entries returned there, do
>>> >
>>> > redis-cli -n 21 hgetall $entry
>>> >
>>> > with $entry being the individual entries returned in the smembers
>>> command. Let’s see then what’s really in your database.
>>> >
>>> > Andreas
>>> >
>>> > > On 30.08.2018, at 12:03, qabane me <qabaneitsolutions at gmail.com>
>>> wrote:
>>> > >
>>> > > redis-cli -n 21 type acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>> > > set
>>> > >
>>> > > Does that mean something to you?
>>> > >
>>> > > On Thu, Aug 30, 2018 at 11:56 AM Andreas Granig <agranig at sipwise.com>
>>> wrote:
>>> > > Hi,
>>> > >
>>> > > > On 30.08.2018, at 10:34, qabane me <qabaneitsolutions at gmail.com>
>>> wrote:
>>> > > >
>>> > > > root at spce:~# redis-cli -n 21 hgetall
>>> acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>> > > > (error) WRONGTYPE Operation against a key holding the wrong kind
>>> of value
>>> > >
>>> > >
>>> > > What type is the data structure then? You can figure it out by
>>> > >
>>> > > redis-cli -n 21 type acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>> > >
>>> > > Best,
>>> > > Andreas
>>> >
>>>
>>> _______________________________________________
> Spce-user mailing list
> Spce-user at lists.sipwise.com
> https://lists.sipwise.com/listinfo/spce-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sipwise.com/mailman/private/spce-user_lists.sipwise.com/attachments/20180920/7459e00c/attachment.html>