[Spce-user] mediator fails

[qabane me]

Hi

Yeah I have tried this a number of times as well - no joy unfortunately.

On Thu, Sep 20, 2018 at 11:40 AM Ferds Calma <ferds.calma at gmail.com> wrote:

> Hi
>
> I have the same issue with qabane. i have manually deleted the entries
> multiple times, but mediator keeps stopping with the same error "Failed to
> convert redis reply to entry "
>
> I'm on 6.3.2
>
>
> On Mon, Sep 3, 2018 at 5:27 PM qabane me <qabaneitsolutions at gmail.com>
> wrote:
>
>> Hi
>>
>> I ran the cleanup commands but that didn't work - the results were as
>> follows:
>>
>> root at spce:~# redis-cli -n 21 srem acc:meth::INVITE
>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>> (integer) 1
>> root at spce:~# redis-cli -n 21 srem acc:meth::INVITE
>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>> (integer) 0
>> root at spce:~# redis-cli -n 21 srem acc:meth::BYE
>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>> (integer) 0
>> root at spce:~# redis-cli -n 21 srem acc:meth::BYE
>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>> (integer) 1
>> root at spce:~# redis-cli -n 21 del
>> acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>> (integer) 1
>>
>> Mediator still fails with the same error though?
>>
>> On Mon, Sep 3, 2018 at 8:25 PM qabane me <qabaneitsolutions at gmail.com>
>> wrote:
>>
>>> Hi Andreas
>>>
>>> Thanks for the response.
>>>
>>> To my knowledge I did nothing but apply hot-fixes to fix the other issue
>>> with mediator (the one I provided backtraces for). The only other thing
>>> that stands out from this is that I cloned this from another VM - changed
>>> the MAC of the NIC and changed the IP address. I have no idea if that could
>>> be related or not. Other than that I cannot think of anything.
>>>
>>> I will give the cleanup a go and let you know how it goes.
>>>
>>>
>>> On Mon, Sep 3, 2018 at 4:39 PM Andreas Granig <agranig at sipwise.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> We’ve pushed a code change to master for mediator to automatically
>>>> trash and skip records with a mapping entry in acc:cid::$callid and
>>>> acc:meth::$method, so starting at least with mr6.5 mediator won’t bail out
>>>> on this anymore.
>>>>
>>>> For now, you can manually clean this up and let mediator continue by
>>>> executing
>>>>
>>>> $ redis-cli -n 21 srem acc:meth::INVITE
>>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>>>> $ redis-cli -n 21 srem acc:meth::INVITE
>>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>>>> $ redis-cli -n 21 srem acc:meth::BYE
>>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>>>> $ redis-cli -n 21 srem acc:meth::BYE
>>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>>>> $ redis-cli -n 21 del acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>>>
>>>> However, this and the scheduled mediator fix are actually just removing
>>>> the sympthom by masking the root of the problem, namely how you ended up in
>>>> the situation of having the acc:cid and acc:meth mappings pointing to
>>>> non-existent hash keys in redis. So any insights into what happened to the
>>>> system before that would be beneficial. In our case, there was one instance
>>>> of that situation where it happened in the labs, namely when kamailio and
>>>> sems crashed on some experimental code.
>>>>
>>>> If this actual problem persists and happens for others as well, we’ll
>>>> consider back-porting the mediator trashing feature to mr6.3 and mr6.4,
>>>> otherwise the advise is to remove the offending keys manually for now and
>>>> report it on the list.
>>>>
>>>> Andreas
>>>>
>>>> > On 30.08.2018, at 13:26, qabane me <qabaneitsolutions at gmail.com>
>>>> wrote:
>>>> >
>>>> > root at spce:~# redis-cli -n 21 hgetall
>>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>>>> > (empty list or set)
>>>> > root at spce:~# redis-cli -n 21 hgetall
>>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>>>> > (empty list or set)
>>>> >
>>>> >
>>>> > On Thu, Aug 30, 2018 at 1:26 PM qabane me <
>>>> qabaneitsolutions at gmail.com> wrote:
>>>> > Hi
>>>> >
>>>> >
>>>> > 1) "acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455"
>>>> > 2) "acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037"
>>>> >
>>>> > root at spce:~# redis-cli -n 21 hgetall
>>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092647.008037
>>>> > (empty list or set)
>>>> > root at spce:~# redis-cli -n 21 hgetall
>>>> acc:entry::011cd8e3-220b-1237-d9a8-a6958035e26f:1535092756.025455
>>>> >
>>>> >
>>>> > On Thu, Aug 30, 2018 at 1:09 PM Andreas Granig <agranig at sipwise.com>
>>>> wrote:
>>>> > Hi,
>>>> >
>>>> > Ah, right.
>>>> >
>>>> > So, please do
>>>> >
>>>> > redis-cli -n 21 smembers acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>>> >
>>>> > and for each of the entries returned there, do
>>>> >
>>>> > redis-cli -n 21 hgetall $entry
>>>> >
>>>> > with $entry being the individual entries returned in the smembers
>>>> command. Let’s see then what’s really in your database.
>>>> >
>>>> > Andreas
>>>> >
>>>> > > On 30.08.2018, at 12:03, qabane me <qabaneitsolutions at gmail.com>
>>>> wrote:
>>>> > >
>>>> > > redis-cli -n 21 type acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>>> > > set
>>>> > >
>>>> > > Does that mean something to you?
>>>> > >
>>>> > > On Thu, Aug 30, 2018 at 11:56 AM Andreas Granig <
>>>> agranig at sipwise.com> wrote:
>>>> > > Hi,
>>>> > >
>>>> > > > On 30.08.2018, at 10:34, qabane me <qabaneitsolutions at gmail.com>
>>>> wrote:
>>>> > > >
>>>> > > > root at spce:~# redis-cli -n 21 hgetall
>>>> acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>>> > > > (error) WRONGTYPE Operation against a key holding the wrong kind
>>>> of value
>>>> > >
>>>> > >
>>>> > > What type is the data structure then? You can figure it out by
>>>> > >
>>>> > > redis-cli -n 21 type acc:cid::011cd8e3-220b-1237-d9a8-a6958035e26f
>>>> > >
>>>> > > Best,
>>>> > > Andreas
>>>> >
>>>>
>>>> _______________________________________________
>> Spce-user mailing list
>> Spce-user at lists.sipwise.com
>> https://lists.sipwise.com/listinfo/spce-user
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sipwise.com/pipermail/spce-user_lists.sipwise.com/attachments/20180921/a0ad827a/attachment-0001.html>