<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-ligatures:none;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Afternoon all!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Been a while since I last posted (back in the v2.6 days)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Question…<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We have a few boxes running mr9.5.4 that will at times stop processing registrations. We have alerts internally on systems that ping us on flapping SIP registrations.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">On said boxes in the kamailio-lb.log we see:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">lb[634146]: NOTICE: REPLY_FROM_INBOUND <script>: Reply from Inbound - S=500 - I'm terribly sorry, server error occurred (1/SL) M=REGISTER IP=«udp»:«127.0.0.1»:«5062» ID=«000001e4-27c6afde096c100099040080f0a581fc@IP-REMOVED» UA='<null>'
DESTIP=«127.0.0.1»:«5060»<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In kamailio-options-lb.log we see:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Line 8307: Sep 1 09:05:56 box01 proxy[634207]: ERROR: ROUTE_REG_HANDLE db_redis [redis_dbase.c:2043]: db_redis_perform_update(): Failed to get reply for query: Connection reset by peer<o:p></o:p></p>
<p class="MsoNormal">Line 8308: Sep 1 09:05:56 box01 proxy[634207]: ERROR: ROUTE_REG_HANDLE db_redis [redis_dbase.c:2298]: db_redis_perform_update(): failed to perform the update<o:p></o:p></p>
<p class="MsoNormal">Line 8309: Sep 1 09:05:56 box01 proxy[634207]: ERROR: ROUTE_REG_HANDLE db_redis [redis_dbase.c:2850]: db_redis_update(): failed to do the query<o:p></o:p></p>
<p class="MsoNormal">Line 8310: Sep 1 09:05:56 box01 proxy[634207]: ERROR: ROUTE_REG_HANDLE usrloc [ucontact.c:1147]: db_update_ucontact_ruid(): updating database failed<o:p></o:p></p>
<p class="MsoNormal">Line 8311: Sep 1 09:05:56 box01 proxy[634207]: ERROR: ROUTE_REG_HANDLE usrloc [ucontact.c:1663]: update_contact_db(): failed to update database<o:p></o:p></p>
<p class="MsoNormal">Line 8312: Sep 1 09:05:56 box01 proxy[634207]: ERROR: ROUTE_REG_HANDLE registrar [save.c:796]: update_contacts(): failed to update contact<o:p></o:p></p>
<p class="MsoNormal">Line 8313: Sep 1 09:05:56 box01 proxy[634207]: ERROR: ROUTE_REG_HANDLE <script>: Contacts failed - R=«sip:box01.easyipt.co.uk;transport=udp» ID=«0216ED4BE98140000000375C@92.207.169.202» UA='NEC SL2100'<o:p></o:p></p>
<p class="MsoNormal">Line 8314: Sep 1 09:05:56 box01 proxy[634207]: NOTICE: ROUTE_REG_HANDLE <script>: Sending reply S=500 Server Error fs='«127.0.0.1»:«5062»' du='«127.0.0.1»:«5060»' - R=«sip:REMOVED;transport=udp» ID=«0216ED4BE98140000000375C@REMOVED» UA='NEC
SL2100'<o:p></o:p></p>
<p class="MsoNormal">Line 8315: Sep 1 09:05:56 box01 proxy[634207]: ERROR: ROUTE_REG_HANDLE sl [sl_funcs.c:414]: sl_reply_error(): stateless error reply used: I'm terribly sorry, server error occurred (1/SL)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Redis.log does not show anything:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Sep 1 09:08:57 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:08:57.481 - DB 3: 11 keys (0 volatile) in 64 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:08:57 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:08:57.481 - DB 4: 2 keys (0 volatile) in 8 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:08:57 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:08:57.481 - DB 19: 24 keys (24 volatile) in 32 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:08:57 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:08:57.481 - DB 20: 57 keys (0 volatile) in 64 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:08:57 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:08:57.481 - DB 21: 31 keys (0 volatile) in 64 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:08:57 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:08:57.481 - DB 23: 32 keys (0 volatile) in 32 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:08:57 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:08:57.481 - Closing idle client<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:08:59 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:08:59.290 - Closing idle client<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:01 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:01.252 - Accepted 127.0.0.1:51798<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:01 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:01.258 - Client closed connection<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:01 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:01.517 - Accepted 127.0.0.1:51802<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:01 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:01.518 - Accepted 127.0.0.1:51804<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:01 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:01.519 - Accepted 127.0.0.1:51806<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:02 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:02.312 - Accepted 127.0.0.1:51808<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:02 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:02.314 - Accepted 127.0.0.1:51810<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:02 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:02.506 - DB 3: 7 keys (0 volatile) in 64 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:02 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:02.506 - DB 4: 1 keys (0 volatile) in 8 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:02 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:02.506 - DB 19: 24 keys (24 volatile) in 32 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:02 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:02.506 - DB 20: 57 keys (0 volatile) in 64 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:02 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:02.506 - DB 21: 32 keys (0 volatile) in 64 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:02 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:02.506 - DB 23: 32 keys (0 volatile) in 32 slots HT.<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:03 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:03.553 - Client closed connection<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:03 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:03.553 - Client closed connection<o:p></o:p></p>
<p class="MsoNormal">Sep 1 09:09:03 box01 (info) redis[634122]: 634122:M 01 Sep 2023 09:09:03.553 - Client closed connection<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Restarting REDIS does clear the problem for a period of time however It does return. Typically, if the box is physically restarted it goes away for a few months..<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Boxes are all VM on ESXi hosts.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Just wondering what I we can do to debug this a little further, or if anyone knows what would be at catalyst for this?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Thank you in advance
</span><span style="font-family:"Segoe UI Emoji",sans-serif;mso-fareast-language:EN-GB">😊</span><span style="mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Kind Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Adam<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>