[Spce-user] Kamailio crashing

Tue Mar 14 16:49:49 EDT 2023

Hello Mika

Virtual server has 2 vCPU and 4GB RAM.

# free -m
               total        used        free      shared  buff/cache
available
Mem:            3931        3030         538          64         362
  568
Swap:              0           0           0

It has 2 NIC (eth0 and eth1) (one public and one private) apart than
loopback (lo)

Today I found out that now it's kamailio-proxy the one failed, along with
ngcp-rate-o-mat:
[image: image.png]

Latest lines in rate-o-mat log shows:
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: Trying to
connect to billing db...
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: Successfully
connected to billing db...
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: Trying to
connect to provisioning db...
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: Successfully
connected to provisioning db...
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: Trying to
connect to accounting db...
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: Successfully
connected to accounting db...
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: No duplication
db credentials, disabled.
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: local cdr tag
column model loaded
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: local cdr time
balance column model loaded
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: local cdr
relation column model loaded
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: local cdr cash
balance column model loaded
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: Grabbed 2 CDRs
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: INFO: 1/2 - rate CDR
ID 21
Mar 14 01:18:11 ngcp (info) ngcp-rate-o-mat[126434]: FATAL: 1/2 - rating
CDR ID 21 aborted (failed 3 times), please fix it manually: FATAL: No
outbound fee info for profile 3 and source user 'XXXXXXXXX' or user/domain
'XXXXXXXX at sudomain.domain.tld' and destination user 'XXXXXXXX' or
user/domain 'XXXXXXXXX at Y.Y.Y.Y' found

And kamailio-proxy log shows:
Mar 14 16:44:53.280723 ngcp proxy[113297]: WARNING: db_redis
[redis_dbase.c:1715]: db_redis_perform_delete():   scan key 1 is 'expires'
Mar 14 16:45:45.432182 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 5
Mar 14 16:45:45.432230 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 30
Mar 14 16:45:45.432257 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 9
Mar 14 16:45:45.432281 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 6
Mar 14 16:45:45.432352 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 33
Mar 14 16:45:45.432375 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 31
Mar 14 16:45:45.432701 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 34
Mar 14 16:45:45.433959 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 38
Mar 14 16:45:45.433975 ngcp proxy[113328]: CRITICAL: <core>
[core/pass_fd.c:277]: receive_fd(): EOF on 36

I only have 1 customer, 1 subscriber and 4 trunks, and it's not under
production. I'm only using for testing.
Does it need more resources?

On Tue, Mar 14, 2023 at 9:25 AM Michael Prokop via Spce-user <
spce-user at lists.sipwise.com> wrote:

> * Alejandro Mejia Evertsz [Sun Mar 12, 2023 at 09:03:09PM -0600]:
>
> > I installed C5 Community a couple of days ago, and there's been a couple
> of
> > times in which Administrative GUI has stopped responding and when
> checking
> > systemctl, ngcp-panel.service it's stopped. So I restarted the service
> and
> > back to normal, but the second time it happened I also noticed
> > kamailio-lb.service being down.
> > As the GUI is not a critical part of the service, as long as SIP keeps
> > running, I didn't pay much attention to it, but in 2 days of running the
> > server, I've noticed kamailio down twice.
> >
> > I noticed the following on it's logs after the last crash:
> >
> > Mar 13 01:49:06.084243 ngcp lb[687]: WARNING: tls [tls_init.c:807]:
> > tls_h_mod_init_f(): openssl bug #1491 (crash/mem leaks on low memory)
> > workaround enabled (on low memory tls operations will fail preemptively)
> > with free memory thresholds 22020096 and 11010048 bytes
> > Mar 13 01:49:06.099174 ngcp lb[687]: CRITICAL: <core>
> [core/ip_addr.c:234]:
> > ip_addr2sbuf(): unknown address family 0
> > Mar 13 01:49:06.099222 ngcp lb[687]: NOTICE: tls [tls_domain.c:1144]:
> > ksr_tls_fix_domain(): registered server_name callback handler for socket
> > [:0], server_name='<default>' ...
> >
> > I'm running latest version published on the installation instructions on
> a
> > Debian 11 system.
>
> How much memory does your system have? Output of `free -m` please.
> What's the output of `ip a` of your system?
>
> regards
> -mika-
>
> --
> Spce-user mailing list
> Spce-user at lists.sipwise.com
> http://lists.sipwise.com/mailman/listinfo/spce-user_lists.sipwise.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sipwise.com/pipermail/spce-user_lists.sipwise.com/attachments/20230314/41716637/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 387523 bytes
Desc: not available
URL: <http://lists.sipwise.com/pipermail/spce-user_lists.sipwise.com/attachments/20230314/41716637/attachment-0001.png>