<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>Hi, we are having a performance problem with ngcp-witnessd. It
starts after the cpu load is higher during nightly cron jobs like
backup. Port 9800 is opened multiple times (>100) like this:</p>
<p><font size="2" face="monospace">root@spce:~# lsof -i TCP| grep
9800<br>
victoria- 607 _victoria-metrics 28u IPv4 34459176
0t0 TCP localhost:45000->localhost:9800 (ESTABLISHED)<br>
ngcp-witn 1300009 root 5u IPv4 33636424
0t0 TCP localhost:9800 (LISTEN)<br>
ngcp-witn 1300009 root 9u IPv4 34442250
0t0 TCP localhost:9800->localhost:49954 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 11u IPv4 34442251
0t0 TCP localhost:9800->localhost:49964 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 16u IPv4 34442256
0t0 TCP localhost:9800->localhost:51952 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 17u IPv4 34442257
0t0 TCP localhost:9800->localhost:51958 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 20u IPv4 34442260
0t0 TCP localhost:9800->localhost:45752 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 21u IPv4 34442261
0t0 TCP localhost:9800->localhost:45764 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 24u IPv4 34442264
0t0 TCP localhost:9800->localhost:48518 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 25u IPv4 34442265
0t0 TCP localhost:9800->localhost:48520 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 28u IPv4 34442268
0t0 TCP localhost:9800->localhost:58756 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 29u IPv4 34442269
0t0 TCP localhost:9800->localhost:58758 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 30u IPv4 34442270
0t0 TCP localhost:9800->localhost:45948 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 31u IPv4 34442271
0t0 TCP localhost:9800->localhost:45954 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 34u IPv4 34442274
0t0 TCP localhost:9800->localhost:50588 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 35u IPv4 34442275
0t0 TCP localhost:9800->localhost:50590 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 37u IPv4 34442277
0t0 TCP localhost:9800->localhost:52274 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 52u IPv4 34442290
0t0 TCP localhost:9800->localhost:56316 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 54u IPv4 34442292
0t0 TCP localhost:9800->localhost:56322 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 55u IPv4 34442293
0t0 TCP localhost:9800->localhost:56332 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 56u IPv4 34442294
0t0 TCP localhost:9800->localhost:58530 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 57u IPv4 34442295
0t0 TCP localhost:9800->localhost:58532 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 60u IPv4 34442298
0t0 TCP localhost:9800->localhost:56404 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 61u IPv4 34442299
0t0 TCP localhost:9800->localhost:56414 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 62u IPv4 34442300
0t0 TCP localhost:9800->localhost:46702 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 63u IPv4 34442301
0t0 TCP localhost:9800->localhost:46708 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 64u IPv4 34442302
0t0 TCP localhost:9800->localhost:50426 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 65u IPv4 34442303
0t0 TCP localhost:9800->localhost:50432 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 68u IPv4 34442306
0t0 TCP localhost:9800->localhost:50944 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 69u IPv4 34442307
0t0 TCP localhost:9800->localhost:50956 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 70u IPv4 34442308
0t0 TCP localhost:9800->localhost:45806 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 71u IPv4 34442309
0t0 TCP localhost:9800->localhost:45812 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 73u IPv4 34442311
0t0 TCP localhost:9800->localhost:53530 (CLOSE_WAIT)<br>
ngcp-witn 1300009 root 74u IPv4 34442312
0t0 TCP localhost:9800->localhost:47184 (CLOSE_WAIT)</font><br>
</p>
<p>Restarting ngcp-witnessd fixes the problem temporary. The
witnessd log doesn't give a clue, only some uninitialized values:</p>
<p><font size="2" face="monospace">Dec 28 16:47:36 spce (warning)
ngcp-witnessd[1548076]: Use of uninitialized value $lv in
substitution (s///) at /usr/share/perl5/Prometheus/Tiny.pm line
28.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in substitution (s///) at
/usr/share/perl5/Prometheus/Tiny.pm line 29.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in concatenation (.) or string at
/usr/share/perl5/Prometheus/Tiny.pm line 30.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in substitution (s///) at
/usr/share/perl5/Prometheus/Tiny.pm line 28.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in substitution (s///) at
/usr/share/perl5/Prometheus/Tiny.pm line 29.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in concatenation (.) or string at
/usr/share/perl5/Prometheus/Tiny.pm line 30.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in substitution (s///) at
/usr/share/perl5/Prometheus/Tiny.pm line 28.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in substitution (s///) at
/usr/share/perl5/Prometheus/Tiny.pm line 29.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in concatenation (.) or string at
/usr/share/perl5/Prometheus/Tiny.pm line 30.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in substitution (s///) at
/usr/share/perl5/Prometheus/Tiny.pm line 28.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in substitution (s///) at
/usr/share/perl5/Prometheus/Tiny.pm line 29.<br>
Dec 28 16:47:36 spce (warning) ngcp-witnessd[1548076]: Use of
uninitialized value $lv in concatenation (.) or string at
/usr/share/perl5/Prometheus/Tiny.pm line 30.</font></p>
<p>Sipwise version is mr9.5.8 with the latest patches, a <font size="2">dual-core with 8 GB</font><font size="2"> memory</font><font size="2">.</font></p>
<p><font size="2">Any hints?</font></p>
<p><font size="2">Thanks<br>
</font></p>
<p><font size="2"><br>
</font></p>
<p><font size="2" face="monospace"><br>
</font></p>
</body>
</html>