[Spce-user] Spce-user Digest, Vol 10, Issue 16
cappellari at connectlife.it
cappellari at connectlife.it
Fri Apr 17 11:23:43 EDT 2020
Ok, the problem of the MySQL ibdata file is solved, it was the mysql encription key. There remains a problem, the ngcpcfg apply script does not update the passwords in the respective configuration files (kamailio es).
The correct procedure is as follows:
copy the ngcp-config directory
restore the mysql.encriptyon.key key on the constants.yml file
import the database
RESTART MARIADB <IMPORTANT OTHERWISE ngcpcfg apply fails
run ngcpcfg apply
RESULT:
ngcp-service: info: Action restart skipped, service kannel-bearerbox is not enab led
ngcp-service: info: Action restart skipped, service kannel-smsbox is not enabled
ngcp-service: info: Action restart skipped, service rtpengine-recording is not and nabled
2020-04-14 13:04:26 sbc: Synchronizing current with expected services state
2020-04-14 13:04:27 sbc: Executing sync-db:
rtp_interface ext = ext is sync between config and db
rtp_interface int = int is sync between config and db
smsc_peer default_smsc = default_smsc is sync between config and db
2020-04-14 13:04:27 sbc: OK
2020-04-14 13:04:27 sbc: Synchronizing MySQL grants / credentials
----- Messaggio originale -----
Da: "spce-user-request" <spce-user-request at lists.sipwise.com>
A: "spce-user" <spce-user at lists.sipwise.com>
Inviato: Giovedì, 16 aprile 2020 9:35:26
Oggetto: Spce-user Digest, Vol 10, Issue 16
Send Spce-user mailing list submissions to
spce-user at lists.sipwise.com
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.sipwise.com/mailman/listinfo/spce-user_lists.sipwise.com
or, via email, send a message with subject or body 'help' to
spce-user-request at lists.sipwise.com
You can reach the person managing the list at
spce-user-owner at lists.sipwise.com
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Spce-user digest..."
Today's Topics:
1. Re: Spce-user Digest, Vol 10, Issue 15 (cappellari at connectlife.it)
2. Re: Spce-user Digest, Vol 10, Issue 15 (cappellari at connectlife.it)
3. Re: Spce-user Digest, Vol 10, Issue 15 (Alex Lutay)
----------------------------------------------------------------------
Message: 1
Date: Wed, 15 Apr 2020 20:06:19 +0200 (CEST)
From: "cappellari at connectlife.it" <cappellari at connectlife.it>
To: spce-user <spce-user at lists.sipwise.com>
Subject: Re: [Spce-user] Spce-user Digest, Vol 10, Issue 15
Message-ID:
<280459404.10484.1586973979989.JavaMail.zimbra at connectlife.it>
Content-Type: text/plain; charset=utf-8
Hi, thanks for replying. I confirm that DB encryption is active. I installed sipwise on the new server from the ISO. After the installation I launched the ngcp-initial-configuration command. Then I restarted the server and replaced the ngcp-config, import db etc. Do I need to also import /etc/mysql/debian.cnf?
----- Messaggio originale -----
Da: spce-user-request at lists.sipwise.com
A: "spce-user" <spce-user at lists.sipwise.com>
Inviato: Mercoled?, 15 aprile 2020 18:00:04
Oggetto: Spce-user Digest, Vol 10, Issue 15
Send Spce-user mailing list submissions to
spce-user at lists.sipwise.com
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.sipwise.com/mailman/listinfo/spce-user_lists.sipwise.com
or, via email, send a message with subject or body 'help' to
spce-user-request at lists.sipwise.com
You can reach the person managing the list at
spce-user-owner at lists.sipwise.com
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Spce-user digest..."
Today's Topics:
1. Re: Move Sipwise 8.3.1 to another server (Alex Lutay)
----------------------------------------------------------------------
Message: 1
Date: Tue, 14 Apr 2020 22:37:50 +0200
From: Alex Lutay <alutay at sipwise.com>
To: spce-user at lists.sipwise.com
Subject: Re: [Spce-user] Move Sipwise 8.3.1 to another server
Message-ID: <9d803207-0125-01cd-20da-31860e584886 at sipwise.com>
Content-Type: text/plain; charset=utf-8
Hi all,
re: /etc/mysql/sipwise.cnf
mr8.3.1 doesn't operate with /etc/mysql/sipwise.cnf
It was deprecated and removed in mr8.1.1+:
> https://www.sipwise.com/doc/mr8.1.1/spce/?_href=ar01s14.html
> Deprecated and removed /etc/mysql/sipwise.cnf, please use /etc/mysql/sipwise_extra.cnf [TT#68021]
Anyway both those files are generated from templates in a modern release
~mr6.5+, so no need to copy it manually.
Please report if you still see in a modern documentation:
> The system configuration files such as /etc/mysql/sipwise.cnf and the
> /etc/ngcp-config/ directory should be included in the backup
On my side:
> 22:36:27 ? taurus:(master)~/sipwise/git/documentation$ ack "sipwise.cnf"
> debian/changelog
> 67: * [80136c1] TT#68021 Add remark about Deprecated /etc/mysql/sipwise.cnf to mr8.1 release notes
> 68: * [736174d] TT#68035 Remove /etc/mysql/sipwise.cnf from documentation
> 22:36:30 ? taurus:(master)~/sipwise/git/documentation$
============
re: mysql start issue
Good to check mysql error log: /var/log/mysql/error.log
also systemd can print something to /var/log/syslog
Maybe it is somehow related to DB encryption, is it enabled for
your installation? It is enabled by default for mr8.3.1:
> root at spce:~# grep crypt /etc/mysql/my.cnf
> # Encryption options
> innodb_encrypt_tables = force
> innodb_encrypt_log = 1
> encrypt_binlog = 1
> innodb_encryption_threads = 1
> innodb_tablespaces_encryption
> root at spce:~#
Or there could be the issue with credential in /etc/mysql/debian.cnf for
user debian-sys-maint.
Please share your findings.
On 4/14/20 12:12 PM, Robert Cuaresma wrote:
> On the Handbook:
>
> ? System configuration
>
> The system configuration files such as /etc/mysql/sipwise.cnf and the
> /etc/ngcp-config/ directory should be included in the backup
> Is it possible that you didn?t copy /etc/mysql/sipwise.cnf?
>
> Saludos,
>
> Robert Cuaresma
>
> ?
>
> *De:* cappellari at connectlife.it <cappellari at connectlife.it>
> *Enviado el:* martes, 14 de abril de 2020 12:06
> *Para:* spce-user at lists.sipwise.com
> *Asunto:* [Spce-user] Move Sipwise 8.3.1 to another server
>
> ?
>
> Hello everyone! I need to move a sipwise installation to another server.
> I copied the ngcp-config folder to the new server, exported the DB with
> mysqldump and reimported to the new server, applied the configuration
> with ngcpcfg apply. Everything works, but when I restart the server the
> mysql service does not start, as a consequence also the kamailio proxy
> and dependent services. I need to delete the idbdata file from /
> ngcp-data / mysql to restart mysql. It looks like an innodb corruption
> issue. Some advice?
>
--
Alex Lutay
------------------------------
Subject: Digest Footer
Spce-user mailing list
Spce-user at lists.sipwise.com
http://lists.sipwise.com/mailman/listinfo/spce-user_lists.sipwise.com
------------------------------
End of Spce-user Digest, Vol 10, Issue 15
*****************************************
------------------------------
Message: 2
Date: Wed, 15 Apr 2020 20:20:42 +0200 (CEST)
From: "cappellari at connectlife.it" <cappellari at connectlife.it>
To: spce-user <spce-user at lists.sipwise.com>
Subject: Re: [Spce-user] Spce-user Digest, Vol 10, Issue 15
Message-ID:
<1554521931.10510.1586974842114.JavaMail.zimbra at connectlife.it>
Content-Type: text/plain; charset=utf-8
After executing the ngcpcfg apply command I get:
ngcp-service: info: Action restart skipped, service kannel-bearerbox is not enabled
ngcp-service: info: Action restart skipped, service kannel-smsbox is not enabled
ngcp-service: info: Action restart skipped, service rtpengine-recording is not enabled
Job for ngcp-mediator.service failed because the control process exited with error code.
See "systemctl status ngcp-mediator.service" and "journalctl -xe" for details.
Job for ngcp-rate-o-mat.service failed because the control process exited with error code.
See "systemctl status ngcp-rate-o-mat.service" and "journalctl -xe" for details.
Job for kamailio-proxy.service failed because the control process exited with error code.
See "systemctl status kamailio-proxy.service" and "journalctl -xe" for details.
2020-04-14 13:11:12 sbc: Synchronizing current with expected services state
Job for ngcp-mediator.service failed because the control process exited with error code.
See "systemctl status ngcp-mediator.service" and "journalctl -xe" for details.
Job for ngcp-rate-o-mat.service failed because the control process exited with error code.
See "systemctl status ngcp-rate-o-mat.service" and "journalctl -xe" for details.
Job for kamailio-proxy.service failed because the control process exited with error code.
See "systemctl status kamailio-proxy.service" and "journalctl -xe" for details.
MYSQL LOG:
2020-04-14 13:01:39 0 [Note] InnoDB: Creating #1 encryption thread id 140460823652096 total threads 1.
2020-04-14 13:01:39 0 [Note] InnoDB: Loading buffer pool(s) from /ngcp-data/mysql/ib_buffer_pool
2020-04-14 13:01:39 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-04-14 13:01:39 0 [Note] Server socket created on IP: '127.0.0.1'.
2020-04-14 13:01:39 0 [Note] InnoDB: Buffer pool(s) load completed at 200414 13:01:39
2020-04-14 13:01:39 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.3.22-MariaDB-0+deb10u1-log' socket: '/run/mysqld/mysqld.sock' port: 3306 Debian 10
2020-04-14 13:11:30 0 [Note] /usr/sbin/mysqld (initiated by: unknown): Normal shutdown
2020-04-14 13:11:30 0 [Note] Event Scheduler: Purging the queue. 0 events
2020-04-14 13:11:30 0 [Note] InnoDB: FTS optimize thread exiting.
2020-04-14 13:11:30 0 [Note] InnoDB: Starting shutdown...
2020-04-14 13:11:30 0 [Note] InnoDB: Dumping buffer pool(s) to /ngcp-data/mysql/ib_buffer_pool
2020-04-14 13:11:30 0 [Note] InnoDB: Instance 0, restricted to 1024 pages due to innodb_buf_pool_dump_pct=25
2020-04-14 13:11:30 0 [Note] InnoDB: Buffer pool(s) dump completed at 200414 13:11:30
2020-04-14 13:11:32 0 [Note] InnoDB: Shutdown completed; log sequence number 172493761; transaction id 23307
2020-04-14 13:11:32 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-04-14 13:11:32 0 [Note] /usr/sbin/mysqld: Shutdown complete
2020-04-14 13:11:35 0 [Note] InnoDB: Using Linux native AIO
2020-04-14 13:11:35 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-04-14 13:11:35 0 [Note] InnoDB: Uses event mutexes
2020-04-14 13:11:35 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-04-14 13:11:35 0 [Note] InnoDB: Number of pools: 1
2020-04-14 13:11:35 0 [Note] InnoDB: Using SSE2 crc32 instructions
2020-04-14 13:11:35 0 [Note] InnoDB: Initializing buffer pool, total size = 64M, instances = 1, chunk size = 64M
2020-04-14 13:11:35 0 [Note] InnoDB: Completed initialization of buffer pool
2020-04-14 13:11:35 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-04-14 13:11:35 0 [Warning] InnoDB: Log scan aborted at LSN 172493312
2020-04-14 13:11:35 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2020-04-14 13:11:36 0 [Note] InnoDB: Starting shutdown...
2020-04-14 13:11:36 0 [ERROR] Plugin 'InnoDB' init function returned error.
2020-04-14 13:11:36 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2020-04-14 13:11:36 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-04-14 13:11:36 0 [ERROR] Unknown/unsupported storage engine: InnoDB
2020-04-14 13:11:36 0 [ERROR] Aborting
----- Messaggio originale -----
Da: spce-user-request at lists.sipwise.com
A: "spce-user" <spce-user at lists.sipwise.com>
Inviato: Mercoled?, 15 aprile 2020 18:00:04
Oggetto: Spce-user Digest, Vol 10, Issue 15
Send Spce-user mailing list submissions to
spce-user at lists.sipwise.com
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.sipwise.com/mailman/listinfo/spce-user_lists.sipwise.com
or, via email, send a message with subject or body 'help' to
spce-user-request at lists.sipwise.com
You can reach the person managing the list at
spce-user-owner at lists.sipwise.com
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Spce-user digest..."
Today's Topics:
1. Re: Move Sipwise 8.3.1 to another server (Alex Lutay)
----------------------------------------------------------------------
Message: 1
Date: Tue, 14 Apr 2020 22:37:50 +0200
From: Alex Lutay <alutay at sipwise.com>
To: spce-user at lists.sipwise.com
Subject: Re: [Spce-user] Move Sipwise 8.3.1 to another server
Message-ID: <9d803207-0125-01cd-20da-31860e584886 at sipwise.com>
Content-Type: text/plain; charset=utf-8
Hi all,
re: /etc/mysql/sipwise.cnf
mr8.3.1 doesn't operate with /etc/mysql/sipwise.cnf
It was deprecated and removed in mr8.1.1+:
> https://www.sipwise.com/doc/mr8.1.1/spce/?_href=ar01s14.html
> Deprecated and removed /etc/mysql/sipwise.cnf, please use /etc/mysql/sipwise_extra.cnf [TT#68021]
Anyway both those files are generated from templates in a modern release
~mr6.5+, so no need to copy it manually.
Please report if you still see in a modern documentation:
> The system configuration files such as /etc/mysql/sipwise.cnf and the
> /etc/ngcp-config/ directory should be included in the backup
On my side:
> 22:36:27 ? taurus:(master)~/sipwise/git/documentation$ ack "sipwise.cnf"
> debian/changelog
> 67: * [80136c1] TT#68021 Add remark about Deprecated /etc/mysql/sipwise.cnf to mr8.1 release notes
> 68: * [736174d] TT#68035 Remove /etc/mysql/sipwise.cnf from documentation
> 22:36:30 ? taurus:(master)~/sipwise/git/documentation$
============
re: mysql start issue
Good to check mysql error log: /var/log/mysql/error.log
also systemd can print something to /var/log/syslog
Maybe it is somehow related to DB encryption, is it enabled for
your installation? It is enabled by default for mr8.3.1:
> root at spce:~# grep crypt /etc/mysql/my.cnf
> # Encryption options
> innodb_encrypt_tables = force
> innodb_encrypt_log = 1
> encrypt_binlog = 1
> innodb_encryption_threads = 1
> innodb_tablespaces_encryption
> root at spce:~#
Or there could be the issue with credential in /etc/mysql/debian.cnf for
user debian-sys-maint.
Please share your findings.
On 4/14/20 12:12 PM, Robert Cuaresma wrote:
> On the Handbook:
>
> ? System configuration
>
> The system configuration files such as /etc/mysql/sipwise.cnf and the
> /etc/ngcp-config/ directory should be included in the backup
> Is it possible that you didn?t copy /etc/mysql/sipwise.cnf?
>
> Saludos,
>
> Robert Cuaresma
>
> ?
>
> *De:* cappellari at connectlife.it <cappellari at connectlife.it>
> *Enviado el:* martes, 14 de abril de 2020 12:06
> *Para:* spce-user at lists.sipwise.com
> *Asunto:* [Spce-user] Move Sipwise 8.3.1 to another server
>
> ?
>
> Hello everyone! I need to move a sipwise installation to another server.
> I copied the ngcp-config folder to the new server, exported the DB with
> mysqldump and reimported to the new server, applied the configuration
> with ngcpcfg apply. Everything works, but when I restart the server the
> mysql service does not start, as a consequence also the kamailio proxy
> and dependent services. I need to delete the idbdata file from /
> ngcp-data / mysql to restart mysql. It looks like an innodb corruption
> issue. Some advice?
>
--
Alex Lutay
------------------------------
Subject: Digest Footer
Spce-user mailing list
Spce-user at lists.sipwise.com
http://lists.sipwise.com/mailman/listinfo/spce-user_lists.sipwise.com
------------------------------
End of Spce-user Digest, Vol 10, Issue 15
*****************************************
------------------------------
Message: 3
Date: Thu, 16 Apr 2020 09:34:42 +0200
From: Alex Lutay <alutay at sipwise.com>
To: spce-user <spce-user at lists.sipwise.com>
Subject: Re: [Spce-user] Spce-user Digest, Vol 10, Issue 15
Message-ID: <55c38e91-6ad1-bc31-af5a-4a9ea51d030b at sipwise.com>
Content-Type: text/plain; charset=utf-8
Hi all,
On 4/15/20 8:20 PM, cappellari at connectlife.it wrote:
> After executing the ngcpcfg apply command I get:
...
> Job for kamailio-proxy.service failed because the control process exited with error code.
> See "systemctl status kamailio-proxy.service" and "journalctl -xe" for details.
All those errors are results of failed MariaDB start.
> MYSQL LOG:
...
> 2020-04-14 13:11:35 0 [Warning] InnoDB: Log scan aborted at LSN 172493312
> 2020-04-14 13:11:35 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
...
> 2020-04-14 13:11:36 0 [Note] InnoDB: Starting shutdown...
...
Which means MariaDB cannot read binlogs because they are encrypted. See
/etc/ngcp-config/constants.yml
> mysql:
> encryption:
> enable: yes
> encrypt_binlog: yes
> key: 1;somesecretkeyhere
I believe your problem in wrong OLD key on NEW server:
1) you have installed NEW system and NEW MariaDB key is generated
2) those NEW key is in use for current MariaDB encryption
2) you have copied constants.yml from OLD system and saved it AS AS
including the old encryption key. "ngcpcfg apply" updated MariaDB
configs and restarted MariaDB -> MariaDB cannot start
(binlogs are encrypted with NEW key but my.cnf has OLD key now).
IMHO, you should replace OLD mysql.encryption.key in constants.yml to
NEW one after copying constants.yml from old system (before you call
"ngcpcfg apply").
Please share your results and we will update public documentation to
highlight this use case!
--
Alex Lutay
------------------------------
Subject: Digest Footer
Spce-user mailing list
Spce-user at lists.sipwise.com
http://lists.sipwise.com/mailman/listinfo/spce-user_lists.sipwise.com
------------------------------
End of Spce-user Digest, Vol 10, Issue 16
*****************************************
More information about the Spce-user
mailing list