[Spce-user] Unable to creat large invoices

Thu Oct 26 16:30:47 EDT 2017

After doing some reading i made the following changes and got it working. 

In /etc/mysql/my.cnf i changed wait_timeout from 60 to 600

I also increased server RAM from 4GB to 6GB.

Now invoice is generated. It is 818 pages long. It takes almost 3mins to generate and uses 2GB RAM. This seems exessive. Anyway i can deacrease amount of RAM utilised by generating invoice? 

Get Outlook for Android

On Thu, Oct 26, 2017 at 11:46 AM +0200, "Koot Pienaar" <koot at ntelecom.co.za> wrote:

Hi Irina

I do not know how many pages the invoice is, as it never generates. It also does not generate the output in panel-debug.log

I have the following in panel-debug.log

Oct 26 11:37:22 ntel-sipsw1 ngcp-panel: DEBUG: *** Root::auto grant access for authenticated user

Oct 26 11:37:22 ntel-sipsw1 ngcp-panel: DEBUG: NGCP::Panel::Controller::Invoice::auto

Oct 26 11:37:24 ntel-sipsw1 ngcp-panel: DEBUG: *** Root::auto grant access for authenticated user

Oct 26 11:37:24 ntel-sipsw1 ngcp-panel: DEBUG: NGCP::Panel::Controller::Invoice::auto

Oct 26 11:37:26 ntel-sipsw1 ngcp-panel: DEBUG: *** Root::auto grant access for authenticated user

Oct 26 11:37:26 ntel-sipsw1 ngcp-panel: DEBUG: NGCP::Panel::Controller::Invoice::auto

Oct 26 11:37:26 ntel-sipsw1 ngcp-panel: DEBUG: *** Root::auto grant access for authenticated user

Oct 26 11:37:26 ntel-sipsw1 ngcp-panel: DEBUG: NGCP::Panel::Controller::InvoiceTemplate::auto

Oct 26 11:37:26 ntel-sipsw1 ngcp-panel: DEBUG: *** Root::auto grant access for authenticated user

Oct 26 11:37:26 ntel-sipsw1 ngcp-panel: DEBUG: NGCP::Panel::Controller::Customer::auto

Oct 26 11:38:45 ntel-sipsw1 ngcp-panel: DEBUG: *** Root::auto grant access for authenticated user

Oct 26 11:38:45 ntel-sipsw1 ngcp-panel: DEBUG: NGCP::Panel::Controller::Invoice::auto

Oct 26 11:38:45 ntel-sipsw1 ngcp-panel: DEBUG: catchup contract 15 contract_balances (now = 2017-10-26 11:37:37)

Oct 26 11:38:45 ntel-sipsw1 ngcp-panel: DEBUG: *** Root::auto grant access for authenticated user

Oct 26 11:38:45 ntel-sipsw1 ngcp-panel: DEBUG: NGCP::Panel::Controller::Invoice::auto

Oct 26 11:38:47 ntel-sipsw1 ngcp-panel: DEBUG: *** Root::auto grant access for authenticated user

Oct 26 11:38:47 ntel-sipsw1 ngcp-panel: DEBUG: NGCP::Panel::Controller::Invoice::auto

In panel.log I get the following:

Oct 26 11:38:45 ntel-sipsw1 ngcp-panel: ERROR: IP=192.168.7.2 CALLED=Controller::Invoice::__ANON__ TX=14F1154286488030 USER=administrator DATA={ 'save' => 'Save', 'contract.id' => '15', 'submitid' => '', 'period' => '2017-09', 'template.id' => '2'} MSG="Failed to save invoice meta data. (DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: MySQL server has gone away [for Statement "INSERT INTO `billing`.`invoices` ( `amount_net`, `amount_total`, `amount_vat`, `contract_id`, `data`, `period_end`, `period_start`, `serial`) VALUES ( ?, ?, ?, ?, ?, ?, ?, ? )" with ParamValues: 0=5923.9, 1=6753.25, 2=829.346, 3="15", 4=undef, 5='2017-09-30 23:59:59', 6='2017-09-01 00:00:00', 7='tmp150901072587348'] at /usr/share/perl5/NGCP/Panel/Controller/Invoice.pm line 290#012)" LOG=""

Oct 26 11:38:45 ntel-sipsw1 ngcp-panel: ERROR: IP=192.168.7.2 CALLED=Controller::Invoice::create TX=14F115428D02C2A0 USER=administrator DATA={ 'period' => '2017-09', 'template.id' => '2', 'contract.id' => '15', 'save' => 'Save', 'submitid' => ''} MSG="Failed to create invoice. ({UNKNOWN}: Use of uninitialized value $contract_id in concatenation (.) or string at /usr/share/perl5/NGCP/Panel/Controller/Invoice.pm line 169. at /usr/share/perl5/Catalyst/Model/DBIC/Schema.pm line 526#012)" LOG=""

It seems still that mysql is “going away” for some reason.

Regards 

From: Irina Peshinskaya [mailto:ipeshinskaya at sipwise.com] 
Sent: Wednesday, 25 October 2017 17:32
To: Koot Pienaar <koot at ntelecom.co.za>; Spce-user at lists.sipwise.com
Subject: Re: [Spce-user] Unable to creat large invoices

Dear Koot,

I just created invoice with 112 pages on the NGCP VM:  5.3.1 pro.

In your /var/log/ngcp/panel-debug.log you can see debug looks like:

           rsvg-convert -a -f pdf -z 0.8 /tmp/Rd13zp00eh/111.svg /tmp/Rd13zp00eh/112.svg

you can find it also using:

           grep -r rsvg-convert /var/log/ngcp/.

This debug is produced before attempt to insert data into DB.

My question is: how many pages are in the invoice that makes panel to fail?

Also, do I understand it correct that you use CE NGCP version? I will try to reproduce it there too.

Best regards,

Irina peshinskaya

On 25.10.2017 12:07, Koot Pienaar wrote:

I am still unable to create invoices that have large size for calculation.

Any assistance would be appreciated, seems to be mysql related

From: Spce-user [mailto:spce-user-bounces at lists.sipwise.com] On Behalf Of Koot Pienaar
Sent: Wednesday, 18 October 2017 12:56
To: 'Irina Peshinskaya' <ipeshinskaya at sipwise.com>; Spce-user at lists.sipwise.com
Subject: Re: [Spce-user] Unable to creat large invoices

HI

I followed your guide and increase the size limit to 512M but still get the same error.

From: Irina Peshinskaya [mailto:ipeshinskaya at sipwise.com] 
Sent: Thursday, 12 October 2017 16:18
To: Koot Pienaar <koot at ntelecom.co.za>; Spce-user at lists.sipwise.com
Subject: Re: [Spce-user] Unable to creat large invoices

Hello Koot,

yes, I see now.

The root reason could be big "data" size for the invoice, considering "MySQL server has gone away" error.

Could you please share output of the:

    grep max_allowed_packet /etc/mysql/my.cnf

Values in my config are:

max_allowed_packet                = 128M
max_allowed_packet                = 128M

Do these values look like close to your generated invoices size?
If yes, you can increase config value 

www_admin.filesize_limit in your /etc/ngcp-config/config.yml and apply these changes.

Best regards,
Irina Peshinskaya

On 12.10.2017 12:43, Koot Pienaar wrote:

HI Irina

Current version is 5.3.1

ngcp-panel 5.3.1.2+0~mr5.3.1.2

regards 

Koot Pienaar

VoIP Coordinator

Ntelecom Central

Tel: 051 412 6300

Email: koot at ntelecom.co.za

Website: www.ntelecom.co.za

Confidentiality Notice: 

This e-mail may contain confidential information and is intended only for the use of the recipient named above. 

Should you receive this e-mail in error, please forward it to koot at ntelecom.co.za  and delete from your in-box. 

Any disclosure, copying, distribution or action on the contents of this e-mail is strictly prohibited. 

Please consider the environment before printing this email and/or any related attachments.

From: Irina Peshinskaya [mailto:ipeshinskaya at sipwise.com] 
Sent: 12 October 2017 11:13 AM
To: Koot Pienaar; Spce-user at lists.sipwise.com
Subject: Re: [Spce-user] Unable to creat large invoices

Dear Koot,

Could you please remind me your ngcp-panel version? 
All concatenations I see in the Invoice.pm are too far from line 169.

Thanks in advance.

Best regards,
Irina Peshinskaya

On 12.10.2017 11:36, Koot Pienaar wrote:

Hi Team

I get the following error in panel.log when generating invoices with large amounts of calls. 

Oct 12 10:28:20 ntel-sipsw1 ngcp-panel: ERROR: IP=2.1.1.86 CALLED=Controller::Invoice::__ANON__ TX=14ECC54ADC9EF410 USER=administrator DATA={ 'template.id' => '2', 'period' => '2017-09', 'submitid' => '', 'contract.id' => '15', 'save' => 'Save'} MSG="Failed to save invoice meta data. (DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: MySQL server has gone away [for Statement "INSERT INTO `billing`.`invoices` ( `amount_net`, `amount_total`, `amount_vat`, `contract_id`, `data`, `period_end`, `period_start`, `serial`) VALUES ( ?, ?, ?, ?, ?, ?, ?, ? )" with ParamValues: 0=5923.9, 1=6753.25, 2=829.346, 3="15", 4=undef, 5='2017-09-30 23:59:59', 6='2017-09-01 00:00:00', 7='tmp150779690053912'] at /usr/share/perl5/NGCP/Panel/Controller/Invoice.pm line 290#012)" LOG=""

Oct 12 10:28:20 ntel-sipsw1 ngcp-panel: ERROR: IP=2.1.1.86 CALLED=Controller::Invoice::create TX=14ECC54AE303C380 USER=administrator DATA={ 'submitid' => '', 'contract.id' => '15', 'save' => 'Save', 'template.id' => '2', 'period' => '2017-09'} MSG="Failed to create invoice. ({UNKNOWN}: Use of uninitialized value $contract_id in concatenation (.) or string at /usr/share/perl5/NGCP/Panel/Controller/Invoice.pm line 169. at /usr/share/perl5/Catalyst/Model/DBIC/Schema.pm line 526#012)" LOG=""

I do not see any logs in Mysql.log or mysql.err 

Koot Pienaar

VoIP Coordinator

Ntelecom Central

Tel: 051 412 6300

Email: koot at ntelecom.co.za

Website: www.ntelecom.co.za

Confidentiality Notice: 

This e-mail may contain confidential information and is intended only for the use of the recipient named above. 

Should you receive this e-mail in error, please forward it to koot at ntelecom.co.za  and delete from your in-box. 

Any disclosure, copying, distribution or action on the contents of this e-mail is strictly prohibited. 

Please consider the environment before printing this email and/or any related attachments.

_______________________________________________Spce-user mailing listSpce-user at lists.sipwise.comhttps://lists.sipwise.com/listinfo/spce-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sipwise.com/pipermail/spce-user_lists.sipwise.com/attachments/20171026/190edf4d/attachment-0001.html>