[Spce-user] Observations/issues after upgrading to 3.6.1 - nginx

Mon Nov 17 08:55:19 EST 2014

Dear Thomas,

First of all, thank you for the bug report!

Please find my in-line reply:

On 14/11/14 17:23, Thomas Odorfer wrote:
> Just upgraded from 3.5.1 to 3.6.1 - got major issues concerning apache2 replecement for panel:
>
> 1) after upgrade nginx start failed due to missing port numbers in nginx conf file sites-enabled/ngcp-panel
>
> Reason: config.yml had not been adapted during the upgrade to reflect the new entries within www_admin (http_admin and http_csc).
> I had to adapt config.yml manually (started a fresh vagrant setup to compare with a reference config.yml)

This is really strange, because is should be done in statement 15060:
>> https://github.com/sipwise/cfg-schema/blob/mr3.6.1/cfg_scripts/config/15060_www_admin_apache_move.up

Please check that is was executed during upgrade:
>> mysql ngcp -e "select * from cfg_schema where revision=15060"

Can you please send me privately your config.yml from backup
if 15060 was executed but resulted config.yml had no http_admin and 
http_csc. You can find it in /var/backup/ngcp-mr3.6.1/ngcp-config
Tnx!

------------------------------------------

> BTW:  /etc/nginx/sites-enabled still contained a link for discarded ngcp-www-csc

Yes, it contains. It is correct because ngcp-www-csc was not deleted,
it has been disabled only. It is going to be deleted in upcoming release(s).

------------------------------------------

> 2) Login of users successful , however any activity including dashboard immediately results in "500 Internal server error (admin is working).
> The error log from panel.log:
>
> Nov 14 17:14:04 ngcptest ngcp-panel: ERROR: fatal error, id=13A693CDA07AF0B0, timestamp=2014-11-14T16:14:04.224590000Z, error=DBIx::Class::ResultSet::all(): DBI Exception: DBD::mysql::st execute failed: Unknown column 'me.source_gpp0' in 'field list' [for Statement "SELECT `me`.`id`, `me`.`update_time`, `me`.`source_user_id`, `me`.`source_provider_id`, `me`.`source_external_subscriber_id`, `me`.`source_external_contract_id`, `me`.`source_account_id`, `me`.`source_user`, `me`.`source_domain`, `me`.`source_cli`, `me`.`source_clir`, `me`.`source_ip`, `me`.`source_gpp0`, `me`.`source_gpp1`, `me`.`source_gpp2`, `me`.`source_gpp3`, `me`.`source_gpp4`, `me`.`source_gpp5`, `me`.`source_gpp6`, `me`.`source_gpp7`, `me`.`source_gpp8`, `me`.`source_gpp9`, `me`.`destination_user_id`, `me`.`destination_provider_id`, `me`.`destination_external_subscriber_id`, `me`.`destination_external_contract_id`, `me`.`destination_account_id`, `me`.`destination_user`, `me`.`destination_domain`, `me`.`destinatio
n_user_dialed`, `me`.`destination_user_in`, `me`.`destination_domain_in`, `me`.`destination_gpp0`, `me`.`destination_gpp1`, `me`.`destination_gpp2`, `me`.`destination_gpp3`, `me`.`destination_gpp4`, `me`.`destination_gpp5`, `me`.`destination_gpp6`, `me`.`destination_gpp7`, `me`.`destination_gpp8`, `me`.`destination_gpp9`, `me`.`peer_auth_user`, `me`.`peer_auth_realm`, `me`.`call_type`, `me`.`call_status`, `me`.`call_code`, `me`.`duration`, `me`.`call_id`, `me`.`source_carrier_cost`, `me`.`source_reseller_cost`, `me`.`source_customer_cost`, `me`.`source_carrier_free_time`, `me`.`source_reseller_free_time`, `me`.`source_customer_free_time`, `me`.`source_carrier_billing_fee_id`, `me`.`source_reseller_billing_fee_id`, `me`.`source_customer_billing_fee_id`, `me`.`source_carrier_billing_zone_id`, `me`.`source_reseller_billing_zone_id`, `me`.`source_customer_billing_zone_id`, `me`.`destination_carrier_cost`, `me`.`destination_reseller_cost`, `me`.`destination_customer_cost`, `me`.`destinatio
n_carrier_free_time`, `me`.`destination_reseller_free_time`, `me`.`destination_customer_free_time`, `me`.`destination_carrier_billing_fee_id`, `me`.`destination_reseller_billing_fee_id`, `me`.`destination_customer_billing_fee_id`, `me`.`destination_carrier_billing_zone_id`, `me`.`destination_reseller_billing_zone_id`, `me`.`destination_customer_billing_zone_id`, `me`.`frag_carrier_onpeak`, `me`.`frag_reseller_onpeak`, `me`.`frag_customer_onpeak`, `me`.`is_fragmented`, `me`.`split`, `me`.`rated_at`, `me`.`rating_status`, `me`.`exported_at`, `me`.`export_status`, `me`.`init_time`, `me`.`start_time` FROM (SELECT `me`.`id`, `me`.`update_time`, `me`.`source_user_id`, `me`.`source_provider_id`, `me`.`source_external_subscriber_id`, `me`.`source_external_contract_id`, `me`.`source_account_id`, `me`.`source_user`, `me`.`source_domain`, `me`.`source_cli`, `me`.`source_clir`, `me`.`source_ip`, `me`.`source_gpp0`, `me`.`source_gpp1`, `me`.`source_gpp2`, `me`.`source_gpp3`, `me`.`source_gpp4`, `m
e`.`source_gpp5`, `me`.`source_gpp6`, `me`.`source_gpp7`, `me`.`source_gpp8`, `me`.`source_gpp9`, `me`.`destination_user_id`, `me`.`destination_provider_id`, `me`.`destination_external_subscriber_id`, `me`.`destination_external_contract_id`, `me`.`destination_account_id`, `me`.`destination_user`, `me`.`destination_domain`, `me`.`destination_user_dialed`, `me`.`destination_user_in`, `me`.`destination_domain_in`, `me`.`destination_gpp0`, `me`.`destination_gpp1`, `me`.`destination_gpp2`, `me`.`destination_gpp3`, `me`.`destination_gpp4`, `me`.`destination_gpp5`, `me`.`destination_gpp6`, `me`.`destination_gpp7`, `me`.`destination_gpp8`, `me`.`destination_gpp9`, `me`.`peer_auth_user`, `me`.`peer_auth_realm`, `me`.`call_type`, `me`.`call_status`, `me`.`call_code`, `me`.`duration`, `me`.`call_id`, `me`.`source_carrier_cost`, `me`.`source_reseller_cost`, `me`.`source_customer_cost`, `me`.`source_carrier_free_time`, `me`.`source_reseller_free_time`, `me`.`source_customer_free_time`, `me`.`sourc
e_carrier_billing_fee_id`, `me`.`source_reseller_billing_fee_id`, `me`.`source_customer_billing_fee_id`, `

 From one side I suspect cfg/db schema update was not executed during 
the upgrade, from another side I see in your log:
> Running CFG update...
> Running DB update...
So, no idea for the moment.... Can you please check max ID in cfg_schema 
and db_schema tables? They should have
>> root at spce:~# mysql ngcp -e "select * from cfg_schema order by revision desc limit 1"
>> +----+----------+------+---------------------+
>> | id | revision | node | applied_at          |
>> +----+----------+------+---------------------+
>> | 90 |    15067 | spce | 2014-11-17 02:55:08 |
>> +----+----------+------+---------------------+
>> root at spce:~# mysql ngcp -e "select * from db_schema order by revision desc limit 1"
>> +-----+----------+------+---------------------+
>> | id  | revision | node | applied_at          |
>> +-----+----------+------+---------------------+
>> | 325 |    15111 | spce | 2014-11-17 02:55:01 |
>> +-----+----------+------+---------------------+
>> root at spce:~# cat /etc/ngcp_version
>> mr3.6.1

You can try to upgrade them once more (already executed statements will 
be skipped):
> VERBOSE=yes ngcp-update-cfg-schema
> VERBOSE=yes ngcp-update-db-schema

Please share your results here. Tnx!

------------------------------------------

> after a quick investigation, I think ngcp-upgrade for mr3.5.1 to mr3.6.1 has the following issues:
> a) config.yml has not been upgraded (easy to fix)

It should be upgraded automatically... maybe you have more then one 
config.yml into /etc/ngcp-config/ ? Or some customtt files?

------------------------------------------

> b) the database schema has not been updated to 3.6.1 although ngcp-upgrade-schema had been executed  (e.g. accounting.cdr is missing the new fields source_gpp0 …) (not so easy to fix)
> The files within /usr/share/ngcp-db-schema are all dated 30th Oct.

The DB update statement #15109 had to do this:
> https://github.com/sipwise/db-schema/blob/mr3.6.1/db_scripts/diff/15109.up
Please check the content of ngcp.db_schema table...

Also please check that content of field 'node' from ngcp.db_schema equal 
to content of file /etc/ngcp_nodename.

Tnx! I hope those comments were useful for you.

-- 
Alexander Lutay
Head of Quality Assurance
Sipwise GmbH, Campus 21/Europaring F15
AT-2345 Brunn am Gebirge

Office: +43(0)13012036
Email: alutay at sipwise.com
Website: http://www.sipwise.com

Meet us @ AFRICACOM : Booth A17/A18
11-13 November 2014 : CTICC Cape Town, South Africa