[Spce-user] Tuning MySQL due to slow queries on mr4.5.4

Fri Jun 30 11:32:17 EDT 2017

Hi,

Sent you some queries in a separate email. Did some more tuning and now tuning-primer.sh shows this (probably have to run it tomorrow again and monitor the slow logs).
The temp tables seem to still be created mostly on disk though. Why ?  Are there any tables with TEXT and BLOB we should optimise to VARCHAR ?

Note this instance is running off a CEPH storage system. (Redhat)

	-- MYSQL PERFORMANCE TUNING PRIMER --
	     - By: Matthew Montgomery -

MySQL Version 5.5.55-0+deb8u1-log x86_64

Uptime = 0 days 3 hrs 15 min 10 sec
Avg. qps = 22
Total Questions = 258193
Threads Connected = 46

Warning: Server has not been running for at least 48hrs.
It may not be safe to use these recommendations

To find out more information on how each of these
runtime variables effects performance visit:
http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html
Visit http://www.mysql.com/products/enterprise/advisors.html
for info about MySQL's Enterprise Monitoring and Advisory Service

SLOW QUERIES
The slow query log is enabled.
Current long_query_time = 5.000000 sec.
You have 21 out of 258214 that take longer than 5.000000 sec. to complete
Your long_query_time seems to be fine

BINARY UPDATE LOG
The binary update log is enabled

WORKER THREADS
Current thread_cache_size = 64
Current threads_cached = 14
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine

MAX CONNECTIONS
Current max_connections = 128
Current threads_connected = 46
Historic max_used_connections = 60
The number of used connections is 46% of the configured maximum.
Your max_connections variable seems to be fine.

INNODB STATUS
Current InnoDB index space = 579 M
Current InnoDB data space = 654 M
Current InnoDB buffer pool free = 93 %
Current innodb_buffer_pool_size = 6.00 G
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory

MEMORY USAGE
Max Memory Ever Allocated : 37.72 G
Configured Max Per-thread Buffers : 67.39 G
Configured Max Global Buffers : 6.13 G
Configured Max Memory Limit : 73.53 G
Physical Memory : 31.36 G

Max memory limit exceeds 90% of physical memory

KEY BUFFER
Current MyISAM index space = 156 K
Current key_buffer_size = 64 M
Key cache miss rate is 1 : 519
Key buffer free ratio = 81 %
Your key_buffer_size seems to be fine

QUERY CACHE
Query cache is enabled
Current query_cache_size = 32 M
Current query_cache_used = 29 M
Current query_cache_limit = 4 M
Current Query cache Memory fill ratio = 91.17 %
Current query_cache_min_res_unit = 4 K
MySQL won't cache query results that are larger than query_cache_limit in size

SORT OPERATIONS
Current sort_buffer_size = 512 M
Current read_rnd_buffer_size = 8 M
Sort buffer seems to be fine

JOINS
Current join_buffer_size = 16.00 M
You have had 57 queries where a join could not use an index properly
join_buffer_size >= 4 M
This is not advised
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.

OPEN FILES LIMIT
Current open_files_limit = 4234 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine

TABLE CACHE
Current table_open_cache = 2048 tables
Current table_definition_cache = 400 tables
You have a total of 343 tables
You have 392 open tables.
The table_cache value seems to be fine

TEMP TABLES
Current max_heap_table_size = 512 M
Current tmp_table_size = 512 M
Of 13208 temp tables, 39% were created on disk
Perhaps you should increase your tmp_table_size and/or max_heap_table_size
to reduce the number of disk-based temporary tables
Note! BLOB and TEXT columns are not allow in memory tables.
If you are using these columns raising these values might not impact your
ratio of on disk temp tables.

TABLE SCANS
Current read_buffer_size = 2 M
Current table scan ratio = 264 : 1
read_buffer_size seems to be fine

TABLE LOCKING
Current Lock Wait ratio = 0 : 258520
Your table locking seems to be fine

Warmest Regards,

 <https://myrepublic.com.sg/>	
Walter Klomp
Head of Voice
MyRepublic Limited
T: +65 6816 1120
F: +65 6717 2031

MyRepublic Limited
33 Ubi Ave 3, Vertex Tower B, #04-13 Singapore 408868

myrepublic.com.sg <https://myrepublic.com.sg/>
Follow us on: Twitter <https://twitter.com/myrepublic> | Facebook <https://facebook.com/myrepublicsg> | LinkedIn <https://www.linkedin.com/company/myrepublic>

> On 29 Jun 2017, at 11:06 PM, Rene Krenn <rkrenn at sipwise.com> wrote:
> 
> hi walter,
> 
> the thing is that the my.cnf params have less effect compared to make
> queries hit properly prepared indexes, or refactoring queries
> explicitly. mysql's creation of temp tables during reqular queries is
> typical if it has hard times to solve a select.
> 
> probably this is also the case with your setup and queries against
> kamailio.location (again).
> 
> can you sent me which queries show up as slow?
> 
> regards, rene
> 
> On Thu, 2017-06-29 at 22:46 +0800, Walter Klomp wrote:
>> Hi,
>> 
>> 
>> I am having some serious performance issues where MySQL server comes
>> to a halt temporarily and registrations fail (SPCE returns 100 trying,
>> then 500 and then 200 after that on the same register request - which
>> is probably not RFC compliant either) 
>> 
>> 
>> … I have been tweaking the my.cnf configuration (by modifying the
>> my.cnf.customtt.tt2 template) a bit to alleviate it with some success,
>> but I still see the below:
>> 
>> 
>> using
>> 
>> 
>> -- MYSQL PERFORMANCE TUNING PRIMER —
>> 
>> 
>> 
>> 
>> TEMP TABLES
>> Current max_heap_table_size = 128 M
>> Current tmp_table_size = 512 M
>> Of 71563 temp tables, 45% were created on disk
>> Effective in-memory tmp_table_size is limited to max_heap_table_size.
>> Perhaps you should increase your tmp_table_size and/or
>> max_heap_table_size
>> to reduce the number of disk-based temporary tables
>> Note! BLOB and TEXT columns are not allow in memory tables.
>> If you are using these columns raising these values might not impact
>> your
>> ratio of on disk temp tables.
>> 
>> 
>> How big should this be, noting the whole database size is currently
>> around 960Mb (6000+ subscribers).  This is running on Redhat Openstack
>> with CEPH storage (with 20 spinning high speed drives).
>> 
>> 
>> 12 cores and 32GB RAM (of which currently less than half is used).
>> Increasing bufferpoolsize does nothing (as it only needs about 1 Gig
>> and it is set to 4G at the moment)
>> 
>> 
>> Hope to find some MySQL tuning experts here…
>> 
>> 
>> Thanks
>> Walter.
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Spce-user mailing list
>> Spce-user at lists.sipwise.com
>> https://lists.sipwise.com/listinfo/spce-user
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sipwise.com/pipermail/spce-user_lists.sipwise.com/attachments/20170630/aa3ff786/attachment-0001.html>