[Spce-user] Logfile mysqld.err 17GB big

[Thomas Georg]

Hi list,

we had an issue when our spce has been upgraded to 6.5.4.
At this time our harddisk went out of space.

We are planning to move our production system to a new machine so I made some investigations where the disk space has gone.

First I found the mysqld.err file in /var log/mysql/ with a size of 17GB!

I wondered about the following:
########## mysqld.err ########
2019-05-30 23:08:27 140628650698752 [ERROR] mysqld: Can't lock aria control file '/ngcp-data/mysql/aria_log_control' for exclusive use, error: 11. Will retry for 30 secon
ds
2019-05-30 23:08:27 140112560863232 [ERROR] mysqld: Can't lock aria control file '/ngcp-data/mysql/aria_log_control' for exclusive use, error: 11. Will retry for 30 secon
ds
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff,
 increase buffer pool at least up to 20MB.

2019-05-30 23:08:27 140253069703168 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: The InnoDB memory heap is disabled
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: Compressed tables use zlib 1.2.8
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: Using Linux native AIO
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: Using SSE crc32 instructions
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: Initializing buffer pool, size = 64.0M
2019-05-30 23:08:27 140253069703168 [Note] InnoDB: Completed initialization of buffer pool
2019-05-30 23:08:29 140253069703168 [Note] InnoDB: Setting log file /ngcp-data/mysql/ib_logfile101 size to 128 MB
2019-05-30 23:08:31 140253069703168 [Note] InnoDB: Setting log file /ngcp-data/mysql/ib_logfile1 size to 128 MB
2019-05-30 23:08:35 140253069703168 [Note] InnoDB: Setting log file /ngcp-data/mysql/ib_logfile2 size to 128 MB
2019-05-30 23:08:39 140253069703168 [Note] InnoDB: Setting log file /ngcp-data/mysql/ib_logfile3 size to 128 MB
2019-05-30 23:08:40 140253069703168 [Note] InnoDB: Renaming log file /ngcp-data/mysql/ib_logfile101 to /ngcp-data/mysql/ib_logfile0
2019-05-30 23:08:40 140253069703168 [Warning] InnoDB: New log files created, LSN=74857028238
2019-05-30 23:08:40 140253069703168 [Note] InnoDB: Highest supported file format is Barracuda.
2019-05-30 23:08:40 7f8f3665f800 InnoDB: Error: page 3 log sequence number 75407322724
InnoDB: is in the future! Current system log sequence number 74857028620.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
2019-05-30 23:08:40 7f8f3665f800 InnoDB: Error: page 2 log sequence number 98893166510
InnoDB: is in the future! Current system log sequence number 74857028620.
InnoDB: Your database may be corrupt or you may have copied the InnoDB
InnoDB: tablespace but not the InnoDB log files. See
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: for more information.
.......
#########################

The log is full of these InnoDB errors.

Has anyone an explanation for this and how to fix it / do more investigation?

Our disk is currently at 91%, is it possible to move the file to another place (scp to our central backup storage) and restart the logging without service interruption?

Regards
Thomas