【发布时间】：2020-08-25 01:38:10
【问题描述】：
我正在使用 wordpress、php 7.2 和 mysql Ver 15.1 Distrib 10.1.44-MariaDB 对生产服务器进行负载测试，具有 4 个 Core i7 CPU 和 10GB Ram 在 ubuntu 18.04 LTS 上运行。问题是即使负载较低，例如每秒 5 个用户，所有 4 个 CPU 都达到 100% 并且它们仍然存在。一段时间后 CPU 已满，我只收到错误 500。此外，我的所有模块都是最新的。
我尝试禁用 wordpress 中的所有插件并运行相同的测试，但问题仍然存在。
我还使用了一些插件来优化数据库，例如wp-optimize，但没有运气。 /var/log/mysql/mysql-slow.log 文件也是空的。
Result of load testing after only few sec including htop
Datadog's monitoring dashboard of the related test
如上图所示，你可以理解mysqld无缘无故占用了过多的CPU。
我也运行了 mysqltuner，结果如下：
>> MySQLTuner 1.7.19 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
[--] Skipped version check for MySQLTuner script
[OK] Logged in using credentials passed on the command line
[OK] Currently running supported MySQL version 10.1.44-MariaDB-0ubuntu0.18.04.1
[OK] Operating on 64-bit architecture
-------- Log file Recommendations ------------------------------------------------------------------
[OK] Log file /var/log/mysql/error.log exists
[--] Log file: /var/log/mysql/error.log(7K)
[OK] Log file /var/log/mysql/error.log is readable.
[OK] Log file /var/log/mysql/error.log is not empty
[OK] Log file /var/log/mysql/error.log is smaller than 32 Mb
[!!] /var/log/mysql/error.log contains 8 warning(s).
[!!] /var/log/mysql/error.log contains 3 error(s).
[--] 3 start(s) detected in /var/log/mysql/error.log
[--] 1) 2020-05-09 12:36:17 139638727916672 [Note] /usr/sbin/mysqld: ready for connections.
[--] 2) 2020-05-09 12:28:42 140719406492800 [Note] /usr/sbin/mysqld: ready for connections.
[--] 3) 2020-05-09 10:52:43 139784491093120 [Note] /usr/sbin/mysqld: ready for connections.
[--] 2 shutdown(s) detected in /var/log/mysql/error.log
[--] 1) 2020-05-09 12:32:02 140713171822336 [Note] /usr/sbin/mysqld: Shutdown complete
[--] 2) 2020-05-09 10:53:00 139784442357504 [Note] /usr/sbin/mysqld: Shutdown complete
-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +Aria +CSV +InnoDB +MEMORY +MRG_MyISAM +MyISAM +PERFORMANCE_SCHEMA +SEQUENCE
[--] Data in InnoDB tables: 112.0K (Tables: 4)
[--] Data in MyISAM tables: 38.7M (Tables: 52)
[OK] Total fragmented tables: 0
-------- Analysis Performance Metrics --------------------------------------------------------------
[--] innodb_stats_on_metadata: OFF
[OK] No stat updates during querying INFORMATION_SCHEMA.
-------- Security Recommendations ------------------------------------------------------------------
[OK] There are no anonymous accounts for any database users
[OK] All database users have passwords assigned
[!!] There is no basic password file list!
-------- CVE Security Recommendations --------------------------------------------------------------
[--] Skipped due to --cvefile option undefined
-------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 48m 13s (73K q [25.317 qps], 1K conn, TX: 978M, RX: 11M)
[--] Reads / Writes: 96% / 4%
[--] Binary logging is disabled
[--] Physical Memory : 9.8G
[--] Max MySQL memory : 10.6G
[--] Other process memory: 0B
[--] Total buffers: 4.7G global + 6.9M per thread (800 max threads)
[--] P_S Max memory usage: 511M
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 6.5G (67.07% of installed RAM)
[!!] Maximum possible memory usage: 10.6G (108.28% of installed RAM)
[!!] Overall possible memory usage with other process exceeded memory
[OK] Slow queries: 0% (0/73K)
[OK] Highest usage of available connections: 25% (204/800)
[!!] Aborted connections: 10.65% (193/1812)
[OK] Query cache is disabled by default due to mutex contention on multiprocessor machines.
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 12K sorts)
[OK] No joins without indexes
[!!] Temporary tables created on disk: 83% (2K on disk / 3K total)
[OK] Thread cache hit rate: 84% (284 created / 1K connections)
[OK] Table cache hit rate: 97% (222 open / 228 opened)
[OK] table_definition_cache(400) is upper than number of tables(303)
[OK] Open file limit used: 6% (304/4K)
[!!] Table locks acquired immediately: 84%
-------- Performance schema ------------------------------------------------------------------------
[--] Memory used by P_S: 511.6M
[--] Sys schema is installed.
-------- ThreadPool Metrics ------------------------------------------------------------------------
[--] ThreadPool stat is enabled.
[--] Thread Pool Size: 4 thread(s).
[--] Using default value is good enough for your version (10.1.44-MariaDB-0ubuntu0.18.04.1)
-------- MyISAM Metrics ----------------------------------------------------------------------------
[!!] Key buffer used: 18.4% (24M used / 134M cache)
[OK] Key buffer size / total MyISAM indexes: 128.0M/10.0M
[OK] Read Key buffer hit rate: 100.0% (695K cached / 152 reads)
[OK] Write Key buffer hit rate: 100.0% (18 cached / 18 writes)
-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 0
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 3.9G/112.0K
[OK] Ratio InnoDB log file size / InnoDB Buffer pool size: 500.0M * 2/3.9G should be equal to 25%
[OK] InnoDB buffer pool instances: 3
[--] InnoDB Buffer Pool Chunk Size not used or defined in your version
[!!] InnoDB Read buffer efficiency: 85.36% (1493 hits/ 1749 total)
[!!] InnoDB Write Log efficiency: 0% (1 hits/ 0 total)
[OK] InnoDB log waits: 0.00% (0 waits / 1 writes)
-------- AriaDB Metrics ----------------------------------------------------------------------------
[--] AriaDB is enabled.
[OK] Aria pagecache size / total Aria indexes: 128.0M/1B
[!!] Aria pagecache hit rate: 89.4% (26K cached / 2K reads)
-------- TokuDB Metrics ----------------------------------------------------------------------------
[--] TokuDB is disabled.
-------- XtraDB Metrics ----------------------------------------------------------------------------
[--] XtraDB is disabled.
-------- Galera Metrics ----------------------------------------------------------------------------
[--] Galera is disabled.
-------- Replication Metrics -----------------------------------------------------------------------
[--] Galera Synchronous replication: NO
[--] No replication slave(s) for this server.
[--] Binlog format: STATEMENT
[--] XA support enabled: ON
[--] Semi synchronous replication Master: Not Activated
[--] Semi synchronous replication Slave: Not Activated
[--] This is a standalone server
-------- Recommendations ---------------------------------------------------------------------------
General recommendations:
Control warning line(s) into /var/log/mysql/error.log file
Control error line(s) into /var/log/mysql/error.log file
MySQL was started within the last 24 hours - recommendations may be inaccurate
Reduce your overall MySQL memory footprint for system stability
Dedicate this server to your database for highest performance.
Reduce or eliminate unclosed connections and network issues
Temporary table size is already large - reduce result set size
Reduce your SELECT DISTINCT queries without LIMIT clauses
Optimize queries and/or use InnoDB to reduce lock wait
mysql 配置在/etc/mysqld/my.cnf:
[mysqld]
general_log = on
skip-name-resolve
innodb_read_io_threads=4
innodb_write_io_threads=4
general_log_file = /var/log/mysql/mysql.log
log_error=/var/log/mysql/mysql_error.log
#
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
#log_slow_queries = /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
#
# * InnoDB
innodb_buffer_pool_size= 4000M
max_connections =800
wait_timeout=100
interactive_timeout=100
query_cache_size=0
query_cache_type=0
query_cache_limit=2M
tmp_table_size=500M
max_heap_table_size=500M
thread_cache_size=4
performance_schema=ON
innodb_log_file_size=500M
innodb_buffer_pool_instances=3
innodb_autoinc_lock_mode = 2
/var/log/mysql/error.log
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: The InnoDB memory heap is disabled
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: Using Linux native AIO
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: Using SSE crc32 instructions
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: Initializing buffer pool, size = 3.9G
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: Completed initialization of buffer pool
2020-05-09 12:36:16 139638727916672 [Note] InnoDB: Highest supported file format is Barracuda.
2020-05-09 12:36:17 139638727916672 [Note] InnoDB: 128 rollback segment(s) are active.
2020-05-09 12:36:17 139638727916672 [Note] InnoDB: Waiting for purge to start
2020-05-09 12:36:17 139638727916672 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.46-86.2 started; log sequence number 1832982
2020-05-09 12:36:17 139632989665024 [Note] InnoDB: Dumping buffer pool(s) not yet started
2020-05-09 12:36:17 139638727916672 [Note] Plugin 'FEEDBACK' is disabled.
2020-05-09 12:36:17 139638727916672 [Note] Server socket created on IP: '::'.
2020-05-09 12:36:17 139638727916672 [Warning] 'user' entry 'wp_user@server' ignored in --skip-name-resolve mode.
2020-05-09 12:36:17 139638727916672 [Warning] 'db' entry 'db_name wp_user@server' ignored in --skip-name-resolve mode.
2020-05-09 12:36:17 139638727916672 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.1.44-MariaDB-0ubuntu0.18.04.1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Ubuntu 18.04
【问题讨论】：
-
@WilsonHauck 感谢您的帮助，结果分别如下：Global Status、Global Variables、Full Process List、InnoDb Stat。还有关于 SSD，不，MySQL 主机服务器上没有这样的设备。
-
@WilsonHauck 不！
-
@WilsonHauck
SCSI storage controller这是我服务器上唯一的存储空间！ -
@WilsonHauck 这也是你说的iostat的结果：iostat before running the test,iostat after running the test
-
@WilsonHauck 哦，哈哈，我的错。这里是变量：Global Vars 实际上我昨天读了你的评论，等了 24 小时才得到结果，但是当我想进入数据库时，它看起来好像已经关闭了。我认为负载测试已经这样做了，所以我不得不重新启动它并运行测试然后获取结果。在我之前的评论中，我提供了 iostat。
标签： php mysql wordpress nginx cpu