如何从 Zabbix 对 MySQL 执行更快的查询？

【问题标题】：How to perform a faster query on MySQL from Zabbix?如何从 Zabbix 对 MySQL 执行更快的查询？
【发布时间】：2021-04-12 20:34:16
【问题描述】：

我正在尝试为我的自定义网络构建一个监控仪表板，在 Zabbix 中实现的那个不符合我的要求。

我为提取此信息而执行的查询。

SELECT  ll.UF
       ,ll.unidade AS metric
       ,if(AVG(ll.ONLINE) = 0 AND AVG(ll.degra) = 0,3,if(AVG(ll.ONLINE) = 0 AND AVG(ll.degra) = 1,3,if(AVG(ll.ONLINE) = 0 AND AVG(ll.degra) > 0 AND AVG(ll.degra) < 1,3,if(AVG(ll.ONLINE) = 1 AND AVG(ll.degra) = 0,0,if(AVG(ll.ONLINE) = 1 AND AVG(ll.degra) = 1,1,if(AVG(ll.ONLINE) = 1 AND AVG(ll.degra) > 0 AND AVG(ll.degra) < 1,1,if(AVG(ll.degra) = 0 AND AVG(ll.ONLINE) > 0 AND AVG(ll.ONLINE) < 1,2,if(AVG(ll.degra) = 1 AND AVG(ll.ONLINE) > 0 AND AVG(ll.ONLINE) < 1,2,if(AVG(ll.degra) > 0 AND AVG(ll.degra) < 1 AND AVG(ll.ONLINE) > 0 AND AVG(ll.ONLINE) < 1,2,3))))))))) AS value
FROM 
(
    SELECT  hh.*
           ,if(hh.falha=1,0,1) AS ONLINE
           ,if(hh.falha=2,1,0) AS degra
    FROM 
    (
        SELECT  dfd1.*
               ,SUM(if(dfd2.name = 'Perda de Pacote' OR dfd2.name = 'Tempo de resposta de ping ICMP alto',2,if(dfd2.name = 'ICMP Ping Indisponível',1,0))) AS falha
        FROM 
        (
            SELECT  AT1.*
                   ,AT2.itemid
            FROM 
            (
                SELECT  hb.*
                FROM 
                (
                    SELECT  HOST
                           ,LEFT(HOST,2)                                        AS UF
                           ,SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',2),'_',-1) AS Operadora
                           ,SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1) AS unidade
                           ,SUBSTRING_INDEX(HOST,'_',-1)                        AS tipo
                           ,if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),3) = 'HS.' ,
                       'hospital',if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),3) = 'VI.' ,
                       'vidaeimagem',if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),7) = 'HAPCLI.' ,
                       'hapclinica',if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),3) = 'PA.' ,
                       'prontoatendimento',if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),3) = 'CD.' ,
                       'centrodedistribuicao',if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),8) = 'MEDPREV.' ,
                       'medprev',if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),4) = 'LAB.' ,
                       'laboratorio',if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),4) = 'ADM.' ,
                       'administrativo',if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),4) = 'AMB.' ,
                       'ambulatorio','outros' ) ) ) ) ) ) ) ) ) AS modalidade
                           ,hostid
                    FROM hosts
                    WHERE STATUS = 0  
                ) AS hb
                LEFT JOIN hosts_groups AS idgr
                ON hb.hostid = idgr.hostid
                WHERE idgr.groupid = 15 
            ) AS AT1
            LEFT JOIN items AS AT2
            ON AT1.hostid = AT2.hostid
        ) AS dfd1
        LEFT JOIN 
        (
            SELECT  gn2.itemid
                   ,gn1.name
            FROM 
            (
                SELECT  a1.*
                FROM events AS a1
                LEFT JOIN event_recovery AS a2
                ON a1.eventid = a2.eventid
                WHERE a2.r_eventid IS NULL 
                AND a1.value = 1 
            ) AS gn1
            LEFT JOIN functions AS gn2
            ON gn1.objectid = gn2.triggerid
            WHERE gn2.itemid IS NOT NULL 
        ) AS dfd2
        ON dfd1.itemid = dfd2.itemid
        GROUP BY  dfd1.host
    ) AS hh
) AS ll
GROUP BY  ll.unidade
ORDER BY ll.UF ASC
         ,VALUE desc

我能够打开 Zabbix 数据库并进行我需要的查询，但是查询大约需要 15 秒并且消耗了大量处理（我在银行表之间使用了很多 INNER JOIN）。

CREATE TABLE `hosts` (
    `hostid` BIGINT(20) UNSIGNED NOT NULL,
    `proxy_hostid` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
    `host` VARCHAR(128) NOT NULL DEFAULT '' COLLATE 'utf8_bin',
    `status` INT(11) NOT NULL DEFAULT '0',
    `name` VARCHAR(128) NOT NULL DEFAULT '' COLLATE 'utf8_bin',
    `flags` INT(11) NOT NULL DEFAULT '0',
    `templateid` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
    `description` TEXT NOT NULL COLLATE 'utf8_bin',
    `proxy_address` VARCHAR(255) NOT NULL DEFAULT '' COLLATE 'utf8_bin',
    `auto_compress` INT(11) NOT NULL DEFAULT '1',
    `discover` INT(11) NOT NULL DEFAULT '0',
    PRIMARY KEY (`hostid`) USING BTREE,
    INDEX `hosts_1` (`host`) USING BTREE,
    INDEX `hosts_2` (`status`) USING BTREE,
    INDEX `hosts_3` (`proxy_hostid`) USING BTREE,
    INDEX `hosts_4` (`name`) USING BTREE,
    INDEX `c_hosts_3` (`templateid`) USING BTREE,
    CONSTRAINT `c_hosts_1` FOREIGN KEY (`proxy_hostid`) REFERENCES `zabbix`.`hosts` (`hostid`) ON UPDATE RESTRICT ON DELETE RESTRICT,
    CONSTRAINT `c_hosts_3` FOREIGN KEY (`templateid`) REFERENCES `zabbix`.`hosts` (`hostid`) ON UPDATE RESTRICT ON DELETE CASCADE
)
COLLATE='utf8_bin'
ENGINE=InnoDB
;

表链接组与主机名

CREATE TABLE `hosts_groups` (
    `hostgroupid` BIGINT(20) UNSIGNED NOT NULL,
    `hostid` BIGINT(20) UNSIGNED NOT NULL,
    `groupid` BIGINT(20) UNSIGNED NOT NULL,
    PRIMARY KEY (`hostgroupid`) USING BTREE,
    UNIQUE INDEX `hosts_groups_1` (`hostid`, `groupid`) USING BTREE,
    INDEX `hosts_groups_2` (`groupid`) USING BTREE,
    CONSTRAINT `c_hosts_groups_1` FOREIGN KEY (`hostid`) REFERENCES `zabbix`.`hosts` (`hostid`) ON UPDATE RESTRICT ON DELETE CASCADE,
    CONSTRAINT `c_hosts_groups_2` FOREIGN KEY (`groupid`) REFERENCES `zabbix`.`hstgrp` (`groupid`) ON UPDATE RESTRICT ON DELETE CASCADE
)
COLLATE='utf8_bin'
ENGINE=InnoDB
;

链接要触发的项目的表

CREATE TABLE `items` (
    `itemid` BIGINT(20) UNSIGNED NOT NULL,
    `hostid` BIGINT(20) UNSIGNED NOT NULL,
    `name` VARCHAR(255) NOT NULL DEFAULT '' COLLATE 'utf8_bin',
    PRIMARY KEY (`itemid`) USING BTREE,
    INDEX `items_1` (`hostid`, `key_`(255)) USING BTREE,
    INDEX `items_3` (`status`) USING BTREE,
    CONSTRAINT `c_items_1` FOREIGN KEY (`hostid`) REFERENCES `zabbix`.`hosts` (`hostid`) ON UPDATE RESTRICT ON DELETE CASCADE
)
COLLATE='utf8_bin'
ENGINE=InnoDB
;

将 event_recovery 链接到事件的表

CREATE TABLE `event_recovery` (
    `eventid` BIGINT(20) UNSIGNED NOT NULL,
    `r_eventid` BIGINT(20) UNSIGNED NOT NULL,
    `c_eventid` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
    `correlationid` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
    `userid` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
    PRIMARY KEY (`eventid`) USING BTREE,
    INDEX `event_recovery_1` (`r_eventid`) USING BTREE,
    INDEX `event_recovery_2` (`c_eventid`) USING BTREE,
    CONSTRAINT `c_event_recovery_1` FOREIGN KEY (`eventid`) REFERENCES `zabbix`.`events` (`eventid`) ON UPDATE RESTRICT ON DELETE CASCADE,
    CONSTRAINT `c_event_recovery_2` FOREIGN KEY (`r_eventid`) REFERENCES `zabbix`.`events` (`eventid`) ON UPDATE RESTRICT ON DELETE CASCADE,
    CONSTRAINT `c_event_recovery_3` FOREIGN KEY (`c_eventid`) REFERENCES `zabbix`.`events` (`eventid`) ON UPDATE RESTRICT ON DELETE CASCADE
)
COLLATE='utf8_bin'
ENGINE=InnoDB
;

事件

CREATE TABLE `events` (
    `eventid` BIGINT(20) UNSIGNED NOT NULL,
    `source` INT(11) NOT NULL DEFAULT '0',
    `object` INT(11) NOT NULL DEFAULT '0',
    `objectid` BIGINT(20) UNSIGNED NOT NULL DEFAULT '0',
    `clock` INT(11) NOT NULL DEFAULT '0',
    `value` INT(11) NOT NULL DEFAULT '0',
    `acknowledged` INT(11) NOT NULL DEFAULT '0',
    `ns` INT(11) NOT NULL DEFAULT '0',
    `name` VARCHAR(2048) NOT NULL DEFAULT '' COLLATE 'utf8_bin',
    `severity` INT(11) NOT NULL DEFAULT '0',
    PRIMARY KEY (`eventid`) USING BTREE,
    INDEX `events_1` (`source`, `object`, `objectid`, `clock`) USING BTREE,
    INDEX `events_2` (`source`, `object`, `clock`) USING BTREE
)
COLLATE='utf8_bin'
ENGINE=InnoDB
;

表函数

CREATE TABLE `functions` (
    `functionid` BIGINT(20) UNSIGNED NOT NULL,
    `itemid` BIGINT(20) UNSIGNED NOT NULL,
    `triggerid` BIGINT(20) UNSIGNED NOT NULL,
    `name` VARCHAR(12) NOT NULL DEFAULT '' COLLATE 'utf8_bin',
    `parameter` VARCHAR(255) NOT NULL DEFAULT '0' COLLATE 'utf8_bin',
    PRIMARY KEY (`functionid`) USING BTREE,
    INDEX `functions_1` (`triggerid`) USING BTREE,
    INDEX `functions_2` (`itemid`, `name`, `parameter`) USING BTREE,
    CONSTRAINT `c_functions_1` FOREIGN KEY (`itemid`) REFERENCES `zabbix`.`items` (`itemid`) ON UPDATE RESTRICT ON DELETE CASCADE,
    CONSTRAINT `c_functions_2` FOREIGN KEY (`triggerid`) REFERENCES `zabbix`.`triggers` (`triggerid`) ON UPDATE RESTRICT ON DELETE CASCADE
)
COLLATE='utf8_bin'
ENGINE=I

所以我发现，如果我分别进行查询并通过 PHP（我用来构建仪表板的语言）将表格放在一起，效率会高得多。但是，我正在使用 PHP 将两个查询关联为一个。

使用 PHP 我提出以下要求：

<?PHP
//error_reporting(E_ERROR | E_PARSE);



$path2 = $_SERVER['DOCUMENT_ROOT'];
$path2 .= "/SCL/db/protect.php";
include($path2);
    
   //this code is better identified at the beginning of this StackOverflow forum

    $geral = "select ll.UF, ll.unidade as metric, 
if(AVG(ll.ONLINE) = 0 AND avg(ll.degra) = 0,3,if(AVG(ll.ONLINE) = 0 AND avg(ll.degra) = 1,3,if(AVG(ll.ONLINE) = 0 AND avg(ll.degra) > 0 AND avg(ll.degra) < 1,3,if(AVG(ll.ONLINE) = 1 AND avg(ll.degra) = 0,0,if(AVG(ll.ONLINE) = 1 AND avg(ll.degra) = 1,1,if(AVG(ll.ONLINE) = 1 AND avg(ll.degra) > 0 AND avg(ll.degra) < 1,1,if(AVG(ll.degra) = 0 AND avg(ll.ONLINE) > 0 AND avg(ll.ONLINE) < 1,2,if(AVG(ll.degra) = 1 AND avg(ll.ONLINE) > 0 AND avg(ll.ONLINE) < 1,2,if(avg(ll.degra) > 0 AND avg(ll.degra) < 1 AND avg(ll.ONLINE) > 0 AND avg(ll.ONLINE) < 1,2,3))))))))) AS value 
FROM (SELECT hh.*, if(hh.falha=1,0,1) AS ONLINE, 
if(hh.falha=2,1,0) AS degra 
FROM (SELECT dfd1.*, sum(if(dfd2.name = 'Perda de Pacote' OR dfd2.name = 'Tempo de resposta de ping ICMP alto',2,if(dfd2.name = 'ICMP Ping Indisponível',1,0))) AS falha 
FROM (SELECT AT1.*, AT2.itemid FROM (SELECT hb.*
FROM (
            SELECT HOST,
            LEFT(HOST,2) AS UF,
            SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',2),'_',-1) AS Operadora, 
            SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1) AS unidade, 
            SUBSTRING_INDEX(HOST,'_',-1) as tipo, 
            if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),3) = 'HS.' , 'hospital', if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),3) = 'VI.' , 'vidaeimagem', if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),7) = 'HAPCLI.' , 'hapclinica', if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),3) = 'PA.' , 'prontoatendimento', if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),3) = 'CD.' , 'centrodedistribuicao', if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),8) = 'MEDPREV.' , 'medprev', if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),4) = 'LAB.' , 'laboratorio', if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),4) = 'ADM.' , 'administrativo', if( LEFT( SUBSTRING_INDEX(SUBSTRING_INDEX(HOST,'_',-2),'_',1),4) = 'AMB.' , 'ambulatorio', 'outros' ) ) ) ) ) ) ) ) )   as modalidade, 
            hostid 
            FROM hosts WHERE STATUS = 0 ) AS hb 
LEFT JOIN hosts_groups AS idgr 
ON hb.hostid = idgr.hostid 
WHERE idgr.groupid = 15) AS AT1 
LEFT JOIN items AS AT2 ON AT1.hostid = AT2.hostid) AS dfd1 
LEFT JOIN (SELECT gn2.itemid, gn1.name from (SELECT a1.* FROM events AS a1 LEFT JOIN event_recovery AS a2 ON a1.eventid = a2.eventid WHERE a2.r_eventid IS NULL AND a1.value = 1) AS gn1 LEFT JOIN functions AS gn2 ON gn1.objectid = gn2.triggerid WHERE gn2.itemid IS NOT NULL) AS dfd2
ON dfd1.itemid = dfd2.itemid
GROUP BY dfd1.host) AS hh) AS ll";

if(!empty($_POST["UF"]) && empty($_POST["modalidade"])){
     
    $uf = " '" . str_replace(" ","','",$_POST["UF"]) . "' ";
    
    $sql2 = $geral . "
    where ll.uf in (" . $uf . ")
    GROUP BY ll.unidade";

}elseif (empty($_POST["UF"]) && !empty($_POST["modalidade"])){

    $modalidade = " '" . str_replace(" ","','",$_POST["modalidade"]) . "' ";
    
    $sql2 = $geral . "
    where ll.modalidade in (" . $modalidade . ")
    GROUP BY ll.unidade";

}elseif (!empty($_POST["UF"]) && !empty($_POST["modalidade"])){

    $uf = " '" . str_replace(" ","','",$_POST["UF"]) . "' ";
    $modalidade = " '" . str_replace(" ","','",$_POST["modalidade"]) . "' ";
    
    $sql2 = $geral . "
    where ll.uf in (" . $uf . ")
    and ll.modalidade in (" . $modalidade . ")
    GROUP BY ll.unidade";

}else{

    $sql2 = $geral . "
    GROUP BY ll.unidade";

}

if(!empty($_POST["separa"])){
    $sql2 = $sql2 . "
    ORDER BY ll.UF ASC, VALUE desc";
}else{
    $sql2 = $sql2 . "
    ORDER BY VALUE desc";
}

if(!empty($_POST["falha"])){
    $sql2 = 'SELECT i.* FROM (' . $sql2 . ') AS i WHERE i.VALUE != 0';
}

                $stmt2 = $connect2->prepare($sql2);
                $result2 = $stmt2->execute();
                $val = $stmt2->rowCount();
                $ds = "";
                echo '<div class="lista">';

                if($val > 0){
                    while($row = $stmt2->fetch(PDO::FETCH_OBJ)){
                        
                        if(!empty($_POST["separa"])){
                            if ($ds <> $row->UF) {
                                $ds = $row->UF;
                                echo '</div><h2><span  class="line-center">' . $row->UF . '</span></h2><div class="lista">';
                            }
                        }
                        

                        if($row->value == 0){
                            $cor = 'green';
                        }elseif($row->value == 1){
                            $cor = 'orange';
                        }elseif($row->value == 2){
                            $cor = 'yellow';
                        }else{
                            $cor = 'red';
                        }
                        echo '<div class="usu"><span class="textON">' . $row->UF . "-" . str_replace("MEDPREV.","FMED.",str_replace("HS.","HOSP.",str_replace("HAPCLI.","HAP.",$row->metric))) . '</span><div class="circulo ' . $cor . '"></div></div>';
                    }
                    echo '</div>';
                }else{
                    echo "Sem Falha";
                    echo '</div>';
                }

?>

如果有人知道任何可以帮助我在提取信息后操纵查询结果的工具，我是否会在这里寻求帮助以找到解决我这个困境的方法？

【问题讨论】：

  • 显示 SQL。在查询中的表中显示show create table {tablename}。为您的 SQL 显示 EXPLAIN {query}。表格的图像没有足够的信息来评估。假设您的数据集是有限的，正确的indexing 应该可以解决性能问题。在 PHP 中编写加入是试图完成数据库的工作。正确索引它，您不需要这样做。
  • 为什么不用zabbix php api？

标签： php mysql zabbix


【解决方案1】：

这些可能会有所帮助：

functions:  INDEX(triggerid, itemid)  -- replacing INDEX(triggered)
hosts:  INDEX(status, hosted, host)   -- replacing INDEX(status)
hosts_groups:  INDEX(groupid, hosted) -- replacing INDEX(grouped)

（可能还有其他我错过的复合索引。）

展平查询。例如：

                SELECT  hb.*
                    FROM ( SELECT  HOST, LEFT(HOST,2) AS UF, hostid
                            FROM  hosts
                            WHERE  STATUS = 0 
                         ) AS hb
                    LEFT JOIN  hosts_groups AS idgr  ON hb.hostid = idgr.hostid
                    WHERE  idgr.groupid = 15

-->

         SELECT hb.HOST, LEFT(hb.HOST,2) AS UF, hb.hostid
             FROM hosts AS hb
             JOIN hosts_groups AS idgr
                   ON hb.hostid = idgr.hostid
             WHERE  idgr.groupid = 15
               AND  hb.STATUS = 0

（我没有为其他扁平化循环。）

对于hosts_groups，可以完全去掉hostgroupid，将UNIQUE键提升为PRIMARY KEY。请参阅：http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table 对任何其他多对多映射表执行类似操作。

【讨论】：

  • 您的提示很有帮助，但是，我仍然遇到性能问题，mysql 执行的整个查询持续大约 15 秒，我最多需要 5 秒。我不确定是否更改数据库，因为它不是我的开发。
  • @AlbertoVitoriano - 让我们看看它降到了多少。 （我只举了一个例子，感觉可以做几个。）
猜你喜欢
  • 1970-01-01
  • 2019-11-30
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2011-12-06
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode