MySQL 运维管理工具--pt(Percona-toolkit)

MySQL 运维管理工具--pt(Percona-toolkit)

Table of Contents

一.检查和安装与Perl相关的模块二.下载和安装percona toolkit的包2.1 下载和安装percona toolkit的包2.2 安装percona-toolkit工具包2.3 percona-toolkit工具介绍2.3.1 pt-archiver(归档表)2.3.2 pt-online-schema-change(在线修改表结构)2.3.3 pt-table-checksum2.3.4 pt-table-sync2.3.5 pt-deadlock-logger(死锁检测)2.3.6 pt-duplicate-key-checker（主键冲突检测）2.3.7 pt-kill（杀进程）2.3.7.1 测试kill空闲链接2.3.7.2 kill查询时间超过20秒的进程2.3.7.3 Kill掉 select IFNULl.*语句开头的SQL参考:

一.检查和安装与Perl相关的模块

PT工具是使用Perl语言编写和执行的，所以需要系统中有Perl环境。依赖包检查命令为：

yum -y install perl-DBI perl-DBD-MySQL perl-Time-HiRes perl-IO-Socket-SSL

二.下载和安装percona toolkit的包

2.1 下载和安装percona toolkit的包

参照：https://percona.com/doc/percona-toolkit/3.0/installation.html#installing-percona-toolkit-on-red-hat-or-centos

yum -y install http://percona.com/downloads/percona-release/redhat/0.1-4/percona-release-0.1-4.noarch.rpm

查看可以安装的包

[root@10-31-1-119 src]# yum list | grep percona-toolkitpercona-toolkit.noarch 2.2.20-1 percona-release-noarchpercona-toolkit.x86_64 3.2.1-1.el7 percona-release-x86_64percona-toolkit-debuginfo.x86_64 3.0.13-1.el7 percona-release-x86_64[root@10-31-1-119 src]#

2.2 安装percona-toolkit工具包

yum install percona-toolkit

遇到报错

[root@10-31-1-119 src]# yum install percona-toolkit -y已加载插件：fastestmirror, langpacksLoading mirror speeds from cached hostfile正在解决依赖关系--> 正在检查事务---> 软件包 percona-toolkit.x86_64.0.3.2.1-1.el7 将被 安装--> 正在处理依赖关系 perl(Digest::MD5)，它被软件包 percona-toolkit-3.2.1-1.el7.x86_64 需要--> 正在检查事务---> 软件包 perl-Digest-MD5.x86_64.0.2.52-3.el7 将被 安装--> 正在处理依赖关系 perl(Digest::base) >= 1.00，它被软件包 perl-Digest-MD5-2.52-3.el7.x86_64 需要--> 正在检查事务---> 软件包 perl-Digest.noarch.0.1.17-245.el7 将被 安装--> 解决依赖关系完成依赖关系解决============================================================================================================================================================================================================ Package 架构 版本 源 大小============================================================================================================================================================================================================正在安装: percona-toolkit x86_64 3.2.1-1.el7 percona-release-x86_64 17 M为依赖而安装: perl-Digest noarch 1.17-245.el7 base 23 k perl-Digest-MD5 x86_64 2.52-3.el7 base 30 k事务概要============================================================================================================================================================================================================安装 1 软件包 (+2 依赖软件包)总下载量：17 M安装大小：17 MDownloading packages:(1/3): perl-Digest-MD5-2.52-3.el7.x86_64.rpm | 30 kB 00:00:00 (2/3): perl-Digest-1.17-245.el7.noarch.rpm | 23 kB 00:00:00 percona-toolkit-3.2.1-1.el7.x8 FAILED 51% [=======================================- ] 4.8 B/s | 8.6 MB 487:30:28 ETA http://repo.percona.com/release/7/RPMS/x86_64/percona-toolkit-3.2.1-1.el7.x86_64.rpm: [Errno 12] Timeout on http://repo.percona.com/release/7/RPMS/x86_64/percona-toolkit-3.2.1-1.el7.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')正在尝试其它镜像。percona-toolkit-3.2.1-1.el7.x8 FAILED 96% [========================================================================== ] 5.7 B/s | 16 MB 31:35:32 ETA http://repo.percona.com/release/7/RPMS/x86_64/percona-toolkit-3.2.1-1.el7.x86_64.rpm: [Errno 12] Timeout on http://repo.percona.com/release/7/RPMS/x86_64/percona-toolkit-3.2.1-1.el7.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')正在尝试其它镜像。warning: /var/cache/yum/x86_64/7/percona-release-x86_64/packages/percona-toolkit-3.2.1-1.el7.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 8507efa5: NOKEY==========-] 3.0 kB/s | 17 MB 00:00:01 ETA percona-toolkit-3.2.1-1.el7.x86_64.rpm 的公钥尚未安装(3/3): percona-toolkit-3.2.1-1.el7.x86_64.rpm | 17 MB 00:01:59 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------总计 4.8 kB/s | 17 MB 00:59:47 从 file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Percona 检索密钥导入 GPG key 0xCD2EFD2A: 用户ID : "Percona MySQL Development Team " 指纹 : 430b df5c 56e7 c94e 848e e60c 1c4c bdcd cd2e fd2a 软件包 : percona-release-0.1-4.noarch (@/percona-release-0.1-4.noarch) 来自 : /etc/pki/rpm-gpg/RPM-GPG-KEY-Perconapercona-toolkit-3.2.1-1.el7.x86_64.rpm 的公钥尚未安装 失败的软件包是：percona-toolkit-3.2.1-1.el7.x86_64 GPG 密钥配置为：file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Percona

解决方案

[root@10-31-1-119 src]# yum update percona-release已加载插件：fastestmirror, langpacksLoading mirror speeds from cached hostfile正在解决依赖关系--> 正在检查事务---> 软件包 percona-release.noarch.0.0.1-4 将被 升级---> 软件包 percona-release.noarch.0.1.0-24 将被 更新--> 解决依赖关系完成依赖关系解决============================================================================================================================================================================================================ Package 架构 版本 源 大小============================================================================================================================================================================================================正在更新: percona-release noarch 1.0-24 percona-release-noarch 19 k事务概要============================================================================================================================================================================================================升级 1 软件包总下载量：19 kIs this ok [y/d/N]: yDownloading packages:No Presto metadata available for percona-release-noarchpercona-release-1.0-24.noarch.rpm | 19 kB 00:00:00 Running transaction checkRunning transaction testTransaction test succeededRunning transaction 正在更新 : percona-release-1.0-24.noarch 1/2 * Enabling the Percona Original repository<*> All done!* Enabling the Percona Release repository<*> All done!The percona-release package now contains a percona-release script that can enable additional repositories for our newer products.For example, to enable the Percona Server 8.0 repository use: percona-release setup ps80Note: To avoid conflicts with older product versions, the percona-release setup command may disable our original repository for some products.For more information, please visit: https://percona.com/doc/percona-repo-config/percona-release.html 清理 : percona-release-0.1-4.noarch 2/2 验证中 : percona-release-1.0-24.noarch 1/2 验证中 : percona-release-0.1-4.noarch 2/2 更新完毕: percona-release.noarch 0:1.0-24 完毕！[root@10-31-1-119 src]# [root@10-31-1-119 src]# yum install percona-toolkit -y已加载插件：fastestmirror, langpacksLoading mirror speeds from cached hostfilepercona-release-noarch | 2.9 kB 00:00:00 percona-release-x86_64 | 2.9 kB 00:00:00 prel-release-x86_64 | 2.9 kB 00:00:00 prel-release-x86_64/7/primary_db | 1.1 kB 00:00:00 正在解决依赖关系--> 正在检查事务---> 软件包 percona-toolkit.x86_64.0.3.2.1-1.el7 将被 安装--> 正在处理依赖关系 perl(Digest::MD5)，它被软件包 percona-toolkit-3.2.1-1.el7.x86_64 需要--> 正在检查事务---> 软件包 perl-Digest-MD5.x86_64.0.2.52-3.el7 将被 安装--> 正在处理依赖关系 perl(Digest::base) >= 1.00，它被软件包 perl-Digest-MD5-2.52-3.el7.x86_64 需要--> 正在检查事务---> 软件包 perl-Digest.noarch.0.1.17-245.el7 将被 安装--> 解决依赖关系完成依赖关系解决============================================================================================================================================================================================================ Package 架构 版本 源 大小============================================================================================================================================================================================================正在安装: percona-toolkit x86_64 3.2.1-1.el7 percona-release-x86_64 17 M为依赖而安装: perl-Digest noarch 1.17-245.el7 base 23 k perl-Digest-MD5 x86_64 2.52-3.el7 base 30 k事务概要============================================================================================================================================================================================================安装 1 软件包 (+2 依赖软件包)总计：17 M安装大小：17 MDownloading packages:warning: /var/cache/yum/x86_64/7/percona-release-x86_64/packages/percona-toolkit-3.2.1-1.el7.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 8507efa5: NOKEY从 file:///etc/pki/rpm-gpg/PERCONA-PACKAGING-KEY 检索密钥导入 GPG key 0x8507EFA5: 用户ID : "Percona MySQL Development Team (Packaging key) " 指纹 : 4d1b b29d 63d9 8e42 2b21 13b1 9334 a25f 8507 efa5 软件包 : percona-release-1.0-24.noarch (@percona-release-noarch) 来自 : /etc/pki/rpm-gpg/PERCONA-PACKAGING-KEYRunning transaction checkRunning transaction testTransaction test succeededRunning transaction 正在安装 : perl-Digest-1.17-245.el7.noarch 1/3 正在安装 : perl-Digest-MD5-2.52-3.el7.x86_64 2/3 正在安装 : percona-toolkit-3.2.1-1.el7.x86_64 3/3 验证中 : perl-Digest-1.17-245.el7.noarch 1/3 验证中 : percona-toolkit-3.2.1-1.el7.x86_64 2/3 验证中 : perl-Digest-MD5-2.52-3.el7.x86_64 3/3 已安装: percona-toolkit.x86_64 0:3.2.1-1.el7 作为依赖被安装: perl-Digest.noarch 0:1.17-245.el7 perl-Digest-MD5.x86_64 0:2.52-3.el7 完毕！[root@10-31-1-119 src]#

2.3 percona-toolkit工具介绍

有的32个命令，可以分为7大类

2.3.1 pt-archiver(归档表)

# 重要参数--limit 100 每次取100行数据用pt-archive处理 --txn-size 100 设置100行为一个事务提交一次， --where 'id<3000' 设置操作条件 --progress 5000 每处理5000行输出一次处理信息 --statistics 输出执行过程及最后的操作统计。（只要不加上--quiet，默认情况下pt- archive都会输出执行过程的） --charset=UTF8 指定字符集为UTF8—这个最后加上不然可能出现乱码。 --bulk-delete 批量删除source上的旧数据(例如每次1000行的批量删除操作)使用案例：1.归档到数据库pt-archiver --source h=10.0.0.11,D=world,t=city,u=root,p=123 --dest h=10.0.0.11,D=world,t=city2,u=root,p=123 --where 'id<1000' --no-check-charset --no-delete --limit=100 --commit-each --progress 200 --statistics2.只清理数据pt-archiver --source h=127.0.0.1,D=world,t=city2,u=root,p=123 --where 'id<100' --purge --limit=1 --no-check-charset3.只把数据导出到外部文件，但是不删除源表里的数据pt-archiver --source h=10.0.0.11,D=world,t=city,u=root,p=123 --where '1=1' --no-check-charset --no-delete --file="/tmp/archiver.dat"

2.3.2 pt-online-schema-change(在线修改表结构)

MySQL 5.6之后支持online DDL之后，可以直接在线修改表结构

pt-osc工作流程：1、检查更改表是否有主键或唯一索引，是否有触发器2、检查修改表的表结构，创建一个临时表，在新表上执行ALTER TABLE语句3、在源表上创建三个触发器分别对于INSERT UPDATE DELETE操作4、从源表拷贝数据到临时表，在拷贝过程中，对源表的更新操作会写入到新建表中5、将临时表和源表rename（需要元数据修改锁，需要短时间锁表）6、删除源表和触发器，完成表结构的修改。##=====================================================##pt-osc工具限制1、源表必须有主键或唯一索引，如果没有工具将停止工作2、如果线上的复制环境过滤器操作过于复杂，工具将无法工作3、如果开启复制延迟检查，但主从延迟时，工具将暂停数据拷贝工作4、如果开启主服务器负载检查，但主服务器负载较高时，工具将暂停操作5、当表使用外键时，如果未使用--alter-foreign-keys-method参数，工具将无法执行6、只支持Innodb存储引擎表，且要求服务器上有该表1倍以上的空闲空间。pt-osc之alter语句限制1、不需要包含alter table关键字，可以包含多个修改操作，使用逗号分开，如"drop clolumn c1, add column c2 int"2、不支持rename语句来对表进行重命名操作3、不支持对索引进行重命名操作4、如果删除外键，需要对外键名加下划线，如删除外键fk_uid, 修改语句为"DROP FOREIGN KEY _fk_uid"pt-osc之命令模板## --execute表示执行## --dry-run表示只进行模拟测试## 表名只能使用参数t来设置，没有长参数pt-online-schema-change \--host="127.0.0.1" \--port=3358 \--user="root" \--password="root@root" \--charset="utf8" \--max-lag=10 \--check-salve-lag='xxx.xxx.xxx.xxx' \--recursion-method="hosts" \--check-interval=2 \--database="testdb1" \ t="tb001" \--alter="add column c4 int" \--execute例子：pt-online-schema-change --user=root --password=123 --host=10.0.0.11 --alter "add column age int default 0" D=test,t=t1 --print --execute

2.3.3 pt-table-checksum

创建数据库Create database pt CHARACTER SET utf8;创建用户checksum并授权GRANT ALL ON *.* TO 'checksum'@'10.0.0.%' IDENTIFIED BY 'checksum';flush privileges;--[no]check-replication-filters：是否检查复制的过滤器，默认是yes，建议启用不检查模式。--databases | -d：指定需要被检查的数据库，多个库之间可以用逗号分隔。--[no]check-binlog-format：是否检查binlog文件的格式，默认值yes。建议开启不检查。因为在默认的row格式下会出错。--replicate`：把checksum的信息写入到指定表中。--replicate-check-only：只显示不同步信息pt-table-checksum --nocheck-replication-filters --no-check-binlog-format --replicate=pt.checksums --create-replicate-table --databases=test --tables=t1 h=10.0.0.11,u=checksum,p=checksum,P=3306#!/bin/bashdate >> /root/db/checksum.logpt-table-checksum --nocheck-binlog-format --nocheck-plan--nocheck-replication-filters --replicate=pt.checksums --set-varsinnodb_lock_wait_timeout=120 --databases UAR_STATISTIC -u'checksum' -p'checksum'-h'10.0.0.11' >> /root/db/checksum.logdate >> /root/db/checksum.log

用途： 可以用来检测主、 从数据库中数据的一致性。原理： 在主库上运行， 对同步的表进行checksum， 记录下来。 然后对比主从中各个表的checksum是否一致， 从而判断数据是否一致。

测试数据准备

-- 120mysql> select * from zqs;+----+------+| id | name |+----+------+| 1 | a || 2 | b || 3 | c || 4 | d |+----+------+4 rows in set (0.00 sec)-- 121 mysql> select * from zqs;+----+------+| id | name |+----+------+| 1 | a || 2 | b || 3 | c || 4 | d |+----+------+4 rows in set (0.00 sec)

测试

[root@10-31-1-119 ~]# pt-table-checksum --nocheck-replication-filters --no-check-binlog-format --replicate=pt.checksums --create-replicate-table --databases=test --tables=zqs h=10.31.1.120,u=root,p=abc123,P=3306Checking if all tables can be checksummed ...Starting checksum ...******************************************************************* Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER possibly with SSL_ca_file|SSL_ca_path for verification. If you really don't want to verify the certificate and keep the connection open to Man-In-The-Middle attacks please set SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.******************************************************************* at /usr/bin/pt-table-checksum line 332.******************************************************************* Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER possibly with SSL_ca_file|SSL_ca_path for verification. If you really don't want to verify the certificate and keep the connection open to Man-In-The-Middle attacks please set SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.******************************************************************* at /usr/bin/pt-table-checksum line 332.# A software update is available: TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE08-31T11:53:32 0 0 4 0 1 0 0.044 test.zqs[root@10-31-1-119 ~]#

-- 121mysql> delete from zqs where id = 4;Query OK, 1 row affected (0.01 sec)mysql> select * from zqs;+----+------+| id | name |+----+------+| 1 | a || 2 | b || 3 | c |+----+------+3 rows in set (0.00 sec)

重新检查

[root@10-31-1-119 ~]# pt-table-checksum --nocheck-replication-filters --no-check-binlog-format --replicate=pt.checksums --create-replicate-table --databases=test --tables=zqs h=10.31.1.120,u=root,p=abc123,P=3306Checking if all tables can be checksummed ...Starting checksum ... TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE08-31T11:56:09 0 1 4 1 1 0 0.040 test.zqs

2.3.4 pt-table-sync

主要参数介绍--replicate ：指定通过pt-table-checksum得到的表.--databases : 指定执行同步的数据库。--tables ：指定执行同步的表，多个用逗号隔开。--sync-to-master ：指定一个DSN，即从的IP，他会通过show processlist或show slave status 去自动的找主。h= ：服务器地址，命令里有2个ip，第一次出现的是Master的地址，第2次是Slave的地址。u= ：帐号。p= ：密码。--print ：打印，但不执行命令。--execute ：执行命令。pt-table-sync --replicate=pt.checksums h=10.0.0.11,u=root,p=123,P=3306 --print

使用对两个库不一致的数据进行同步，他能够自动发现两个实例间不一致的数据，然后进行sync操作，pt-table-sync无法同步表结构，和索引等对象，只能同步数据。

数据准备

-- 120主库mysql> create table zqs(id int primary key,name varchar(100));Query OK, 0 rows affected (0.02 sec)mysql> insert into zqs values (1,'a'),(2,'b'),(3,'c'),(4,'d');Query OK, 4 rows affected (0.05 sec)Records: 4 Duplicates: 0 Warnings: 0mysql> select * from zqs;+----+------+| id | name |+----+------+| 1 | a || 2 | b || 3 | c || 4 | d |+----+------+4 rows in set (0.00 sec)-- 121 从库mysql> select * from zqs;+----+------+| id | name |+----+------+| 1 | a || 2 | b || 3 | c || 4 | d |+----+------+4 rows in set (0.00 sec)mysql> delete from zqs where id = 4;Query OK, 1 row affected (0.00 sec)mysql> select * from zqs;+----+------+| id | name |+----+------+| 1 | a || 2 | b || 3 | c |+----+------+3 rows in set (0.00 sec)

测试记录

[root@10-31-1-119 ~]# pt-table-sync --charset=utf8 --ignore-databases=mysql,sys --databases=test --tables=zqs --no-check-slave dsn=u=root,p=abc123,h=10.31.1.120 dsn=u=root,p=abc123,h=10.31.1.121 --execute --printINSERT INTO `test`.`zqs`(`id`, `name`) VALUES ('4', 'd') /*percona-toolkit src_db:test src_tbl:zqs src_dsn:A=utf8,h=10.31.1.120,p=... dst_db:test dst_tbl:zqs dst_dsn:A=utf8,h=10.31.1.121,p=... lock:0 transaction:1 changing_src:0 replicate:0 bidirectional:0 pid:20478 user:root host:10-31-1-119*/;[root@10-31-1-119 ~]#

查看121从库上的数据

mysql> select * from zqs;+----+------+| id | name |+----+------+| 1 | a || 2 | b || 3 | c || 4 | d |+----+------+4 rows in set (0.00 sec)mysql>

也可以同步库和同步整个实例，简直不要太方便

-- 同步除mysql、sys两个库外的所有数据库pt-table-sync --charset=utf8 --ignore-databases=mysql,sys --no-check-slave dsn=u=root,p=abc123,h=10.31.1.120 dsn=u=root,p=abc123,h=10.31.1.121 --execute --print-- 同步整个test数据库pt-table-sync --charset=utf8 --ignore-databases=mysql,sys --databases=test --no-check-slave dsn=u=root,p=abc123,h=10.31.1.120 dsn=u=root,p=abc123,h=10.31.1.121 --execute --print

2.3.5 pt-deadlock-logger(死锁检测)

参数

--create-dest-table ：创建指定的表。--dest ：创建存储死锁信息的表。--database ：-D，指定链接的数据库。--table ：-t，指定存储的表名。--log ：指定死锁日志信息写入到文件。--run-time ：运行次数，默认永久--interval ：运行间隔时间，默认30s。u，p，h，P ：链接数据库的信息。

用法模拟一个死锁的场景

-- 关闭死锁检测mysql> show variables like '%deadlock%';+----------------------------+-------+| Variable_name | Value |+----------------------------+-------+| innodb_deadlock_detect | ON || innodb_print_all_deadlocks | OFF |+----------------------------+-------+2 rows in set (0.00 sec)mysql> mysql> mysql> set global innodb_deadlock_detect = OFF;Query OK, 0 rows affected (0.00 sec)mysql> show variables like '%deadlock%';+----------------------------+-------+| Variable_name | Value |+----------------------------+-------+| innodb_deadlock_detect | OFF || innodb_print_all_deadlocks | OFF |+----------------------------+-------+2 rows in set (0.00 sec)

pt-deadlock-logger --ask-pass --run-time=10 --interval=3 --create-dest-table --dest D=test,t=deadlocks u=root,P=3306,h=10.31.1.120

输出信息

[root@10-31-1-120 ~]# pt-deadlock-logger --ask-pass --run-time=10 --interval=3 --create-dest-table --dest D=test,t=deadlocks u=root,P=3306,h=10.31.1.120Enter MySQL password: Enter MySQL password: server ts thread txn_id txn_time user hostname ip db tbl idx lock_type lock_mode wait_hold victim query10.31.1.120 2020-08-28T17:17:39 7448 0 32 root localhost test t1 idx_id RECORD X w 1 update t1 set name='xxx' where id = 210.31.1.120 2020-08-28T17:17:39 7457 0 43 root localhost test t1 idx_id RECORD X w 0 update t1 set name = 'www' where id = 3[root@10-31-1-120 ~]# [root@10-31-1-120 ~]# mysql -uroot -pEnter password: Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 7486Server version: 5.7.31-log MySQL Community Server (GPL)Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> mysql> use test;Reading table information for completion of table and column namesYou can turn off this feature to get a quicker startup with -ADatabase changedmysql> show tables;+----------------+| Tables_in_test |+----------------+| deadlocks || t1 |+----------------+2 rows in set (0.00 sec)mysql> select * from deadlocks;+-------------+---------------------+--------+--------+----------+------+-----------+----+------+-----+--------+-----------+-----------+-----------+--------+-----------------------------------------+| server | ts | thread | txn_id | txn_time | user | hostname | ip | db | tbl | idx | lock_type | lock_mode | wait_hold | victim | query |+-------------+---------------------+--------+--------+----------+------+-----------+----+------+-----+--------+-----------+-----------+-----------+--------+-----------------------------------------+| 10.31.1.120 | 2020-08-28 17:17:39 | 7448 | 0 | 32 | root | localhost | | test | t1 | idx_id | RECORD | X | w | 1 | update t1 set name='xxx' where id = 2 || 10.31.1.120 | 2020-08-28 17:17:39 | 7457 | 0 | 43 | root | localhost | | test | t1 | idx_id | RECORD | X | w | 0 | update t1 set name = 'www' where id = 3 |+-------------+---------------------+--------+--------+----------+------+-----------+----+------+-----+--------+-----------+-----------+-----------+--------+-----------------------------------------+2 rows in set (0.00 sec)mysql>

2.3.6 pt-duplicate-key-checker（主键冲突检测）

pt-duplicate-key-checker --database=world h='127.0.0.1' --user=root --password=123456

2.3.7 pt-kill（杀进程）

常用参数说明--daemonize 放在后台以守护进程的形式运行；--interval 多久运行一次，单位可以是s,m,h，d等默认是s –不加这个默认是5秒--victims 默认是oldest,只杀最古老的查询。这是防止被查杀是不是真的长时间运行的查询，他们只是长期等待 这种种匹配按时间查询，杀死一个时间最高值。--all 杀掉所有满足的线程--kill-query 只杀掉连接执行的语句，但是线程不会被终止--print 打印满足条件的语句--busy-time 批次查询已运行的时间超过这个时间的线程；--idle-time 杀掉sleep 空闲了多少时间的连接线程，必须在--match-command sleep时才有效—也就是匹配使用 -- –match-command 匹配相关的语句。----ignore-command 忽略相关的匹配。 这两个搭配使用一定是ignore-commandd在前 match-command在后，--match-db cdelzone 匹配哪个库command有：Query、Sleep、Binlog Dump、Connect、Delayed insert、Execute、Fetch、Init DB、Kill、Prepare、Processlist、Quit、Reset stmt、Table Dump例子：---杀掉空闲链接sleep 5秒的 SQL 并把日志放到/home/pt-kill.log文件中/usr/bin/pt-kill --user=用户名 --password=密码 --match-command Sleep --idle-time 5 --victim all --interval 5 --kill --daemonize -S /tmp/mysql.sock --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &---查询SELECT 超过1分钟路/usr/bin/pt-kill --user=用户名 --password=密码 --busy-time 60 --match-info "SELECT|select" --victim all --interval 5 --kill --daemonize -S -S /tmp/mysql.sock --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &--- Kill掉 select IFNULl.*语句开头的SQLpt-kill --user=用户名 --password=密码 --victims all --busy-time=0 --match-info="select IFNULl.*" --interval 1 -S /tmp/mysqld.sock --kill --daemonize --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &----kill掉state Locked/usr/bin/pt-kill --user=用户名 --password=密码 --victims all --match-state='Locked' --victim all --interval 5 --kill --daemonize -S /tmp/mysqld.sock --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &---kill掉 a库，web为10.0.0.11的链接pt-kill --user=用户名 --password=密码 --victims all --match-db='a' --match-host='10.0.0.11' --kill --daemonize --interval 10 -S /tmp/mysqld.sock --pid=/tmp/ptkill.pid --print-log=/tmp/pt-kill.log &---指定哪个用户killpt-kill --user=用户名 --password=密码 --victims all --match-user='root' --kill --daemonize --interval 10 -S /home/zb/data/my6006/socket/mysqld.sock --pid=/tmp/ptkill.pid --print --log=/home/pt-kill.log &---查询SELECT 超过1分钟路pt-kill --user=用户名 --password=密码 --busy-time 60 --match-info "SELECT|select" --victim all --interval 5 --kill --daemonize -S /tmp/mysqld.sock --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &----kill掉 command query | Executept-kill --user=用户名 --password=密码 --victims all --match-command= "query|Execute" --interval 5 --kill --daemonize -S /tmp/mysqld.sock --pid=/tmp/ptkill.pid --print --log=/home/pt-kill.log &

2.3.7.1 测试kill空闲链接

模拟两个空闲实例

-- session 1[root@10-31-1-120 ~]# mysql -uroot -pEnter password: Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 14177Server version: 5.7.31-log MySQL Community Server (GPL)Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> -- session 2[root@10-31-1-120 ~]# mysql -uroot -pEnter password: Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 14180Server version: 5.7.31-log MySQL Community Server (GPL)Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> mysql> mysql> mysql> use test;Reading table information for completion of table and column namesYou can turn off this feature to get a quicker startup with -ADatabase changedmysql> show tables;+----------------+| Tables_in_test |+----------------+| deadlocks || t1 |+----------------+2 rows in set (0.00 sec)mysql> begin;Query OK, 0 rows affected (0.00 sec)mysql> select * from t1;+------+------+| id | name |+------+------+| 1 | a || 2 | b || 3 | c |+------+------+3 rows in set (0.00 sec)

-- 等待10秒，再执行这个语句/usr/bin/pt-kill --user=root --password=abc123 --match-command Sleep --idle-time 5 --victim all --interval 5 --kill --daemonize -S /var/lib/mysql/mysql.sock --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &

测试记录

[root@10-31-1-119 ~]# /usr/bin/pt-kill --user=root --password=abc123 --host=10.31.1.120 --match-command Sleep --idle-time 5 --victim all --interval 5 --kill --daemonize -S /var/lib/mysql/mysql.sock --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &[2] 13170[root@10-31-1-119 ~]# Overwriting PID file /tmp/ptkill.pid because the PID that it contains, 13000, is not running at /usr/bin/pt-kill line 2420.Overwriting PID file /tmp/ptkill.pid because the PID that it contains, 13000, is not running at /usr/bin/pt-kill line 2420.[2]+ 完成 /usr/bin/pt-kill --user=root --password=abc123 --host=10.31.1.120 --match-command Sleep --idle-time 5 --victim all --interval 5 --kill --daemonize -S /var/lib/mysql/mysql.sock --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log[root@10-31-1-119 ~]# [root@10-31-1-119 ~]# ps -ef | grep pt-killroot 13171 1 0 10:35 ? 00:00:00 perl /usr/bin/pt-kill --user=root --password=abc123 --host=10.31.1.120 --match-command Sleep --idle-time 5 --victim all --interval 5 --kill --daemonize -S /var/lib/mysql/mysql.sock --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.logroot 13193 31644 0 10:35 pts/1 00:00:00 grep --color=auto pt-kill[root@10-31-1-119 ~]# [root@10-31-1-119 ~]# more /tmp/pt-kill.log******************************************************************* Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER possibly with SSL_ca_file|SSL_ca_path for verification. If you really don't want to verify the certificate and keep the connection open to Man-In-The-Middle attacks please set SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.******************************************************************* at /usr/bin/pt-kill line 5761.# 2020-08-31T10:35:03 KILL 14177 (Sleep 233 sec) NULL# 2020-08-31T10:35:03 KILL 14180 (Sleep 167 sec) NULL[root@10-31-1-119 ~]#

查看2个session是否被kill

-- session 1[root@10-31-1-120 ~]# mysql -uroot -pEnter password: Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 14177Server version: 5.7.31-log MySQL Community Server (GPL)Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> mysql> select sysdate;ERROR 2006 (HY000): MySQL server has gone awayNo connection. Trying to reconnect...Connection id: 14188Current database: *** NONE ***ERROR 1054 (42S22): Unknown column 'sysdate' in 'field list'mysql> mysql> -- session 2[root@10-31-1-120 ~]# mysql -uroot -pEnter password: Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 14180Server version: 5.7.31-log MySQL Community Server (GPL)Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> mysql> mysql> mysql> use test;Reading table information for completion of table and column namesYou can turn off this feature to get a quicker startup with -ADatabase changedmysql> show tables;+----------------+| Tables_in_test |+----------------+| deadlocks || t1 |+----------------+2 rows in set (0.00 sec)mysql> begin;Query OK, 0 rows affected (0.00 sec)mysql> select * from t1;+------+------+| id | name |+------+------+| 1 | a || 2 | b || 3 | c |+------+------+3 rows in set (0.00 sec)mysql> select current_timestamp();ERROR 2006 (HY000): MySQL server has gone awayNo connection. Trying to reconnect...Connection id: 14189Current database: test+---------------------+| current_timestamp() |+---------------------+| 2020-08-31 10:35:54 |+---------------------+1 row in set (0.00 sec)

可以看到，未完成的事务因为空闲5秒，也被kill了，生产环境需谨慎

2.3.7.2 kill查询时间超过20秒的进程

数据准备

mysql> select count(*) from t1;+----------+| count(*) |+----------+| 5889738 |+----------+1 row in set (30.54 sec)

kill 语句准备

/usr/bin/pt-kill --user=lepus --password=lepus --host=10.31.1.112 --busy-time 20 --match-info "SELECT|select" --victim all --interval 5 --kill --daemonize -S --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &

测试记录

[root@10-31-1-119 ~]# /usr/bin/pt-kill --user=lepus --password=lepus --host=10.31.1.112 --busy-time 20 --match-info "SELECT|select" --victim all --interval 5 --kill --daemonize -S --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &[2] 15172[root@10-31-1-119 ~]# [2]+ 完成 /usr/bin/pt-kill --user=lepus --password=lepus --host=10.31.1.112 --busy-time 20 --match-info "SELECT|select" --victim all --interval 5 --kill --daemonize -S --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log[root@10-31-1-119 ~]# [root@10-31-1-119 ~]# ps -ef | grep pt-killroot 15173 1 0 10:51 ? 00:00:00 perl /usr/bin/pt-kill --user=lepus --password=lepus --host=10.31.1.112 --busy-time 20 --match-info SELECT|select --victim all --interval 5 --kill --daemonize -S --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.logroot 15208 31644 0 10:52 pts/1 00:00:00 grep --color=auto pt-kill[root@10-31-1-119 ~]#

mysql> select count(*) from t1 order by name;ERROR 2013 (HY000): Lost connection to MySQL server during querymysql>

2.3.7.3 Kill掉 select IFNULl.*语句开头的SQL

kill语句

pt-kill --user=lepus --password=lepus --host=10.31.1.112 --victims all --busy-time=0 --match-info="select IFNULl.*" --interval 1 --kill --daemonize --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &

测试记录

[root@10-31-1-119 ~]# pt-kill --user=lepus --password=lepus --host=10.31.1.112 --victims all --busy-time=0 --match-info="select IFNULl.*" --interval 1 --kill --daemonize --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &[2] 15609[root@10-31-1-119 ~]# Overwriting PID file /tmp/ptkill.pid because the PID that it contains, 13171, is not running at /usr/bin/pt-kill line 2420.Overwriting PID file /tmp/ptkill.pid because the PID that it contains, 13171, is not running at /usr/bin/pt-kill line 2420.[2]+ 完成 pt-kill --user=lepus --password=lepus --host=10.31.1.112 --victims all --busy-time=0 --match-info="select IFNULl.*" --interval 1 --kill --daemonize --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log

mysql> select IFNULl(id,'NULL'),name from t1;ERROR 2013 (HY000): Lost connection to MySQL server during querymysql>

查看日志

[root@10-31-1-119 ~]# more /tmp/pt-kill.log ******************************************************************* Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER possibly with SSL_ca_file|SSL_ca_path for verification. If you really don't want to verify the certificate and keep the connection open to Man-In-The-Middle attacks please set SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.******************************************************************* at /usr/bin/pt-kill line 5761.# 2020-08-31T10:35:03 KILL 14177 (Sleep 233 sec) NULL# 2020-08-31T10:35:03 KILL 14180 (Sleep 167 sec) NULL# 2020-08-31T10:35:48 KILL 14188 (Sleep 9 sec) NULL# 2020-08-31T10:36:03 KILL 14189 (Sleep 9 sec) NULL******************************************************************* Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER possibly with SSL_ca_file|SSL_ca_path for verification. If you really don't want to verify the certificate and keep the connection open to Man-In-The-Middle attacks please set SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.******************************************************************* at /usr/bin/pt-kill line 5761.******************************************************************* Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER possibly with SSL_ca_file|SSL_ca_path for verification. If you really don't want to verify the certificate and keep the connection open to Man-In-The-Middle attacks please set SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.******************************************************************* at /usr/bin/pt-kill line 5761.# A software update is available:# 2020-08-31T10:53:06 KILL 10663 (Query 22 sec) select count(*) from t1 order by name# 2020-08-31T10:56:29 KILL 10680 (Query 0 sec) select IFNULl(id,'NULL'),name from t1[root@10-31-1-119 ~]#

参考:

1.https://cnblogs.com/zishengY/p/6852280.html2.https://jianshu.com/p/36ace5c2bc8b3.https://cnblogs.com/zhoujinyi/p/3392800.html