记一次线上bug：crontab 被意外清空
目录问题概述
同事反馈，某台服务器的crontab 被清空了.
看了cron运行日志来看，问题出在这里：
Jul 3 10:01:24 10-10-65-235 crontab[19333]: (root) REPLACE (root)
这一时刻被替换掉了，后续就没有任务执行了。
然后紧急恢复备份，并从日志中检查遗漏项。
问题排查
定位出问题后，就从history中找执行的操作。
可从history中没有看到 crontab -r 的操作被执行。
后在Google搜索问题找到一些类似的情况，是由于远程登录、加空格之类的导致的(https://cloud.tencent.com/developer/article/2222953)。
我看了后，就发现这一定是我的锅了，并且受影响的还有其他两台设备。
问题复现
我的操作是从远程服务器通过 ssh 直接执行的命令，卡住之后 ctrl+C中断了操作，于是任务被清空。
复现流程：
-
创建任务（保证任务列表中有内容，不为空）。
-
╭─ ~/cmd ▓▒░·······························································░▒▓ ✔ 10:29:52 ─╮ ╰─ crontab -l ─╯ # Minute Hour Day of Month Month Day of Week who Command # (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat) 20 17 * * 1-5 open "https://tengyun.qianxin-inc.cn/home/workspace/worklog" ╭─ ~/cmd ▓▒░·······························································░▒▓ ✔ 10:29:54 ─╮ ╰─ ─╯
-
-
使用
ssh命令执行能卡住的命令，如crontab或crontab --
╭─ ~/cmd ▓▒░·······························································░▒▓ ✔ 10:29:54 ─╮ ╰─ ssh [email protected] crontab ─╯
-
-
命令卡住，此刻查看
crontab -l还是有的。-
╭─ ~/cmd ▓▒░·······························································░▒▓ ✔ 10:30:59 ─╮ ╰─ crontab -l ─╯ # Minute Hour Day of Month Month Day of Week who Command # (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat) 20 17 * * 1-5 open "https://tengyun.qianxin-inc.cn/home/workspace/worklog" ╭─ ~/cmd ▓▒░·······························································░▒▓ ✔ 10:31:02 ─╮ ╰─ ─╯
-
-
使用
ctrl+C结束掉后检查任务列表。-
╭─ ~/cmd ▓▒░·······························································░▒▓ ✔ 10:29:54 ─╮ ╰─ ssh [email protected] crontab ─╯ ^C% ╭─ ~/cmd ▓▒░··························································░▒▓ ✔ 55s 10:31:30 ─╮ ╰─ crontab -l ─╯ ╭─ ~/cmd ▓▒░·······························································░▒▓ ✔ 10:31:33 ─╮ ╰─ ─╯
-
问题复现成功。
其他测试
- 后又分别在本机直接测试了这些指令，均没有清空任务列表。只有在
ssh操作时，会被清空。 ssh操作中ssh XXX@XXX bash -c "crontab -l"该命令也会被卡住。
总结
产生的操作，弄明白了，但是原理还是没搞明白。
避坑方法：
- 定时备份。
- 在卡住时，不要中断，先备份，在停止掉。