【问题标题】:Crontab stopping - broken pipe with Ubuntu serverCrontab 停止 - Ubuntu 服务器的管道损坏
【发布时间】:2020-03-06 00:28:45
【问题描述】:

AWS/EC2/ubuntu 的第一个用户在这里。我在使用 crontab 的 ubuntu 服务器上运行 python 脚本(网络爬虫,使用 chromedriver、selenium、beautifulsoup 等)时遇到问题。这是 AWS 上的 EC2 实例。 我已经让 crontab 作为一个简单的测试工作,创建了一个简单的 txt 文件,上面写着“你好”。但是,当前的 cronjob 一直失败:

很简单,每天中午 12:17 执行一个脚本:

    ubuntu@ip-172-31-31-200:~$ crontab -l
17 12 * * * python3 BandsInTown_Scraper_SF.py

作业未执行。我的 crontab 状态为:

    ubuntu@ip-172-31-31-200:~$ sudo service cron status
● cron.service - Regular background program processing daemon
   Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2019-11-02 00:18:36 UTC; 1 weeks 0 days ago
     Docs: man:cron(8)
 Main PID: 830 (cron)
    Tasks: 1 (limit: 1152)
   CGroup: /system.slice/cron.service
           └─830 /usr/sbin/cron -f

Nov 09 16:17:01 ip-172-31-31-200 CRON[27857]: pam_unix(cron:session): session opened for user root by (uid=0)
Nov 09 16:17:01 ip-172-31-31-200 CRON[27858]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Nov 09 16:17:01 ip-172-31-31-200 CRON[27857]: pam_unix(cron:session): session closed for user root
Nov 09 17:17:01 ip-172-31-31-200 CRON[27908]: pam_unix(cron:session): session opened for user root by (uid=0)
Nov 09 17:17:01 ip-172-31-31-200 CRON[27909]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Nov 09 17:17:01 ip-172-31-31-200 CRON[27908]: pam_unix(cron:session): session closed for user root
Nov 09 18:15:01 ip-172-31-31-200 cron[830]: (ubuntu) RELOAD (crontabs/ubuntu)
Nov 09 18:17:01 ip-172-31-31-200 CRON[28242]: pam_unix(cron:session): session opened for user root by (uid=0)
Nov 09 18:17:01 ip-172-31-31-200 CRON[28243]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Nov 09 18:17:01 ip-172-31-31-200 CRON[28242]: pam_unix(cron:session): session closed for user root

而且,当我挖掘过去失败的 cron 作业的日志时,我得到:

    Nov  5 02:17:01 ip-172-31-31-200 CRON[14524]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Nov  5 02:26:58 ip-172-31-31-200 systemd[1]: Starting Message of the Day...
Nov  5 02:26:58 ip-172-31-31-200 50-motd-news[14532]:  * Kata Containers are now fully integrated in Charmed Kubernetes 1.16!
Nov  5 02:26:58 ip-172-31-31-200 50-motd-news[14532]:    Yes, charms take the Krazy out of K8s Kata Kluster Konstruction.
Nov  5 02:26:58 ip-172-31-31-200 50-motd-news[14532]:      https://ubuntu.com/kubernetes/docs/release-notes
Nov  5 02:26:58 ip-172-31-31-200 systemd[1]: Started Message of the Day.
Nov  5 02:36:28 ip-172-31-31-200 systemd-timesyncd[577]: Network configuration changed, trying to establish connection.
Nov  5 02:36:28 ip-172-31-31-200 systemd-timesyncd[577]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
Nov  5 03:06:28 ip-172-31-31-200 systemd-timesyncd[577]: Network configuration changed, trying to establish connection.
Nov  5 03:06:28 ip-172-31-31-200 systemd-timesyncd[577]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
Nov  5 03:17:01 ip-172-31-31-200 CRON[14621]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Nov  5 03:36:28 ip-172-31-31-200 systemd-timesyncd[577]: Network configuration changed, trying to establish connection.
Nov  5 03:36:28 ip-172-31-31-200 systemd-timesyncd[577]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
Nov  5 03:39:01 ip-172-31-31-200 CRON[14646]: (root) CMD (   test -x /etc/cron.daily/popularity-contest && /etc/cron.daily/popularity-contest --crond)
Nov  5 04:06:27 ip-172-31-31-200 systemd-timesyncd[577]: Network configuration changed, trying to establish connection.
Nov  5 04:06:27 ip-172-31-31-200 systemd-timesyncd[577]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
Nov  5 04:17:01 ip-172-31-31-200 CRON[14675]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Nov  5 04:27:25 ip-172-31-31-200 systemd[1]: Started Session 144 of user ubuntu.
Nov  5 04:36:27 ip-172-31-31-200 systemd-timesyncd[577]: Network configuration changed, trying to establish connection.
Nov  5 04:36:27 ip-172-31-31-200 systemd-timesyncd[577]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
Nov  5 04:41:04 ip-172-31-31-200 systemd[1]: Started Session 145 of user ubuntu.
Nov  5 04:41:09 ip-172-31-31-200 crontab[14897]: (ubuntu) LIST (ubuntu)

Network configuration changed, trying to establish connection. 行令人担忧并想知道这是否是其根本原因。 ubuntu 服务器是否需要启动并运行才能从 crontab 工作?我问是因为当我在终端上登录 ubuntu,然后让它运行一段时间并且我的计算机进入睡眠状态时,我收到以下错误:

ubuntu@ip-172-31-31-200:~/bandsintown$ packet_write_wait: Connection to 3.132.213.71: Broken pipe

这是我的问题的根本原因吗?我认为在 AWS 上运行代码的全部意义在于它可以独立于我的本地机器运行(即,如果互联网中断,如果终端退出 Ubuntu,等等)。

【问题讨论】:

    标签: python selenium ubuntu amazon-ec2 cron


    【解决方案1】:

    这可能是路径问题,我会尝试:

    17 12 * * * cd /path/to/script && /path/to/python3 BandsInTown_Scraper_SF.py

    也许保存输出以供以后查看:

    17 12 * * * cd /path/to/script && /path/to/python3 BandsInTown_Scraper_SF.py > /tmp/out 2>&1

    【讨论】:

    • 我很好奇,为什么需要 && 语句和两个单独的文件路径?以下工作不应该(这是我在 ubuntu 上的 .py 文件的路径):17 12 * * * cd /home/ubuntu/bandsintown/python3 BandsInTown_Scraper_SF.py
    • 要清楚,上面的代码不起作用 - 因此问题!
    • 不幸的是,我也尝试了最初的建议,但也失败了...... 17 12 * * * cd /home/ubuntu/bandsintown && /usr/bin/python3.6 BandsInTown_Scraper_SF.py
    • 如果你像我建议的那样重定向输出,你可以看到哪里出了问题。
    • 好的。这是输出,当我将结果输出到 tmp 文件夹时:ubuntu@ip-172-31-31-200:/tmp$ dir systemd-private-081cd2f421de4eacb0bb40217b0a9e0f-systemd-timesyncd.service-scldmU
    【解决方案2】:

    所提供的答案均无效,但确实有效:

    17 12 * * * /usr/bin/python3.6 /home/ubuntu/bandsintown/BandsInTown_Scraper_SF.py
    

    【讨论】:

      猜你喜欢
      • 2011-10-17
      • 1970-01-01
      • 2019-09-08
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多