Page 1 of 1

crontab

Posted: Tue Mar 06, 2018 4:43 am
by nhdesign
Copied my crontab from bolt to sh yesterday.
31 emails from (cron daemon) last night.
Not even a touch command worked
touch: cannot touch ???/nfs/ftp/pub/users/nhdesign/outgoing/33660-52-foot-shift-crank.jpg???: No such file or directory
Crontab is gone.
Presume someone deleted it.

When can I reload the crontab?

Thanks,
Vic

Re: crontab

Posted: Tue Mar 06, 2018 10:48 am
by scott
nhdesign wrote:
Copied my crontab from bolt to sh yesterday.
31 emails from (cron daemon) last night.
Not even a touch command worked
touch: cannot touch ???/nfs/ftp/pub/users/nhdesign/outgoing/33660-52-foot-shift-crank.jpg???: No such file or directory
Crontab is gone.
Presume someone deleted it.

When can I reload the crontab?

Thanks,
Vic


Hi Vic,

I'm looking into this now. Last night, the server had another problem with resources. Two things were going on: first, your jobs in your crontab were being denied by SELinux (security enhanced Linux), and second, certain mounts for the resource management system known as "cgroups" had somehow disappeared from the system. One of our system admins rebooted the system last night, and it very shortly ended up in the same situation. He moved two crontabs aside (including yours), then tried again, and the system worked as expected.

So it seems there is something amiss with the way the system sets up crontab environments, which with busy crontabs can cause the system to lose its mind. So I'm bird-dogging that now.

-Scott

Re: crontab

Posted: Tue Mar 06, 2018 7:44 pm
by scott
scott wrote:
nhdesign wrote:
Copied my crontab from bolt to sh yesterday.
31 emails from (cron daemon) last night.
Not even a touch command worked
touch: cannot touch ???/nfs/ftp/pub/users/nhdesign/outgoing/33660-52-foot-shift-crank.jpg???: No such file or directory
Crontab is gone.
Presume someone deleted it.

When can I reload the crontab?

Thanks,
Vic


Hi Vic,

I'm looking into this now. Last night, the server had another problem with resources. Two things were going on: first, your jobs in your crontab were being denied by SELinux (security enhanced Linux), and second, certain mounts for the resource management system known as "cgroups" had somehow disappeared from the system. One of our system admins rebooted the system last night, and it very shortly ended up in the same situation. He moved two crontabs aside (including yours), then tried again, and the system worked as expected.

So it seems there is something amiss with the way the system sets up crontab environments, which with busy crontabs can cause the system to lose its mind. So I'm bird-dogging that now.

-Scott


Your crontab is back in place.

This new change to the system that I worked on all day should mitigate a lot of the problems. There's still one corner case I need to fix with the automated unmounting, but I think we should last overnight with this setup (fingers crossed, knock on wood).

-Scott

Re: crontab

Posted: Wed Mar 07, 2018 11:27 pm
by lr
Copied my crontab over today. Completely painless. Boring. Worked perfectly.

Re: crontab

Posted: Thu Mar 08, 2018 5:31 am
by nhdesign
I deleted mine and put it back on bolt until the dust settles. Oddly I have had an issue with bolt. Is there a chance you changed something on bolt? Both nights, two nights, fortunately I run backups. Maybe it is related to the missing files thing?

--
Vic

Re: crontab

Posted: Thu Mar 08, 2018 10:17 am
by patty1
I've been running crontab on both machines. I only use it to grab entries from a calendar file in my home directory. I put a three-times-a-week entry in my calendar just to see how things are going. Two nights ago, sh.sonic.net returned a "file not found" error, but last night, both machines worked fine. Since I only run one cron job a day, I'm not as apt to catch problems as heavier users are.

Re: crontab

Posted: Thu Mar 08, 2018 11:19 am
by goetsch
I've got a job that runs hourly; last error seen yesterday (Wednesday) in the 12:00pm hour, so we're coming up on 23 error-free hours...

Re: crontab

Posted: Thu Mar 08, 2018 12:05 pm
by patty1
Thanks, goetsch, your hourly cron job is a much better indicator of sh's health than my daily one.

Re: crontab

Posted: Thu Mar 08, 2018 12:57 pm
by goetsch
Another cron job, another fail...

Every day at 12:31pm I copy the previous day's webserver access log for a Sonic-hosted domain to a folder in my home directory. Today's error was for a file not found. This server log should live at

Code: Select all

/var/log/httpd/[user name]/[hosted domain name]/access_log


When I ssh into sh.sonic.net and start walking down the directory tree, I find that there is no httpd directory under /var/log

Code: Select all

$ ls /var
account  cache  cvs  empty  gopher    lib    lock  mail  opt       run    tmp
adm      crash  db   games  kerberos  local  log   nis   preserve  spool  yp


Code: Select all

$ ls /var/log
anaconda           btmp           denyhosts-20180203  mail               messages-20180304  secure-20180225   wtmp
audit              btmp-20180301  denyhosts-20180220  maillog            munin-node         secure-20180304   wtmp-20180304
boot.log           chrony         denyhosts-20180301  maillog-20180211   none               shell             yum.log
boot.log-20180302  cron           denyhosts-20180307  maillog-20180218   ntpstats           spooler           yum.log-20180101
boot.log-20180303  cron-20180211  dmesg               maillog-20180225   ppp                spooler-20180211
boot.log-20180304  cron-20180218  dmesg.old           maillog-20180304   rhsm               spooler-20180218
boot.log-20180305  cron-20180225  grubby              messages           sa                 spooler-20180225
boot.log-20180306  cron-20180304  grubby_prune_debug  messages-20180211  secure             spooler-20180304
boot.log-20180307  custweb        iptraf-ng           messages-20180218  secure-20180211    tallylog
boot.log-20180308  denyhosts      lastlog             messages-20180225  secure-20180218    tuned


Code: Select all

$ ls /var/log/httpd
ls: cannot access /var/log/httpd: No such file or directory


I'm running the same cron jobs on bolt, so the copy is taking place (I'm getting my file), but no thanks to sh.sonic.net.

Re: crontab

Posted: Thu Mar 08, 2018 1:28 pm
by scott
goetsch wrote:
Another cron job, another fail...

Every day at 12:31pm I copy the previous day's webserver access log for a Sonic-hosted domain to a folder in my home directory. Today's error was for a file not found. This server log should live at

Code: Select all

/var/log/httpd/[user name]/[hosted domain name]/access_log


When I ssh into sh.sonic.net and start walking down the directory tree, I find that there is no httpd directory under /var/log

Code: Select all

$ ls /var
account  cache  cvs  empty  gopher    lib    lock  mail  opt       run    tmp
adm      crash  db   games  kerberos  local  log   nis   preserve  spool  yp


Code: Select all

$ ls /var/log
anaconda           btmp           denyhosts-20180203  mail               messages-20180304  secure-20180225   wtmp
audit              btmp-20180301  denyhosts-20180220  maillog            munin-node         secure-20180304   wtmp-20180304
boot.log           chrony         denyhosts-20180301  maillog-20180211   none               shell             yum.log
boot.log-20180302  cron           denyhosts-20180307  maillog-20180218   ntpstats           spooler           yum.log-20180101
boot.log-20180303  cron-20180211  dmesg               maillog-20180225   ppp                spooler-20180211
boot.log-20180304  cron-20180218  dmesg.old           maillog-20180304   rhsm               spooler-20180218
boot.log-20180305  cron-20180225  grubby              messages           sa                 spooler-20180225
boot.log-20180306  cron-20180304  grubby_prune_debug  messages-20180211  secure             spooler-20180304
boot.log-20180307  custweb        iptraf-ng           messages-20180218  secure-20180211    tallylog
boot.log-20180308  denyhosts      lastlog             messages-20180225  secure-20180218    tuned


Code: Select all

$ ls /var/log/httpd
ls: cannot access /var/log/httpd: No such file or directory


I'm running the same cron jobs on bolt, so the copy is taking place (I'm getting my file), but no thanks to sh.sonic.net.


Take a look in /logs/by_user -- you'll find your domain's directories in there.

-Scott