# Endpad and Cron



## kev160967 (Jan 7, 2005)

Hi,
I have a recurring problem where endpad stops working every few days. If I go to hackman it thinks it is still running, but when I hit the stop button it tells me it has encountered a stale PID. I can then start it and it will run again for a few days. I came up with the idea of stopping and starting it each day using cron, but it doesn't appear to work. Here are the lines from my crontab:

# shut down endpad
0 2 * * * /var/hack/endpad.tcl -stop >> /dev/null &

# start endpad up again
0 3 * * * /var/hack/endpad.tcl -s 1 -e 5 -auto >> /var/hack/cron/eps.log &

So at 2am it stops it, which works okay if it is running, then at 3am it should start it again. 

The problem is that by 5am (when daily mail runs, from same crontab file) it has stopped again. Any ideas what's going on? The following is the recording of the startup I directed to eps.log:

EndPad v1.5.4 (c) 2005 Stuart Anderton
Strictly no commercial distribution
See file header for terms of use and distribution
Switching to background...

Not sure what other logs might have anything useful in them? Endpad.log and endpad.msg have both been empty since I changed this - only thing I can see is the initial cron kill being accepted

Kev


----------



## TCM2007 (Dec 25, 2006)

I'd try to find out why EndPad is crashing, rather than this workaround.

Can you post the log?


----------



## kev160967 (Jan 7, 2005)

Okay, I've removed the crontab entries and started it again. Next time it fails I'll post the error from hackman and the logs. Are there any particular logs you think would be helpful? Under logs on tivoweb I have an endpad.log and an endpad.msg. I don't remember seeing anything in either of these in the past though.

Kev


----------



## TCM2007 (Dec 25, 2006)

EndPad.log should have useful info.


----------



## kev160967 (Jan 7, 2005)

Okay, lasted for longer than usual this time. There's not much in the log though. Here it is:

Thursday 00:55:01 : No recording in progress
Thursday 00:55:01 : Next end padding setting due 04:00
Thursday 00:55:01 : Not time to add start padding yet
Thursday 00:55:01 : Next start padding setting due 03:38
Thursday 00:55:01 : Next wake up will be 01:25
Sleeping, checking every minute for kill signal............................... 
Thursday 01:25:01 : Woken up
Thursday 01:25:01 : No recording in progress
Thursday 01:25:01 : Next recording is {The Cramp Twins} on CNTOO (Suggestion) at 03:40
Thursday 01:25:01 : No end padding required:
Thursday 01:25:01 : No recording in progress
Thursday 01:25:01 : Next end padding setting due 04:00
Thursday 01:25:01 : Not time to add start padding yet
Thursday 01:25:01 : Next start padding setting due 03:38
Thursday 01:25:01 : Next wake up will be 01:55
Sleeping, checking every minute for kill signal............................... 
Thursday 01:55:01 : Woken up
Thursday 01:55:01 : No recording in progress
Thursday 01:55:01 : Next recording is {The Cramp Twins} on CNTOO (Suggestion) at 03:40
Thursday 01:55:02 : No end padding required:
Thursday 01:55:02 : No recording in progress
Thursday 01:55:02 : Next end padding setting due 04:00
Thursday 01:55:02 : Not time to add start padding yet
Thursday 01:55:02 : Next start padding setting due 03:38
Thursday 01:55:02 : Next wake up will be 02:25
Sleeping, checking every minute for kill signal.................

Also, here's the error message I got when I tried to stop endpad from hackman. At this point Dailymail had reported endpad wasn't running, but hackman thought that it was. I pressed the stop button and after the usual text I got this:

A stale PID file was found and has been deleted. This means that this script terminated unexpectedly last time. Loading now. The errors logged (if any) before the last termination were: No instance of EndPad running

Any ideas?


----------



## AMc (Mar 22, 2002)

If endpad isn't auto starting from a boot then it might be a symptom of something else?
What's your uptime? Are there any other clues in the main logs at the same(ish) time.


----------



## kev160967 (Jan 7, 2005)

Endpad starts okay from a boot, it's just every so often it will stop for no apparent reason. That's why I tried the cron approach, since it would invariably run for a day or so each time before it stopped working - by stopping and starting every night I was hoping to avoid the issue


----------



## spitfires (Dec 19, 2006)

Hmm, nothing wrong there. Other than "tcp_keepalive" messages is there anything in your kernel log between 01:55 and 02:25 which might suggest a problem? (Or in the 'messages' log?)

Edit: oops I see your endpad log was for Thursday so that would probably be "Okernel" and "Omessages"


----------



## kev160967 (Jan 7, 2005)

spitfires said:


> Hmm, nothing wrong there. Other than "tcp_keepalive" messages is there anything in your kernel log between 01:55 and 02:25 which might suggest a problem? (Or in the 'messages' log?)
> 
> Edit: oops I see your endpad log was for Thursday so that would probably be "Okernel" and "Omessages"


Already rolled of the end of the log, unfortunately - Okernel goes back to 6am on Thursday. Next time it happens I'll grab a segment of the kernel log for the appropriate time. I'll also scan any other logs for events in that timeframe.


----------



## kev160967 (Jan 7, 2005)

Interesting (well maybe). Tivoweb.log rolled over at 02:13 on Thursday - looking at the log entry, tivoweb restarted at that point.

Okay, my bad - it seems that the tivo rebooted on Thursday morning. Now as far as i can recall, in the past endpad failing hasn't aligned with the tivo rebooting. That being said, i didn't think it had this time, so who knows. To back it up though, I've noticed the tivo is more prone to rebooting since moving to altepg (and endpad failures became more common), but prior to that I can state as a fact that the tivo went for months without rebooting but endpad was failing (very) occasionally. Not as much, by far, but occasionally.

I guess there could be two issues here then - an intermittent/rare problem not tied to a reboot, and a more frequent problem tied to a reboot. Has anyone any thoughts on why endpad wouldn't start after a reboot, but hackman would think it was started?

I'll continue to monitor for endpad failures and reboots and see how they tie together. Since going to altepg it's rare to see endpad stay up for more than a couple of days at a time, so it shouldn't take long

Thanks folks,
Kev


----------



## TCM2007 (Dec 25, 2006)

Your TiVo rebooted, but EndPad should recover cleanly from that if it's in rc.sysinit.author - which it's not because you've done your cron stuff. If it was set up conventionally it would have restarted.

In this particular case, it seems your attempt to fix the problem has caused it!

I think you need to put everything back to a standard setup and keep checking for errors!

When was your daily call; 2.23 perchance?

You could use cron as belt and braces; don't stop EndPad but just try to start it *without the -auto argument*. If it's not running it will start, if it is running it will do nothing.


----------



## kev160967 (Jan 7, 2005)

TCM2007 said:


> Your TiVo rebooted, but EndPad should recover cleanly from that if it's in rc.sysinit.author - which it's not because you've done your cron stuff. If it was set up conventionally it would have restarted.
> 
> In this particular case, it seems your attempt to fix the problem has caused it!
> 
> ...


I didn't actually take endpad out of rc.sysinit.author when I added it to cron. There wasn't a conflict so I didn't see the need - sysinit was there for reboots, cron for daily maintenance.

This is the relevant part of my rc.sysinit.author:

/var/local/TivoWebPlus/tivoweb &
/var/hack/endpad.tcl -s 1 -e 5 -auto >> /dev/null &
/var/hack/suggs.tcl -threshold 1 <</dev/null

The tivoweb bit worked, but not endpad. Also I notice that suggs doesn't have an ampersand so is being started synchronously, which doesn't sound right - it's after the endpad though, so probably not relevant? Interesting to know if it too didn't start. Are there any bits of that that you would change? Is suggs still needed?


----------



## kev160967 (Jan 7, 2005)

TCM2007 said:


> You could use cron as belt and braces; don't stop EndPad but just try to start it *without the -auto argument*. If it's not running it will start, if it is running it will do nothing.


Just to be explicit, endpad is no longer in crontab, and still is in sysinit


----------



## TCM2007 (Dec 25, 2006)

Is that copy and pasted, because youve got << and >> mixed up, and no space


----------



## kev160967 (Jan 7, 2005)

Tivo's been annoyingly )) stable since I brought this up, but i did a manual reboot over the weekend, and endpad did not start up (although hackman seemed to think that it did??! and again reported the stale PID when I stopped it).



TCM2007 said:


> Is that copy and pasted, because youve got << and >> mixed up, and no space


It was copy and pasted - though it was strange, but I assumed I'd copied that from a set of installation instructions. Going into /etc/rc.d, and manually executing rc.sysinit.author does start endpad though, so I'm still not sure what's going on.

On the face of it, endpad only seems to start from an interactive bash shell. From both a startup script and from cron it fails. I'll comment out the suggs line from sysinit.author, and see if that makes a difference

Kev


----------



## kev160967 (Jan 7, 2005)

This is the end of my rc.sysinit file - does anything look out of order:


```
if [ ! "$vmstat" = "" ]; then
  echo "Starting memory statistic gathering"
  vmstat 10 &
fi

#echo "[ ! -f /etc/rc.d/rc.sysinit.author ] || /etc/rc.d/rc.sysinit.author"
/etc/rc.d/rc.sysinit.tpm

echo "rc.sysinit is complete"
source /etc/rc.d/rc.net

# Call the Tivo Package Manager Startup Scripts
#/etc/rc.d/rc.sysinit.tpm

/bin/bash </dev/ttyS3 >& /dev/ttyS3 &
/sbin/tnlited 23 /bin/bash -login &
/sbin/tivoftpd
/var/hack/bin/cron >> /dev/null &

# Set the path
PATH=/bin:/sbin:/tvbin:/devbin:/var/hack:/var/hack/bin
export PATHbash-2.02#
```
I'm not sure what the commented out line near the top do with rc.sysinit.author is doing. The first bit, I think, is saying if the file doesn't exist, but I'm not sure what the second half does.

Should there be something in rc.sysinit that calls rc.sysinit.author, or does the tivo boot sequence do it automatically?

Kev


----------



## kev160967 (Jan 7, 2005)

Just rebooted again. Here is my current rc.sysinit.author - I added the echo line to see if it would appear in the log on a reboot. It doesn't appear to.


```
echo "running rc.sysinit.author"
/var/local/TivoWebPlus/tivoweb &
/var/hack/endpad.tcl -s 1 -e 5 -auto >> /dev/null &
#/var/hack/bin/cron >> /dev/null &
#/var/hack/suggs.tcl -threshold 1 <</dev/null
```
Here's the log from a reboot I did after adding the echo line


```
Nov 21 09:58:29 (none) kernel: Scanning for phase4 repair scripts 
Nov 21 09:58:30 (none) kernel: MCP startup complete 
Nov 21 09:58:30 (none) kernel: rc.sysinit is complete 
Nov 21 09:58:32 (none) kernel: Warning: kernel-module version mismatch 
Nov 21 09:58:32 (none) kernel: ^I/lib/modules/ax88796.o was compiled for kernel version 2.1.24-TiVo.1 
Nov 21 09:58:32 (none) kernel: ^Iwhile this kernel is version 2.1.24-TiVo-2.5 
Nov 21 09:58:32 (none) kernel: ax88796 version 20030223-0 
Nov 21 09:58:32 (none) kernel: ax88796: updated tlb entry 2 to 1M 
Nov 21 09:58:32 (none) kernel: ax88796: using MAC address 00:0B:AD:69:C6:67 
Nov 21 09:58:32 (none) kernel: ax88796: driver installed 
Nov 21 09:58:32 (none) kernel: 
Nov 21 09:58:33 (none) kernel: IP struct was not filled in! 
Nov 21 09:58:33 (none) kernel: sa is: 0x20000 0xc0a8c806 0x7ffffce4 0x7ffffd00 
Nov 21 09:58:33 (none) kernel: sa.sin_addr = 0x7ffffc48 a sockaddr is 16 bytes 
Nov 21 09:58:33 (none) kernel: route.tivo forgot to specify route netmask. 
Nov 21 09:58:41 (none) kernel: EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended 
Nov 21 09:58:44 (none) last message repeated 2 times
Nov 21 09:59:01 (none) kernel: Microcode version 00034 
Nov 21 09:59:04 (none) kernel: Initialize with 1 live caches 
Nov 21 09:59:04 (none) kernel: mediaswitch: resetting adjustments 
Nov 21 09:59:04 (none) kernel: tivoblaster.C::ReadTivoFormatKey(), Line : 356, Found 350 ms. Prefix. 
Nov 21 09:59:05 (none) kernel: osd buffer size = 1080000 
Nov 21 09:59:30 (none) kernel: mediaswitch: returning -1 from standin tune after tuning to ch -3 with adjust 0 
Nov 21 09:59:30 (none) kernel: tuning:lock 1 
Nov 21 09:59:31 (none) kernel: Done with this packet
```
Starting to think the author file isn't getting executed?? But then what would be starting tivoweb?

Another funny, which may be a clue - after the reboot my tivo's filesystem was read/write. I'd specifically set it to read only before rebooting (since I had a suspicion something was doing this at a reboot).

I'm going to try changing the order of endpad and tivoweb in the rc.sysinit.author file, and rebooting

Kev
Kev


----------



## kev160967 (Jan 7, 2005)

Stranger and stranger - I commented out the tivoweb startup from rc.sysinit.author, but when I rebooted tivoweb was up and running. Also, the path reported by "ps -ax" isn't the one from the author file. Note also that the pid (112) is lower than things like the ftp daemon and cron, which are started at the end of rc.sysinit:


```
109  ?  S    0:00 Mcp event
  110  ?  S    0:00 tcphonehome
  112  ?  R    0:14 tivosh /var/local/tivoweb-tcl/httpd-tt.tcl
  115  ?  S    0:00 tcphonehome
  116  ?  S    0:00 PipeListener
  117  ?  S    0:00 FsMpStream
  118  ?  S    0:00 PhoneHome event hd
  120  ?  S    0:00 /bin/bash
  123  ?  S    0:01 /sbin/tnlited 23 /bin/bash -login
  126  ?  S    0:00 /sbin/tivoftpd
  128  ?  S    0:00 /var/hack/bin/cron
  142  ?  S    0:00 dbgc-mcp
  143  ?  S    0:00 dbgc-mcp
  144  ?  S    0:00 PipeListener
```
Any idea what might be starting tivoweb? I can't see anything else in rc.d.

Kev


----------



## TCM2007 (Dec 25, 2006)

[ ! -f /etc/rc.d/rc.sysinit.author ] || /etc/rc.d/rc.sysinit.author

is the line that runs rc.sysinit.author.

No idea a) why it's commented out and b) why it's been edited to be in an echo statement.

Both of those things will stop rc.sysinit.author running.

So EndPad is only being started by Hackman.

I'm afraid I've no experience of using that,.


----------



## kev160967 (Jan 7, 2005)

Should this be in /etc/init.d/TivoWebPlus:


```
bash-2.02# cat TivoWebPlus
#!/bin/sh
RUN_ONCE=0
if [ "$1" == "start" ]; then
    if [ $RUN_ONCE -eq 1 ]; then
        mount -o remount,rw /
        rm -f /etc/rcS.d/S90TivoWebPlus
        mount -o remount,ro /
    fi
/var/local/tivoweb-tcl/tivoweb
fi
```
The file /etc/rcS.d/S90TivoWebPlus has the same contents as above??


----------



## kev160967 (Jan 7, 2005)

TCM2007 said:


> [ ! -f /etc/rc.d/rc.sysinit.author ] || /etc/rc.d/rc.sysinit.author
> 
> is the line that runs rc.sysinit.author.
> 
> No idea a) why it's commented out and b) why it's been edited to be in an echo statement.


Hmm, I've got no recollection of commenting it out or putting the echo statement in. I've also got a file rc.sysinit.bak dated October 2006 which has no additionals startup commands in it, but also has this line commented out. I'm wondering if I might have taken a backup the very first time I made an edit, and this is it.

Looking at the stuff I found that starts tivoweb, I'm wondering if I may have run some install script for an early tivoweb, which made these changes.

Kev


----------



## kev160967 (Jan 7, 2005)

Okay, changed the line back and rebooted, and I'm happy to announce that endpad has now started. I'd still welcome any thoughts on why my filesystem is writable at the end of the bootup, or whether there's something I need to change about the way tivoweb is starting (see posts above)

Thanks for your help so far!!
Kev

p.s. Could I remove the TivoWebPlus file in /etc/init.d and replace it with an entry in sysinit.author? Perhaps if I did, I'd be better commenting out its content, in case something is calling it and wouldn't be happy if it was removed. Also, I have the latest tivoweb2 installed, and was wondering why it wouldn't start, so I guess I'd need to remove this anyway??


----------



## kev160967 (Jan 7, 2005)

Aha! Found the following:


```
bash-2.02# cat rc.sysinit.tpm
#!/bin/sh
for i in /etc/rcS.d/S[0-9][0-9]*
do
    # Check if the script is there
    [ ! -f $i ] && continue
    $i start
done
```
This gets called near the end of rc.sysinit, and calls the stuff in /etc/rcS.d. Not sure what the copy in /etc/init.d is for though. I'm definitely inclined to get rid of this script now, and do it from sysinit.author - does anyone think that's a bad idea?

Kev


----------



## RichardJH (Oct 7, 2002)

I'm sure that Hackman does something to the files so that you can have the ability to edit from within Tivoweb Hackman module without the need to use Telnet. It may be worth having a read up of the Hackman guide found here http://www.tivohackman.com/


----------



## kev160967 (Jan 7, 2005)

RichardJH said:


> I'm sure that Hackman does something to the files so that you can have the ability to edit from within Tivoweb Hackman module without the need to use Telnet. It may be worth having a read up of the Hackman guide found here http://www.tivohackman.com/


sysinit.tpm is apparently Tivo Package Manager. A bit of reading on the hackman site throws up that as of November 2005 "if you install the only official version of TWP (although now deprecated) it installs as a tpm package". Guess that's how I originally installed TWP. I'm certainly of a mind to use sysinit.author instead, just to keep things simple.


----------



## RichardJH (Oct 7, 2002)

kev160967 said:


> I'm certainly of a mind to use sysinit.author instead, just to keep things simple.


Its a long time since I have fiddled with my one remaining S1 Tivo and thats the way I went


----------

