# Linux based Home Media applications



## lrhorer

I decided to start this thread because while there are not a huge number of people running applications such as Galleon, pyTivo, or HME for Python under Linux, there are still some of us, and perhaps a few lurking in the bushes who would consider dipping their toe into building a Linux server because of the speed, simplicity, expanded feature set, and stability of a Linux server. Others, perhaps may be thinking of hacking their Linux-based NAS boxes so they can run one of the aforementioned apps.

Basically, anything related to Linux and a networked TiVo is open for discussion, here. Anything related to specific features of one of the popular HMO or HME applications is probably better suited to one of the application specific threads elsewhere in the forum, but if your need is to write a script for handling downloaded videos or for starting pyTivo automatically at boot time, running an application under wine, or even getting a greenfield Linux installation set up for use with a TiVo, this is the place to be.

There are several members of this forum who have extensive experience with *nix based systems, and I invite them to participate. I am sure they will be happy to offer assistance to TiVo owners with limited (or no) Linux experience. While I am no Linux expert, I do have some experience with the OS and with interfacing it to TiVo DVRs, and I will be happy to offer my own help as far as I am able.

So, anyone who is interested or needs help with any aspect of Linux and TiVo, speak up!


----------



## markmarz

lrhorer said:


> So, anyone who is interseted or needs help with any aspect of Linux and TiVo ,speak up!


Just want to be the first to express my gratitude for this thread. I'm sure I'll be a frequent supplicant. In time, perhaps a resource as well.


----------



## lrhorer

markmarz said:


> Just want to be the first to express my gratitude for this thread.


'Just couldn't resist, could you? 



markmarz said:


> I'm sure I'll be a frequent supplicant. In time, perhaps a resource as well.


Yep. That's the idea. I have to say it really is satisfying to hear of people who benefit from my suggestions, and I think many others are the same. It's not unlikely that one day your expertise may well exceed mine, and that will be just fine by me.


----------



## Stormspace

I think what would really help people using linux are installation scripts for pyTiVo and that do all the grunt work for you. I'm running an old version of the program and have resisted upgrading simply because I don't want to have to spend an hour or two reconfiguring the interface. As it stands now I have only the media server part working and most of the time in order to recognize new shows on the server I have to restart the service. 

All of this might have been addressed in later versions, but I haven't seen a place that details what's now involved in the installation or configuration of pyTivo.

Good thread though, I'll be following it.


----------



## lrhorer

I don't know of any that have been puslished, but then installation under Linux is distro dependant and also highly personal. For pyTiVo, HME for Python, and vidmgr, it's pretty trivial though, except for the dependance based init scripts, if any. I have a couple of update scripts that I use, and if you will give me a couple of days or so, I can spiff them up for general distribution. Perhpaps someone else will contribute, as well.

Note the CONFIGURATION of pyTivo, etc., is beyond the scope of this thread, and is handled by other, platform independant utilities. Installation under Linux is another matter, however, and fits very nicely in this thread.


----------



## lrhorer

OK, here is the first script. It is designed to upgrade to the newest version of vidmgr, whenever Jeff may decide to release one. It could also be used to partially install vidmgr the first time. Using this script requires the following:

1. HME for Python must already be installed in the <Target> directory.
2. At least one version of vidmgr must be downloaded into the directory.
3. While in the directory, issue the command



Code:


tar -xzvf jbernardis-pytivo-video-mgr2-XXXXXXX.tar.gz

where XXXXXXX is the version number of the downloaded revision of vidmgr.

4. In the following script, update the value of SourceDir to 
5. Update the value of TargetDir to <Target>/vidmgr
6. Run the script.



Code:


#! /bin/bash

# Modify the following three lines to match your local system and software
TargetDir="/usr/share/pyhme/vidmgr/"
SourceDir="/RAID/Server-Main/Downloads/TiVo_Upgrade/pyHME"
AppName="vidmgr"

LastDir=""
SourceTemp=""

while read NewDir
do
	[[ $NewDir -nt $LastDir ]] && SourceTemp=$NewDir
	LastDir=$NewDir
done < <( find $SourceDir -type d -name "$AppName" )
SourceDir="$SourceTemp/"
if [[ -n $SourceTemp ]]
then
	cp -r "$SourceDir"* "$TargetDir"
	[[ $? -eq 0 ]] && echo Files Copied
	[[ $? -eq 0 ]] && /etc/init.d/pyHME restart && echo HME for Python restarted
else
	echo Source directory not found.  Vidmgr not upgraded.
fi
echo


----------



## lrhorer

Next is the script that actually runs HME for Python.

pyHME:



Code:


#! /bin/bash
PIDFILE=/var/run/pyHME.pid
RUNDIR=/usr/share/pyhme
LOGFILE=/var/log/pyhme.log
cd $RUNDIR
/usr/bin/nohup /usr/bin/python2.6 $RUNDIR/start.py > $LOGFILE &
echo $! > $PIDFILE
exit 0

This creates a file in /var/run named pyHME.pid that contains the PID of the running HME for Python process and a log file named pyhme.log in /var/log containg HME for Python's stdout stream. It puts the program into the background and divorces it from the parent script, allowing the app to run even after the terminal which spawned it is gone. It's parent process then becomes PID 1, the system init app.

To automatically run HME for Python at startup on a Debian based system employing dependency based booting, one can create a file such as the following in /etc/init.d.

/etc/init.d/pyHME:



Code:


#! /bin/sh
### BEGIN INIT INFO
# Provides:          pyHME
# Required-Start:    $remote_fs $syslog $network $pyTivo
# Required-Stop:     $remote_fs $syslog $network $pyTivo
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: HME Services
# Description:       Provides HME services for TiVo
#
### END INIT INFO

# Author: Leslie Rhorer

# Do NOT "set -e"

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="HME Services"
NAME=pyHME
DAEMON=/usr/share/pyhme/$NAME
DAEMON_ARGS=""
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
	# Return
	#   0 if daemon has been started
	#   1 if daemon was already running
	#   2 if daemon could not be started
	if [ -e "$PIDFILE" ];
	then
		PIDVAL=$( cat $PIDFILE )
		ps -ef | grep $PIDVAL | grep -qv grep && return 1
		rm $PIDFILE
	fi
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON || return 2
}

#
# Function that stops the daemon/service
#
do_stop()
{
	# Return
	#   0 if daemon has been stopped
	#   1 if daemon was already stopped
	#   2 if daemon could not be stopped
	#   other if a failure occurred
	if [ -e $PIDFILE ];
	then
		PIDVAL=$( cat $PIDFILE )
		rm -f $PIDFILE
		kill -15 $PIDVAL 2> /dev/null
		return "$?"
	else
		return 1
	fi
}

case "$1" in
  start)
	[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
	do_start
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  stop)
	[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
	do_stop
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  restart)
	#
	# If the "reload" option is implemented then remove the
	# 'force-reload' alias
	#
	log_daemon_msg "Restarting $DESC" "$NAME"
	do_stop
	case "$?" in
	  0|1)
		do_start
		case "$?" in
			0) log_end_msg 0 ;;
			1) log_end_msg 1 ;; # Old process is still running
			*) log_end_msg 1 ;; # Failed to start
		esac
		;;
	  *)
	  	# Failed to stop
		log_end_msg 1
		;;
	esac
	;;
  *)
	echo "Usage: $SCRIPTNAME {start|stop|restart}" >&2
	exit 3
	;;
esac


----------



## lrhorer

For upgrading / installing pyTivo, the essentially the same script used for vidmgr can be used, after updating the SourceDir, TargetDir, and AppName variables.

In my installation, for example, the values for the three are:

SourceDir="/RAID/Server-Main/Downloads/TiVo_Upgrade/pyTiVo"
TargetDir="/usr/share/pyTivo"
AppName="lucasnz"

Once again, simply download the git tarball into , and untar the file. William uses a slightly different directory format for his git, so if you are using his fork, you should have:



Code:


AppName="wmcbine-pytivo-*"

The script, when run, will find all directories under that match the string $AppName, select the newest, and copy all the files in that directory to <Target>.

In the gits, the configuration files are named XXXX.dist. That way, when the script runs, the default configuration should not overwrite your custom settings. The first time the software is installed, the user should run the command



Code:


cp XXXX.dist XXXX

and then edit the resulting configuration file accordingly.


----------



## lrhorer

This is the script I use to run pyTivo. Note it is nearly identical to the one that runs HME for Python, except that it redirects both stdout and stderr to the log file.

pyTivo:



Code:


#! /bin/bash
PIDFILE=/var/run/pyTivo.pid
RUNDIR=/usr/share/pyTivo
LOGFILE=/var/log/pyTivo.log
/usr/bin/nohup /usr/bin/python2.6 $RUNDIR/pyTivo.py > $LOGFILE 2>&1 &
RetVal=$?
if [[ RetVal == 0 ]]
then
     echo $! > $PIDFILE
else
     rm -f $PIDFILE
fi
exit $RetVal

The init script is also very similar:

/etc/init.d/pyTivo:



Code:


#! /bin/sh
### BEGIN INIT INFO
# Provides:          pyTivo
# Required-Start:    $remote_fs $syslog $network
# Required-Stop:     $remote_fs $syslog $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: pyTivo Multimedia Server
# Description:       Provides the pyTivo video server for TiVo
#                    
### END INIT INFO

# Author: Leslie Rhorer
#
# Please remove the "Author" lines above and replace them
# with your own name if you copy and modify this script.

# Do NOT "set -e"

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="pyTivo Multimedia Server"
NAME=pyTivo
DAEMON=/usr/share/pyTivo/$NAME
DAEMON_ARGS=""
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
	# Return
	#   0 if daemon has been started
	#   1 if daemon was already running
	#   2 if daemon could not be started
	if [ -e "$PIDFILE" ];
	then
		PIDVAL=$( cat $PIDFILE )
		ps -ef | grep $PIDVAL | grep -qv grep && return 1
		rm $PIDFILE
	fi
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON || return 2
}

#
# Function that stops the daemon/service
#
do_stop()
{
	# Return
	#   0 if daemon has been stopped
	#   1 if daemon was already stopped
	#   2 if daemon could not be stopped
	#   other if a failure occurred
	if [ -e $PIDFILE ];
	then
		PIDVAL=$( cat $PIDFILE )
		rm -f $PIDFILE
		kill -15 $PIDVAL 2> /dev/null
		return "$?"
	else
		return 1
	fi
}



case "$1" in
  start)
	[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
	do_start
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  stop)
	[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
	do_stop
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  restart)
	log_daemon_msg "Restarting $DESC" "$NAME"
	do_stop
	case "$?" in
	  0|1)
		do_start
		case "$?" in
			0) log_end_msg 0 ;;
			1) log_end_msg 1 ;; # Old process is still running
			*) log_end_msg 1 ;; # Failed to start
		esac
		;;
	  *)
	  	# Failed to stop
		log_end_msg 1
		;;
	esac
	;;
  *)
	echo "Usage: $SCRIPTNAME {start|stop|restart}" >&2
	exit 3
	;;
esac


----------



## lrhorer

The following only applies if the user is employing dependency based booting on a Debian or Debian derivative system. If one is employing legacy (manual) boot script ordering, it does not apply, and of course it does not apply to any distro that is not Debian based, although the scripts themselves should work with just about any distro.

Notice in the LSB header of the /etc/init.d/pyHME script there are two lines:



Code:


# Required-Start:    $remote_fs $syslog $network $pyTivo
# Required-Stop:     $remote_fs $syslog $network $pyTivo

This tells insserv that the script in question should not be run until after the "pyTivo" service is up and running. I don't actually know that this is strictly necessary in this case, but there might be some unexpected results if one tries to run vidmgr or jukebox before pyTivo, upon which they depend, is active. It also makes sure HME for Python is shut down prior to shutting down pyTivo when the sytstem is shutting down or rebooting. I'm pretty sure this is not necessary in this case, but it is a good practice in general.

In order for this to work properly, one must register pyTivo as a service. To do so, one simply edits the file /etc/insserv.conf and adds the line



Code:


$pyTivo		pyTivo

This tells insserv the script /etc/init.d/pyTivo implements the pyTivo service. When ordering the boot scripts, then, update-rc.d and insserv make sure the /etc/init.d/pyHME script is run after the pyTivo script and is shut down prior to pyTivo being shut down.


----------



## lrhorer

Both pyTivo and the TiVos themselves have web interfaces. Consequently, as most of you know, a web browser can be used to control them. In particular, pyTivo has a web interface that not only allows one to configure pyTivo, but also to push videos from the server to a TiVo or to download videos from a TiVo to the server. Similarly, the TiVos themselves allow downloading of videos to the host machine via secure HTTP using a browser. There is a utility out there, however, that allows a person writing a script (or an app like kmttg, as a matter of fact) to automate web operations. That utility is curl. Strictly speaking, curl is not a Linux app, because there are ports for other Operating Systems, including DOS/Windows. I mention it here, however, because there is a Linux port, and I think those of us running Linux servers may well want to implement curl to handle various aspects of Home Media.

In short, curl will take a CL argument and pass it to any web server. This means essentially anything that can be done using a browser can also be done using curl. For example, I wrote a web application that itself uses curl to control pyTivo. From anywhere in the world, I can log in to my sever and view almost all of the videos on the server. From there I can select one and have pyTivo push the video to one of my TiVos. The command used to implement this is:



Code:


curl -d "File=$DirName%2F$MetaFile&Command=Push&Container=$Share&tsn=HD+Theater" http://raid-server:9032

Where $DirName is the fully qualified file name of the video to push and $MetaFile is the fully qualified file name of the metafile.


----------



## lrhorer

If one redirects the stdout and stderr of pyTivo to a log file, after a while that log file can grow quite large and unweildly, as is the tendency with log files. Many Linux distros include the logrotate utility for this very reason.

When logrotate runs, it checks its configuration directory (typically /etc/logrotate.d) for target applications to manage. Each managed application will have its own configuration file. In this case, one might create a file named "pyTivo", with its configuration options. Given the nature of pyTivo's output and the nature of what it does, it is probably sufficient to keep no more than 1 - 2 weeks of logs, and it is probably not necessary to compress the logs. Due to the way logrotate works, this means deleting the previously saved log file, renaming the current log file, and creating a new, empty log file.

Because logging of pyTivo is accomplished by redirecting stderr and stdout, the application does not open and close the log file every time it needs to write, but rather the file is opened and kept open until the application is terminated. Thus, even if an external process like logrotate renames the log file (or even deletes it), the application continues to write to the same file. Thus, the old log file will continue to grow, even though it has a new name (or even has been deleted), and nothing will ever be written to the new log file. This is good in that even if the application is currently writing to the log file, renaming the file doesn't interrupt or corrupt the logging. It is bad, of course, in that logrotate hasn't really done anything very useful at this point.

In such cases, it is necessary to stop and restart the application so that it will stop writing to the old log file and start writing to the new one. Well, that is simple enough.

First of all, logrotate provides a directive for just such a purpose. It is called "postrotate". When logrotate encounters this directive and its terminator, which is named "endscript", it executes all the commands between the two markers sequentially. One could then just put something like this in the config file:



Code:


postrotate
     /etc/init.d/pyTivo restart
endscript

This will certainly work, but there is a bit of a hitch.

The second thing is that using this elementary approach is liable to kill pyTivo while it is right in the middle of sending a show from the server to a TiVo. That wouldn't be very nice, although depending on one's viewing habits it probably would not happen all that often. One way around this would be to send pyTivo a signal that tells it to shut down gracefully after it is done transferring all its current jobs. I do not know if William has implemented or would be interested in implementing a trap in his code to handle this sort of signal, however. If he reads this and responds in the affirmative, then there is a fairly slick way to handle the situation. Not knowing if that is or ever will be the case, then, we have to handle it in a little more of a brute force fashion. The following script will check to see if pyTivo is transferring any files, and if so wait 60 seconds to check again and do so continuously until it isn't transferring anything, and then restarts the application.

Edit 11/06/2012: I ran across what is effectively a bug in the following routine this morning. If one of the transfer processes hangs (or one runs Jukebox, turns off the TV, and forgets about it), the script below could loop forever. I've added a timer that kills the process aftrer 6 hours. If that isn't long enough, increase the value of the Trans_Elapse variable.

logcheck:


Code:


#! /bin/bash
VideoDir=/RAID
PIDFile=/var/run/pyTivo.pid
InitFile=/etc/init.d/pyTivo
Trans_Elapse=360

check_transfer () {
        lsof | grep -f "$PIDFile" | grep -q "$VideoDir"
}

check_transfer
Test=$?
while [[ $Test == 0 ]]
do
        echo Found $( date )
        sleep 60
        Trans_Elapse=$(( $Trans_Elapse - 1 ))
        [[ $Trans_Elapse == 0 ]] && break
        check_transfer
        Test=$?
done
"$InitFile" restart

Of course, the $VideoDir, $PIDFile, and $InitFile variables would need to be adjusted to match the individual parameters of the user's system. With the above script in place in /usr/share/pyTivo/logcheck, the logrotate config file could be:



Code:


/var/log/pyTivo.log {
rotate 2
weekly
postrotate
        /usr/share/pyTivo/logcheck
endscript
}

Now, there are a couple of vulnerabilities to this approach I need to mention. The first is this creates a bit of a race condition, where between the time the script decides pyTivo isn't busy and the time the init script kills the current instance of pyTivo, pyTivo could possibly start transferring a video. Given this period of time is at most a few milliseconds, it's quite unlikely, but it is possible. Perhaps the most likely thing is a push to one of the TiVos is scheduled, but has not yet actually started to transfer when pyTivo is killed, and the Tivo then looks for a non-existent pyTivo server. The good news is this would be an exceedingly rare occurrence, and it would only happen at the very beginning of a transfer, not half-way through or almost at the end.

The other vulnerability has to do with the user creating multiple shares on a system. The $VideoDir variable points to the highest directory where all the shares are found. Thus, in my system, all the shares are subdirectories of the /RAID mount point. I have /RAID/Recordings, /RAID/Music, /RAID/DVD, etc. All of the files are either somewhere in /RAID/xxx[/yyy/zzz...] themselves or are symlinks pointing to files in /RAID/xxx[/yyy/zzz...]. Thus, although the `lsof` utility finds a fair number of open files throughout the system (see below), the filter `grep -q "VideoDir"` insures that only files somewhere in /RAID are considered to be ones being transferred by pyTivo. If the user has multiple pyTivo shares configured and any of them are not in the subdirectory tree of a single parent directory, then those files will not be noticed by the script. For example, if my share were not /RAID/Recordings, /RAID/Music, /RAID/DVD, etc. but rather /Recordings, /Music, /DVD, etc, then no single value of $VideoDir will suffice to catch all the shares except for "/". The problem with using "/" as the filter, however, is while it will without question produce a positive result if any file is being transferred, it will in fact always produce a positive result because pyTivo always has a number of files open that are not transferring videos:



Code:


RAID-Server:/etc/logrotate.d# PIDFile=/var/run/pyTivo.pid
RAID-Server:/etc/logrotate.d# VideoDir=/
RAID-Server:/etc/logrotate.d# lsof | grep -f "$PIDFile" | grep "$VideoDir"
python2.6  9129       root  cwd       DIR                9,2      4096          2 /
python2.6  9129       root  rtd       DIR                9,2      4096          2 /
python2.6  9129       root  txt       REG                9,2   2617520   10616897 /usr/bin/python2.6
python2.6  9129       root  mem       REG                9,2    165744   10619143 /usr/lib/libexpat.so.1.5.2
python2.6  9129       root  mem       REG                9,2     61856   10725091 /usr/lib/python2.6/lib-dynload/pyexpat.so
python2.6  9129       root  mem       REG                9,2     80712   10379349 /lib/libresolv-2.11.3.so
python2.6  9129       root  mem       REG                9,2     22928   10379368 /lib/libnss_dns-2.11.3.so
python2.6  9129       root  mem       REG                9,2      9456   10379508 /lib/libnss_mdns4_minimal.so.2
python2.6  9129       root  mem       REG                9,2     85920   10725101 /usr/lib/python2.6/lib-dynload/datetime.so
python2.6  9129       root  mem       REG                9,2     51728   10379285 /lib/libnss_files-2.11.3.so
python2.6  9129       root  mem       REG                9,2     22256   10725086 /usr/lib/python2.6/lib-dynload/_heapq.so
python2.6  9129       root  mem       REG                9,2   1527584   10642137 /usr/lib/locale/locale-archive
python2.6  9129       root  mem       REG                9,2   1437064   10379360 /lib/libc-2.11.3.so
python2.6  9129       root  mem       REG                9,2    530736   10379373 /lib/libm-2.11.3.so
python2.6  9129       root  mem       REG                9,2     93936   10617266 /usr/lib/libz.so.1.2.3.4
python2.6  9129       root  mem       REG                9,2   1693344   10617102 /usr/lib/libcrypto.so.0.9.8
python2.6  9129       root  mem       REG                9,2    349248   10618803 /usr/lib/libssl.so.0.9.8
python2.6  9129       root  mem       REG                9,2     10648   10379361 /lib/libutil-2.11.3.so
python2.6  9129       root  mem       REG                9,2     14696   10379372 /lib/libdl-2.11.3.so
python2.6  9129       root  mem       REG                9,2    131258   10379355 /lib/libpthread-2.11.3.so
python2.6  9129       root  mem       REG                9,2    128744   10379356 /lib/ld-2.11.3.so
python2.6  9129       root    0r      CHR                1,3       0t0        580 /dev/null
python2.6  9129       root    1w      REG                9,2     13697    4661339 /var/log/pyTivo.log
python2.6  9129       root    2w      REG                9,2     13697    4661339 /var/log/pyTivo.log

Making sure pyTivo is not killed while it is transferring content from shares in directories that have no common ancestor other than / requires special handling.

Note a graceful shutdown of pyTivo will avoid this vulnerability, but not the first one above, or at least not directly.


----------



## lrhorer

<moved from the pyTivo thread>



Soapm said:


> There are two scripts there, what is the first one for?


The first script, contained in the directory pointed to by the $DAEMON variable in the second script, is what actually runs and daemonizes pyTivo, setting up the log file and the PID file, spawing pyTivo as a detached process in the background, and returning a value that reflects whether this process was successful or not. It can be run by itself to start pyTivo, provided of course pyTivo is not already running. This file can easily be modified to meet changing situations specific to pyTivo.

The second script is the init script, which should be much more static than the pyTivo script, merely calling the daemon script and putting the result into the startup logs.

Doing it this way makes things more modular and more future-proof, allowing for there to be changes in the particulars of the pyTivo daemon without having to monkey with the init script while also allowing for there to be future changes to insserv or the init process without having to deal with any particulars of the pyTivo daemon. It also can make trouble-shooting one or the other much easier.


----------



## Soapm

Whoops... Didn't notice you brought my question over and answered it. Thanks...


----------



## Soapm

Ok, I made the first script a file in the pyTivo directory then altered the second script to point to that file. It works but I still get the same results, I have 9 in my movies folder until I restart pyTivo. 

I'm can't even figure out what's common about the 9 that show up but it's consistently the same 9 shows???


----------



## justen_m

Any chance this thread could be perma posted as an FAQ?


----------



## lrhorer

Soapm said:


> Ok, I made the first script a file in the pyTivo directory then altered the second script to point to that file. It works but I still get the same results, I have 9 in my movies folder until I restart pyTivo.
> 
> I'm can't even figure out what's common about the 9 that show up but it's consistently the same 9 shows???


It's been a while, so I'm fuzzy on the details, but I seem to recall something similar happening to me in the past. I'm not certain, but I think it had to do with zero length files, or maybe broken symlinks. Are any of the video files or metafiles in any of your shares zero length or otherwise corrupted?

Another guess would be files with problematical file names. Do any of the file names (including directory names) on any of your shares have a name with any of the following characters:

: \ ' " ` ? & /

A leading period ( "." ) could also cause trouble.

Symlinks to non-existent files can also be trouble. (In fact, IIRC, I think that is what caused my trouble.) Do an `ls -l` in all of your shares to make absolutely certain there are no symlinks and if there are, make absolutely certain they are not broken.

Also, you are not specific whether this is in the NPL or under vidmgr. Are you running vidmgr? If so, are the results there the same as in the NPL? If you aren't running vidmgr, you might consider running it if for no other reason than troubleshooting this issue. Also, it occurs to me it might help with diagnosis if you run pyTivo in debug mode. Set "debug = True" in the config file, restart the server, and try to duplicate the issue. Look at the log file, and perhaps reproduce it here.


----------



## lrhorer

justen_m said:


> Any chance this thread could be perma posted as an FAQ?


It has been stickied in this forum.


----------



## windracer

PaJo said:


> What is causing this resolution error and how do I fix it? The server keeps running and togo will work but the Galleon icon disappears from the HDtivo.


This probably belongs in one of the old Galleon threads and not this one, but ...

I don't think the resolution line is necessarily the error. What version of Galleon are you using? You could also try to turn on debugging so there's more information in the log that might help determine the cause of that "event opcode 10."


----------



## lrhorer

PaJo said:


> I recently install Galleon on my kubuntu 12.04 system, it has been several years since I originally installed it but want to use Galleon so my wife can monitor the traffic cams during my drive home and alert me if the roads are closed.


Hey, that's kinda cool.

It would be easier to read your logs if you did not use a tiny font and wrapped them in code tags:



Code:


19:43:07,426  INFO [Acceptor] HDApplication - Received resolution event: ResolutionInfo[current=Resolution[width=640,height=480,aspectNumerator=1,aspectDenominator=1],supported=Resolution[width=1280,height=720,aspectNumerator=1,aspectDenominator=1]Resolution[width=704,height=480,aspectNumerator=40,aspectDenominator=33]Resolution[width=640,height=480,aspectNumerator=1,aspectDenominator=1]]
19:43:07,426  INFO [Acceptor] AppHost - unknown event opcode : 10
19:43:18,810  INFO [Acceptor] AppHost - connection to receiver closed




PaJo said:


> What is causing this resolution error and how do I fix it? The server keeps running and togo will work but the Galleon icon disappears from the HDtivo.


Have you checked your system logs for an event right about the same time?

The log you posted looks to me like input to log.txt. Have you checked wrapper.log?


----------



## PaJo

Sorry, I did not realize the question should be put in an old thread. The first paragraph mention Galleon so I thought this was suitable. I will take your advice and see what else I can learn.

thanks


----------



## JeffDwork

I'm about to take the TiVo plunge now that I can get Comcast's OnDemand on TiVo (yes!) and my 2nd Comcast DVR just crashed. I've been using BeyondTV for a long time recording off SD cable boxes while using Comcast's box for HD recordings, so I'm really interested in moving files between TiVo and PC. I think it's really cool that there's all this 3rd party open-source software for TiVo, but I need some help understanding what everything does.

Can someone please put together a post with a short description of the currently available programs including:
program name
what does it do
where to download it

It would be really nice if this could be put into a closed sticky thread so newbies like me could easily find it.

TIA,
Jeff


----------



## windracer

There are a few posts like that around here already, like this one.


----------



## lrhorer

JeffDwork said:


> I'm about to take the TiVo plunge now that I can get Comcast's OnDemand on TiVo (yes!) and my 2nd Comcast DVR just crashed. I've been using BeyondTV for a long time recording off SD cable boxes while using Comcast's box for HD recordings, so I'm really interested in moving files between TiVo and PC.


This topic is not really suited to this thread. It would be best moved elsewhere. If you have questions or comments on getting any of the applications you seek working under Linux, then come on back here.

Note I do not believe you will be able to move any OnDemand content to a PC.



JeffDwork said:


> Can someone please put together a post with a short description of the currently available programs including:
> program name
> what does it do
> where to download it


As you can see from the link provided by windracer, there are something over 100 3rd party applications available for the Tivo, and some of it has quite an extensive array of features. Writing up a "what does it do" synopsis for every one of them would be quite an undertaking for any one individual - especially an unpaid one.



JeffDwork said:


> It would be really nice if this could be put into a closed sticky thread so newbies like me could easily find it.


Um, why a closed thread? New apps for TiVos come out on a regular basis. Some of the very best have been developed within the last few months. Any closed thread of this nature would soon be obsolete, sticky or not.



windracer said:


> There are a few posts like that around here already, like this one.


Unfortunately, while that thread is close to being comprehensive in terms of the names and locations of apps, it is not so when it comes to functional descriptions. I'm afraid JeffDwork is just going to have to do some research, but that thread is an excellent place to start.


----------



## PaJo

lrhorer said:


> Hey, that's kinda cool.
> 
> Have you checked your system logs for an event right about the same time?
> 
> The log you posted looks to me like input to log.txt. Have you checked wrapper.log?


Thanks again, I think the resolution errors may have been associated with the video setting on the HD Tivo after I switched to 1080i hybrid it seemed to work better.

I am now using a slightly overclocked Raspberry Pi (900mhz) running the 32 bit 2.3.1 version of Galleon for the traffic cams. The Raspberry Pi is slower but once the traffic cam is up it seems pretty good. I added a call to galleon/bin/run.sh to the /etc/rc.local so if my wife has any problems she can just switch the Raspberry Pi off and back on again. If and when I want to use xbmc I can just switch SD cards and reboot. The only thing setup in Galleon is the Internet Images, and I do not plan on using it for anything more at this time.

thanks again and I apologize for being off topic

joe


----------



## windracer

PaJo said:


> I am now using a slightly overclocked Raspberry Pi (900mhz) running the 32 bit 2.3.1 version of Galleon for the traffic cams.


That's pretty cool! :up:


----------



## PaJo

The traffic cams via Raspberry Pi with Galleon has been working very well for a couple days. I have ordered a second $35.00 Raspberry Pi to use with an old, usb wifi adapter & usb cam currently gathering dust in hopes to send my own cam images. It may be possible to use the wireless webcam setup to do video and later push it to the Tivo but I need to do more research, I read someone had motion working with the Raspberry Pi but have a lot to learn. 

It's ironic, a few years ago we used Pytivo a lot but after getting Roku and a HD media player we stopped using it, but if I can get my wireless usb cam video working with motion might install pytivo again and try pushing home made videos to our Tivo, it will be a nice addition for our current surveillance/dvrs setup and having an additional capabilty of security surveillance videos would be much more useful to me than many of the features on Tivo these days. It's a long way off, now just an idea, but if I could get security videos on the Tivo in folders by date it would be really nice addition to the Tivo for me. Most stand alone dvr systems for security surveillance only work with windows, not much for linux unless you do it yourself.


----------



## lrhorer

PaJo said:


> It may be possible to use the wireless webcam setup to do video and later push it to the Tivo


If you are going to be pushing the video to the TiVo, I suggest you try vidmgr. The curl example above may be useful, as well.


----------



## PaJo

Thanks for the suggestions, and you script examples. I have used pytivo & and also curl in the past to link a ftp directory on my media player to my Kubuntu file system. I will take a long look at it all, may just use the HD Media player for the new video setup because it will allow me to zoom in on areas of the video, which will be better for security videos. Either way I prefer view them on my big screen TV rather than a 24" monitor. Galleon still seems to be working OK and so far appears to be a unique program that must be used on Tivo.

thanks again to all


joe


----------



## 72morgan

I am currently using WHS and streambaby. I had a drive go bad and it was a real pain to save and move my data and videos. I am thinking of going to a new setup with either a Netgear or Synology server. I am thinking the netgear / sinology setup might be easier to maintain, ie hot swappable. Netgear has some Tivo support, but in some ways Synology seems better. Can anyone give me an idea which is better and how hard would it be to set up either one with PyTivo so I can stream videos from the server to my Tivos. Any input would be appreciated.

I have installed Windows thousands of times , built all of my computers and the WHS. I have* NO *experience with Linux.


----------



## lrhorer

Welcome to the thread. A number of users on this forum have experience with either the Synology or Netgear systems. The advantages of a NAS are they are usually small in footprint, comparatively speaking, and they usually enjoy very low power consumption. On the down side, they tend to be a bit more expensive than a roll-your-own, and unless you purchase a high-end model, expandability can be a real challenge.

I will take the time to enter into a little more lengthy discourse this evening, but in the mean time I request that you think a bit about your needs in terms of capacity and usage both now and in the near future, and let us know what they are. The answers will help determine a good path for you to follow. Note that you need to think about not only how much capacity you will need, but also what types of files ( video only or audiom photos, video, and perhaps general purpose files? MPEG-II? MKV? h.264? ), and what you will be doing with them (transferring to a handheld device? Burning to DVD?). It also encompasses how much you want to spend.


----------



## lrhorer

Well, the latest version of Debian Linux, named "Wheezy", was released just a few days ago. I haven't upgraded, yet, but it looks pretty nice. I'm probably going to upgrade one of my desktop PCs this weekend. I'll post my findings here.


----------



## timckelley

Say, I have a general question about this. I've been running pyTiVo on an old computer with Windows XP on it, and I've always assumed the slowness of file transfers were due to me using wifi. Well I ran a long ethernet cable as an experiment, and a 2-3 GB file still took hours to transfer, so I'm guessing the bottleneck must be whatever pyTiVo is internally doing during the transfer. It must be doing a bunch of data crunching or something.

It seems like I remember you or somebody saying that pyTiVo could be installed on my NAS directly. (Currently I have the XP computer transferring files between the TiVos and the NAS.) I actually now have two NAS's in my house, both made by Synology, which I think might be linux-based, if I understand correctly, which made me think of this thread. (One NAS contains 2 x 3TB in RAID 0 configuration, and the other has 4 x 3TB in RAID 5.)

So my question is this: If I were to install pyTiVo on one of the NAS's, do you think there would be a significant boost in how fast I can transfer files?

On another subject, pyTiVo freezes on me pretty much daily, and I think it's because of the wifi network. My attempts to use MOCA have failed, but my BIL thinks I should install a high quality wireless access point (instead of relying on the router's built-in wifi), and he's offered to help me run ethernet to the center of the house through my attic to get this set up, so hopefully that will work. I'd consider running ethernet to the TiVos, but that would involve parts of my attic that are very hard to get to. Getting to a central location is easy though, as that part of my attic is very accessible.


----------



## windracer

timckelley said:


> So my question is this: If I were to install pyTiVo on one of the NAS's, do you think there would be a significant boost in how fast I can transfer files?


My initial response is no. The "data crunching" is pyTivo transcoding the file (via ffmpeg) to a format the TiVo can play. You can minimize transcoding by keeping your files in formats the newer boxes can play natively so that really you're just copying the file instead of re-encoding it (which is CPU intensive). NASes usually have slower CPUs than a full-blown desktop machine which is why I say you probably won't see a performance boost moving pyTivo to the NAS (although I have no experience with the Synology units).

As for the networking setup, wired ethernet would be the way to go, then MoCA (which I personally love and use with all of my TiVos), and then WiFi.


----------



## timckelley

The kind of transferring we're doing is mainly just moving a file from the TiVo to the NAS. Should that really even involve transcoding the files? It looks like the files are showing up on the NAS with a .TiVo suffix, plus a small meta data file in text format. Are the files stored natively on our TiVos in .TiVo format? By the way, the 3 TiVos we use with pyTiVo are 2 S2, plus one Premier. (My TiVo is a TiVoHD, but I don't bother transferring files from that one.)


----------



## wmcbrine

S2s are slow as hell. They're almost always the limiting factor in a transfer. You should get pretty good speeds to a Premiere, though.

The TiVo's internal representation of recordings doesn't really resemble a conventional filesystem. But .TiVo files are the transport mechanism in and out of the TiVo.

A .TiVo file might still need transcoding, e.g. if you were attempting to watch an HD recording on a Series 2.


----------



## timckelley

Well my wife almost never attempts to watch an HD recording on her S2's, but it sounds like even without transcoding, slowness can be caused by the hardware inside the S2 itself, which I assume must participate in the work of the file transfer.

So it's sounding like whether I continue with my XP box, or house pyTiVo on an NAS, I'm still going to be limited by the S2. And my BIL's suggestion I install a high speed 5GHz wireless access point (to get higher speed wifi) wouldn't necessarily help that much either.


----------



## lrhorer

timckelley said:


> Say, I have a general question about this. I've been running pyTiVo on an old computer with Windows XP on it, and I've always assumed the slowness of file transfers were due to me using wifi. Well I ran a long ethernet cable as an experiment, and a 2-3 GB file still took hours to transfer


It should not take that long, but using WiFi, it might. WiFi speeds can be unspeakably low, but one should be able to get at least a couple of Mbps out of even a weak link. Two Mbps is 900 MB per hour, so if your link is only managing 1 or 2 Mbps, then a couple of hours or so is not unexpected for a 2G file.



timckelley said:


> so I'm guessing the bottleneck must be whatever pyTiVo is internally doing during the transfer. It must be doing a bunch of data crunching or something.


No, pyTiVo doesn't do anything of note during a transfer from a TiVo to the server. The TTG transfer protocol is actually just a standard Secure HTTP transfer as far as the clients are concerned.



timckelley said:


> It seems like I remember you or somebody saying that pyTiVo could be installed on my NAS directly.


It surely can, as long as the platform wither allows alien software, or else can be hacked.



timckelley said:


> (Currently I have the XP computer transferring files between the TiVos and the NAS.) I actually now have two NAS's in my house, both made by Synology, which I think might be linux-based


Yes, I believe that is true of most, if not all Synology units.



timckelley said:


> So my question is this: If I were to install pyTiVo on one of the NAS's, do you think there would be a significant boost in how fast I can transfer files?


Well, it depends on the topology, but I expect not.



timckelley said:


> On another subject, pyTiVo freezes on me pretty much daily, and I think it's because of the wifi network.


That is the best first guess. It could be something else, however. Without more information, it is not possible to refine the guess.



timckelley said:


> My attempts to use MOCA have failed, but my BIL thinks I should install a high quality wireless access point (instead of relying on the router's built-in wifi), and he's offered to help me run ethernet to the center of the house through my attic to get this set up, so hopefully that will work.


It may or may not help. Again, we would need more details to guess more effectively.


----------



## lrhorer

timckelley said:


> The kind of transferring we're doing is mainly just moving a file from the TiVo to the NAS. Should that really even involve transcoding the files?


Well, yes and no. The transcoding, if any, is not done by the server in that case, however.



timckelley said:


> It looks like the files are showing up on the NAS with a .TiVo suffix, plus a small meta data file in text format. Are the files stored natively on our TiVos in .TiVo format?


No, they are tystreams. On S1 and S2 TiVos, the tystreams are an extension of MPEG2. On the S3 and S4, I believe they can be coded as either MPEG2 or h.264 extensions.



timckelley said:


> By the way, the 3 TiVos we use with pyTiVo are 2 S2, plus one Premier. (My TiVo is a TiVoHD, but I don't bother transferring files from that one.)


Do you experience the same low transfer rate to / from the Premiere? If so, then it lends more support to the notion of your problem being related mainly to the WiFi.


----------



## lrhorer

timckelley said:


> Well my wife almost never attempts to watch an HD recording on her S2's, but it sounds like even without transcoding, slowness can be caused by the hardware inside the S2 itself


It's possible, but the rates you mention sort of sound more like something else. It's hard to tell without quantitative data.



timckelley said:


> which I assume must participate in the work of the file transfer.


There are ways to speed it up - a lot. Not, however, if your wireless loop is the main culprit.



timckelley said:


> So it's sounding like whether I continue with my XP box, or house pyTiVo on an NAS, I'm still going to be limited by the S2.


At some point, yes. Whether you are at that point now, or not, is another matter. We need to know what segments are wireless and which ones are wired. Is the link from the NAS to the XP machine wired or wireless? From the XP machine to the S2s? To the Premiere? Remember, when you host the download server (pyTivo in this case) on the XP machine, the data has to transfer from the TiVo to the server, and then from the server to the NAS. Add to that the fact the wireless links are half duplex, and not very fast in the first place, and it would be no wonder the transfers are slow if the data is passing across a wireless segment twice.



timckelley said:


> And my BIL's suggestion I install a high speed 5GHz wireless access point (to get higher speed wifi) wouldn't necessarily help that much either.


If you can get us some real numbers, we might be able to tell you better. In partcular see what speeds you get to and from the Premiere. The Premiere is not slow, comparatively speaking. Most routers have stats for their wireless clients. Posting those would help, as well.


----------



## lrhorer

windracer said:


> My initial response is no. The "data crunching" is pyTivo transcoding the file (via ffmpeg) to a format the TiVo can play.


He seems to be using pyTivo to transfer from the TiVos to the NAS, rather than the other way around. If so, your point is moot for his use.



windracer said:


> As for the networking setup, wired ethernet would be the way to go, then MoCA (which I personally love and use with all of my TiVos), and then WiFi.


Wireline fits in there somewhere, definitely above WiFi.


----------

