# .TIVO with CAPTIONS on Mac



## deac33 (Sep 28, 2013)

How can I save a high quality .TIVO file to my Mac that can be later uploaded to the Tivo Roamio Plus preserving Closed Captions?

I have cTivo, kmttg, pytivo and would buy Tivo Desktop to run in Windows 7 under Parallels if that works best.

I just want the best quality .TIVO file on my computer hard disk to put back to the TIVO with closed captions. Then I'll be ecstatic!

If Tivo Desktop does that and runs on a Mac in Parallels then I'll buy it.

Thanks much,
-deac


----------



## moyekj (Jan 24, 2006)

All you need to do is when downloading from your TiVos with kmttg under config->Program Options make sure following is enabled:
"Download TiVo files in Transport Stream format"

Then, using pyTivo if you transfer these TS .TiVo files back to your TiVo the captions will be intact.
(Be advised though that by turning on above option you won't be able to decrypt the .TiVo files or do anything else with them other than transfer back to your TiVos).


----------



## deac33 (Sep 28, 2013)

moyekj said:


> All you need to do is when downloading from your TiVos with kmttg under config->Program Options make sure following is enabled:
> "Download TiVo files in Transport Stream format"
> 
> Then, using pyTivo if you transfer these TS .TiVo files back to your TiVo the captions will be intact.
> (Be advised though that by turning on above option you won't be able to decrypt the .TiVo files or do anything else with them other than transfer back to your TiVos).


Kevin,
Thanks much for the tips. I've tried it now several times and am unable to copy the .TIVO files up to the TiVo: "Transfer prohibited by the copyright holder". How do I get around this error so I can upload a .TIVO with captions?

Here is a total recap of the various settings and results. -------
1. New: Your suggestion to check TS:
My settings for the the result described above are:
..... Check "Download TiVo files in Transport Stream format"
..... and uncheck all checkboxes on the main kmttg page. 
I select the show, click START JOBS and it does download to the Mac but gives the copyright error above when I try to upload back to the TiVo.

2. TS and all other boxes checked - they fail as you said they would.

3. When I uncheck TS and recheck decrypt, etc. , the resulting mp4 file does upload, but has no captions as before.


----------



## moyekj (Jan 24, 2006)

You should use an updated version of wmcbrine pyTivo to pull TS .TiVo files back to your TiVos and that will work. If you're using an old pyTivo it won't work.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> You should use an updated version of wmcbrine pyTivo to pull TS .TiVo files back to your TiVos and that will work. If you're using an old pyTivo it won't work.


Failed again:
I just downloaded the latest mcbrine fork zip and unzipped it, copied my old pytivo.conf file (which uses the same ffmpeg as kmttg), relaunched and still get the copyright error.

It's using python 2.7.6.

Is there another configuration parameter I need to check?


----------



## moyekj (Jan 24, 2006)

The "copyright" error for pyTivo means that ffmpeg is not able to extract information from the .TiVo file. Post here the results of running:
ffmpeg -i yourFile.TiVo

If it's not returning basic video + audio information then that's the problem as pyTivo uses ffmpeg to get that information. Perhaps updating ffmpeg to a recent version will solve the problem. (I don't recall what version of ffmpeg for Mac is in kmttg package but it's likely old).


----------



## deac33 (Sep 28, 2013)

moyekj said:


> The "copyright" error for pyTivo means that ffmpeg is not able to extract information from the .TiVo file. Post here the results of running:
> ffmpeg -i yourFile.TiVo


1. Problem solved: 
You nailed it, as usual, I found the latest ffmpeg, 2.2.1, on ffmpeg.org and it does preserve captions when uploaded back to the Tivo. (It took a day of my making various mistakes, but finally got it right.  Thanks much).
... ffmpeg -i '2 Broke Girls.TIVO' ... shows lines such as
[mpeg2video @ 0x7fef19808000] ac-tex damaged at 20 0
[mpeg2video @ 0x7fef19808000] ac-tex damaged at 19 1

but none with "audio". And these lines says "damaged" and "skipped", is that a problem? If so, any suggestions to fix it?

2. New Problem:
I got it to work fine on a couple of 30 minute programs. But on 2-3 hour movies it most often does not download the whole recording, most are only a few minutes of the show. It appears to be incomplete download since the .TIVO files are much smaller than the recording size shown in kmttg.

Settings:
...I have two Macs similarly configured and both have the same premature behavior.
...Neither Mac had a premature download problem with the old ffmpeg.
...I am using the same new ffmpeg for both kmttg and pyTivo. 
...I just updated to kmttg v1p0r.

I didn't have this problem before when I was losing the captions and the only change I can think of is the new ffmpeg and update to kmttg v1p0r.

Any ideas what might cause this premature truncation in downloads?


----------



## moyekj (Jan 24, 2006)

ffmpeg is not used at all for downloads so is not a factor for truncated .TiVo files.
Sounds like TS downloads (with "Download TiVo files in Transport Stream format" turned on) is the difference that matters. If you turn that off and try downloading same show that is currently being truncated you might find you get the whole show that way. If that's the case then it's a TiVo side bug where it's not serving up TS .TiVo files properly, and nothing can be done on client side to fix it.

However, note that PS .TiVo files do have captions, and if you decrypt them and then transfer back to your Roamio using pyTivo with ts=on setting then you will still get captions just fine on the TiVo. i.e. The difference is you'll have to decrypt them before transferring back to TiVo compared to the case of TS .TiVo files where you don't need to decrypt.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> Note that PS .TiVo files do have captions, and if you decrypt them and then transfer back to your Roamio using pyTivo with ts=on setting then you will still get captions just fine on the TiVo.


I find now that sometimes my kmttg downloaded .mpg and .mp4 files do retain the closed captions in the original show on the Tivo and sometimes they do not.

Is there a setting in kmttg that I'm inadvertently changing to disable download of captions present in the original?

CURRENT SETTINGS:
Is it true that unchecking TS reverts back to PS, Program Stream, so you're using one or the other?

I did uncheck TS and checked decrypt and then set pytivo Server Settings to include "ts = on".

Now I need a new .mpg file that contains captions from the original, so that's where I'm stuck at the moment.


----------



## moyekj (Jan 24, 2006)

OK, let's take a step back to review now:

1. With kmttg "Download TiVo files in Transport Stream format" setting off (default setting), when you download you get PS .TiVo files. With it turned on you get TS .TiVo files.

2. Because of a TiVo side bug, video files with captions have to be transferred back in TS container in order for captions to work on the TiVo.

3. TiVos can only play back video files with captions very specific requirements, so unless the video originated from a TiVo recording itself, chances are you will not get captions to work when transferring to a TiVo regardless of any pyTivo settings.

4. With pyTivo, when you transfer back .TiVo files using pull method (initiated from the TiVo), it will leave the .TiVo files as is regardless any settings. So if you transfer back PS .TiVo files you will lose captions. If you transfer back TS .TiVo files you will retain captions.

5. tivodecode which is used to decrypt .TiVo files only works properly for PS .TiVo downloads, so if you want to be able to decrypt .TiVo files you must download in PS .TiVo format.

6. Once you decrypt PS .TiVo file to .mpg it will be PS container , so if you transfer that back to TiVo it must be remuxed to TS container to preserve captions (because of bug listed in point #2). Setting pyTivo ts=on setting will make that happen automatically when transferring back, thus preserving captions.

7. If you do further processing of .mpg file such as encoding to H.264 in mp4 container, then in most cases even if the encoder can retain captions, it won't be in proper format that TiVo understands, so transferring that back to TiVo even with pyTivo with ts=on setting won't yield captions on the TiVo.

8. If you enable "QS Fix" task in kmttg and are using ProjectX for it (instead of VideoRedo), ProjectX does not retain captions, so the resulting .mpg file will lose captions. Using VideoRedo for "QS Fix" tasks does properly retain captions.


----------



## deac33 (Sep 28, 2013)

KEVIN:
I cant adequately express my appreciation for the time and effort you have expended to continue to guide me. Your last message is especially well written and full of useful information; I have saved it to my notes file. I will continue to search documentation also, but so far your help has been the absolute most help.

Q: Is your point 3,  unless the video originated from a TiVo recording itself, chances are you will not get captions to work . saying that home videos or other sources wont have captions when uploaded? If so, thats not a problem for me. All videos Im dealing with now are recorded on the Tivo and downloaded using kmttg. 

My goal is to download a Tivo Roamio Plus recording to a Mac disk file such that I can later upload the file back to the same Tivo box with the best quality video possible and preserving captions. It seems to be a fairly straight-forward goal, but still I have yet to achieve it.

Here is my summary of my understanding of my options to do this - I would greatly appreciate your correcting any errors:

1. = = = = Using PS , no QS Fix , mpg file, ts=on, Pull : 
Download in PS (TS not checked) & decrypt to mpg, then Pull the mpg file using pytivo with ts = on. I have failed to get this to work yet.

1a. I tried on several recordings using kmttg, however the resultant .mpg file did not have captions, so, 
of course, the uploaded file did not have captions either.

1b. I tried the download with cTivo set to MPEG-2 HD format, but it did not leave the mpg file that I could find. The mp4 had captions on the computer in Quicktime but not when uploaded.


2. = = = = Using TS , .TIVO file, VideoRedo: 
Buy and use VideoRedo instead of ProjectX, download with TS checked and which allows QS Fix. Wont the $50 version of VideoRedo work for keeping captions? (Still keeping this one open.)


Q: Are there any ways to Push from the computer to the Tivo that might be more successful, or does my inability to Pull a file mean that file wouldnt Push either?

Q: Is there a way to start kmttg on the decrypt step using a previously downloaded .TIVO file?

Thanks so much for your help.
-deac


----------



## moyekj (Jan 24, 2006)

deac33 said:


> 1. = = = = Using PS , no QS Fix , mpg file, ts=on, Pull :
> Download in PS (TS not checked) & decrypt to mpg, then Pull the mpg file using pytivo with "ts = on". I have failed to get this to work yet.


 What's failing in this sequence exactly? The pyTivo doesn't pull or that is working but you're not getting captions on the TiVo?



> 1a. I tried on several recordings using kmttg, however the resultant .mpg file did not have captions, so,
> of course, the uploaded file did not have captions either.


 If the recording on TiVo has captions then the decrypted .mpg file will have captions too. Not sure how you are determining there are no captions, but there almost certainly is. Use MediaInfo to examine the .mpg file and I think you'll see captions listed. You can even run MediaInfo on the .TiVo file itself to examine that for captions. Also if you run kmttg "captions" task on the .mpg file you'll most likely get captions out of it. Post the Mediainfo information on the .TiVo and .mpg file here if you are not sure.



> 2. = = = = Using TS , .TIVO file, VideoRedo:
> Buy and use VideoRedo instead of ProjectX, download with TS checked and which allows QS Fix. Won't the $50 version of VideoRedo work for keeping captions? (Still keeping this one open.)


 Yes, the $50 version can decrypt and run QS fix on either TS or PS .TiVo files and will preserve captions unlike projectX which strips them out.



> Q: Are there any ways to Push from the computer to the Tivo that might be more successful, or does my inability to Pull a file mean that file wouldn't Push either?


 Need more details here on what is failing exactly, along with pyTivo log with debug=true set.



> Q: Is there a way to start kmttg on the decrypt step using a previously downloaded .TIVO file?


 Of course, that's what the FILES tab is for. Switch to FILES tab and find the .TiVo file and add it to the table, then select the entry in the table and enable the "decrypt" task and press START.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> deac33 said:
> 
> 
> > 1. = = = = Using PS , no QS Fix , mpg file, ts=on, Pull : Fails
> ...


Great question, I always check that the Tivo recording has captions (old 1930s movies dont, o.w. most do). But recordings with captions on the Tivo dont download to mpg with captions. (BTW, cTivo mp4s have captions and I upload them, but with no captions, at least I don't yet know how to get them up with captions. I guessed from your comments that cTivo uses ProjectX, but I don't know.)



moyekj said:


> deac33 said:
> 
> 
> > e: 1a. I tried on several recordings [w/captions on Tivo] using kmttg, however the resultant .mpg file did not have captions, .
> ...


Excellent, Ill try MediaInfo and the other approaches you described soon and respond with what I find. Ive been looking at the mpg in QuickTime and uploading back to Tivo - both have no captions. Lately I just look at the Quicktime run of the mpg, but I'll find out about and use MediaInfo.

Also I'll find out how to do a pyTivo log with debug=true set and report on that.



moyekj said:


> deac33 said:
> 
> 
> > Q: Is there a way to start kmttg on the decrypt step using a previously downloaded .TIVO file?
> ...


Great news, Ill give that a go also.

Thanks again for your invaluable help. Im getting more hope that we can do this. Ha!

-deac


----------



## deac33 (Sep 28, 2013)

Kevin,

Quick questions: 

Which encoding format gives the highest quality output? 
I&#8217;ve been using ff_tivo_hd or hb_tivo_hd, which of these is better or does it matter? 
Will that choice affect captions?

Is this correct: Setting ts=on allows a Pull upload of a PS mpg file to retain captions, but ts=on will not retain captions on the same PS .TIVO file since Tivo requires TS container for .TIVO files? So ts=on saves captions for mpg, but not for .TIVO.

I&#8217;ve now turned on pytivo logging with debug=true. I was redirecting both stdout and stderr to it but it seemed verbose so now I&#8217;m just redirecting stderr. Is there a way to force a pytivo error to check that it captures it?

I got MediaInfo and am doing some tests on my .TIVO and mpg files with captions and will post them shortly.

thank you very much,
-deac


----------



## moyekj (Jan 24, 2006)

deac33 said:


> Kevin,
> Which encoding format gives the highest quality output?
> I've been using ff_tivo_hd or hb_tivo_hd, which of these is better or does it matter?
> Will that choice affect captions?


 The HB one will probably take longer but give you a smaller file. Neither one will preserve captions - most HB and ffmpeg profiles encoding to mp4 container will kill captions. VideoRedo preserves them even for mp4 container. What's your target playback device? If it's a TiVo I wouldn't encode these at all.



> Is this correct: Setting ts=on allows a Pull upload of a PS mpg file to retain captions, but ts=on will not retain captions on the same PS .TIVO file since Tivo requires TS container for .TIVO files? So ts=on saves captions for mpg, but not for .TIVO.


 Yes. You always have to return to TiVo in TS container to preserve captions. .TiVo files in either container don't get touched by pyTivo on pulls, so PS .TiVo files get pulled back as is and captions won't work. PS .mpg files are remuxed to TS container on pulls with ts=on resulting in working captions on the TiVo.



> I've now turned on pytivo logging with debug=true. I was redirecting both stdout and stderr to it but it seemed verbose so now I'm just redirecting stderr. Is there a way to force a pytivo error to check that it captures it?


 A useful log needs to contain all messages, but I believe most useful stuff is on stdout anyway for pyTivo so you shouldn't throw that out. When posting here even if it's long just enclose in "code" tags.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> If the recording on TiVo has captions then the decrypted .mpg file will have captions too. Not sure how you are determining there are no captions, but there almost certainly is. Use MediaInfo to examine the .mpg file and I think you'll see captions listed. You can even run MediaInfo on the .TiVo file itself to examine that for captions. Also if you run kmttg "captions" task on the .mpg file you'll most likely get captions out of it. Post the Mediainfo information on the .TiVo and .mpg file here if you are not sure.


How do I run "captions" task on a file? I used "FILES" to choose the .mpg file but don't see a choice in configuration or a button to check captions.

I downloaded MediaInfo as you suggested and have been experimenting with it. Here is an example for a movie that has great captions on the original recording on the Tivo but consistently has corrupt captions - garbled text with occasional correct words mixed in using VLC and no captions for Quicktime Player.

My kmttg settings are: PS (TS not checked), and "decrypt" checked. Should I add "metadata" or does that matter?

Thank you for looking at this MediaInfo output. Here is MediaInfo output for both files:

-------------------------------------------------------------------------------
momma mia .TiVo:
Text
ID : 224 (0xE0)-CC1
Format : EIA-608
Muxing mode : A/53 / DTVCC Transport
Muxing mode, more info : Muxed in Video #1
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)

-----------------------------------------
momma mia .mpg:
Text
ID : 224 (0xE0)-CC1
Format : EIA-608
Muxing mode : A/53 / DTVCC Transport
Muxing mode, more info : Muxed in Video #1
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)
-------------------------------------------------------------------------------


----------



## moyekj (Jan 24, 2006)

As expected, both the .TiVo and .mpg files have captions as shown by Mediainfo. You need to install "ccextractor" and enter path to it in kmttg in order for the "captions" task to be enabled in kmttg.

If you transfer back the .mpg file (not the .TiVo file) back to your TiVo with pyTivo with ts=on setting you should have functioning captions on the TiVo.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> As expected, both the .TiVo and .mpg files have captions as shown by Mediainfo. You need to install "ccextractor" and enter path to it in kmttg in order for the "captions" task to be enabled in kmttg.
> 
> If you transfer back the .mpg file (not the .TiVo file) back to your TiVo with pyTivo with ts=on setting you should have functioning captions on the TiVo.


I did install ccextractor and confirmed that the .srt file matches the VLC captions for the mpg file. Thanks for that.

ISSUE 1:
Some Tivo recordings with good captions on the Tivo download fine with good captions and some I have been unable to get good captions after many attempts. Here's an example of Mamma Mia .srt captions:
------------------
1
00:00:14,414 --> 00:00:26,058
_[ Won ma]_ 
_ I he ava eadrm _

2
00:00:37,320 --> 00:00:46,412
A sg ontohelp pe Wan 
_seu e th_

3
00:00:46,414 --> 00:00:50,499
a fa 
_ c tantufure_ 
------------------

Sometimes a clear word shows up and sometimes it's close, e.g., "I he ava eadrm" should be "I have a dream", but it's mostly too garbled to make out.

I have the same recording on two Tivos that play clearly and have tried to download to 3 different Macs with the same result. Using PS down and "decrypt" only.

Do you have any ideas on how to fix this?

ISSUE 2:
I've been unable to Pull the mpg using pytivo with ts=on. The name shows up on the Tivo and I select "Transfer this recording" but when I try to play it, either right away or hours later, instead of playing it just says "Delete now" or "Keep this recording" as if the upload was actually empty.

Again I've tried this with different Macs and both Tivos and with several different .mpg files. For fun I tried to upload the .Tivo file and it succeeded but without captions as expected.

Any ideas on this one?

thanks so much,
-deac


----------



## moyekj (Jan 24, 2006)

ISSUE 1. Could be a PS download which requires remuxing by the TiVo is garbling the captions in some cases. As an experiment, download a show that has the garbled problem in TS format, then use pyTivo to transfer that TS .TiVo file back to your TiVo. If the captions survive that without getting garbled then you have some options to explore.

ISSUE 2. Are you using a recent version of ffmpeg? Execute the following to get version information and post here:
ffmpeg -version


----------



## deac33 (Sep 28, 2013)

moyekj said:


> ISSUE 2. Are you using a recent version of ffmpeg? Execute the following to get version information and post here:
> ffmpeg -version


ffmpeg version 2.2.1 (From ffmpeg.org recently)



moyekj said:


> ISSUE 1. Could be a PS download which requires remuxing by the TiVo is garbling the captions in some cases. As an experiment, download a show that has the garbled problem in TS format, then use pyTivo to transfer that TS .TiVo file back to your TiVo. If the captions survive that without getting garbled then you have some options to explore.


YES! TS down (and no decrypt since I'm still using ProjectX) does upload retaining beautiful captions. This seems to have solved both Issues 1 and 2.

(Does leaving ts=on in pytivo hurt anything even though it was already in a TS container? It seems to have worked fine.)

So the .TIVO made with TS does work. What were some options you had in mind?


----------



## moyekj (Jan 24, 2006)

If you go back to my 1st post in this thread, the suggestion was to download TiVo files in TS format then use pyTivo to send it back:


moyekj said:


> All you need to do is when downloading from your TiVos with kmttg under config->Program Options make sure following is enabled:
> "Download TiVo files in Transport Stream format"
> 
> Then, using pyTivo if you transfer these TS .TiVo files back to your TiVo the captions will be intact.
> (Be advised though that by turning on above option you won't be able to decrypt the .TiVo files or do anything else with them other than transfer back to your TiVos).


The ts=on setting doesn't affect transfers for .TiVo files as they are left as is by pyTivo.

If the only goal is to archive shows for your TiVo then TS .TiVo files are the best option.

If you want to do something else with the downloaded files then the problem is TS .TiVo files can't be decrypted properly by tivodecode. VideoRedo (which runs on Windows only) with a minimal TiVo Desktop installation can handle decryption of these TS .TiVo files, so you'd need to find a way to be able to run Parallels or something with Windows to pursue that option.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> If the only goal is to archive shows for your TiVo then TS .TiVo files are the best option.
> 
> If you want to do something else with the downloaded files then the problem is TS .TiVo files can't be decrypted properly by tivodecode. VideoRedo (which runs on Windows only) with a minimal TiVo Desktop installation can handle decryption of these TS .TiVo files, so you'd need to find a way to be able to run Parallels or something with Windows to pursue that option.


I would really like to be able to download one file that can be both played on other devices with captions like a .mpg as well as be uploaded back to the Tivo still retaining the captions. (We have this second goal with the TS .TIVO file).

Will a VideoRedo decoded .mpg file serve both of these purposes?

Is there possibly another decoder that does run on a Mac that can generate such a .mpg file from a TS .TIVO file?

I don't know why some PS decoded .mpg files have good captions and some do not even when played on the computer.

I fear I may have to resort to trying VideoRedo in Parallels, but at least that may give me one .mpg file that can also be uploaded.

Thank you for your long suffering advice to stay with me on this exploratory journey. You are an absoute champ.


----------



## moyekj (Jan 24, 2006)

Decrypting TS .TiVo file to a TS .mpg file and then returning the that .mpg file to TiVo with ts=on setting should work. In that case, pyTivo will know it doesn't need to remux the file so ffmpeg won't be in the picture so you shouldn't have an issue related to ffmpeg.

The only available solutions for decrypting .TiVo files that I know of are:
1. tivodecode - a cross-platform C-based program that can only really handle PS .TiVo files properly. (There are versions of tivodecode with TS support, but from my experience and others it has problems and doesn't work without issues for most TS .TiVo files. It either doesn't work at all or if it does, generates a .mpg file that has improperly decrypted segments resulting in heavy pixelation during playback).
2. Windows DirectShow with at least partial TiVo Desktop installation (VideoRedo uses DirectShow to decrypt .TiVo files).

So for TS .TiVo files there's no reliable way to decrypt them on platforms other than Windows that I know of.


----------



## moyekj (Jan 24, 2006)

Of course you always have the option to:
1. Download shows as TS .TiVo files for sole purpose of transferring back to TiVos
2. Download same shows as PS .TiVo files to different location as above for purposes other than transferring back to TiVos.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> If you go back to my 1st post in this thread, the suggestion was to download TiVo files in TS format then use pyTivo to send it back:
> 
> If you want to do something else with the downloaded files then the problem is TS .TiVo files can't be decrypted properly by tivodecode. VideoRedo (which runs on Windows only) with a minimal TiVo Desktop installation can handle decryption of these TS .TiVo files, so you'd need to find a way to be able to run Parallels or something with Windows to pursue that option.


Thanks for this and your subsequent comments that suggest that to do what I want to do requires trying your suggestions here: 
... Using VideoRedo on Windows.

So I created a fresh Windows 7 virtual machine in Parallels and installed the 15 day trial version of the $50 VideoRedo Plus 3.1, the latest on their website. In kmttg Configure... under VideoRedo I entered the VRD path and checked both:
... Decrypt using VideoRedo instead of tivodecode
... Show VideoRedo encoding profiles

When I saved the configuration an error message showed up:
... Configuration saved to file: C:\kmttg\config.ini 
... Refreshing encoding profiles
... Encoding support requires VRD version 4.20.2.595 or later
... Version you are running is: 3.10.3.629 - Nov 9 2011
... Encountered problems obtaining encoding profiles from VideoRedo

Of the 3 versions of VideoRedo on their site, the only one at version 4.2 is the $96 TVSuite H.264. Since I can live with mpg and not requite mp4, I don't need the H.264, correct?

I unchecked "Show VideoRedo encoding profiles" and it saved okay. To just get uploadable mpg's with captions can I get along with the kmttg built in encoding profiles and not need the ones from VideoRedo?


----------



## moyekj (Jan 24, 2006)

If you are not encoding to H.264 you only need the $50 VideoRedo. Using VideoRedo kmttg will decrypt TS .TiVo files to a mpeg2 .ts file which you can subsequently transfer back to your TiVos with captions intact using pyTivo and/or further process for other devices using kmttg ffmpeg or handbrake encoding profiles.


----------



## wmcbrine (Aug 2, 2003)

DirectShow Dump will decode TS .TiVo files, you don't need VideoReDo.


----------



## moyekj (Jan 24, 2006)

I don't believe you can batch a bunch of jobs at once (or even queue up a bunch at once to process 1 at a time) using DirectShow Dump though right?


----------



## deac33 (Sep 28, 2013)

wmcbrine said:


> DirectShow Dump will decode TS .TiVo files, you don't need VideoReDo.





moyekj said:


> I don't believe you can batch a bunch of jobs at once (or even queue up a bunch at once to process 1 at a time) using DirectShow Dump though right?


Actually I can live with one at a time at this point if it just works to give an mpg with captions that can also be uploaded. I've been running into new problems with VideoRedo, so I'll put that on hold and I'll check out DirectShow Dump.

Many thanks to you both. You are both super rock stars.

-deac


----------



## moyekj (Jan 24, 2006)

FYI, it looks like there is a command line version available in this zip file which could be used to batch a bunch of jobs. I guess I should think about adding that to kmttg as an option for Windows users.


----------



## wmcbrine (Aug 2, 2003)

I've actually only used the command-line version of DSD, which probably says a lot about me.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> FYI, it looks like there is a command line version available in this zip file which could be used to batch a bunch of jobs. I guess I should think about adding that to kmttg as an option for Windows users.


Thanks much for the link, I hadn't been able to find a download of DirectShow Dump. I did use your link to get DSDCmd.exe which I tried in my Windows 7 virtual machine.

The only documentation I could find was the /? switch which shows -s and -t. I used a kmttg TS .TIVO file for -s and gave a directory for -t (and then a filename I made up as MM.mpg). There was no output in either case. Possible a permissions issue though I opened up the destination disk with chmod 777 -R and then checked that it's all writeable.

I assume that it wants a .TIVO as input and produces a .mpg?

Does it need other installations, I see references to Tivo Desktop?

thanks,
-deac


----------



## moyekj (Jan 24, 2006)

Yes, you need at least a partial install of TiVo Desktop with TiVoDirectShowFilter.dll installed and registered for DirectShow Dump or VideoRedo to work. A partial install is achieved by first installing TiVoDesktop and configuring it with MAK, etc. then uninstall and choose partial uninstall to keep the components needed to decrypt .TiVo files. TiVo is hiding TiVo Desktop these days, but the link is:
http://assets.tivo.com/assets/exe/tivotogo/TiVoDesktop2.8.3.exe

Note that I have incorporated DirectShow dump in beta version of kmttg. You can substitute your current kmttg.jar with the one in this zip file:
https://drive.google.com/file/d/0B0SMFC97ymdEWFZ1Y2tlSVg0eWc/edit?usp=sharing

Then if running kmttg on Windows run Help->Update tools to download updated tools package which contains DirectShow dump.
Then you can find an option to enable under config->Program Options: "Decrypt using DirectShow Dump instead of tivodecode"

The command line for DSDCmd.exe is quite simple:
"\path\to\DSDCmd.exe" "-s:\path\to\file.TiVo" "-t:\path\to\file.mpg"
(The quotes around everything is in case your paths and/or file names have spaces).

(Note that DSDCmd.exe is not setup to accept stdin instead of a file, so combined download/decrypt is not possible).


----------



## wmcbrine (Aug 2, 2003)

BTW, I use (and pyTivo uses) ".ts" for transport streams, and ".mpg" only for program streams. If you tell FFmpeg to make a new stream, and you don't specify the format, it will infer the same way, based on the filename.


----------



## moyekj (Jan 24, 2006)

wmcbrine said:


> BTW, I use (and pyTivo uses) ".ts" for transport streams, and ".mpg" only for program streams. If you tell FFmpeg to make a new stream, and you don't specify the format, it will infer the same way, based on the filename.


 For kmttg encodes the format is always explicitly defined, so the file extension name doesn't really matter. Having said that, when using VideoRedo for decrypt I do actually use .ts extension for TS .TiVo decrypt/qsfix instead of .mpg. (Also use .ts for H264 .TiVo decrypt).


----------



## deac33 (Sep 28, 2013)

moyekj said:


> Yes, you need at least a partial install of TiVo Desktop with TiVoDirectShowFilter.dll installed and registered for DirectShow Dump or VideoRedo to work. A partial install is achieved by first installing TiVoDesktop and configuring it with MAK, etc. then uninstall and choose partial uninstall to keep the components needed to decrypt .TiVo files. TiVo is hiding TiVo Desktop these days, but the link is:
> http://assets.tivo.com/assets/exe/tivotogo/TiVoDesktop2.8.3.exe
> 
> Note that I have incorporated DirectShow dump in beta version of kmttg. You can substitute your current kmttg.jar with the one in this zip file:
> ...


YES, IT WORKS!!!! End to end, I was able to download with TS to .TIVO and decrypt to .mpg which can be played in VLC with captions and the mpg uploaded back to the Tivo with the same beautiful captions as the original. THANK YOU SO MUCH FOR ADDING THIS AND FOR ALL OF THE ASSISTANCE!!!

This is a complete win for my goals. I am going to test comskip which is still running but I'll be quite content even if that doesn't work out. I'm about 4 days into the 15 day VideoRedo trial so it's available if needed.

Quicktime player does not show the captions, but that's okay. I have your "captions" task queued and will put the .srt file in the same folder as the .mpg just to test, but VLC is probably a better player anyway.

Testing so far has been successful on two recordings which failed every other approach, and their upload succeeded to two different Tivos.

After my other tests I'll copy my Parallels virtual machine and put it on a second Mac to go into production mode with the 2 Macs to download all of my recordings to date. Hopefully never again will I have to be concerned about filling the Tivo hard disk. And I can take the mpg's with me to play on the road.

It's so nice to have everything working in a single mpg so I don't have to save every .TIVO (though I probably will as a form of backup.  )

As always, your instructions were perfectly clear and complete. You have a great talent for not only imagining what needs to be done and writing the code but also for explaining it clearly. THANK YOU, THANK YOU, THANK YOU!!

I'll use this for a few days and clean up my notes and come back with any new results.

BUT MISSION ACCOMPLISHED for what I really wanted to do. Anything else is gravy.

Regads,
-deac


----------



## deac33 (Sep 28, 2013)

wmcbrine said:


> BTW, I use (and pyTivo uses) ".ts" for transport streams, and ".mpg" only for program streams. If you tell FFmpeg to make a new stream, and you don't specify the format, it will infer the same way, based on the filename.


William, thank you so much for joining the conversation. You and Kevin are doing wonderful things for humanity. Many thanks to you both!

-deac


----------



## deac33 (Sep 28, 2013)

deac33 said:


> YES, IT WORKS!!!! End to end, I was able to download with TS to .TIVO and decrypt to .mpg which can be played in VLC with captions and the mpg uploaded back to the Tivo with the same beautiful captions as the original. THANK YOU SO MUCH FOR ADDING THIS AND FOR ALL OF THE ASSISTANCE!!!
> Regards,
> -deac


First failure - a little more testing shows that the mpg's often only have about 70% of the captions of the original recording. The .TIVO files when uploaded so far do have perfect video, audio, and captions, so I will be keeping both .TIVO & mpg.

I suspect that the DirectShow Dump may have some. I understand that it's no longer being supported, that would be a shame because it seems powerful.

I just started a new test using VideoRedo to decrypt. I'll see how that comes out.


----------



## deac33 (Sep 28, 2013)

deac33 said:


> I just started a new test using VideoRedo to decrypt. I'll see how that comes out.


After a tiny sample it appears that VideoRedo only misses a few sentences in the captions and then picks up again. And the captions are much less noisy than with DirectShow Dump. Unless you can think of a way to get DirectShow Dump cleaned up, buying VideoRedo may be the best system.

Next to test AdCut.

-deac


----------



## wmcbrine (Aug 2, 2003)

Are you testing DSD and VRD on the _same_ videos?


----------



## moyekj (Jan 24, 2006)

deac33, it doesn't really make much sense. Both VideoRedo and DSD are using the EXACT same TiVoDirectShowFilter.dll from TiVo Desktop to decrypt the .TiVo file, so decrypting with either one should give same result. The difference if there is indeed one is that if you are using kmttg set to VideoRedo for decrypt, it runs decrypt + qsfix in one shot, in which case it's cleaning up glitches in timestamps compared to the .mpg produced by DSD. But if in kmttg flow you use DSD to decrypt and then run qsfix using VideoRedo you should get the same .mpg file as a result.


----------



## deac33 (Sep 28, 2013)

wmcbrine said:


> Are you testing DSD and VRD on the _same_ videos?


Good point. Yes, I have one video that fails (or not) in a given place making comparisons easy; overall I'm trying to keep my testing as consistent as possible, changing only one parameter at a time.



moyekj said:


> deac33, it doesn't really make much sense. Both VideoRedo and DSD are using the EXACT same TiVoDirectShowFilter.dll from TiVo Desktop to decrypt the .TiVo file, so decrypting with either one should give same result. The difference if there is indeed one is that if you are using kmttg set to VideoRedo for decrypt, it runs decrypt + qsfix in one shot, in which case it's cleaning up glitches in timestamps compared to the .mpg produced by DSD. But if in kmttg flow you use DSD to decrypt and then run qsfix using VideoRedo you should get the same .mpg file as a result.


Great info, thanks.

My plan is to just test just the decrypt step starting with a known good .TIVO file using FILES tab with settings 1. and 2. here:

BOTH: .... Windows 7, kmttg v1.1a_beta, Front Panel: only decrypt checked

1. DSD Test -- with Configure... settings as: 
Program Options tab: Decrypt using DirectShow Dump - CHECKED
VideorRedo tab: Decrypt using VideoRedo ... - - NOT CHECKED

2. VRD Test -- with Configure... settings as: 
Program Options tab: Decrypt using DirectShow Dump - NOT CHECKED
VideorRedo tab: Decrypt using VideoRedo ... - - CHECKED

--------
However, I'm now unable to get a new .TIVO file, I've tried 5 times to do just the download which has succeeded before but now I only get the first 3 minutes of the video. The original on the Tivo box still shows good all the way through. I have no clue why this new glitch came up.

BUT, as soon as some other jobs finish I'll reboot the virtual machine, reboot the Mac and try again.

Later,
-deac


----------



## moyekj (Jan 24, 2006)

May have to reboot the TiVo too.

Note on your test #2: Even with just "decrypt" checked when you configure kmttg to do decrypt with VRD, it will always run qsfix regardless if you check "qsfix" or not. That's why I suggested if you want to compare both, then for DSD test #1 you should check both "decrypt" and "qsfix".

BTW, there is no question that VRD is a superior solution to just having DSD, because qsfix fixes a lot of issues that just DSD alone won't. So you should at least spend the money for the $50 version if not the $100 version.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> May have to reboot the TiVo too.
> 
> Note on your test #2: Even with just "decrypt" checked when you configure kmttg to do decrypt with VRD, it will always run qsfix regardless if you check "qsfix" or not. That's why I suggested if you want to compare both, then for DSD test #1 you should check both "decrypt" and "qsfix".


Re-reading your earlier message I see that you did say that before. Thanks for repeating. 



moyekj said:


> BTW, there is no question that VRD is a superior solution to just having DSD, because qsfix fixes a lot of issues that just DSD alone won't. So you should at least spend the money for the $50 version if not the $100 version.


Is qsfix only available with VRD, I see the box on the front panel for QS Fix is enabled on my computer without VRD, but is it a no-op without VRD?

So using DSD plus QS Fix is essentially (or exactly) equivalent to using VRD decrypt?

I see in the VRD tab in Configuration the choice for using VRD AdScan, is it usually a bit better than com skip?

I am inclined to take your suggestion and buy VRD. It's great to have it in trial version for now.

(My downloads are working again and my testing is proceeding. Things are generally looking pretty good.)

thanks.
-deac


----------



## moyekj (Jan 24, 2006)

qsfix without VRD uses ProjectX which is free but nowhere as good as VRD and totally useless to you as it throws out captions completely.

I don't use commercial detection but Comskip is much more advanced than VRD AdScan and can be customized for different shows.

DSD + VRD qsfix is not identical to just VRD qsfix for decrypt but should yield same results, but if you have VRD anyway then it's pointless to use DSD and make a 2 step process instead of 1 step.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> qsfix without VRD uses ProjectX which is free but nowhere as good as VRD and totally useless to you as it throws out captions completely.


Right, I had in the back of my head something I had read about qsfix not working for what I want. But the VRD version of their qsfix does what I want. Good, another reason to buy it.



moyekj said:


> I don't use commercial detection but Comskip is much more advanced than VRD AdScan and can be customized for different shows.
> 
> DSD + VRD qsfix is not identical to just VRD qsfix for decrypt but should yield same results, but if you have VRD anyway then it's pointless to use DSD and make a 2 step process instead of 1 step.


So I'll settle in on using VRD decrypt. Didn't you say that with VRD chosen for decrypt that automatically uses its qsfix also, so would it be true that it doesn't matter whether the QS Fix box is checked?


----------



## moyekj (Jan 24, 2006)

deac33 said:


> So I'll settle in on using VRD decrypt. Didn't you say that with VRD chosen for decrypt that automatically uses its qsfix also, so would it be true that it doesn't matter whether the QS Fix box is checked?


 Yes, with VRD configured for decrypt, if you check "decrypt" and "qsfix" only a "qsfix" job will be run (since qsfix will decrypt anyway). Similarly if you check "qsfix" and not "decrypt" it will also just run "qsfix".

With VRD configured, the "AdCut" task also uses VRD, else projectX would be used for that thereby throwing away your captions again. VRD is just a great tool to have period.


----------



## deac33 (Sep 28, 2013)

Kevin,
Thanks again for holding my hand through all of this. I'm in production mode for downloading and fitting in some testing. Results are totally good enough to accomplish my mission, thank you so much!!!!!

Captions are holding up extremely well, mucho thanks for that. My hearing is just bad enough that it really helps me to fill in dialog and that would otherwise be missed. Yayyy for you and captions!!!!

I have a couple of sporadic problems that I'm still testing, but if you have any insights I'd love to hear them.

1. Some mpgs upload with "Description not available" when I press the Info button on the Tivo remote. It appears that the .TIVO upload does still have the info however. Not a big deal since IMDB has all of the info. 

At times the Program Details says "... container mpegts not compatible...", not sure what that means and if it's related to "Description not available". The video does upload and play fine, just no description.

2. Still sometimes get "Transferring prohibited by the copyright holder". I seem to recall that was usually a problem with old ffmpeg, but I have 2.2.1 which is the latest on ffmpeg.org and works with most recordings.

I just downloaded 2 new videos and in trying to upload them back both of them failed when attempting to upload .TIVO but the .mpg does upload (without Description, but it plays, and with captions). Strange, no?

thank you,
-deac


----------



## deac33 (Sep 28, 2013)

Kevin,

All is well here thanks to you. I am very indebted for all of the time and expertise you put into helping me get what I was looking for, thank you very much.

I'd love to invite you and a guest to a fine dinner at a restaurant of your choice. Or send you a gift card for same, any way we could arrange that. Please drop me a message at [email protected] so I can arrange it.

You are an excellent and patient instructor. Thank you for all of your help.

-deac


----------



## moyekj (Jan 24, 2006)

Glad it's working for you. Thanks for the kind words but no need for gifts/dinners. Enjoy.


----------



## deac33 (Sep 28, 2013)

moyekj said:


> Glad it's working for you. Thanks for the kind words but no need for gifts/dinners. Enjoy.


You are the best. We can close this thread from my viewpoint.

THANK YOU!!!! :up:


----------



## orinaccio (Sep 18, 2003)

I just wanted to pipe in here and add that I am presently dealing with the exact same corruption issues with captions on videos downloaded from Tivo units. 

The corruption to the captioning is exactly as you have described and oddly, it has grown much worse in the last couple of months. I have downloaded videos from Tivo units for years and have never seen them this bad. 

I am grateful that this thread seems to produce the answer as to the "why" - program .PS downloading seems to be the culprit. 

I read through this thread with great interest and my only disappointment is that the solution here involves a workflow that uses a windows emulator. 

I will keep this thread bookmarked but in the meantime I am keen to investigate further and see if there is *somehow* a solution to be had that can be 100% OS X (i.e. without resorting to windows emulation). 

My goals are similar to deac33 but differs in that I am not looking to return videos back to Tivo units, but rather to convert/encode them to videos with hardcoded subtitles for viewing on the various iOS and OS X devices around the house (Apple TV, iPhone, iPad, Macs, etc)

By the way, great thread guys - if there was a /bestof/ category on tivocommunity.com I'd nominate this


----------



## deac33 (Sep 28, 2013)

orinaccio said:


> ... see if there is *somehow* a solution to be had that can be 100% OS X (i.e. without resorting to windows emulation).
> 
> My goals are similar to deac33 but differs in that I am not looking to return videos back to Tivo units, but rather to convert/encode them to videos with hardcoded subtitles for viewing on the various iOS and OS X devices around the house (Apple TV, iPhone, iPad, Macs, etc)
> 
> By the way, great thread guys - if there was a /bestof/ category on tivocommunity.com I'd nominate this


I'm glad to hear someone else got as much value from this thread as I did. I don't know how much time Kevin spent on his responses, but they are great. We should put them in a book since they cover so much.

You're right that I now use Parallels to get the Windows solution, but I did try Mac only approaches for probably two months first. I have a vague recollection that it was my need to re-upload the files back to the TIVO that caused me to resort to the emulator approach.

I'm not sure, but I think I may have succeeded with cTivo to do what you want to do. I'll try it again and see. It seemed to be pretty easy to get mp4 files that run on the iOS devices, I'll have to re-do some to see if they retain captions. I think the TS container is only required to re-upload, but PS should be okay for mp4's on the Mac itself and iOS devices. I think. We'll have to check.

I just dug out an old mp4 file I'm almost sure was from cTivo, it does have captions on the Mac and I just made sure that putting it i iTunes transfers to my iPod and plays with captions. Please give that a try. Please post your results, good or bad.

With any luck at all Kevin will also see your post and be able to add clarity on your needs using kmttg. I wouldn't be surprised if that can be done using OS X only. (Hopefully).

BTW, I'm not sure what you mean by "hard coded captions". I just played my (I think) cTivo mp4 on my iPod and needed to click the caption button (looks like an old comic book dialog cloud). But it showed the captions well.

Good luck.
-deac33


----------



## orinaccio (Sep 18, 2003)

I was actually successful in working out a workflow that allows me to create videos using cTivo solely in a OS X environment. 

However, for the occasional video (which is occurring with increasing frequency) where the entire .SRT file is corrupted (the same corruption issue that you shared earlier in this thread), i have found no solution yet. For some reason this issue occurs with some channels but not others. I am not sure if this has anything to do with it, but the videos where I have no problems tend to be 30 minute shows, whereas, lately the movies I have downloaded have been badly corrupted. 

I determined that the corruption was occurring on the source .tivo file, so it could not be a decoding/ccextractor/encoding issue. There is something going on that is causing this while the video is being downloaded to the computer. 

I tried something just now and thought I'd share my results. 

I selected one movie that has the severe captioning corruption problem when downloaded (both on KMTTG and cTivo), and tried downloading it, this time as a .ts Transport Stream file. 

As expected, KMTTG could not decode it to .mpg, so I had just the raw .tivo file to work with. 

I ran the raw .tivo file through ccextractor with the -ts (to read transport stream captions) option, and it created a .srt file, which I examined and found was ALSO corrupted badly. I do wish I could decode the .tivo file and see if it might generate a .mpg with a less corrupted caption/subtitle track, but i think this is where it becomes necessary to have a windows emulator. 


I was hoping that a transport stream download could produce a perfect .srt file so that my theoretical workflow could be adjusted to : 

1) download the video as PS (program stream), for further encoding, and then
2) download the video again, this time, in TS (transport stream), for the sole purpose of generating uncorrupted .srt file, after which I could toss the TS video download. 
3) encode the video generated from #1, along with .srt file generated from #2

I know it sounds terribly inconvenient, but if that idea worked, I would have been happy as it is better than anything I have now for dealing with these videos with corrupted caption tracks.


----------



## deac33 (Sep 28, 2013)

orinaccio said:


> I determined that the corruption was occurring on the source .tivo file, so it could not be a decoding/ccextractor/encoding issue. There is something going on that is causing this while the video is being downloaded to the computer.
> 
> I tried something just now and thought I'd share my results. I selected one movie that has the severe captioning corruption problem when downloaded (both on KMTTG and cTivo), and tried downloading it, this time as a .ts Transport Stream file.
> 
> ...


Yes, now I recall that my caption corruption problems with cTivo were intermittent. I like your analysis that the corruption seems to happen more on certain channels and more on full length movies than the 30 minute shows. Also your attempts to download with TS to see if you could get a clean .TIVO file with captions was insightful. Good thinking, sorry it didn't work out.

It appears that you've narrowed it down to an inability to download a clean .TIVO file with captions using either cTivo or kmttg and either PS or TS.

I'm sure that you did check that the original recordings do play on the Tivo box with captions to show that the original is good, yes?

I think I'm at the limit of my knowledge, sorry. You've done more than I would have thought of. All I can suggest at this point is to see if Kevin picks up this thread again with ideas on how to better get a clean .TIVO file on MacOS kmttg. If not, then perhaps starting a new thread with many of the notes you've posted here might reinvigorate the discussion.

I'll keep watching and try to contribute if I actually can think of anything useful. 
Best,
-deac


----------



## orinaccio (Sep 18, 2003)

Perhaps I should create a new thread to track this corruption issue here, but I just wanted to respond to your questions and add a new discovery I just made that might be of interest to you:

1) Yes the Tivo recordings have captions that display properly without corruptions on TV. 

2) I have narrowed it down somewhat, yes. The corruption is occurring either during the transfer from the Tivo box to the computer although I do not know if it is the Tivo box's OS software that is causing it or the Tivo decoding process (i.e. perhaps an issue with how tivodecode processes the caption track). I do know that the same issue occurs both with KMTTG and CTIVO. 

3) I recently recorded a movie from the same channel that has consistently produced corruption issues (Cartoon Network), but this time I recorded the movie from Cartoon Network's HD channel. After I downloaded the HD movie, I see that it produced perfectly uncorrupted captions. 

Up until now, I have always recorded animated programming from analog channels with no issue - as I mentioned the badly corrupted captions are a recent phenomenon. I will now record from HD channels going forward, and see if that "fixes" the problem. It would be interesting to see if the corruption is due to some change to how analog caption tracks are stored and/or transmitted.


----------

