# No Streaming (MRS) Discovery thread?



## bradleys

I am surprised we haven't seen an MRS discovery thread similar to the iPad thread. I would love to see MRS from the server integrated into pytivo!

Just trying a friendly nudge... 

I do realize that this may not be as easy to discover.


----------



## puffdaddy

What were you looking to accomplish?

*Edit: saw that you wanted to have pytivo serve up MRS. I can't saw for sure, but drawing an analogy to the original MRV introduced back in sw 4.0+, the reversing done there (which ultimately lead to the creation of "tivoserver") required hacked units (both for the reversing process as well as to use tivoserver to transfer shows), so that wouldn't bode well. That said, it would be quite snazzy to have pytivo allow MRS access to its videos.*

After streaming a video, you can simply pull the drives from the MRS client and server units to look through their logs to find the request URL to initiate the stream.

IIRC, the URL was just a suffix permutation of the MRV URLs, but I can't recall if the stream setup first required a mutual certificate authentication or not. Any such authentication (if present) plus the stream encryption means there's little that can be done, unless you have a way to crack those.


----------



## bradleys

> mutual certificate authentication


I suspect this is correct. As I said, I suspect this is a lot easier said then done.

It was fun to follow the iPad discovery thread and I was surprised that we didn't see a similar thread evaluating some of the new functionality.

I suppose it might be happening in the shadows - or - it might just be a whole different animal and not as discoverable.

Either way - have I said that I love pyTiVo!


----------



## wmcbrine

Well, I don't have 20.2 yet.  Even then, I only have one Premiere.

Of course, on the extreme end of optimism, it's possible that implementing streaming will be as simple as adding "<StreamingPermission>Yes</StreamingPermission>" to the container XML.


----------



## bradleys

Well, I suppose I will just have to be patient! 

Thanks for all the work wmcbrine - it is appreciated.


----------



## gonzotek

wmcbrine said:


> Of course, on the extreme end of optimism, it's possible that implementing streaming will be as simple as adding "<StreamingPermission>Yes</StreamingPermission>" to the container XML.


Just tried that, after modifying your latest commit to add the tag, it's easy to see that pyTiVo is definitely sending the StreamingPermission tag in the right place (correctly CamelCased), but no change on the TiVo. I also noticed the TiVo sends some new stuff for QueryContainer:


Code:


<?xml version="1.0" encoding="utf-8"?>
<TiVoContainer xmlns="http://www.tivo.com/developer/calypso-protocol-1.6/">
<Details>
<ContentType>x-tivo-container/tivo-server</ContentType>
<SourceFormat>x-tivo-container/tivo-dvr</SourceFormat>
<Title>Mercury</Title>
<TotalItems>2</TotalItems>
</Details>
<ItemStart>0</ItemStart>
<ItemCount>2</ItemCount>
<Item>
<Details>
<ContentType>x-tivo-container/tivo-videos</ContentType>
<SourceFormat>x-tivo-container/tivo-dvr</SourceFormat>
<Title>Mercury</Title>
<UniqueId>Mercury</UniqueId>
</Details>
<Links>
<Content>
<Url>https://192.168.1.42:443/TiVoConnect?Command=QueryContainer&amp;Container=%2FNowPlaying</Url>
<ContentType>x-tivo-container/tivo-videos</ContentType>
</Content>
</Links>
</Item>
<Item>
<Details>
<ContentType>x-tivo-container/tivo-videostream</ContentType>
<SourceFormat>x-tivo-container/tivo-dvr</SourceFormat>
<Title>Mercury</Title>
<UniqueId>Mercury</UniqueId>
</Details>
<Links>
<Content>
<Url>https://192.168.1.42:443/TiVoConnect?Command=QueryContainer&amp;Container=%2FNowPlaying</Url>
<ContentType>x-tivo-container/tivo-videostream</ContentType>
</Content>
</Links>
</Item>
</TiVoContainer>

So I also tried hacking the root template to send an extra Item with a 'x-tivo-container/tivo-videostream' ContentType and with the UniqueId set to the same as the Title, but still no change. Everything looks correctly formatted, and pytivo still functions without complaint fine for push and pull transfers.

Haven't had the time to play more than that.


----------



## moyekj

As is the norm for TiVo these days the entire MRS communication between TiVos is SSL encrypted, so packet sniffing MRS communication didn't yield anything useful for me (unlike MRV which did show useful info in the past).


----------



## wmcbrine

Gotta get me some MITM for that.

The annoying thing for me right now is that I can't even get my Premiere to accept a transport-stream .TiVo file (from the same unit) via pyTivo. Same file via TiVo Desktop, no problem. I've got the TiVo to _request_ the file from pyTivo, but as soon as the file starts to transfer, the TiVo drops the connection. I've copied (almost) all the TD behavior I can see by hitting it with a browser, and clearly it's not enough.

But, I digress.


----------



## moyekj

Not sure how one would MITM easily with both sides of the communications being TiVos. Guess you would have to set Gateway for both TiVo network setups to go through a computer implementing MITM instead of a router.


----------



## wmcbrine

wmcbrine said:


> The annoying thing for me right now is that I can't even get my Premiere to accept a transport-stream .TiVo file (from the same unit) via pyTivo.


Figured it out.

80+ Mbps, here we come.


----------



## bradleys

Now the fun begins!


----------



## moyekj

wmcbrine said:


> Figured it out.
> 
> 80+ Mbps, here we come.


 Does mpeg2 Transport Stream container with H.264 video work as well?


----------



## wmcbrine

So far no, it transfers but I get a blank screen. This is with 14.9.


----------



## moyekj

wmcbrine said:


> So far no, it transfers but I get a blank screen. This is with 14.9.


 Will be interesting to see if 20.2 works any differently. With 20.2 I noticed that choosing secondary audio (SAP) from the Info screen for TV recordings now actually works. Also txporter recently discovered that mp4 with H.264 and multiple audio streams also allows you to switch audio streams. i.e. If you want to make a video that plays on a portable player that requires 2-channel AAC but also plays on a TiVo with the original 6-channel AC3 now it's possible to do so. (maybe that already worked before 14.9/20.2 but I'm not sure). It also looks like TiVo decoder actively looks for Dolby audio stream as first choice regardless if it's the 1st or 2nd audio stream.

The interesting thing about TS container with H.264 would be to eliminate the need for MOOV atom nonsense that mp4 container requires which would also open up possibility/option for pyTivo to transcode to H.264 instead of mpeg2.


----------



## gonzotek

moyekj said:


> Will be interesting to see if 20.2 works any differently. With 20.2 I noticed that choosing secondary audio (SAP) from the Info screen for TV recordings now actually works. Also txporter recently discovered that mp4 with H.264 and multiple audio streams also allows you to switch audio streams. i.e. If you want to make a video that plays on a portable player that requires 2-channel AAC but also plays on a TiVo with the original 6-channel AC3 now it's possible to do so. (maybe that already worked before 14.9/20.2 but I'm not sure). It also looks like TiVo decoder actively looks for Dolby audio stream as first choice regardless if it's the 1st or 2nd audio stream.
> 
> The interesting thing about TS container with H.264 would be to eliminate the need for MOOV atom nonsense that mp4 container requires which would also open up possibility/option for pyTivo to transcode to H.264 instead of mpeg2.


That's interesting about the multi-audio streams. Now I have to do some tests and see if I can come up with a handbrake or ffmpeg recipe that produces a file both the Roku and TiVo will accept and play.

If we get streaming enabled from pytivo, I'll be happy with mpeg2. It's faster/easier to encode when using general purpose cpus. I guess if I wanted to store content on the box, h.264 would still be preferable .


----------



## wmcbrine

My triumph was pitifully short-lived. 20.2 appears to throttle all connections, in and out, to around 20 Mbps. Perversely, MPEG-2 transfers are now faster than MP4, and program streams are the fastest of all (though not by much), turning everything on its head.

They also broke the transfer of many metadata items, even via real .TiVo files. Apart from all that, it doesn't seem much different (for pyTivo's purposes) from 14.9 -- same kinds of weirdness with transport streams, MP4 pulls still look like they're going to work but don't, etc.


----------



## wmcbrine

OK, I'm an idiot. Or, put it down to being tired... I was testing from my laptop, which was connected via G. That's where the throttle was.

Metadata is still broken, though.


----------



## wmcbrine

Naive attempt:



Code:


http://downloadurl&Format=x-tivo-container/tivo-videostream

does not work. I didn't expect it to, but I thought it was worth a try, since I just got streaming enabled. 

In case you're wondering, x-tivo-container/tivo-videostream comes from QueryFormats:



Code:


<TiVoFormats>
  <Format>
    <ContentType>video/x-tivo-mpeg</ContentType>
    <Description/>
  </Format>
  <Format>
    <ContentType>video/x-tivo-mpeg-ts</ContentType>
    <Description/>
  </Format>
  <Format>
    <ContentType>x-tivo-container/tivo-videostream</ContentType>
    <Description/>
  </Format>
  <Format>
    <ContentType>video/x-tivo-raw-tts</ContentType>
    <Description/>
  </Format>
</TiVoFormats>

It's the only new one. Interesting that it's "x-tivo-container" where the others are "video".


----------



## wmcbrine

There's a Zeroconf announcement of a service associated with streaming. It looks exactly like the "tivo-videos" service, except that it's called "tivo-videostream" (other fields are identical AFAICT).

I tried having pyTivo put out an announcement for tivo-videostream, while adding to QueryFormats and QueryContainer as outlined above. So far, no luck.


----------



## moyekj

I assume you already have <StreamingPermission>Yes</StreamingPermission> added to the XML container for pyTivo video shares right?


----------



## wmcbrine

That was included under "adding to ... QueryContainer as outlined above".


----------



## gonzotek

wmcbrine said:


> Gotta get me some MITM for that.


I don't know if this'll be useful to you for Tivo/pyTivo hacking or not, but thought I'd share on the off-chance it could be.
http://mitmproxy.org/


> mitmproxy is an SSL-capable man-in-the-middle HTTP proxy. It provides a console interface that allows traffic flows to be inspected and edited on the fly.
> 
> mitmdump is the command-line version of mitmproxy, with the same functionality but without the frills. Think tcpdump for HTTP.
> 
> Intercept and modify HTTP traffic on the fly
> Save HTTP conversations for later replay and analysis
> Replay both HTTP clients and servers
> Make scripted changes to HTTP traffic using Python
> SSL interception certs generated on the fly


----------



## moyekj

I gave MITM a serious go over the weekend. Ultimately I was not able to decrypt SSL traffic as intended but I post in details the steps I took in the hopes to encourage others to give it a shot and perhaps find a way to get it working. I think I'm close but perhaps need a different tool for SSL stripping.

NOTE: One of the most important things I learned is it's not necessary to have a hub to monitor your network traffic, since ARP poisoning can take care of making sure you can see all your switched activity from your PC.

NOTE: I also don't have linux installed at home so I used a linux installation on a thumb drive (4GB thumb drive in my case). The nice thing about that approach is if you currently have only Windows or Mac you can just install and run everything from a thumb drive without interfering at all with your Windows or Mac installation. It's better if you have a more permanent linux installation to play with, but steps below don't require that.

STEP 1 - INSTALL LINUX ON A THUMB DRIVE
(You can use Ubuntu if you want, but that means some hacking tools missing you would have to install. Backtrack 5 has most of the hacking tools needed already installed)
a. Download Backtrack Linux iso file from:


Code:


 http://www.backtrack-linux.org/downloads/

Release = BackTrack 5
WM Flavor = GNOME
Arch = 32 bit
Image = ISO
Download = Direct

(This gets you iso file BT5-GNOME-32.iso)

b. Download and install UNetbootin (to install iso file to thumb drive)


Code:


http://unetbootin.sourceforge.net/

c. Install iso on thumb drive.
1. Insert your thumb drive in USB slot
2. Start UNetbootin and select Diskimage = ISO and browse to the BT5-GNOME-32.iso
3. Leave Space used to preserve... as 0 unless you are using Ubuntu ISO instead
4. Make sure Type = USB drive and Drive is the correct thumb drive volume

STEP 2 - BOOT LINUX FROM THUMB DRIVE
a. Make sure thumb drive is in a USB slot and reboot/start your PC
b. During boot up go to your boot options screen. For my laptop running Windows I press Esc during bootup and then F9 to choose which device to boot from. Here I then choose the thumb drive

STEP 3 - GET LINUX UP AND RUNNING WITH NETWORKING ENABLED
a. At prompt type the following to start x-windows:
*startx*
b. Start networking as follows:
Applications-Internet-Wicd Network Manager
- If you have wired network then simply choose connect on 1st entry.
- If you have wireless network then choose "Properties" and in "Key" field enter you WPA2 password under (or whichever protection you are using). Then click on "Connect".

STEP 4 - DOWNLOAD AND INSTALL sslstrip
a. Start firefox: Applications-Internet-Firefox
b. Download sslstrip from:


Code:


 http://www.thoughtcrime.org/software/sslstrip/

c. Simply choose to save sslstrip-0.9.tar.gz to root folder
d. Unpack and install it:
*gunzip -c sslstrip-0.9.tar.gz | tar xvf -*
*cd sslstrip-0.9*
*python setup.py install*
*cd ..*

STEP 5 - COLLECT NECESSARY NETWORK INFORMATION
a. Determine the IP addresses of your Premieres on your home network. For me this is:
192.168.10.196 = LR Premiere (This is my MRS host)
192.168.10.199 = Premiere (This is my MRS client)

b. Determine name of your network interface device.
If using wired this is "eth0"
If using wireless this is "wlan0"

STEP 6a - SETUP AND RUN THE MITM ATTACK USING ettercap
a. Start a new shell by clicking on the terminal icon to the right of System
b. Install ettercap:
*apt-get install ettercap*
c. Edit the /etc/etter.conf file. I usually use "vi" as editor but you can use xedit graphical editor:
*xedit /etc/etter.conf*
d. Scroll down to section entitled "Linux" and then uncomment (remove the leading #) from the following 2 entries under "# if you use iptables"
redir_command_on = "iptables -t nat -A PREROUTING -i %iface -p tcp --dport %port -j REDIRECT --to-port %rport"
redir_command_off = "iptables -t nat -D PREROUTING -i %iface -p tcp --dport %port -j REDIRECT --to-port %rport"
e. Click on Save and then Quit
f. Now we are ready to start ettercap (Use *wlan0* or *eth0* interface according to wireless or wired, and replace the IP names with your Premiere IPs):
*ettercap -Tqdi wlan0 -w etter.pcap -M arp:remote /192.168.10.196/ /192.168.10.199/*
g. The traffic is now logged to *etter.pcap* file which can then be viewed using wireshark:
*wireshark etter.pcap*
NOTE: Stop ettercap by pressing 'q' in the ettercap window.

STEP 6b - MORE COMPLEX ALTERNATIVE TO 6a: SETUP AND RUN THE MITM ATTACK USING arpspoof & sslstrip
a. Enable ip forwarding
*echo 1 > /proc/sys/net/ipv4/ip_forward*

b. Use iptables to setup forwarding of port 443 traffic to port 8080:
*iptables -t nat -A PREROUTING -p tcp --destination-port 443 -j REDIRECT --to-port 8080*

c. arp poison traffic on your network so that it routes through your PC. Specifically I choose to poison my 2 Premieres:
1. Start a new shell by clicking on the terminal icon to the right of System
2. Execute following command in that shell (use *eth0* if wired, *wlan0* if wireless which is my case):
*arpspoof -i wlan0 -t 192.168.10.196 192.168.10.199*
(Obviously substitute the 2 IPs above for whatever your 2 Premiere IPs are)

d. Start sslstrip monitoring port 8080 and logging to file strip.log:
1. Start a new shell by clicking on the terminal icon to the right of System
2. Execute following command in that shell:
*sslstrip -a -k -l 8080 -w strip.log*
3. Now on your client Premiere browse to your other Premiere and push inside of show details of your host Premiere. That is enough to generate traffic on port 443 (without actually starting MRS).
4. If you want to monitor the strip.log file you can open another shell and execute the following:
*tail -f strip.log*

NOTES:
- Ideally if this worked properly at this point strip.log would contain unencrypted traffic.
- You can use the following iptables command to actually check if any traffic is being port forwarded:
*iptables -t nat -L -v*
(Even though for me this shows there is some traffic on port 443 sslstrip is not doing anything with it)
- Use Ctrl-C to stop arpspoof and/or ssltrip
- If instead of 443 I repeat the above with port 80 then I do see all the traffic using sslstrip (kind of interesting to see). In order to remove forwarding you simply use -D instead of -A in the iptables command. i.e. To remove the 443 forwarding:
*iptables -t nat -D PREROUTING -p tcp --destination-port 443 -j REDIRECT --to-port 8080*
Then to add port 80 forwarding instead use:
*iptables -t nat -A PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-port 8080*

VIEWING TRAFFIC WITH WIRESHARK
After you setup the arpspoof poisoning you can actually start wireshark to monitor network traffic as follows:
1. From command prompt start wireshark:
*wireshark*
2. Choose the appropriate network interface, in my case wlan0
3. Confirm there is a bunch of traffic generated between your 2 Premiere units when browsing remote Premiere and pushing inside of show details. Specifically you should look for SSLv3 and "Server Hello" which is the SSL handshaking that happens when you push into show details on remote Premiere.
4. NOTE: Click on the red 'x' to stop capturing network traffic.

In my case the arp poisoning is working fine since I can see all the traffic using wireshark. But unfortunately sslstrip is not doing what I expected which is to decrypt https traffic. I think this is probably because it was designed for web based ssl decryption (clients using web browsers) as opposed to SSL between 2 local LAN machines.

FINAL NOTE
If using Backtrack 5 thumb drive remember that because there is no perpetual file store defined as soon as you shutdown then any and all changes you made to the linux installation will be lost and need to be repeated. I made a script that does most of the above tasks for me so I don't have to repeat every time. I save the script as part of an email attachment so I can get to the script through Firefox while in Backtrack 5. i.e. A permanent linux install would be better if you have an available machine to do it or if you setup dual boot or VMWare instead.


----------



## tomhorsley

Don't know how many Live USB installations this works for, but with a Fedora USB,
you can make the usb stick with an "overlay" storage (whatever that means , but
the upshot is that you actually get a modifiable USB installation so you can
add packages, etc and they will be there the next time you plug in a boot from
the USB.


----------



## moyekj

tomhorsley said:


> Don't know how many Live USB installations this works for, but with a Fedora USB,
> you can make the usb stick with an "overlay" storage (whatever that means , but
> the upshot is that you actually get a modifiable USB installation so you can
> add packages, etc and they will be there the next time you plug in a boot from
> the USB.


 Yes, for Ubuntu you can do that as well (define persistence space for a thumb drive installation that survives reboots). I actually started with and have another USB stick with persistent Ubuntu on it. It was just easier to summarize with Backtrack 5 because it required minimal amount of extra package installations to get going. Pretty much any recent linux installation should work though. Note also that if it was just ARP poisoning necessary then something like Cain & Abel on Windows works fine for that task. However I didn't find much in the way of transparent proxy + ssl decryption tools available for Windows so became clear pretty quickly Linux was way to go, plus for me I like command line tools better anyway so Linux was a better fit. Actually I'm open to anything that will just work at this point - don't really care if it's Windows or Linux.

As a side note doing sniffing on port 80 actually provides a lot of insight on how HME applications (for Showcases menus and in TiVo My Shows screen). With help of some DNS spoofing it may be possible to get your own HME applications showing up on My Shows screen which would be interesting, but I don't want to be side-tracked at the moment.


----------



## moyekj

FYI I got ettercap running properly, but unfortunately it also doesn't seem to decrypt SSL properly for the TiVo communication either. I updated the instructions above indicating how to use ettercap which is actually simpler than arpspoof + sslstrip.


----------



## reneg

Interested novice here and certainly no security expert. Doesn't sslstrip present http to one side of the conversation? Is it possible that tivo(s) only accepts https (tls+http)? 

I'm guessing a sucessful trace will involve spoofing a cert and then feeding that cert into wireshark to decrypt the data stream.

Cheering you on from the sideline.


----------



## moyekj

reneg said:


> Interested novice here and certainly no security expert. Doesn't sslstrip present http to one side of the conversation? Is it possible that tivo(s) only accepts https (tls+http)?


 Well in my example, theoretically because of my iptables rule all port 443 traffic is redirected to port 8080 which sslstrip is then processing and passing off to the actual destination. The traffic is reaching my Premiere so it looks like sslstrip is just leaving everything alone and just passing traffic through. If it were actually downgrading to http as it's supposed to and the destination TiVo didn't like that then there would be a handshaking failure and I wouldn't be able to get to show details on the host TiVo. I think part of the complication here is that the host TiVo is using port 443 (192.168.10.196 in my example) while the client TiVo is using a different port (not a specific port but varies with each attempt).



> I'm guessing a sucessful trace will involve spoofing a cert and then feeding that cert into wireshark to decrypt the data stream.
> Cheering you on from the sideline.


 Well the problem is we don't have the TiVo certificate which is needed for this and if TiVo won't accept fake certificates then none of these MITM attacks are going to work from my limited understanding.

As an example I tried using similar techniques to see if I could sniff out my login and password for mail.yahoo.com but yahoo is smart enough to recognize it's being compromised and login wouldn't work while I had port redirection turned on.


----------



## reneg

moyekj said:


> Well in my example, theoretically because of my iptables rule all port 443 traffic is redirected to port 8080 which sslstrip is then processing and passing off to the actual destination. The traffic is reaching my Premiere so it looks like sslstrip is just leaving everything alone and just passing traffic through. If it were actually downgrading to http as it's supposed to and the destination TiVo didn't like that then there would be a handshaking failure and I wouldn't be able to get to show details on the host TiVo. I think part of the complication here is that the host TiVo is using port 443 (192.168.10.196 in my example) while the client TiVo is using a different port (not a specific port but varies with each attempt).
> 
> Well the problem is we don't have the TiVo certificate which is needed for this and if TiVo won't accept fake certificates then none of these MITM attacks are going to work from my limited understanding.
> 
> As an example I tried using similar techniques to see if I could sniff out my login and password for mail.yahoo.com but yahoo is smart enough to recognize it's being compromised and login wouldn't work while I had port redirection turned on.


I tried playing around with cain & abel and a self-signed cert generated by the program. Tivo wouldn't take it and then I was quickly over the my head.


----------



## moyekj

reneg said:


> I tried playing around with cain & abel and a self-signed cert generated by the program. Tivo wouldn't take it and then I was quickly over the my head.


 Can you elaborate? I don't remember any option for providing self-signed certificates in Cain & Abel but maybe I missed it.


----------



## ggieseke

Have you tried the .pem certificate files that come with Desktop?


----------



## reneg

moyekj said:


> Can you elaborate? I don't remember any option for providing self-signed certificates in Cain & Abel but maybe I missed it.


It's under the options, you can have the program generate a fake cert or load another cert. After the arp poisoning and trying to establish the session between the tivos, you can view the cert it generated. The program saves the cert in it's install directory in the cert directory.



ggieseke said:


> Have you tried the .pem certificate files that come with Desktop?


I don't use tivo desktop so I didn't even think of it, but I'll give it a try tonight.


----------



## moyekj

reneg said:


> It's under the options, you can have the program generate a fake cert or load another cert. After the arp poisoning and trying to establish the session between the tivos, you can view the cert it generated. The program saves the cert in it's install directory in the cert directory.


 Cain & Abel never generated those certificates automatically for me. However, using openssl and copying the information I saw from wireshark (the important part of the certificate is having the right CN name) I was able to generate self-signed certificates. I then added them to Cain\Certs folder with proper name and edited Cain\CERT.LST to add references to them. Then re-starting Cain I see the certificates in place through the GUI under APR-Cert and lo and behold I actually got some decrypted traffic captured under APR-HTTPS tab. I will need to find some more time to go through it in more detail but a brief analysis this morning showed some interesting stuff in how program details and video formats are obtained. However note that actual streaming happens on port 2191 on the host side and seemingly random port on the client side, so decrypting traffic on port 443 doesn't show the whole story and I couldn't get Cain working to decrypt traffic on non 443 ports.

NOTE: I also tried to use generated pem files (starting with -----BEGIN RSA PRIVATE KEY-----) in wireshark to see if I could get the captured traffic decrypted there but unfortunately that didn't work.


----------



## reneg

Progress. :up:


----------



## moyekj

The 443 port capture which is not really the exciting part related to MRS itself basically boils down to:
(This corresponds to pushing into program details for a show on a remote DVR - the actual start of MRS transfer is on port 2191 on host side so not captured here)


Code:


GET /TiVoConnect?Command=QueryItem&Url=http%3A%2F%2Fa%2Fb%3FContainer%3D%2FNowPlaying%26id%3D260546&SerialNum=74600019xxxxxxx 
GET /TiVoConnect?Command=QueryFormats&SourceFormat=video%2Fx-tivo-raw-tts 
GET /TiVoVideoDetails?id=260546&SerialNum=74600019xxxxxxx 
GET /TiVoConnect?Command=QueryFormats&SourceFormat=video%2Fx-tivo-mpeg

The relevant Cain captures for the above:
(NOTE: Looks like Cain is chopping off part of some responses)



Code:


===========================================
=== Cain's HTTPS sniffer generated file ===
===========================================

[Client-side-data]
GET /TiVoConnect?Command=QueryItem&Url=http%3A%2F%2Fa%2Fb%3FContainer%3D%2FNowPlaying%26id%3D260546&SerialNum=74600019xxxxxxx HTTP/1.1
Authorization: Digest username="tivo", realm="TiVo DVR", nonce="148CF5D20A514266", uri="/TiVoConnect?Command=QueryItem&Url=http%3A%2F%2Fa%2Fb%3FContainer%3D%2FNowPlaying%26id%3D260546&SerialNum=74600019xxxxxxx", qop=auth, nc=00000001, cnonce="tivo tcd", response="89b45a5166a4c41b749cd3cb15f78d22"
Cookie: sid=F7AF634943F80A44
Host: 192.168.10.196:443
User-Agent: TvHttpClient
tsn: 74600019xxxxxxx
Connection: close
TiVo_SW_VER: 20.2.x



[Server-side-data]
HTTP/1.1 200 File Follows
Server: tivo-httpd-1:20.2.x:746
Content-Type: text/xml; charset=UTF-8
Connection: close

<?xml version="1.0" encoding="utf-8"?><TiVoItem xmlns="http://www.tivo.com/developer/calypso-protocol-1.6/"><Item><Details><ContentType>video/x-tivo-raw-tts</ContentType><SourceFormat>video/x-tivo-raw-tts</SourceFormat><Title>White Collar</Title><SourceSize>5305794560</SourceSize><Duration>3599000</Duration><CaptureDate>0x4F445A30</CaptureDate><EpisodeTitle>Stealing Home</EpisodeTitle><Description>Neal joins the crew of a wealthy Yankees fan, tasked with stealing memorabilia from Yankee Stadium.</Description><SourceChannel>1022</SourceChannel><SourceStation>USAHD</SourceStation><HighDefinition>Yes</HighDefinition><ProgramId>EP011775780045</ProgramId><SeriesId>SH01177578</SeriesId><StreamingPermission>Yes</StreamingPermission><TvRating>4</TvRating><ShowingBits>5121</ShowingBits><SourceType>2</SourceType><IdGuideSource>58452</IdGuideSource></Details><Links><Content><Url>http://192.168.10.196:80/download/White%20Collar.TiVo?Container=%2FNowPlaying&amp;id=260546</Url><ContentType>video/x-tivo-raw-tts</ContentType></Content><TiVoVideoDetails><Url>https://192.168.10.196:443/TiVoVideoDetails?id=260546</Url><ContentType>text/xml</ContentType><AcceptsParams>No</AcceptsParams></TiVoVideoDetails></Links></Item></TiVoItem>

===========================================
=== Cain's HTTPS sniffer generated file ===
===========================================

[Client-side-data]
GET /TiVoConnect?Command=QueryFormats&SourceFormat=video%2Fx-tivo-raw-tts HTTP/1.1
Authorization: Digest username="tivo", realm="TiVo DVR", nonce="8523799FE1094C58", uri="/TiVoConnect?Command=QueryFormats&SourceFormat=video%2Fx-tivo-raw-tts", qop=auth, nc=00000001, cnonce="tivo tcd", response="e98fc85a6fbda726b3b406b99d93671b"
Cookie: sid=F7AF634943F80A44
Host: 192.168.10.196:443
User-Agent: TvHttpClient
tsn: 74600019xxxxxxx
Connection: close
TiVo_SW_VER: 20.2.x



[Server-side-data]
HTTP/1.1 200 File Follows
Server: tivo-httpd-1:20.2.x:746
Content-Type: text/xml; charset=UTF-8
Connection: close

<?xml version="1.0" encoding="utf-8"?><TiVoFormats xmlns="http://www.tivo.com/developer/calypso-protocol-1.6/"><Format><ContentType>video/x-tivo-mpeg</ContentType><Description/></Format><Format><ContentType>video/x-tivo-mpeg-ts</ContentType><Description/></Format><Format><ContentType>x-tivo-container/tivo-videostream</ContentType><Description/></Format><Format><ContentType>video/x-tivo-raw-tts</ContentType><Description/></Format></TiVoFormats>

===========================================
=== Cain's HTTPS sniffer generated file ===
===========================================

[Client-side-data]
GET /TiVoVideoDetails?id=260546&SerialNum=74600019xxxxxxx HTTP/1.1
Authorization: Digest username="tivo", realm="TiVo DVR", nonce="CF888C4ADD5AF0DC", uri="/TiVoVideoDetails?id=260546&SerialNum=74600019xxxxxxx", qop=auth, nc=00000001, cnonce="tivo tcd", response="762f2db37d452d9a2ac930ee55b6e2db"
Cookie: sid=F7AF634943F80A44
Host: 192.168.10.196:443
User-Agent: TvHttpClient
tsn: 74600019xxxxxxx
Connection: close
TiVo_SW_VER: 20.2.x



[Server-side-data]
HTTP/1.1 200 File Follows
Server: tivo-httpd-1:20.2.x:746
Content-Type: text/xml; charset=UTF-8
Content-Length: 4752
Connection: close

<?xml version="1.0" encoding="utf-8"?><TvBusMarshalledStruct:TvBusEnvelope xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" xmlns:TvBusMarshalledStruct="http://tivo.com/developer/xml/idl/TvBusMarshalledStruct" xmlns:TvPgdRecording="http://tivo.com/developer/xml/idl/TvPgdRecording" xmlns:TvBusDuration="http://tivo.com/developer/xml/idl/TvBusDuration" xmlns:TvPgdShowing="http://tivo.com/developer/xml/idl/TvPgdShowing" xmlns:TvDbShowingBit="http://tivo.com/developer/xml/idl/TvDbShowingBit" xmlns:TvBusDateTime="http://tivo.com/developer/xml/idl/TvBusDateTime" xmlns:TvPgdProgram="http://tivo.com/developer/xml/idl/TvPgdProgram" xmlns:TvDbColorCode="http://tivo.com/developer/xml/idl/TvDbColorCode" xmlns:TvPgdSeries="http://tivo.com/developer/xml/idl/TvPgdSeries" xmlns:TvDbShowType="http://tivo.com/developer/xml/idl/TvDbShowType" xmlns:TvDbTvRating="http://tivo.com/developer/xml/idl/TvDbTvRating" xmlns:TvPgdChannel="http://tivo.com/developer/xml/idl/TvPgdChannel" xmlns:TvDbBitstreamFormat="http://tivo.com/developer/xml/idl/TvDbBitstreamFormat" xs:schemaLocation="http://tivo.com/developer/xml/idl/TvBusMarshalledStruct TvBusMarshalledStruct.xsd http://tivo.com/developer/xml/idl/TvPgdRecording TvPgdRecording.xsd http://tivo.com/developer/xml/idl/TvBusDuration TvBusDuration.xsd http://tivo.com/developer/xml/idl/TvPgdShowing TvPgdShowing.xsd http://tivo.com/developer/xml/idl/TvDbShowingBit TvDbShowingBit.xsd http://tivo.com/developer/xml/idl/TvBusDateTime TvBusDateTime.xsd http://tivo.com/developer/xml/idl/TvPgdProgram TvPgdProgram.xsd http://tivo.com/developer/xml/idl/TvDbColorCode TvDbColorCode.xsd http://tivo.com/developer/xml/idl/TvPgdSeries TvPgdSeries.xsd http://tivo.com/developer/xml/idl/TvDbShowType TvDbShowType.xsd http://tivo.com/developer/xml/idl/TvDbTvRating TvDbTvRating.xsd http://tivo.com/developer/xml/idl/TvPgdChannel TvPgdChannel.xsd http://tivo.com/deve

[Server-side-data]
loper/xml/idl/TvDbBitstreamFormat TvDbBitstreamFormat.xsd" xs:type="TvPgdRecording:TvPgdRecording"><recordedDuration>PT59M59S</recordedDuration><vActualShowing><element><showingBits value="5121"/><time>2012-02-22T03:00:00Z</time><duration>PT1H</duration><program><vActor><element>Bomer|Matt</element><element>DeKay|Tim</element><element>Thiessen|Tiffani</element><element>Garson|Willie</element><element>Thomason|Marsha</element><element>Atkins|Sharif</element></vActor><vAdvisory/><showingBits value="0"/><vChoreographer/><colorCode value="4">COLOR</colorCode><description>Neal joins the crew of a wealthy Yankees fan, tasked with stealing memorabilia from Yankee Stadium.</description><vDirector><element>DeKay|Tim</element></vDirector><episodeTitle>Stealing Home</episodeTitle><vExecProducer><element>Eastin|Jeff</element></vExecProducer><vProgramGenre><element>Crime Drama</element></vProgramGenre><vGuestStar><element>Carroll|Diahann</element><element>Ozsan|Hal</element></vGuestStar><vHost/><isEpisode>true</isEpisode><originalAirDate>2012-02-21T00:00:00Z</originalAirDate><vProducer/><series><isEpisodic>true</isEpisodic><vSeriesGenre/><seriesTitle>White Collar</seriesTitle></series><showType value="5">SERIES</showType><title>White Collar</title><vWriter/></program><tvRating value="4">PG</tvRating></element></vActualShowing><vBookmark/><showing><showingBits value="5121"/><time>2012-02-22T03:00:00Z</time><duration>PT1H</duration><program><vActor><element>Bomer|Matt</element><element>DeKay|Tim</element><element>Thiessen|Tiffani</element><element>Garson|Willie</element><element>Thomason|Marsha</element><element>Atkins|Sharif</element></vActor><vAdvisory/><showingBits value="0"/><vChoreographer/><colorCode value="4">COLOR</colorCode><description>Neal joins the crew of a wealthy Yankees fan, tasked with stealing memorabilia from Yankee Stadium.</description><vDirector><element>DeKay|Tim</element></vDirector><episodeTitle>Stealing Home</episodeTitle><vExecProducer><element>Eastin|Jeff</element></vExecProducer><vProgramGenre><elem
[Server-side-data]
ent>Crime Drama</element></vProgramGenre><vGuestStar><element>Carroll|Diahann</element><element>Ozsan|Hal</element></vGuestStar><vHost/><isEpisode>true</isEpisode><originalAirDate>2012-02-21T00:00:00Z</originalAirDate><vProducer/><series><isEpisodic>true</isEpisodic><vSeriesGenre/><seriesTitle>White Collar</seriesTitle></series><showType value="5">SERIES</showType><title>White Collar</title><vWriter/></program><channel><callsign/></channel><tvRating value="4">PG</tvRating></showing><startTime>2012-02-22T02:59:58Z</startTime><stopTime>2012-02-22T04:00:00Z</stopTime><bitstreamFormat><vFormat><element><vByte><base64>EjQAAwABAjoBywxXAAAADwAAAAQAAAACAAAAAwAAAA==</base64></vByte></element></vFormat></bitstreamFormat><expirationTime>2012-02-24T03:00:00Z</expirationTime></TvBusMarshalledStruct:TvBusEnvelope>

===========================================
=== Cain's HTTPS sniffer generated file ===
===========================================

[Client-side-data]
GET /TiVoConnect?Command=QueryFormats&SourceFormat=video%2Fx-tivo-mpeg HTTP/1.1
Authorization: Digest username="tivo", realm="TiVo DVR", nonce="3EB8E06DE539D9F2", uri="/TiVoConnect?Command=QueryFormats&SourceFormat=video%2Fx-tivo-mpeg", qop=auth, nc=00000001, cnonce="tivo tcd", response="32a193861f132d84776b451653fd2f01"
Cookie: sid=F7AF634943F80A44
Host: 192.168.10.196:443
User-Agent: TvHttpClient
tsn: 74600019xxxxxxx
Connection: close
TiVo_SW_VER: 20.2.x



[Server-side-data]
HTTP/1.1 200 File Follows
Server: tivo-httpd-1:20.2.x:746
Content-Type: text/xml; charset=UTF-8
Connection: close

<?xml version="1.0" encoding="utf-8"?><TiVoFormats xmlns="http://www.tivo.com/developer/calypso-protocol-1.6/"><Format><ContentType>video/x-tivo-mpeg</ContentType><Description/></Format><Format><ContentType>video/x-tivo-raw-tts</ContentType><Description/></Format></TiVoFormats>


----------



## wmcbrine

Nothing new there, AFAICT. It's nice that you can intercept it, though.


----------



## moyekj

wmcbrine said:


> Nothing new there, AFAICT. It's nice that you can intercept it, though.


Yeah, all the good stuff seems to be on ports I can't decrypt through Cain. Still hoping there's a way to get my self-generated certificates working with Wireshark which would be much more interesting.


----------



## bradleys

Probably not helping at all - you seem to understand these technologies significantly better than I do... But this wiki talks about creating and applying a Security certificate using wireshark.

http://wiki.wireshark.org/SSL

Good luck!


----------



## moyekj

bradleys said:


> Probably not helping at all - you seem to understand these technologies significantly better than I do... But this wiki talks about creating and applying a Security certificate using wireshark.
> 
> http://wiki.wireshark.org/SSL
> 
> Good luck!


 Yes, already tried that with Wireshark last night with the private key portion of my certificate which worked with Cain as follows:


Code:


192.168.10.196,443,http,c:\home\lr_premiere.pem

Where lr_premiere.pem is only the private key portion as stated at web site (the Cain one has both certificate & private key sections).

It's almost as if Wireshark is not even trying to use however since my specified debug file which accompanies above setting is always empty.


----------



## bradleys

I knew I read this somewhere... I will chat with one of my techs tomorrow if this does not help out because I know it is possible...

It describes how to use both the self signed certificate as well as a private key.

http://wirewatcher.wordpress.com/20...raffic-with-wireshark-and-ways-to-prevent-it/


----------



## moyekj

Well, Wireshark is actually spitting out stuff to ssl decrypt debug file and there are a bunch of errors such as:

ssl_decrypt_pre_master_secret wrong pre_master_secret length (128, expected 48)
dissect_ssl3_handshake can't decrypt pre master secret
ssl_generate_keyring_material not enough data to generate key (0x17 required 0x37 or 0x57)
decrypt_ssl3_record: using server decoder
decrypt_ssl3_record: no decoder available
etc.

So obviously it's trying and failing where Cain had no problem...


----------



## bradleys

Have you tried fiddler?

http://www.fiddler2.com/fiddler2/


----------



## moyekj

bradleys said:


> Have you tried fiddler?
> 
> http://www.fiddler2.com/fiddler2/


 No. Since it's a proxy you'd have to find a way to route TiVos through the proxy. It's more of a tool for web browsers or other applications that support proxies directly.


----------



## reneg

moyekj said:


> Well, Wireshark is actually spitting out stuff to ssl decrypt debug file and there are a bunch of errors such as:
> 
> ssl_decrypt_pre_master_secret wrong pre_master_secret length (128, expected 48)
> dissect_ssl3_handshake can't decrypt pre master secret
> ssl_generate_keyring_material not enough data to generate key (0x17 required 0x37 or 0x57)
> decrypt_ssl3_record: using server decoder
> decrypt_ssl3_record: no decoder available
> etc.
> 
> So obviously it's trying and failing where Cain had no problem...


Could the "Private Key Format" Section listed here be an issue?


----------



## moyekj

reneg said:


> Could the "Private Key Format" Section listed here be an issue?


 My private key section does begin with "-----BEGIN RSA PRIVATE KEY-----" so I think it is a decrypted PEM key. Note that the keys themselves are getting loaded find according to the debug log so I don't think it's an issue with the format of the keys. Doing some Googling seemed to imply though that 
"ssl_decrypt_pre_master_secret wrong pre_master_secret length" type errors are because the client side key is being used instead of server side key. Obviously I don't have access to server side key - I'm using a self-generated client side key. So if that indeed is what is required then Wireshark approach is not going to work.


----------



## moyekj

After some more Googling it looks like Wireshark does require the SERVER side private key to decrypt SSL, so for our purposes that will never be available. The reason Cain works is that it is using MITM and a self-signed certificate (which I generated) that it presents to the TiVo for handshaking so in that case our self-generated private key is sufficient to decrypt the communication. So I think I'll abandon any attempts to decrypt via Wireshark.


----------



## reneg

Just thinking out loud here:
1) If I were a tivo developer, would it make sense to do command and control over SSL between the Tivos?
2) Also, would it make sense to stream transport stream format over another set of ports without encrypting it because the transport stream format is already encrypted on the Tivo?

I've got shows archiving tonight from my Tivo, but I think I need to download a transport stream format file from the Tivo and grab another trace to see if this even makes sense.


----------



## moyekj

reneg said:


> Just thinking out loud here:
> 1) If I were a tivo developer, would it make sense to do command and control over SSL between the Tivos?
> 2) Also, would it make sense to stream transport stream format over another set of ports without encrypting it because the transport stream format is already encrypted on the Tivo?


 They are using SSL as can be seen in my post above for getting show details, etc. I think the MRS stream format is most likely same as MRV. I suspect the MRS communication is more like the RPC (iPad) protocol which is also SSL encrypted. I also suspect the remote delete option is same as the RPC (iPad) protocol.



> I've got shows archiving tonight from my Tivo, but I think I need to download a transport stream format file from the Tivo and grab another trace to see if this even makes sense.


 FYI, if you want download the format used by MRV (and presumably MRS) it's actually with the following modifier added to the TTG url:
&Format=video/x-tivo-raw-tts

That's different than "TS" downloads which are:
&Format=video/x-tivo-mpeg-ts
OR "PS" TTG downloads which are:
&Format=video/x-tivo-mpeg


----------



## ggieseke

Would this utility I wrote help? The usage is TiVoGetCert TiVo [MAK].

TiVo is the IP address of the DVR. MAK is optional if you have Desktop installed (it will read it from the registry).


----------



## moyekj

ggieseke said:


> Would this utility I wrote help? The usage is TiVoGetCert TiVo [MAK].
> 
> TiVo is the IP address of the DVR. MAK is optional if you have Desktop installed (it will read it from the registry).


 Tried it as follows:
1. From Details tab used Copy To File to save as DER format as file tivo.crt
2. Used openssl to convert DER to PEM:
openssl x509 in tivo.crt inform DER out tivo.pem outform PEM
Result is only the certificate (-----BEGIN CERTIFICATE----- ...). Wireshark needs the private key (-----BEGIN RSA PRIVATE KEY----- ...).


----------



## ggieseke

Bummer.


----------



## moyekj

It's possible that initiating MRS is done using port 2191 tvbus messaging which is discussed in deal database forums (search for tivoserver 2191) in MRV discovery thread.

From my packet captures SSLv3 is only happening on port 443 of host side and I couldn't find any other SSL handshaking on other ports. Port 2191 does show up extensively in the captures. So it may be useful to go through the old tivoserver code to pursue this further but I'm bowing out of this for now.


----------



## bradleys

Looks like you have hit a brick wall at this point wit the tools you have available ... Your work is appreciated and I hope someone else will feel motivated enough to step in.

Thanks!


----------



## chrispitude

Reviving an old thread - gosh I hope someone figures this out some day. You guys are all awesome for putting the hours into this to get this far.


----------

