# Discovering the iPad interface



## wmcbrine

Let's see how much I can do without actually having an iPad...

Nmap against a Series 3:

PORT STATE SERVICE
80/tcp open http
443/tcp open https
2190/tcp open unknown
2191/tcp open unknown
3791/tcp open unknown
31339/tcp open unknown

Nmap against a Premiere running 14.7RC7:

PORT STATE SERVICE
80/tcp open http
443/tcp open https
1390/tcp open iclpv-sc
1393/tcp open iclpv-nls
1400/tcp open cadkey-tablet
1410/tcp open hiq
1413/tcp open innosys-acl
2190/tcp open unknown
2191/tcp open unknown
8430/tcp open unknown
31339/tcp open unknown

FWIW, port 8430 disappeared when I repeated the scan.

Avahi Zeroconf Browser lists some new services, Premiere only:

_tivo-mindrpc._tcp
_tivo-device._tcp

These are in addition to the existing _tivo-remote._tcp, etc. that other TiVos report. According to this, the "mindrpc" service runs on port 1413. ("tivo-device" just shows as port 80.)

Perhaps the mindrpc service works like mind.tivo.com?


----------



## richwiss

Interesting.. I'd love to play with that if I only had a TiVo premiere...


----------



## innocentfreak

wmcbrine said:


> Avahi Zeroconf Browser lists some new services, Premiere only:
> 
> _tivo-mindrpc._tcp
> _tivo-device._tcp
> 
> These are in addition to the existing _tivo-remote._tcp, etc. that other TiVos report. According to this, the "mindrpc" service runs on port 1413. ("tivo-device" just shows as port 80.)
> 
> Perhaps the mindrpc service works like mind.tivo.com?


Well with the iPad app, you can either use it locally with full functionality or remotely with limited functionality so maybe the mindrpc is for commands while outside your local network.


----------



## tomhorsley

What we really need is something with something like a dd-wrt router they can run tcpdump on to monitor all the traffic between the ipad and the tivo. If it isn't encrypted, it should be easy to start deducing the interface (maybe .


----------



## windracer

I tried messing around with Wireshark to see if I could capture traffic between my iPad and the Premiere, but don't really know what I'm doing/looking for. If anyone can give me some basic guidance on how to set this up, I'll see if I can contribute.


----------



## orangeboy

windracer said:


> I tried messing around with Wireshark to see if I could capture traffic between my iPad and the Premiere, but don't really know what I'm doing/looking for. If anyone can give me some basic guidance on how to set this up, I'll see if I can contribute.


You could setup a filter, something like this:



Code:


(ip.src == 192.168.0.60 and ip.dst == 192.168.0.150) or (ip.dst == 192.168.0.60 and ip.src == 192.168.0.150)

Adjust ip addresses accordingly...

Edit: I seem to recall that Wireshark may not capture the traffic of the different devices unless the NIC on the computer running Wireshark is set to promiscuous mode, and switches are not in between the computer and the target devices...


----------



## davezatz

I'm super busy at the moment (life), but if I can find some time tomorrow I'll set up my promiscuous dumb hub, borrow the wife's laptop (my MBA lacks Ethernet), and see what I can sniff off the wire. No promises though, but it's on my mind (and requested). 

I assume communication to/from TiVo Inc is locked down in some way. But at the very least we should be able to get at some things locally like the scrub bar.


----------



## 11274

davezatz said:


> I'm super busy at the moment (life), but if I can find some time tomorrow I'll set up my promiscuous dumb hub,


I've been looking for a promiscuous dumb hub. Where did you get it?


----------



## davezatz

km said:


> I've been looking for a promiscuous dumb hub. Where did you get it?


Sigh, no work is going to happen today. I grabbed the hub off ebay a couple years back specifically for Ethereal/Wireshark projects. The first one I purchased didn't spew packets all over the place, so I grabbed a second which worked fine. Not sure what model I have, I'm still digging out after the move last month. But it's a small, plain, black no-name brand thing. Probably ran only $10 or $15.

UPDATE: Went digging through boxes, and this is what I have. I'm pretty sure this is the promiscuous one. Dynex DX-EHB4


----------



## Worf

You know, another option instead of the hub (for those with friendly IT departments or good hunting skills) is a smart or managed switch. Used managed switches from Cisco and HP aren't too hard to find (hint: EOL'd ones that work are perfectly fine) and they're often cheap, and brand new smart switches from Netgear and the like aren't out there either.

These switches normally have a monitor feature which lets you tap into another port's traffic, so you get all the benefits of a hub, but it switches and you get to see just the port traffic you want.

And yes, I've had to sniff traffic at GigE speeds. It helps that I only needed a GigE link, and not sniffing at full bandwidth.


----------



## davezatz

Eh, it's small and cheap and my home networking projects/needs are infrequent. I pulled up my ebay receipt - $9.88. Port mirroring is nice, but it usually comes in a larger footprint (if my work gear is any indication). Also, the way I hang a secondary network off my primary one with the dumb hub mostly gives me mostly the same benefits of monitoring a single port.


----------



## tomm1079

Any further update on this? Would be nice if someone would be able to make a Tivo iPad like app


----------



## 11274

Just to conclude the monitoring port tangent, there is a web page with a list of real hubs

http://www.miarec.com/knowledge/hubs

as well as switches with port mirroing.

http://www.miarec.com/knowledge/switches-port-mirroring

The latter includes a small gigabit netgear switch which lists for $70 and goes for about half that on ebay.


----------



## moyekj

My guess is that the interface is SSL encrypted (i.e. uses https), so sniffing the network communication will not yield anything readable. Otherwise I would have expected someone to post about it by now.


----------



## innocentfreak

I would think it might also be more the fact that the people who would know what they are doing don't own iPads. 

It is disappointing to see that no one has made any progress though since I would love to see some of this functionality.


----------



## orangeboy

Yeah, I'm not gonna run out and get an iPad just for the TiVo app. I did however download the app through iTunes (something I did get just for the TiVo app!), and extracted the files to see what I could see. Which amounted to really nothin' of use to me...


----------



## innocentfreak

I think Best Buy got rid of their restocking fees so you could always buy and return. 

I would do it, but I would then need a good wifi router to sniff the packets along with a guide for dummies on what I was doing.


----------



## davezatz

moyekj said:


> My guess is that the interface is SSL encrypted (i.e. uses https), so sniffing the network communication will not yield anything readable. Otherwise I would have expected someone to post about it by now.


Between here and TiVo, Inc I agree it's likely. Between the iPad and my TiVo, I'm not so sure given the previously discussed Creston controls. Sorry I didn't get it done and the loaner iPad is packed up for return.


----------



## windracer

I tried messing around with WireShark, but I just don't have the necessary equipment to analyze the traffic (or the wherewithal to make any sense of it if I did).


----------



## davezatz

TiVo should just make the api/hooks public. We certainly couldn't do any worse than their parter in i.TV who stores our credentials in the clear and I've previously had scheduling issues with.


----------



## moyekj

orangeboy said:


> Yeah, I'm not gonna run out and get an iPad just for the TiVo app. I did however download the app through iTunes (something I did get just for the TiVo app!), and extracted the files to see what I could see. Which amounted to really nothin' of use to me...


 Note that the .ipa file you can just re-name to .zip and then you can unzip it as a normal zip file. This extracts to Payload/Quicksilver-TiVo.app with a whole lot of files under that, including many png image files. Interestingly there is a "tivo.cer" SSL certificate file there which can be converted to PEM format (using openssl) and used with wireshark to decrypt SSL communications.
I do have a hub and wireshark setup, however I'm missing the key piece to the puzzle - the iPad.
I used "grep" and various other utilities to look through the .ipa set of files but didn't find any interesting http/https/telnet looking entries.


----------



## innocentfreak

I know Orangeboy asked TiVodesign previously about a public API prior to the iPad app, but she said he would most likely have to sign some type of agreement.

I do agree though it should be public.


----------



## jbuehl

I captured some iPad-Tivo traffic to try and figure out an issue that I'm having with having to constantly reboot the Tivo to get the iPad app to work. I used an old ethernet hub and captured the traffic using Wireshark.

I can see a socket being opened by the iPad to port 1393 on the Tivo. The messages are binary, but they don't appear to be encrypted. It's definitely not SSL.


----------



## moyekj

jbuehl said:


> I captured some iPad-Tivo traffic to try and figure out an issue that I'm having with having to constantly reboot the Tivo to get the iPad app to work. I used an old ethernet hub and captured the traffic using Wireshark.
> 
> I can see a socket being opened by the iPad to port 1393 on the Tivo. The messages are binary, but they don't appear to be encrypted. It's definitely not SSL.


 Post a couple more times so you have enough to attach a zip file with the wireshark pcap file so we can take a look?


----------



## jbuehl

OK, will do.


----------



## jbuehl

Here is a hex dump of the conversation between the iPad and the Tivo. This is the case where it is failing to work for me.


iPad:
00000000 16 03 01 00 45 01 00 00 41 03 01 4d 69 28 bf c6 ....E... A..Mi(..
00000010 12 38 2e 2a 77 11 96 d7 35 e8 9a 00 70 85 0e b1 .8.*w... 5...p...
00000020 81 80 9f 29 8b dc ed 1d 4b d3 e3 00 00 1a 00 2f ...).... K....../
00000030 00 05 00 04 00 35 00 0a 00 09 00 03 00 08 00 33 .....5.. .......3
00000040 00 39 00 16 00 15 00 14 01 00 .9...... ..

Tivo:
00000000 16 03 01 00 4a 02 00 00 46 03 01 4d 69 27 f6 bb ....J... F..Mi'..
00000010 05 e5 14 dd a5 56 3d 65 fb 3d f1 21 12 a3 87 9d .....V=e .=.!....
00000020 8f 9d 1c 6a 6a 6f e3 88 96 1b 54 20 a1 37 4c c9 ...jjo.. ..T .7L.
00000030 78 8f 21 85 cc 18 75 30 15 5f 08 03 1b 51 d2 91 x.!...u0 ._...Q..
00000040 54 13 4b b3 8b 93 39 6f 48 bd ba 54 00 2f 00 16 T.K...9o H..T./..
00000050 03 01 02 67 0b 00 02 63 00 02 60 00 02 5d 30 82 ...g...c ..`..]0.
00000060 02 59 30 82 01 c2 02 09 00 e9 89 1d ff 4b 58 c7 .Y0..... .....KX.
00000070 20 30 0d 06 09 2a 86 48 86 f7 0d 01 01 05 05 00 0...*.H ........
00000080 30 71 31 1b 30 19 06 03 55 04 03 13 12 37 34 36 0q1.0... U....746
00000090 2d 30 30 30 31 2d 39 30 32 34 2d 32 45 43 31 31 -0001-90 24-2EC11
000000A0 12 30 10 06 03 55 04 0a 13 09 54 69 56 6f 20 49 .0...U.. ..TiVo I
000000B0 6e 63 2e 31 0b 30 09 06 03 55 04 0b 13 02 49 54 nc.1.0.. .U....IT
000000C0 31 0f 30 0d 06 03 55 04 07 13 06 41 6c 76 69 73 1.0...U. ...Alvis
000000D0 6f 31 13 30 11 06 03 55 04 08 13 0a 43 61 6c 69 o1.0...U ....Cali
000000E0 66 6f 72 6e 69 61 31 0b 30 09 06 03 55 04 06 13 fornia1. 0...U...
000000F0 02 55 53 30 1e 17 0d 31 30 30 31 32 36 31 39 32 .US0...1 00126192
00000100 39 34 39 5a 17 0d 32 30 30 31 32 34 31 39 32 39 949Z..20 01241929
00000110 34 39 5a 30 71 31 1b 30 19 06 03 55 04 03 13 12 49Z0q1.0 ...U....
00000120 37 34 36 2d 30 30 30 31 2d 39 30 32 34 2d 32 45 746-0001 -9024-2E
00000130 43 31 31 12 30 10 06 03 55 04 0a 13 09 54 69 56 C11.0... U....TiV
00000140 6f 20 49 6e 63 2e 31 0b 30 09 06 03 55 04 0b 13 o Inc.1. 0...U...
00000150 02 49 54 31 0f 30 0d 06 03 55 04 07 13 06 41 6c .IT1.0.. .U....Al
00000160 76 69 73 6f 31 13 30 11 06 03 55 04 08 13 0a 43 viso1.0. ..U....C
00000170 61 6c 69 66 6f 72 6e 69 61 31 0b 30 09 06 03 55 aliforni a1.0...U
00000180 04 06 13 02 55 53 30 81 9f 30 0d 06 09 2a 86 48 ....US0. .0...*.H
00000190 86 f7 0d 01 01 01 05 00 03 81 8d 00 30 81 89 02 ........ ....0...
000001A0 81 81 00 a1 7f 64 d3 32 d5 15 c8 97 98 4a fe 75 .....d.2 .....J.u
000001B0 e2 8f 13 3e 0a d4 0b f6 c4 77 c0 db df da 96 e1 ...>.... .w......
000001C0 ad 72 4b 1f a8 e6 47 13 e4 0f a5 90 0b 00 fb 0e .rK...G. ........
000001D0 0e cc f5 ca 1c cc 51 47 8e aa fb 79 96 69 d5 3b ......QG ...y.i.;
000001E0 46 31 08 a9 57 96 6d 3d 27 1f 77 2e ac 7c e1 3b F1..W.m= '.w..|.;
000001F0 76 94 ff e1 2f 20 84 fa d2 6e 79 4e e1 49 af d6 v.../ .. .nyN.I..
00000200 e4 b7 e2 e3 77 8f ee da 6d 6c 54 5d 20 9c 39 5e ....w... mlT] .9^
00000210 7f 98 fd a5 4a 43 17 37 2a 9a 77 f2 53 df 1e 8b ....JC.7 *.w.S...
00000220 fa ec c3 02 03 01 00 01 30 0d 06 09 2a 86 48 86 ........ 0...*.H.
00000230 f7 0d 01 01 05 05 00 03 81 81 00 85 fd 97 27 b4 ........ ......'.
00000240 25 9d 18 d1 88 d5 a7 e5 f8 41 f9 e6 40 b5 90 6d %....... [email protected]
00000250 cf d7 e0 66 e0 12 44 9d 96 47 47 76 2c e7 99 66 ...f..D. .GGv,..f
00000260 26 8b cb e8 bb b4 15 a4 05 c6 aa 55 72 ea 67 de &....... ...Ur.g.
00000270 b3 e1 b0 49 17 92 0d af 35 0e 41 54 9c 64 45 21 ...I.... 5.AT.dE!
00000280 2e ff 7d ac 44 03 74 b7 80 40 51 5e 80 17 54 53 ..}.D.t. [email protected]^..TS
00000290 83 0d ad d5 54 68 f3 8c 56 f3 b7 43 08 a5 73 97 ....Th.. V..C..s.
000002A0 8a e2 6c 43 e7 93 c0 ed 39 46 49 37 2f fe 4f 52 ..lC.... 9FI7/.OR
000002B0 20 b8 2f c6 c7 64 39 16 60 8e cb 16 03 01 00 04 ./..d9. `.......
000002C0 0e 00 00 00 ....

iPad:
0000004A 16 03 01 00 86 10 00 00 82 00 80 4f 41 a7 51 7c ........ ...OA.Q|
0000005A de 12 85 80 e5 c5 27 9e ae e5 c9 a7 28 5a 11 4d ......'. ....(Z.M
0000006A 6b ac fa 7d 71 2d 56 20 01 57 28 fd 54 38 8c 75 k..}q-V .W(.T8.u
0000007A 07 4b 32 6b 8f 1e 3d 16 12 ee 5c ac 7a 27 91 2c .K2k..=. ..\.z'.,
0000008A ee cb ec e5 fc eb 67 74 bf 51 12 dc b6 36 8a b1 ......gt .Q...6..
0000009A 92 84 9f b3 df 19 75 f5 ac e6 e5 54 15 9d da 5b ......u. ...T...[
000000AA 66 35 bf 5c ba 03 89 61 f4 b1 82 6f 86 ec 8d 61 f5.\...a ...o...a
000000BA 3b bf 7b f4 a0 5f d5 ee 1d e8 8b ad 46 2d 42 66 ;.{.._.. ....F-Bf
000000CA 94 8f 25 fe ef a3 67 6d 73 d7 af ..%...gm s..
000000D5 14 03 01 00 01 01 16 03 01 00 30 ae 4d 62 0f 03 ........ ..0.Mb..
000000E5 11 35 8f 14 cf 24 3e b7 f2 08 34 d6 26 51 68 5a .5...$>. ..4.&QhZ
000000F5 1e 31 24 a2 21 05 2d e5 92 f6 3f 9a e5 b1 ae 30 .1$.!.-. ..?....0
00000105 f6 4f 1f cc df 9a e7 30 b6 6e c8 .O.....0 .n.

Tivo:
000002C4 14 03 01 00 01 01 16 03 01 00 30 9f 8b 34 49 28 ........ ..0..4I(
000002D4 94 06 dc ca ee 6b b4 37 13 52 09 de b5 77 95 1d .....k.7 .R...w..
000002E4 b5 3c 9e 10 3a 07 d7 60 fc e2 55 18 05 db fd 4c .<..:..` ..U....L
000002F4 37 a0 d0 20 8e 5c ae e0 34 4c bb 7.. .\.. 4L.

iPad:
00000110 17 03 01 00 d0 97 a7 c8 54 b8 a5 54 e5 33 e5 4f ........ T..T.3.O
00000120 97 14 50 64 5e d6 f1 2b fc 0b 16 d5 cf d1 b8 67 ..Pd^..+ .......g

iPad:
00000130 b4 29 a9 ce fb c8 9c e6 1b e4 cd 2e 4b b2 f4 9a .)...... ....K...
00000140 05 03 8a 4e 9d 2b cf d5 b0 4e 83 d4 e5 2e 96 d0 ...N.+.. .N......
00000150 e2 61 75 ab e3 9d fc e6 10 fc 0b 1c 63 ac 05 45 .au..... ....c..E
00000160 cf 37 ad ac f1 88 b2 ec aa e6 d8 0c 09 df 6b cc .7...... ......k.
00000170 6d 78 df 2e 1a 05 73 d8 1d a8 0e bf ad d0 b6 0f mx....s. ........
00000180 c8 c8 ae 10 69 80 a5 ad e9 d7 7a 2c cc 4f 8b 9e ....i... ..z,.O..
00000190 2d 94 d0 db 70 a6 08 a4 08 f2 e0 ff e2 f7 f7 ad -...p... ........
000001A0 5c 27 92 b7 b4 ee 3e 78 23 75 a3 a2 46 fb d2 7a \'....>x #u..F..z
000001B0 6e ec e4 76 c1 ed b5 b0 d2 f6 50 06 f9 25 b6 8c n..v.... ..P..%..
000001C0 27 68 4e 82 ef 89 68 b0 00 68 7d b3 54 cf 04 d8 'hN...h. .h}.T...
000001D0 f5 9c 33 c8 cb ac 0c f9 40 b8 30 b0 f7 c8 d4 fa ..3..... @.0.....
000001E0 54 a5 17 6e 27 T..n'

Tivo:
000002FF 17 03 01 00 20 07 ae 28 d4 4f 73 fe 06 5f d3 2b .... ..( .Os.._.+
0000030F 57 d3 90 04 55 c5 98 c3 03 60 85 84 4d 72 38 80 W...U... .`..Mr8.
0000031F f4 f7 2f d7 06 17 03 01 00 d0 39 65 27 66 f2 86 ../..... ..9e'f..
0000032F c1 54 21 b5 74 48 01 e3 c1 cc de 12 45 1f 95 c4 .T!.tH.. ....E...
0000033F e3 02 47 95 f1 4d 86 b5 c6 86 bf 93 c5 c1 3e 5d ..G..M.. ......>]
0000034F 15 70 75 e6 75 d2 3f 62 63 6b e4 31 52 ca ce 95 .pu.u.?b ck.1R...
0000035F 4a de ff c1 86 8d c1 c4 59 3c 6e 03 1c 3d f7 0d J....... Y<n..=..
0000036F 84 bb 47 c3 7f 56 e9 5c ec 36 6b 89 7e 37 28 a6 ..G..V.\ .6k.~7(.
0000037F 45 bb b1 fe ef ab 8a b7 96 b0 72 f6 82 0c 1f 1c E....... ..r.....
0000038F f8 c4 7b 19 22 c6 0e 51 f8 eb 09 6c 1b fd 7a 0f ..{."..Q ...l..z.
0000039F 85 76 d3 e8 fe 2d 6e 8a b0 27 45 4f c2 0c 72 9b .v...-n. .'EO..r.
000003AF 82 6e 88 50 d4 82 62 74 c9 8a 96 ee 4c 75 46 0a .n.P..bt ....LuF.
000003BF 6a 40 c7 8f 88 b8 db ad 02 a4 e2 0c d4 ad 5e 9c [email protected] ......^.
000003CF 85 c3 7c 4e f0 00 d2 6e f1 0e 85 7a 6c 01 b5 5d ..|N...n ...zl..]
000003DF f9 de f9 7c 31 1e f4 30 b0 bb 97 16 4c 34 fd ea ...|1..0 ....L4..
000003EF fb 8e 74 07 27 d3 2d 82 cd 27 ..t.'.-. .'

iPad:
000001E5 17 03 01 00 90 7f ca 76 29 1f 13 78 f2 16 28 85 .......v )..x..(.
000001F5 bc 66 4a 60 47 6b 21 dc bc e0 d3 5f 78 39 2e ca .fJ`Gk!. ..._x9..
00000205 3a ef 55 bb 1c 64 c0 d5 35 58 da 59 11 1e 46 ec :.U..d.. 5X.Y..F.
00000215 63 1a c7 1a fa 8c 0d d5 f4 df 57 16 27 88 aa 4a c....... ..W.'..J
00000225 38 1a ab a5 b6 87 88 e7 e6 f5 c6 15 72 92 20 f0 8....... ....r. .
00000235 37 76 ff cc 8d c9 dd 06 a1 9b ee 90 ce d2 94 62 7v...... .......b
00000245 65 05 2e 69 8f c5 3e 8c f5 ac d1 b2 f8 7a ca 3c e..i..>. .....z.<
00000255 b7 d6 2f e1 bf 6c 32 45 9c 18 38 e1 ab 15 70 5e ../..l2E ..8...p^
00000265 43 5d df c3 68 ac c4 0a ce dc 07 e6 96 5c 3c 0d C]..h... .....\<.
00000275 b9 30 14 bd 00 .0...

Tivo:
000003F9 17 03 01 00 20 df a1 ea 79 7d 32 1e 9c 37 5e fc .... ... y}2..7^.
00000409 a2 24 bb 47 07 7b 01 ac 8d 24 4a 2d fc 39 e3 88 .$.G.{.. .$J-.9..
00000419 28 dd cc 8e 53 17 03 01 00 f0 71 25 88 5a db fa (...S... ..q%.Z..
00000429 ec b1 40 b6 fb c8 51 e5 07 c4 af 10 9a 79 5c 6c [email protected] .....y\l
00000439 2d 87 67 37 06 e0 e1 fa 31 f7 c1 27 84 c3 51 2c -.g7.... 1..'..Q,
00000449 21 06 2a f0 ca 44 b1 3f 3c 69 d8 25 df 54 9b 09 !.*..D.? <i.%.T..
00000459 d5 f3 f1 fe 98 f4 00 1e af 5a 63 18 2c 94 12 a9 ........ .Zc.,...
00000469 61 de 5c 8c ba a6 52 2b 19 fa 7b 58 7a 0d 14 6f a.\...R+ ..{Xz..o
00000479 18 18 c1 04 5b 73 3f e0 07 59 21 76 24 a2 38 29 ....[s?. .Y!v$.8)
00000489 54 99 46 d7 16 72 62 db 99 d0 0d a5 55 8f 51 2f T.F..rb. ....U.Q/
00000499 8b 1c 79 9e f8 20 23 0e fc ff c2 75 bf c9 b9 1a ..y.. #. ...u....
000004A9 ff 1f dc 7d 37 a0 12 c7 9e 17 b7 b5 1e 78 35 19 ...}7... .....x5.
000004B9 10 6d 1d e0 91 12 f2 af 3b b3 c9 9e 13 b7 98 44 .m...... ;......D
000004C9 bc b9 e1 a2 aa c0 43 57 13 b1 55 e8 6b fb 1d 74 ......CW ..U.k..t
000004D9 f3 60 d7 a6 7d a3 3e c2 14 e6 01 24 cc be f0 47 .`..}.>. ...$...G
000004E9 b3 3a 25 9f ea 05 da 96 76 2a 7b a6 bd c0 d7 53 .:%..... v*{....S
000004F9 3d 87 16 1f 59 6e e4 be 42 27 65 ce 88 0c 3f d9 =...Yn.. B'e...?.
00000509 d2 fc 83 65 9b 80 8e 4e 0e f2 ...e...N ..

iPad:
0000027A 15 03 01 00 20 81 9e b1 7d c7 57 c7 08 ab 69 1a .... ... }.W...i.
0000028A 7d b8 81 aa e6 48 a7 46 78 09 93 9d 43 5a 4d 32 }....H.F x...CZM2
0000029A 86 50 74 73 3c .Pts<
[/SIZE]


----------



## jbuehl

All the messages seem to start with 0x14030100, 0x15030100, 0x16030100, or 0x17030100. When I google those it comes up with some references to Skype. That doesn't make any sense.


----------



## jbuehl

Port 1393 is registered with IANA as network log server, whatever that is.

# iclpv-nls 1393/tcp Network Log Server
# iclpv-nls 1393/udp Network Log Server
# iclpv-nlc 1394/tcp Network Log Client
# iclpv-nlc 1394/udp Network Log Client


----------



## moyekj

Can you attach a zip file with the actual pcap file? Rather than just the raw data the header and packet information may have more clues.


----------



## jbuehl

Yeah, I'm still getting up to 10 posts. Only 2 more to go. I haven't seen this particular restriction in a forum before. I guess they were having a problem with people spamming them with huge uploads or something.


----------



## jbuehl

So how's the weather in Mission Viejo? We had rain and hail here yesterday.


----------



## jbuehl

OK, one more...


----------



## jbuehl

Here is a zip with 2 captures. One is where the iPad app failed to work and one where it succeeded. The successful one doesn't contain the entire exchange between the devices.


----------



## moyekj

OK thanks. As you said it looks like Premiere (192.168.1.151) is communicating on port 1393 with the iPad (192.168.1.138) using TCP/IP communication. I don't see any http/https level communication between them and therefore there is no header information to look at or anything else interesting to look at. So doesn't look like network sniffing is going to be too useful...


----------



## orangeboy

Something to try (and I don't know if it will help any) -

Set up this filter in Wireshark to display only traffic between the iPad and Premiere, regardless of port: 


Code:


(ip.src == 192.168.1.151 and ip.dst == 192.168.1.138) or (ip.src == 192.168.1.138 and ip.dst == 192.168.1.151)

Use the iPad to navigate around the Premiere, as if it were only a remote control, and make note of the action (used TiVo button, Right Arrow, etc). I haven't seen the app to know if what I'm asking is possible.

When saving the capture, save only the _displayed_ packets.


----------



## wmcbrine

moyekj said:


> So doesn't look like network sniffing is going to be too useful...


Man, you give up way too easily.


----------



## moyekj

Who said anything about giving up? The revelation that port 1393 is being used on TiVo is a good start and made the exercise well worth while. Just don't know if anything else useful can be gleamed from the captured raw data.


----------



## jbuehl

> Use the iPad to navigate around the Premiere, as if it were only a remote control, and make note of the action (used TiVo button, Right Arrow, etc). I haven't seen the app to know if what I'm asking is possible.
> 
> When saving the capture, save only the displayed packets.


Good idea, orangeboy. I'll try that in a while.


----------



## orangeboy

Hopefully with some methodical testing, something may come of it. And with methodical, I mean hit the TiVo button, wait about 10 seconds, hit Live TV, wait 10 seconds, hit Pause, wait 10 seconds, etc... Something that will clearly show that the action performed was actually captured. Getting to the more esoteric functions like "scrubbing" the progress bar can be done later *IF* the iPad protocol can be figured out.


----------



## wmcbrine

moyekj said:


> Just don't know if anything else useful can be gleamed from the captured raw data.


Let's see... start with the visible ASCII strings in the initial response from the TiVo (which include the TSN, BTW). I can see that each string is preceded by a length byte, and the length byte is preceded by a byte that may indicate the data type -- it's always 0x13 for the ASCII strings, except for the pure numeric sequences that end in "Z"; they're 0x17. In between these may be sections beginning with 0x30 (followed by a single byte) or 0x31 (followed by eight bytes). Not sure I've parsed that right yet, but it's early. The first group of strings etc. is _almost_ repeated after the numeric strings. (Besides the TSN, the other strings in this group are "TiVo Inc.", "IT", "Alviso", "California", and "US". Admittedly that doesn't seem useful, but it did help confirm the string format.)

Another thing I might look for would be data in HME-like formats (e.g., variable-length integers and packed dicts), which are also used in push requests, so TiVo seems to like them. But the string format doesn't fit that theme.

Anyway, there are patterns here; you just have to keep at it.


----------



## wmcbrine

orangeboy said:


> And with methodical, I mean hit the TiVo button, wait about 10 seconds, hit Live TV, wait 10 seconds, hit Pause, wait 10 seconds, etc... Getting to the more esoteric functions like "scrubbing" the progress bar can be done later


My understanding is that the iPad app uses the known network remote interface for the basic stuff, so the "esoteric" functions are the only ones to worry about. I can't confirm that, though.


----------



## jbuehl

Here are 2 captures of a play and a pause message. When the remote is active on the iPad, the Tivo continuously sends 298 byte messages at 1 second intervals to the iPad, no doubt to update the progress bar. The iPad only sends data when a button is pressed.

Sorry, I couldn't figure out how to save only the filtered packets.


----------



## moyekj

It's pretty easy to filter when viewing with Wireshark. Simply right click on one of the TCP lines and choose "Follow TCP Stream". That will filter out the other network traffic. Other useful thing to note is in the middle pane click on the Data row to show actual data being communicated without the TCP related overhead.


----------



## jbuehl

moyekj said:


> It's pretty easy to filter when viewing with Wireshark. Simply right click on one of the TCP lines and choose "Follow TCP Stream". That will filter out the other network traffic. Other useful thing to note is in the middle pane click on the Data row to show actual data being communicated without the TCP related overhead.


Right, I know how to do that, but when I save it to a file, it includes everything that was captured. The Save As... dialog has an option to save a range of packets, but it didn't seem to work.

The hex dump that I posted earlier is from "Follow TCP Stream" and includes just the data without the overhead.


----------



## orangeboy

Only frame 34 of the "Play" capture contains data from the iPad to the Premiere. Frame 7 of the "Pause" capture is also the only frame with data. Nothing on port 31339.

Edit: the first 5 bytes are common between both Pause and Play: 17 03 01 00 a0


----------



## jbuehl

orangeboy said:


> Only frame 34 of the "Play" capture contains data from the iPad to the Premiere. Frame 7 of the "Pause" capture is also the only frame with data. Nothing on port 31339.
> 
> Edit: the first 5 bytes are common between both Pause and Play: 17 03 01 00 a0


All the messages from the Tivo in those captures also start with 17030100. As I posted earlier, every message I have seen starts with 14030100, 15030100, 16030100, or 17030100. The ones starting with 16 occur when the devices initially connect, and the one starting with 15 is sent when they disconnect.

I am also seeing that signature at the beginning of messages sent from the Tivo to a server at Tivo with the IP address 208.73.181.192. The iPad app won't work unless the Tivo can be talking to a server at the mother ship.


----------



## tomhorsley

Shucks, I was hoping it would be something obvious like XML messages. This is gonna be more work to crack. I wonder if broadcom has some "standard" communications library they sold tivo (like they sold them the flash nonsense).


----------



## tomhorsley

orangeboy said:


> Hopefully with some methodical testing, something may come of it. And with methodical, I mean hit the TiVo button, wait about 10 seconds, hit Live TV, wait 10 seconds, hit Pause, wait 10 seconds, etc... Something that will clearly show that the action performed was actually captured. Getting to the more esoteric functions like "scrubbing" the progress bar can be done later *IF* the iPad protocol can be figured out.


Or setup a video camera to record the iPad with timestamps turned on and compare the time of the IP packets .


----------



## reneg

orangeboy said:


> Only frame 34 of the "Play" capture contains data from the iPad to the Premiere. Frame 7 of the "Pause" capture is also the only frame with data. Nothing on port 31339.
> 
> Edit: the first 5 bytes are common between both Pause and Play: 17 03 01 00 a0


Looks like the data starting with 17 03 01 00 a0 is SSL application data header according to MS Network Monitor.


Code:


  Frame: Number = 7, Captured Frame Length = 231, MediaType = ETHERNET
+ Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[00-11-D9-31-A2-74],SourceAddress:[D8-A2-5E-49-C3-02]
+ Ipv4: Src = 192.168.1.138, Dest = 192.168.1.151, Next Protocol = TCP, Packet ID = 16021, Total IP Length = 217
+ Tcp: Flags=...AP..., SrcPort=49957, DstPort=1393, PayloadLen=165, Seq=2321678643 - 2321678808, Ack=2797587107, Win=32942
  TLSSSLData: Transport Layer Security (TLS) Payload Data
- TLS: TLS Rec Layer-1 SSL Application Data
  - TlsRecordLayer: TLS Rec Layer-1 SSL Application Data 
     ContentType: SSL Application Data (0x17)
   + Version: TLS 1.0 (0x0301)
     Length: 160 (0xA0)
   - ApplicationData: 
      SSLApplicationData: Binary Large Object (160 Bytes)

Makes me wonder if we're dealing with a XMPP interface.


----------



## jbuehl

reneg said:


> Looks like the data starting with 17 03 01 00 a0 is SSL application data header according to MS Network Monitor.
> 
> Makes me wonder if we're dealing with a XMPP interface.


I think you're right. I assumed it wasn't encrypted because Wireshark didn't flag it as something special and I saw the clear text in the second message, but those signatures make sense. I don't know much about SSL, but the text is probably Tivo's SSL certificate.

Here is a description that I found of the protocol

http://publib.boulder.ibm.com/infocenter/tpfhelp/current/index.jsp?topic=/com.ibm.ztpf-ztpfdf.doc_put.cur/gtps5/s5rcd.html

And here's the RFC

http://tools.ietf.org/html/rfc2246


----------



## moyekj

If it's any help, attached is the TiVo SSL certificate from the iPad application.


----------



## moyekj

Some progress. Using openssl as I was able to establish a connection with my Premiere on port 1393:
(NOTE: I x'd out my TSN below)


Code:


C:\OpenSSL-Win32\bin>openssl s_client -connect 192.168.10.199:1393 -state -nbio
Loading 'screen' into random state - done
CONNECTED(000000AC)
turning on non blocking io
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:error in SSLv2/v3 read server hello A
write R BLOCK
SSL_connect:SSLv3 read server hello A
depth=0 CN = 746-0001-xxxx-xxxx, O = TiVo Inc., OU = IT, L = Alviso, ST = Califo
rnia, C = US
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = 746-0001-xxxx-xxxx, O = TiVo Inc., OU = IT, L = Alviso, ST = Califo
rnia, C = US
verify return:1
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:error in SSLv3 read finished A
SSL_connect:error in SSLv3 read finished A
read R BLOCK
SSL_connect:SSLv3 read finished A
read R BLOCK
---
Certificate chain
 0 s:/CN=746-0001-xxxx-xxxx/O=TiVo Inc./OU=IT/L=Alviso/ST=California/C=US
   i:/CN=746-0001-xxxx-xxxx/O=TiVo Inc./OU=IT/L=Alviso/ST=California/C=US
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICWTCCAcICCQCXECFzZRrCKDANBgkqhkiG9w0BAQUFADBxMRswGQYDVQQDExI3
NDYtMDAwMS05MDM1LThCQTgxEjAQBgNVBAoTCVRpVm8gSW5jLjELMAkGA1UECxMC
SVQxDzANBgNVBAcTBkFsdmlzbzETMBEGA1UECBMKQ2FsaWZvcm5pYTELMAkGA1UE
BhMCVVMwHhcNMTAwNDI2MjAzMTI5WhcNMjAwNDIzMjAzMTI5WjBxMRswGQYDVQQD
ExI3NDYtMDAwMS05MDM1LThCQTgxEjAQBgNVBAoTCVRpVm8gSW5jLjELMAkGA1UE
CxMCSVQxDzANBgNVBAcTBkFsdmlzbzETMBEGA1UECBMKQ2FsaWZvcm5pYTELMAkG
A1UEBhMCVVMwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAKF7vQnT0+7Dd5dn
cr+OUVT+urRzKGW4SOOLxd8G0FwdEdYdKKUz5HyuvXczv77+SjThtleaFAaeeLUO
X96uBGOHIv5bjoxsA1GiTizBY5dDJj3qLDS8qnnyY5EsBy2gxElsN/7+rTFson06
Ui5jA75hY4ZId/NKUPI1ayHSEcD3AgMBAAEwDQYJKoZIhvcNAQEFBQADgYEARIgE
mo/0Bj/GwpuQX4X1DleiXvyM75giRIMAqVa6tXHZa/75y0vdsKy30uyj6wXkahnb
X/4GuzxYhADWfiVnVhiTyLdCzALChH3+5rEkUC/AMcNp8NIauEL5NGdkPrifFYhS
n6wpyf1PF3JOcNDEeeiILQyEhjN7cP93zRTs+FU=
-----END CERTIFICATE-----
subject=/CN=746-0001-xxxx-xxxx/O=TiVo Inc./OU=IT/L=Alviso/ST=California/C=US
issuer=/CN=746-0001-xxxx-xxxx/O=TiVo Inc./OU=IT/L=Alviso/ST=California/C=US
---
No client certificate CA names sent
---
SSL handshake has read 767 bytes and written 408 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES256-SHA
    Session-ID: 96F08F7587604AA95B9AA41E878D88011441DE79FF9EC21E0247910CEB447ECD

    Session-ID-ctx:
    Master-Key: A3E16A6834CDEF6A1F6F63BE321BEEE7F78391683678710FDDBB8832DC8CE8D9
C62473BC9E03CAD94AAA09A2DEF8327D
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1298986922
    Timeout   : 300 (sec)
    Verify return code: 18 (self signed certificate)
---
^C

An article on SSL debugging:
http://www.sslshopper.com/article-debugging-ssl-communications.html

I believe we have the client certificate in my previous post obtained from the iPad app.


----------



## jbuehl

That's good news moyekj. It should be possible to write a man-in-the-middle to see the messages in the clear.


----------



## ckrames1234

The traffic is in fact, encrypted with SSL. The fact that every packet above starts with the same data proves that; it is the SSL header. The TiVo uses the MRPC (MindRPC) protocol. I, being an Objective-C hacker, actually went as far as to disable SSL inside the app's binary, and although this disables the app from working, I captured the first request it sends to the TiVo, unencrypted:



Code:


MRPC/2 224 85
Type:request
RpcId:4
SchemaVersion:7
Content-Type:application/json
RequestType:bodyAuthenticate
ResponseCount:single
BodyId:
X-ApplicationName:Quicksilver
X-ApplicationVersion:1.2
X-ApplicationSessionId:0x3bc3f0

{"type":"bodyAuthenticate","credential":{"type":"makCredential","key":"XXXXXXXXXX"}}

Going further, I found the sweet spot in the code, I can now get all requests _before_ they are sent out in the app (i.e. unencrypted). It seems MRPC is a very configurable protocol, you specify a request type (recordingSearch, subscriptionSearch, contentSearch, offerSearch, and collectionSearch are the common ones I see), and then specify what you want to search for, and also how you want to receive the results. All in JSON. An example request to get all your season passes:



Code:


MRPC/2 246 651
Type:request
RpcId:37
SchemaVersion:7
Content-Type:application/json
RequestType:subscriptionSearch
ResponseCount:single
BodyId:tsn:XXXXXXXXXXXXXXX
X-ApplicationName:Quicksilver
X-ApplicationVersion:1.2
X-ApplicationSessionId:0x3a82a0 (Unique, random number, probably based on time, that remains the same per one authentication session)
	
{"type":"subscriptionSearch","noLimit":true,"bodyId":"tsn:XXXXXXXXXXXXXXX","levelOfDetail":"medium","responseTemplate":[{"type":"responseTemplate","fieldName":["subscription"],"typeName":"subscriptionList"},{"type":"responseTemplate","fieldName":["title","subscriptionId","idSetSource","maxRecordings","showStatus","keepBehavior"],"typeName":"subscription"},{"type":"responseTemplate","fieldName":["title","contentId","channel","startTime","duration","hasSignLanguage","hasAudioDescription","screenFormat","offerId","cc","collectionId"],"typeName":"offer"}],"objectIdAndType":["581641651188047","581641651188014","581641651187883","581641651187995"]}

Overall:
You send an authentication request to your TiVo on port 1413 with your MAK, encrypted with SSL. Then you send any request you want to obtain the data you want. I will probably provide a more in depth dissemination of the protocol and some sample code in perl soon. (I wrote this in a rush, sorry )


----------



## orangeboy

Awesome first post!


----------



## ckrames1234

Thanks!

And to clear up some points above:

- SSL was created for, and is used, for the express purpose of preventing people from spying on data in between two ends of a connection. When you found the SSL certificate, you could have used it with Wireshark (if it was built with GnuTLS) to decrypt the packets, but i think the developers changed the certificate, removed the file, and integrated it into the binary, because I couldn't find a tivo.cer file anywhere. Without that certificate, packet captures are useless other than to tell which port is being used.

- It is _very rare_ to see a TCP protocol not based on plaintext.

- The best method, in my opinion, to capture packets between the two devices, is by installing and using tcpdump on a jailbroken iPad 


Code:


tcpdump -w ./tivo_dump.pcap -vvv -s 0 'src or dst 192.168.2.202'

- Are you guys seeing port 1393 being used? My 3 day old, recently updated TiVo Premiere was using port 1413. It may vary per TiVo?

- Random note: Looking through IDA shows that there are some classes specifically for iPhone in the code, so we should be seeing an iPhone version soon


----------



## tomhorsley

Actually, it looks to me like they are not using a certificate based SSL anymore. On my linux box, I setup this entry in my stunnel.conf file:

[tivo]
accept = 1413
connect = tivo-7460001902dac0b.my.lan:1413

(where that ridiculous name is the name the tivo gave itself via dhcp and .my.lan is what I call my local subnet in my dns server).

I then do this:

telnet localhost 1413

and paste in the example authentication request, modified to contain my tivo's mak, and the telnet session prints this:

MRPC/2 75 97
Content-Type: application/json
IsFinal: true
RpcId: 4
Type: response

{"message": "Authentication successful", "status": "success", "type": "bodyAuthenticateResponse"}

So I get a successful authentication back just using the same sort of SSL connection you'd use to connect to an SSL protected mail server, etc. So it looks like things are ready to take off maybe (I can hope .


----------



## orangeboy

Well huh! With no thread activity in two months, I thought this was a dead project!


----------



## innocentfreak

orangeboy said:


> Well huh! With no thread activity in two months, I thought this was a dead project!


Same here lol. Even with an iPad now I would still love to see some of this functionality figured out.

Speaking of dead projects, any progress


----------



## orangeboy

I haven't even looked at python code for months. Christmas break did just that: it broke my habit of further development...


----------



## ckrames1234

Yeah, it's not dead anymore


----------



## innocentfreak

orangeboy said:


> I haven't even looked at python code for months. Christmas break did just that: it broke my habit of further development...


Lol just figured I would give you crap over it no worries.


----------



## unitron

orangeboy said:


> Awesome first post!


ckrames1234 also seems to be breathing a little new life into the "other forum" as well.


----------



## arantius

Wow. I should have checked back here earlier. Here's the post I was about to write elsewhere, and the details I've found (more in my next reply):
----------------------------------------------​There's two parts to my story, one happy and one sad.

Happy: I spent a bit of time, a while back, trying and failing to decode the protocol that the iPad app uses to communicate with the TiVo. I tried again this weekend and succeeded! Details will follow below.

Sad: because I had spend so long not using my iPad (having switched to a Xoom, and keeping the iPad around only because I wanted to get back to figuring this out, so I could write an Android equivalent) the TiVo app on it was still version 1.1, with an upgrade to 1.2 available. I did this work before upgrading, then clicked the 'upgrade all' button. Bad idea. Version 1.2 of the iPad app speaks something different, which I can't decode yet. _(Edit: removed call for old apps -- previous problems seem to be user error on my part!)_ Who knows how long this version will continue to work, given that it changed between 1.1 and 1.2? Hopefully forever. Anyway:

Protocol details, for iPad app version 1.1:

The iPad discovers the TiVo with mDNS.
The iPad then connects to port 1393.
It does so over SSL so the data is impossible to sniff.
(With 1.2, it connects on port 1413, which still appears to be SSL, but somehow changed. Perhaps it's validating certificates?)
Previous attempts to reverse engineer with only tcpdump have seemed to stop here. Bummer!

But I managed to forge some mDNS replies so that the iPad app connected to my desktop instead (this still works with 1.2 -- see https://gist.github.com/1009602#file_mdns_tivo_forgery.py) and used TcpCatcher as a man-in-the middle to decrypt the SSL (this part has failed for me so far with version 1.2).

The result is this proof-of-concept script: https://gist.github.com/1009602#file_remote.py
Open up TiVo search with something playing in the corner and run this script. It will pause the video, type "typed via remote" into the search box, and unpause the video. (Or, at least it does for me.)

Here's both these scripts, plus all the RPCs I managed to dump (with my MAK and TSN stripped -- hope I didn't leave anything else that's too sensitive). As mentioned above, I can't dump any more because I no longer have version 1.1 of the iPad app, and would love it if anyone could help me get it back in case I missed anything.
http://www.mediafire.com/?z40dci9756a7vh7


----------



## arantius

ckrames1234 said:


> The TiVo uses the MRPC (MindRPC) protocol. I, being an Objective-C hacker, actually went as far as to disable SSL inside the app's binary, and although this disables the app from working, I captured the first request it sends to the TiVo, unencrypted:


Is MRPC or MindRPC something you've ever heard of before, or just something you figured out based on context?

Could you perhaps provide the dumps-the-requests binary (my iPad is jailbroken)? This would be likely painful, but enough to continue if it does turn out that I missed some important request types.


----------



## arantius

Nevermind! TcpCatcher was just being a pain. It is still possible to SSL man-in-the-middle version 1.2 of the app, as:

Edit: see the .zip posted later.


----------



## moyekj

Good stuff arantius, thanks for posting!
Have you captured something more interesting, such as deleting a show from the Premiere via the iPad? Sending characters/numbers can already be handled with telnet based remote so to me is not as interesting to discover, though of course makes for a good repeatable prototype.


----------



## orangeboy

There's some interesting captures in the tcp dump:
{"type":"tunerConflictEventRegister"}


----------



## arantius

moyekj said:


> Good stuff arantius, thanks for posting!
> Have you captured something more interesting, such as deleting a show from the Premiere via the iPad?


Yes, download the linked zip. It's chock full of all sorts of kinds of requests and responses. I'm hoping to sort through it all and try to get it documented, so I don't have to keep referring to the raw data. That might be a really big task.

Also as mentioned, I realized I can monitor app 1.2's traffic as well, which appears to be similar but critically different in just enough ways to be a pain.


----------



## arantius

Some protocol v2 requests/responses, as from iPad app version 1.2:
http://www.mediafire.com/?636hz9tb09e26ws


----------



## moyekj

arantius, I finally had a little time this morning to play around with this a little and want to re-iterate thanks for your work on this. I was able to get your prototype working and tried some additional things. For others monitoring this thread, an example of a cool application of this remote:

Jump to a specific location in currently playing program, example:
RpcRequest('videoPlaybackPositionSet', offset=400000)

The above request always jumps to specific offset from start of program given, so you can immediately jump to specific point in a program for advanced navigation controls.

Haven't yet figured out how to delete a specific program directly, but something worth exploring further along with some other things.

One little obstacle I ran into was I didn't have the 'simplejson' python module, but a quick Google search identified where I could download that from, and I just installed in same dir as the remote.py module.

Thanks again!


----------



## wmcbrine

"simplejson" I think is the same as "json" in the standard library as of Python 2.6.


----------



## moyekj

Note that the 'offset' argument of 'videoPlaybackPositionSet' is the time you want to jump to in milliseconds. i.e. To jump to 5 minute mark it would be 60*1000*5. i.e. It's easy to jump exactly to any point in a program that you want.

One can use 'videoPlaybackInfoEventRegister' to get current position information. So based on the above 2 it's simple to implement a simple means of jumping forwards/backwards n minutes with a slight modification to the Key method in remote.py. Below is a sample implementation (with no error checking at all, but just for proof of concept):


Code:


  def Key(self, key, offset=0):
    """Send a key.

    Supported:
      Letters 'a' through 'z'.
      Numbers '0' through '1'.
      Named keys:
        actionA, actionB, actionC, actionD, advance, channelDown,
        channelUp, clear, down, enter, forward, guide, info, left, liveTv,
        pause, play, record, replay, reverse, right, select, slow,
        thumbsDown, thumbsUp, tivo, up, zoom
    """
    if key == ' ':
      key = 'forward'
    elif key == 'jump':
      result = self.Key('position')
      pos = int(result['response']['position'])
      pos += 60000*offset
      req = RpcRequest('videoPlaybackPositionSet', offset=pos)
    elif key == 'position':
      req = RpcRequest('videoPlaybackInfoEventRegister', throttleDelay=1000)
    elif key.lower() in 'abcdefghijklmnopqrstuvwxyz':
      req = RpcRequest('keyEventSend', event='ascii', value=ord(key))
    elif key in '0123456789':
      req = RpcRequest('keyEventSend', event='num' + key)
    else:
      req = RpcRequest('keyEventSend', event=key)

    self.Write(req)
    result = self.Read()
    return result

Then as examples the following work:

# Jump 5 minutes ahead
remote.Key('jump', offset=5)

# Jump 7 minutes backwards
remote.Key('jump', offset=-7)

i.e. There seems to be enough uncovered already to mimic telnet based remotes as well as adding additional functionality such as above.


----------



## orangeboy

arantius said:


> ...Open up TiVo search with something playing in the corner and run this script. It will pause the video, type "typed via remote" into the search box, and unpause the video. (Or, at least it does for me.)


Well, after actually reading the instructions above, yep, "typed via remote" shows up for me, too!



wmcbrine said:


> "simplejson" I think is the same as "json" in the standard library as of Python 2.6.


Indeed it is (2.6.6). And in 2.7.0 and 2.7.1.



moyekj said:


> Note that the 'offset' argument of 'videoPlaybackPositionSet' is the time you want to jump to in milliseconds. i.e. To jump to 5 minute mark it would be 60*1000*5. i.e. It's easy to jump exactly to any point in a program that you want.
> 
> One can use 'videoPlaybackInfoEventRegister' to get current position information. So based on the above 2 it's simple to implement a simple means of jumping forwards/backwards n minutes with a slight modification to the Key method in remote.py...


I haven't looked into all the arguments, but did you notice if there's a 'videoPlaybackEnd' (or something similar) that represents total duration? I'm thinking of a horizontal scroll bar, representing 0 to <end> that mimics the iPad scrub. No clue if the two could be tied together somehow.


----------



## moyekj

OK, another good one. Once you get hold of a recording id you can issue a "play" request as follows:



Code:


req = RpcRequest('uiNavigate', uri='x-tivo:classicui:playback', parameters={'fUseTrioId': 'true', 'recordingId': 'tivo:rc.247339', 'fHideBannerOnEnter': 'true'})

Note that you can choose to display the banner or not as part of request. I really like NOT displaying the banner which is what I used above.

Getting list of recording Id's seems trickier than it should be. By trial and error I couldn't get list of completed recordings, but I did manage to get in progress recordings using:


Code:


req = RpcRequest('recordingSearch', levelOfDetail='medium', bodyId='-', state=['inProgress'])

For the above when I use 'complete' instead of 'inProgress' I get an error response.
(search must have one of the following filters: 'offerId', 'collectionId', 'recordingId', 'state')

To do list searching:


Code:


req = RpcRequest('recordingSearch', levelOfDetail='medium', bodyId='-', state=['scheduled'])


----------



## arantius

wmcbrine said:


> "simplejson" I think is the same as "json" in the standard library as of Python 2.6.


No it's different. It's older, and in my experience (quite surprisingly) much faster. Primarily, I'm used to it. So I stick with it. You can almost definitely substitute the standard library's "json" to no ill effect, assuming your Python is new enough to have that.


----------



## arantius

Two new things to report. One, the proof-of-concept script, updated to speak the newer protocol on port 1413:
https://gist.github.com/1009602#file_remote2.py

Two, the documentation I had previously mentioned:
https://docs.google.com/document/pub?id=1e4ymm7ROwmW6co2pKENjANGT5xM00VzmzybZ4u8yDE8

I'd love to know more about what all the RPCs are and how they relate, but I'm still working on figuring that all out.


----------



## moyekj

It's pretty clear that RPC is just part of the story though. I think the iPad is also communicating with tivo.com servers to get a lot of information and also perhaps using the traditional HMO to communicate with the TiVos as well.

Typically RPC search queries directly to the TiVo only return 1 result and are highly targeted/filtered. For example, you can't get list of all recordings on the TiVo, but you can get additional details on particular shows. When you search for "scheduled" recordings without a filter you only get the first in To Do list as a response. i.e. When using this protocol you should already be armed with specific information about existing and/or scheduled recordings which are obtained with alternate means.

Note also for example that a show 'delete' does not seem to happen through this protocol. The only relevant RPC related to delete I found was:
111637-1229-1-recordingUpdate-recordingUpdate.txt
i.e. Just information that a specific show was deleted. The actual delete itself was performed with a different method, probably using different port and protocol.


----------



## arantius

moyekj said:


> It's pretty clear that RPC is just part of the story though. I think the iPad is also communicating with tivo.com servers to get a lot of information and also perhaps using the traditional HMO to communicate with the TiVos as well.


I'm not certain of that yet. I think it's just complicated, nothing is completely done in one RPC.



moyekj said:


> Typically RPC search queries directly to the TiVo only return 1 result and are highly targeted/filtered. For example, you can't get list of all recordings on the TiVo


Plenty of RPCs response is an "idSequence". I think you just need to take those IDs and fire off another call to get their details. Anyway, this is one of the areas I'd expect to concentrate; getting the list of stuff on the TiVo and interacting with it (play, delete, etc.) is one of the primary nice features to add on top of the apps currently available for Android. (Which all seem to use IRCODE.)

Clearly my docs are missing something, because I don't have any idSequence responses documented. But they're there in the traces. It's gonna take a lot of work to read through all the possible responses and see where I'm missing things. Requests seem to omit optional parameters all the time, making finding them extra tricky.



moyekj said:


> Note also for example that a show 'delete' does not seem to happen through this protocol. The only relevant RPC related to delete I found was:
> 111637-1229-1-recordingUpdate-recordingUpdate.txt
> i.e. Just information that a specific show was deleted. The actual delete itself was performed with a different method, probably using different port and protocol.


That is the request to perform the delete. The response is 111637-1229-2-response-success.txt -- success, that recording ( tivo:rc.181929 ) was deleted.


----------



## moyekj

arantius said:


> That is the request to perform the delete. The response is 111637-1229-2-response-success.txt -- success, that recording ( tivo:rc.181929 ) was deleted.


Ah, yes that should have been obvious - given that there is a response to go along with it. What threw me off was the state='deleted' which implied it already had happened. I was indeed able to delete a show using that method, and oddly enough it worked just using bodyId='-' (i.e. no need for tsn even though bodyId is a required argument). Now to focus next on how to obtain recording id's...


----------



## moyekj

Very simply, to get recordings in My Shows it is:
req = RpcRequest('recordingFolderItemSearch', bodyId='-')
(One can use 'orderBy' argument to change the order)

Also, I am glad to see I was wrong about response being only 1 item at a time. Above seems to return a full list (including folders and #items per folder).


----------



## arantius

Here's a fun (if difficult to read) image to explain the request and response types:

http://img820.imageshack.us/img820/3053/maprequeststoresponsesf.png

You can see that subscriptionSearch could give a subscriptionList, or an idSequence. Then recordingSearch could give recordingList or idSequence, also recordingFolderItemSearch could give recordingFolderItemList or idSequence.

I think you see where I'm going. I happen to see a recordingFolderItemSearch that sets "format": "idSequence", gets back



Code:


{
  "isBottom": false, 
  "isTop": false, 
  "objectIdAndType": [
    "94873", 
    "871425", 
    "870665", 
    "37169", 
    "120769", 
    "46006", 
    "57822", 
    "36231", 
    "36432", 
    "35665", 
    "35648", 
    "603352", 
    "35662", 
    "38264", 
    "35739", 
    "1", 
    "0"
  ], 
  "type": "idSequence"
}

Then very soon issues a recordingFolderItemSearch of



Code:


{
  "orderBy": [
    "startTime"
  ], 
  "bodyId": "tsn:7460001905F9871", 
  "objectIdAndType": [
    "94873", 
    "871425", 
    "870665", 
    "37169", 
    "120769"
  ], 
  "note": [
    "recordingForChildRecordingId"
  ], 
  "type": "recordingFolderItemSearch", 
  "responseTemplate": [...]
}

Notice how those are the first five in the list? It issues one RPC (in this case) to see what the groups are, then another to see what the contents of (some of those) groups are. There's more RPC(s) later to dig into the list of what the recordings are, if one of these results is actually a group. (I see items and groups with lengths mixed into this response. Exactly what you'd see on the "My Shows" section of the TiVo itself, if it's set to show groups, depending on what recordings you have.)


----------



## orangeboy

arantius said:


> Two new things to report. One, the proof-of-concept script, updated to speak the newer protocol on port 1413:
> https://gist.github.com/1009602#file_remote2.py
> 
> Two, the documentation I had previously mentioned:
> https://docs.google.com/document/pub?id=1e4ymm7ROwmW6co2pKENjANGT5xM00VzmzybZ4u8yDE8
> 
> I'd love to know more about what all the RPCs are and how they relate, but I'm still working on figuring that all out.


Thanks. It's obvious that you took (and are taking) a good chunk of time getting this information together. I have little doubt that an iPad like app will be developed based on your efforts.


----------



## moyekj

Haven't had much time to play around with this but did so a little last night. arantius, this is a TiVo gold mine you have uncovered. It doesn't take much effort to get complete "My Shows" and "To Do" lists which I have prototypes for. No doubt season pass management, etc. can also be implemented. I haven't even touched possibilities with HME, show searching etc.

This discovery basically unlocks and makes available a big chunk of the database on the Premiere that was not available previously directly from Premiere. It's akin to having a hacked Premiere without the need for hacking.


----------



## orangeboy

moyekj said:


> Haven't had much time to play around with this but did so a little last night. arantius, this is a TiVo gold mine you have uncovered. It doesn't take much effort to get complete "My Shows" and "To Do" lists which I have prototypes for. No doubt season pass management, etc. can also be implemented. I haven't even touched possibilities with HME, show searching etc.
> 
> This discovery basically unlocks and makes available a big chunk of the database on the Premiere that was not available previously directly from Premiere. It's akin to having a hacked Premiere without the need for hacking.


I am absolutely awestruck by the amount of information that is now available. Using the remote.py script, I've been adding in the discovered RequestTypes as functions in the Remote Class, similar to the following:



Code:


    ...
    # bodyConfigSearch
    def bodyConfigSearch(self):
        req = RpcRequest('bodyConfigSearch', 
                         bodyId='-')
        self.Write(req)
        result = self.Read()
        for item in result:
            print item
        print result['response']
        print " "
        print result
        print " "

    # channelSearch
    def channelSearch(self):
        req = RpcRequest('channelSearch',
                         noLimit='true', 
                         isReceived='false') 
        self.Write(req)
        result = self.Read()
        for item in result:
            print item
        print result['response']
        print " "
        print result
        print " "
    ...

and just simply calling them to see what can be gleaned from my own Premiere:



Code:


    ...
    remote = Remote()

    print 'bodyConfigSearch'
    remote.bodyConfigSearch()

    print 'channelSearch'
    remote.channelSearch()

   ...

Oh. My. God.

My mind is reeling with potential applications that can come from this. I can seriously see elimination of (TiVo) meta-generators that rely on IMDB and TheTVDB. Show and movie cover art and image links can be found, hardware information (userDiskSize, userDiskUsed), software information (hasScheduler(?), softwareVersion, parentalControlsState, clipOverlapping, etc.), Series information, Episode information, the list goes on and on. And I've only defined the first four RequestTypes! It makes me wonder what else is buried under the covers. What would be icing on the cake would be a "QuerySchema" request, that outputs ALL of the RequestTypes and their options, as well as output definitions.


----------



## innocentfreak

Is it strange I am excited about this and don't have clue what orangeboy just said ?


----------



## windracer

He had me at "Oh. My. God."


----------



## orangeboy

I'm not a fan of hyperbole, but arantius' time and effort (I think) was monumental! There's a lot to go through, but I encourage downloading the "Request/Responses" zip found in this post: http://www.tivocommunity.com/tivo-vb/showthread.php?p=8556511#post8556511

If you do get the zip, this file is an example of a collectionSearch: 61706-32-2-response-contentList.txt, that I think would provide an ideal set of data for a new meta-generator. Bah, here's the content of the file:



Code:


MRPC/2 76 2928
Raw Packet=MRPC/2 76 2928
Content-Type: application/json
IsFinal: true
RpcId: 32
Type: response

{
  "content": [
    {
      "category": [
        {
          "image": [
            {
              "width": 360, 
              "imageUrl": "http://i.tivo.com/images-static/bravo/genre-backgrounds/showcaseBanner/Drama.jpg", 
              "type": "image", 
              "imageType": "showcaseBanner", 
              "height": 270
            }, 
            {
              "width": 180, 
              "imageUrl": "http://i.tivo.com/images-static/bravo/genre-backgrounds/moviePoster/Drama.jpg", 
              "type": "image", 
              "imageType": "moviePoster", 
              "height": 270
            }
          ], 
          "displayRank": 0, 
          "type": "category", 
          "label": "Drama"
        }, 
        {
          "displayRank": 1, 
          "type": "category", 
          "label": "TV Show"
        }
      ], 
      "collectionType": "series", 
      "seasonNumber": 4, 
      "description": "Gossip flies about Caroline and a man (Gil Gerard) building her kitchen in exchange for room and board.", 
      "episodic": true, 
      "collectionId": "tivo:cl.19284", 
      "contentId": "tivo:ct.127876", 
      "image": [
        {
          "width": 139, 
          "imageUrl": "http://i.tivo.com/images-production/bravo/banner-med/dd/a7/dda739d69d9893b079ee4479043e9dea.jpg", 
          "type": "image", 
          "imageType": "showcaseBanner", 
          "height": 104
        }, 
        {
          "width": 200, 
          "imageUrl": "http://i.tivo.com/images-production/bravo/banner-lg/dd/a7/dda739d69d9893b079ee4479043e9dea.jpg", 
          "type": "image", 
          "imageType": "showcaseBanner", 
          "height": 150
        }
      ], 
      "episodeNum": [
        4
      ], 
      "originalAirYear": 1977, 
      "credit": [
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Landon", 
          "first": "Michael"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Grassle", 
          "first": "Karen"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Gilbert", 
          "first": "Melissa"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Anderson", 
          "first": "Melissa"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Swenson", 
          "first": "Karl"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Bull", 
          "first": "Richard"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "MacGregor", 
          "first": "Katherine"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Arngrim", 
          "first": "Alison"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Gilbert", 
          "first": "Jonathan"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Hagen", 
          "first": "Kevin"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Greer", 
          "first": "Dabbs"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Stewart", 
          "first": "Charlotte"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Olsen", 
          "first": "Merlin"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Laborteaux", 
          "first": "Patrick"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Parady", 
          "first": "Hersha"
        }, 
        {
          "role": "guestStar", 
          "type": "credit", 
          "last": "Gerard", 
          "first": "Gil"
        }, 
        {
          "role": "guestStar", 
          "type": "credit", 
          "last": "Baddeley", 
          "first": "Hermione"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Greenbush", 
          "first": "Lindsay"
        }, 
        {
          "role": "actor", 
          "type": "credit", 
          "last": "Greenbush", 
          "first": "Sidney"
        }, 
        {
          "role": "writer", 
          "type": "credit", 
          "last": "Heinemann", 
          "first": "Arthur"
        }, 
        {
          "role": "director", 
          "type": "credit", 
          "last": "Claxton", 
          "first": "William F."
        }
      ], 
      "title": "Little House on the Prairie", 
      "subtitle": "The Handyman", 
      "type": "content", 
      "tvRating": "pg"
    }
  ], 
  "type": "contentList"
}

Note that it contains: a couple "imageUrl"s, "seasonNumber", "episodeNum", "originalAirYear", "title" (Series title), and "subtitle" (Episode title), not to mention "description" and "role" for cast and crew, all handy/interesting information. Other than the images, I _believe_ all the other information is *local*. Although the Request looks a bit daunting, I think the payoff is well worth the effort. Yessir, I like it! :up:


----------



## innocentfreak

I guess the next question is there anything us non programmers can do to help?


----------



## ckrames1234

I love the API documentation, great work arantius. I will definitely try to contribute to it. There are so many things I could do with this, projects I could make, the possibilities are literally endless.

I never posted sample code on here because in my sample, I tried using a random X-ApplicationSessionId, which I now know is not a good idea. 

(For some reason I never got notified when someone posted on here :/)


----------



## orangeboy

innocentfreak said:


> I guess the next question is there anything us non programmers can do to help?


Post ideas about what ya want! 

I have a feeling moyekj will have a java version of something out before long. I'm struggling with (personally) interpreting JSON nomenclature, and getting useful Requests put together. For example, when querying the (local) To Do List, I can only get the first item in the list. I'm sure there's something to say "get the rest", but for the time being, I'm just trying to get a grasp of all the things that can be had, and will worry about the details later....


----------



## orangeboy

ckrames1234 said:


> I love the API documentation, great work arantius. I will definitely try to contribute to it. There are so many things I could do with this, projects I could make, the possibilities are literally endless.
> 
> I never posted sample code on here because in my sample, I tried using a random X-ApplicationSessionId, which I now know is not a good idea.
> 
> (For some reason I never got notified when someone posted on here :/)


Nice to hear! More eyes on it could/should mean quicker development of stuff!


----------



## innocentfreak

orangeboy said:


> Post ideas about what ya want!


LOL
For example, does this mean potentially we could actually transfer and automatically rename shows so they end up in the SxxExx format in the name since the current way is hit or miss on season and episode information even though the Premiere screen will display it properly?


----------



## orangeboy

innocentfreak said:


> LOL
> For example, does this mean potentially we could actually transfer and automatically rename shows so they end up in the SxxExx format in the name since the current way is hit or miss on season and episode information even though the Premiere screen will display it properly?


With proper formatting of the information retrieved from the Premiere, I would think that yes, SxxExx could be added to the subtitle (episode name). I'll have to see if Search capabilities extend to sources like Netflix or Amazon for movies and/or documentaries.


----------



## orangeboy

OK, got past the To Do List search only returning one item by including a "count=" option:


Code:


    def ToDoSearch(self):
        req = RpcRequest('recordingSearch', 
                         levelOfDetail='low', 
                         bodyId='-', 
                         count='50',
                         state=['scheduled'])

Setting a count too high results in:


Code:


{'debug': '', 'text': 'Count is out of range. 1 <= count <= 50', 'code': 'badArgument', 'type': 'error'}

I'm looking for some type of "Anchor" statement that could continue listing beyond the first 50 items.

Adding an offset seems to do the trick:


Code:


    def ToDoSearch(self):
        req = RpcRequest('recordingSearch', 
                         levelOfDetail='low', 
                         bodyId='-', 
                         count='50',
                         offset='50',
                         state=['scheduled'])


----------



## tomhorsley

The thing I'm most interested in is correcting rotten bits of the native tivo interface, like by having a cron job to download schedule info off the tivo and schedule recording of alternate showings of episodes you missed due to a conflict (but which are now not "new" anymore, so the tivo won't automatically record them).

Or a way to correct the "new" flag by recording the descriptions of the shows and scheduling recordings for any show with a new description and cancelling recordings of shows marked new that have a description I already remember.

I can also imagine using the tivo to get schedule info, then using it in a myth TV setup to get an additional tuner that will at least work on unencrypted channels.


----------



## arantius

ckrames1234 said:


> I never posted sample code on here because in my sample, I tried using a random X-ApplicationSessionId, which I now know is not a good idea.


I'm generally mystified by that value yet. It just seemed strange that they seemed to be large numbers in a small range, for what samples I had, so I just did random within that range.



ckrames1234 said:


> (For some reason I never got notified when someone posted on here :/)


As far as I can tell, (depending on options) you get one notification, then no more until you visit the site (thread?).


----------



## orangeboy

Just got to "uiNavigate". Nice. Set a URI gathered from "uiDestinationInstanceSearch", and instantly jump to that (HME) app, just like hitting LiveTV or TiVo button.

URIs:


Code:


Amazon:             x-tivo:hme:uuid:35FE011C-3850-2228-FBC5-1B9EDBBE5863
Blockbuster:        x-tivo:hme:uuid:63ED0C3D-D49C-9602-C322-E0A3D3EA5A3D
Browse Web Videos:  x-tivo:hme:uuid:19452283-D00B-C411-595D-447ABF62F37C
Browse Web Videos:  x-tivo:hme:uuid:6784571A-016C-8A46-DF2F-FD842D78006F
Fandango:           x-tivo:hme:uuid:B9DDBECF-BC6D-5E33-388A-4CF840068D79
Music Choice:       x-tivo:hme:uuid:7FDF1BEB-AF92-B8B6-6269-FAE44C8F887D
Netflix:            x-tivo:hme:uuid:7edeb291-0db8-487f-b842-6b97bc71ad9a
One True Media:     x-tivo:hme:uuid:D1AF7BED-325E-5E6E-B087-72381C92D077
Swivel:             x-tivo:hme:uuid:2BD40EF7-D9D2-3CD4-0130-6C18855CE898
TiVoCast RSS:       x-tivo:hme:uuid:863cb78f-efdd-4106-b572-51733983dc76
Youtube:            x-tivo:hme:uuid:9AA364C9-CF8A-1E1D-B50F-CC65C40D4A96

The function:


Code:


    def uiNavigate(self):
        req = RpcRequest('uiNavigate',
                         uri='x-tivo:hme:uuid:2BD40EF7-D9D2-3CD4-0130-6C18855CE898',
                         bodyId='-')
        self.Write(req)


----------



## arantius

innocentfreak said:


> I guess the next question is there anything us non programmers can do to help?


Dump more traffic (with my mdns daemon and tcpcatcher) to find out any details missing from my doc. E.g.how do i make a season pass? How do i cancel it? Etc.


----------



## innocentfreak

arantius said:


> Dump more traffic (with my mdns daemon and tcpcatcher) to find out any details missing from my doc. E.g.how do i make a season pass? How do i cancel it? Etc.


Ok I will have to look into it tomorrow and see if I can figure it out. Never messed with any of these tools.


----------



## orangeboy

Taking the information that is found from "subscriptionSearch", I've come up with these values that should work with "subscribe" (to create a new Season Pass):



Code:


'keepBehavior' : 'forever',         KUID
'keepBehavior' : 'fifo',            Space Needed

'showStatus' : 'everyEpisode',      All with Duplicates
'showStatus' : 'rerunsAllowed',     First Run & Reruns
'showStatus' : 'firstRunOnly',      First Run Only

'startTimePadding' : 0,             On time
'startTimePadding' : 60,            One minute early    

'endTimePadding' : 0                On time
'endTimePadding' : 60               One minute late

"maxRecordings": 0                  All episodes
"maxRecordings": 25                 Keep at Most 25 episodes

'recordingQuality' : 'best',        *Maybe* basic, medium, high, best, device_default if 'isDigital' : False,

The rest I think would have to come from "contentSearch" "gridRowSearch".
gridRowSearch allows searching by Series title.


----------



## orangeboy

More keyword discoveries through trial-n-error:


Code:


mixMappingSearch
'You must specify exactly one of: childMixId, parentMixId, mixMappingId, mixServiceGroup or minUpdateDate'

tunerConflictEventRegister
'This handler only support monitoring.'

tunerStateEventRegister
'Must specify monitoring.'


----------



## puffdaddy

Good work. Here are some additional supported MRPC/2 requests. Note that some of these may not be implemented in the iPad app. 


Code:


BodyAuthenticate
BodyConfigSearch
ChannelSearch
ChannelUpdate
CheckParentalControlsPinRequest
CheckPurchaseControlPinRequest
ControllerIdVctIdGet
EndUserMessageEventRegister
EndUserMessageRemove
EndUserMessageSearch
EndUserMessageUpdate
IdSearch
KeyEventSend
KnownHostItemRemove
KnownHostItemSearch
KnownHostItemStore
LocalApplicationDataRemove
LocalApplicationDataSearch
LocalApplicationDataStore
OfferSearch
OptStatusGet
PartnerLogging
PartnerTicketRemove
PartnerTicketRenewE
PartnerTicketStatus
PartnerTicketStore
Ping
PromotionalMenuItemGet
RecordingFolderItemEmpty
RecordingFolderItemSearch
RecordingSearch
RecordingUpdate
ServiceStateEventRegister
SetParentalControlsPinRequest
SetPurchaseControlPinRequest
SettingsGet
SettingsStore
ShowcaseStateSearch
Subscribe
SubscriptionSearch
SubscriptionsReprioritize
SyncRequest
SystemDiagnosticSearch
TransportStreamIdMappingsCacheUpdate
TransportStreamIdMappingsGet
TransportStreamIdMappingsScanConfigurationGet
TransportStreamIdMappingsScanConfigurationStore
TransportStreamIdMappingsScanControl
TunerConflictEventRegister
TunerConflictResolve
TunerStateEventRegister
UiNavigate
Unsubscribe
UserContentSearch
UserContentStore
VideoPlaybackInfoEventRegister
VideoPlaybackPositionSet
VideoProviderListGet
WhatsOnSearch
CandyBarGet
ClassicUiSegue
FlashSegue
HmeSegue
ReturnSegue


----------



## orangeboy

puffdaddy said:


> Good work. Here are some additional supported MRPC/2 requests. Note that some of these may not be implemented in the iPad app.
> 
> 
> Code:
> 
> 
> BodyAuthenticate
> BodyConfigSearch
> ChannelSearch
> ChannelUpdate
> CheckParentalControlsPinRequest
> CheckPurchaseControlPinRequest
> ControllerIdVctIdGet
> EndUserMessageEventRegister
> EndUserMessageRemove
> EndUserMessageSearch
> EndUserMessageUpdate
> IdSearch
> KeyEventSend
> KnownHostItemRemove
> KnownHostItemSearch
> KnownHostItemStore
> LocalApplicationDataRemove
> LocalApplicationDataSearch
> LocalApplicationDataStore
> OfferSearch
> OptStatusGet
> PartnerLogging
> PartnerTicketRemove
> PartnerTicketRenewE
> PartnerTicketStatus
> PartnerTicketStore
> Ping
> PromotionalMenuItemGet
> RecordingFolderItemEmpty
> RecordingFolderItemSearch
> RecordingSearch
> RecordingUpdate
> ServiceStateEventRegister
> SetParentalControlsPinRequest
> SetPurchaseControlPinRequest
> SettingsGet
> SettingsStore
> ShowcaseStateSearch
> Subscribe
> SubscriptionSearch
> SubscriptionsReprioritize
> SyncRequest
> SystemDiagnosticSearch
> TransportStreamIdMappingsCacheUpdate
> TransportStreamIdMappingsGet
> TransportStreamIdMappingsScanConfigurationGet
> TransportStreamIdMappingsScanConfigurationStore
> TransportStreamIdMappingsScanControl
> TunerConflictEventRegister
> TunerConflictResolve
> TunerStateEventRegister
> UiNavigate
> Unsubscribe
> UserContentSearch
> UserContentStore
> VideoPlaybackInfoEventRegister
> VideoPlaybackPositionSet
> VideoProviderListGet
> WhatsOnSearch
> CandyBarGet
> ClassicUiSegue
> FlashSegue
> HmeSegue
> ReturnSegue


Wow! "SubscriptionsReprioritize" sounds interesting! I'm going to run the 47 or so new Requests to see what the Premiere spits out. :up:

ETA: It appears the Requests are case-sensitive. When the proper case is provided, I get an error indicating "badArgument" if I don't supply the required or correct keywords. When the proper case isn't supplied, I get an error indicating "routingError". Is the list above properly cased?


----------



## orangeboy

Looking at the images supplied with the iPad app, I'm seeing "Thumbs" ratings. Can you give Thumbs Up/Down preferences with the app? If so, can you capture what is being Requested?

From the iPad app:


----------



## arantius

In the app I've seen it display thumbs ratings already made (on the TiVo), but I've never found a place to set them in app.


----------



## arantius

puffdaddy said:


> Good work. Here are some additional supported MRPC/2 requests. Note that some of these may not be implemented in the iPad app.


Awesome. How did you find these? Can we discover valid request formats also?


----------



## puffdaddy

Those came directly from the tivo software itself (e.g., run strings against tivoapp). The case may well be incorrect (lowercase first letter), and I'd have to poke a bit more to see if anything about the request format can be divined from the disassembly.


----------



## orangeboy

This one was scary:

recordingFolderItemEmpty
'success'

I had the TV turned off, and didn't know if My Shows was going to be emptied! Oddly, my FSI (sitting at 100%) disappeared while checking out my recordings. This may be a "Delete Folder/Group" function, opposed to recordingUpdate... I'll play with that later!

Edit: Looking at Recording History, it _does_ look like 5 recordings were deleted. They all appear to have been Suggestions. Interesting...


----------



## arantius

puffdaddy said:


> Those came directly from the tivo software itself (e.g., run strings against tivoapp).


Ah. How did you get that? I certainly am not in the mood to take apart mt TiVo.


----------



## orangeboy

arantius said:


> Ah. How did you get that? I certainly am not in the mood to take apart mt TiVo.


It sounds like it came from a hacked TiVo, a Series 3 or earlier?


----------



## moyekj

puffdaddy said:


> Those came directly from the tivo software itself (e.g., run strings against tivoapp).


 I'd be very interested in looking at tivoapp binary from a Premiere for other reasons as well. Can you make a download available somewhere?


----------



## orangeboy

orangeboy said:


> It sounds like it came from a hacked TiVo, a Series 3 or earlier?


Whoops - I checked the forum who shall not be named. I was thinking tivoapp was yet to be had. I should have realized that with JMFS expansion, my thinking was incorrect...


----------



## orangeboy

FWIW, I added a new project into Google Code: http://code.google.com/p/tivopremiereremote/

It's YATR - Yet Another TiVo remote. But it solely utilizes the iPad interface to send the commands. It's also the first I used tkinter, so it looks a bit different!









You'll note along the top left side I included "jumps" to the various VOD providers in place of TV power/volume control. Edit the Remote.pyw file to include the IP Address of your TiVo, as well as the MAK. I plan on expanding this project to include "scrub" functionality.

ETA: Let me know if the window "Geometry" gives you issues. I used that feature to place my prototype exactly over the iPad "full" background template, so I could match the button positioning more accurately:









ETA: started new thread.


----------



## orangeboy

moyekj said:


> Note that the 'offset' argument of 'videoPlaybackPositionSet' is the time you want to jump to in milliseconds. i.e. To jump to 5 minute mark it would be 60*1000*5. i.e. It's easy to jump exactly to any point in a program that you want.
> 
> One can use 'videoPlaybackInfoEventRegister' to get current position information. So based on the above 2 it's simple to implement a simple means of jumping forwards/backwards n minutes with a slight modification to the Key method in remote.py. Below is a sample implementation (with no error checking at all, but just for proof of concept):
> 
> 
> Code:
> 
> 
> def Key(self, key, offset=0):
> """Send a key.
> 
> Supported:
> Letters 'a' through 'z'.
> Numbers '0' through '1'.
> Named keys:
> actionA, actionB, actionC, actionD, advance, channelDown,
> channelUp, clear, down, enter, forward, guide, info, left, liveTv,
> pause, play, record, replay, reverse, right, select, slow,
> thumbsDown, thumbsUp, tivo, up, zoom
> """
> if key == ' ':
> key = 'forward'
> elif key == 'jump':
> result = self.Key('position')
> pos = int(result['response']['position'])
> pos += 60000*offset
> req = RpcRequest('videoPlaybackPositionSet', offset=pos)
> elif key == 'position':
> req = RpcRequest('videoPlaybackInfoEventRegister', throttleDelay=1000)
> elif key.lower() in 'abcdefghijklmnopqrstuvwxyz':
> req = RpcRequest('keyEventSend', event='ascii', value=ord(key))
> elif key in '0123456789':
> req = RpcRequest('keyEventSend', event='num' + key)
> else:
> req = RpcRequest('keyEventSend', event=key)
> 
> self.Write(req)
> result = self.Read()
> return result
> 
> Then as examples the following work:
> 
> # Jump 5 minutes ahead
> remote.Key('jump', offset=5)
> 
> # Jump 7 minutes backwards
> remote.Key('jump', offset=-7)
> 
> i.e. There seems to be enough uncovered already to mimic telnet based remotes as well as adding additional functionality such as above.


I made a (rough) python/tkinter Scale widget that does "scrubbing":

<Code snipped, see scrubber.pyw.txt attachment>










Like I say, it's rough, but I like the functionality. Currently it displays minutes in float, so 15 minutes 30 seconds will show up as 15.5000. There's something odd too, where when the app launches, it repositions a few milliseconds before or after the current position. The TiVo responds very quickly though, and can navigate to any position in the recording! More work to be had before I go "prime time" with it.


----------



## moyekj

What I would really like is a means to access the Recording History, specifically for items that will *NOT* record. As it is now one has to wade through a huge list of useless reasons on the TiVo itself (usually firstRun or 28 day rule) why something will not record to find 1 or 2 that won't record because of tuner conflicts. It would be nice to:
1. First detect such conflicts
2. If other Premieres are available then possibly attempt to schedule a recording on another Premiere and if failing to do so then email about it.

i.e. A first step at cooperative scheduling, but just conflict monitoring.

In rpcs2 there is only 1 response to a "recordingSearch" request that offers some hope, a cancellationReason that is not "unknown":

111637-1212-2-response-recordingList.txt


Code:


"cancellationReason": "notFirstRun",


----------



## orangeboy

moyekj said:


> What I would really like is a means to access the Recording History, specifically for items that will *NOT* record. As it is now one has to wade through a huge list of useless reasons on the TiVo itself (usually firstRun or 28 day rule) why something will not record to find 1 or 2 that won't record because of tuner conflicts. It would be nice to:
> 1. First detect such conflicts
> 2. If other Premieres are available then possibly attempt to schedule a recording on another Premiere and if failing to do so then email about it.
> 
> i.e. A first step at cooperative scheduling, but just conflict monitoring.
> 
> In rpcs2 there is only 1 response to a "recordingSearch" request that offers some hope, a cancellationReason that is not "unknown":
> 
> 111637-1212-2-response-recordingList.txt
> 
> 
> Code:
> 
> 
> "cancellationReason": "notFirstRun",


I have a few ideas for using a combination of subscriptionSearch (SPM), gridRowSearch (upcoming) and recordingSearch (To Do List). I'm thinking a few fields from those should give an idea of what's _desired_, what's _available_ to record, and what is actually _scheduled_ to record. Importing datetime as dtm, I'm able to dynamically set some required fields in gridRowSearch to look for specific "collections" coming up:



Code:


    # gridRowSearch
    def gridRowSearch(self):
        req = RpcRequest('gridRowSearch',
                         orderBy='channelNumber',
                         maxStartTime=(dtm.datetime.now() + dtm.timedelta(days=14)).strftime("%Y-%m-%d %H-%M-%S"),
                         minEndTime=dtm.datetime.now().strftime("%Y-%m-%d %H-%M-%S"),
                         collectionId=['tivo:cl.78241342'],
                         levelOfDetail='high',
                         bodyId='-')
        self.Write(req)

The following works as well, but may create ambiguity if there are similarly titled shows:



Code:


    # gridRowSearch
    def gridRowSearch(self):
        req = RpcRequest('gridRowSearch',
                         orderBy='channelNumber',
                         maxStartTime=(dtm.datetime.now() + dtm.timedelta(days=14)).strftime("%Y-%m-%d %H-%M-%S"),
                         minEndTime=dtm.datetime.now().strftime("%Y-%m-%d %H-%M-%S"),
                         title='30 Rock',
                         levelOfDetail='high',
                         bodyId='-')
        self.Write(req)

I'll get started!


----------



## orangeboy

The more I look, the more I find:



Code:


    {
      u 'recordingQuality' : u 'best',
      u 'maxRecordings' : 5,
      u 'keepBehavior' : u 'fifo',
      u 'folderingRules' : u 'subscriptionOnly',
      u 'startTimePadding' : 0,
      u 'title' : u 'UNDERCOVER BOSS',
      u 'bodyId' : u 'tsn:746000000000001',
      u 'priority' : 2525,
      u 'bodyGeneratesCandidates' : True,
      u 'showStatus' : u 'rerunsAllowed',
      u 'idSetSource' :
      {
        u 'titleKeywordOp' : [u 'required'],
        u 'titleKeyword' : [u 'Undercover Boss'],
        u 'type' : u 'wishListSource'
      },
      u 'subscriptionId' : u 'tivo:sb.689',
      u 'levelOfDetail' : u 'high',
      u 'type' : u 'subscription',
      u 'endTimePadding' : 0
    },

Could this mean exporting Wishlists?!


----------



## orangeboy

OK, why am I having such a hard time figuring out how to construct a responseTemplate? Given an example of a JSON response:



Code:


"responseTemplate": [
  {
    "fieldName": [
      "gridRow"
    ],
    "type": "responseTemplate",
    "typeName": "gridRowList"
  },
  {
    "fieldName": [
      "offer"
    ],
    "type": "responseTemplate",
    "typeName": "gridRow"
  },
  {
    "fieldName": [
      "title",
      "contentId",
      "startTime",
      "duration",
      "offerId",
      "collectionId"
    ],
    "type": "responseTemplate",
    "typeName": "offer"
  },
  {
    "fieldName": [
      "channelNumber",
      "sourceType",
      "logoIndex",
      "callSign",
      "isDigital"
    ],
    "type": "responseTemplate",
    "typeName": "channel"
  }
]

how do I turn that into python lists and/or dictionaries? My Google-fu is failing me; I'm only finding reference to the actual json or simplejson modules, which is fine because it lets me know JSON <> python translations can occur, but doesn't really help me translate what I'm seeing into something I can interpret.


----------



## orangeboy

Nevermind, figured it out. It's a dictionary of lists of dictionaries that can contain lists (or is it a list of lists that contain dictionaries that can contain lists?). Regardless, the above turns into this:



Code:


responseTemplate=[{"fieldName":["gridRow"], "typeName": "gridRowList"},
                  {"fieldName":["offer"], "typeName": "gridRow"},
                  {"fieldName":["title", "contentId", "startTime", "duration", "offerId", "collectionId"], "typeName": "offer"},
                  {"fieldName":["channelNumber", "sourceType", "logoIndex", "callSign", "isDigital"], "typeName": "channel"}],

One thing to note: omit "type": "responseTemplate" and use only typeName if using arantius' RpcRequest function. TiVo's servers got angry at me, giving back:



Code:


type field of dictionary should have been 'gridRowSearch', but was 'error'
-or-
com.tivo.trio.schema.SchemaException: must specify either typeName or typeNumber


----------



## moyekj

FYI, played around a little with season passes tonight. Changing priority works as in example:


Code:


{"type":"subscriptionsReprioritize","subscriptionId":["tivo:sb.549"],"priority":2516,"bodyId":"-"}

You 1st need to get season pass Ids then specific data on Ids using "subscriptionSearch" to get the current priority numbers. i.e.:


Code:


{"type":"subscriptionSearch","format":"idSequence","noLimit":"true","bodyId":"-"}

Then for each individual id:


Code:


{"type":"subscriptionSearch","objectIdAndType":"581641651692499","levelOfDetail":"medium","bodyId":"-"}

You get the subscriptionId & priority numbers you need from that.

(The highest priority doesn't start at 1 like one might expect).
Also note if you are on the Season Pass Manager screen when you do this it won't show the changes until you exit and come back in (i.e. screen does not refresh).

One can probably make a backup of all the information obtained in these queries and then use the data to re-create season passes, i.e. Like the crippled tivo.com method but hopefully implemented better. i.e. Finally a season pass backup procedure.


----------



## orangeboy

moyekj said:


> FYI, played around a little with season passes tonight. Changing priority works as in example:
> 
> 
> Code:
> 
> 
> {"type":"subscriptionsReprioritize","subscriptionId":["tivo:sb.549"],"priority":2516,"bodyId":"-"}
> 
> You 1st need to get season pass Ids then specific data on Ids using "subscriptionSearch" to get the current priority numbers. i.e.:
> 
> 
> Code:
> 
> 
> {"type":"subscriptionSearch","format":"idSequence","noLimit":"true","bodyId":"-"}
> 
> Then for each individual id:
> 
> 
> Code:
> 
> 
> {"type":"subscriptionSearch","objectIdAndType":"581641651692499","levelOfDetail":"medium","bodyId":"-"}
> 
> You get the subscriptionId & priority numbers you need from that.
> 
> (The highest priority doesn't start at 1 like one might expect).
> Also note if you are on the Season Pass Manager screen when you do this it won't show the changes until you exit and come back in (i.e. screen does not refresh).
> 
> One can probably make a backup of all the information obtained in these queries and then use the data to re-create season passes, i.e. Like the crippled tivo.com method but hopefully implemented better. i.e. Finally a season pass backup procedure.


You should be able to save a step and get subscriptionId and priority numbers by setting the levelOfDetail (medium or high) in the initial subscriptionSearch, and not specifying the idSequence format.

As it happens, I too played around with this Request last night, but only had limited "success". It moved a Season Pass that I had in position 2 (priority=2472 for me) to position 29 (priority 2500). I initially set "priorty=2471" to see if it would just swap positions 1 and 2, but was surprised to see it way down the line. Subsequent tries with other values (2472, 1, -5) left the particular Season Pass at position 29.

Oddly, my priorities were all initially sequential from 2471-2528, then jumped to 2569 for the very last one. After subscriptionsReprioritize, the order was 2471, 2473-2529, with _*two*_ priorities at 2500. Attempts at creating a responseTemplate to gather only subscriptionId and priority only resulted in "Error parsing subscriptionSearch request".


----------



## moyekj

orangeboy said:


> You should be able to save a step and get subscriptionId and priority numbers by setting the levelOfDetail (medium or high) in the initial subscriptionSearch, and not specifying the idSequence format.
> 
> As it happens, I too played around with this Request last night, but only had limited "success". It moved a Season Pass that I had in position 2 (priority=2472 for me) to position 29 (priority 2500). I initially set "priorty=2471" to see if it would just swap positions 1 and 2, but was surprised to see it way down the line. Subsequent tries with other values (2472, 1, -5) left the particular Season Pass at position 29.
> 
> Oddly, my priorities were all initially sequential from 2471-2528, then jumped to 2569 for the very last one. After subscriptionsReprioritize, the order was 2471, 2473-2529, with _*two*_ priorities at 2500. Attempts at creating a responseTemplate to gather only subscriptionId and priority only resulted in "Error parsing subscriptionSearch request".


Yes it does seem a little strange. I too was unable to move back to its original place through programming after the initial move but didn't think much of it at the time and figured I just gave it wrong priority #. I can see already where if there are many changes at once things get complicated - probably why the tivo.com method screws things up too. I'm a little weary about experimenting too much as I don't want to screw things up.

Still think season pass backup is worth investing more time. Probably good way to test that initially is setup a dummy season pass, save the data associated with it, delete it, then see if one can re-create it with saved data.


----------



## orangeboy

moyekj said:


> Yes it does seem a little strange. I too was unable to move back to its original place through programming after the initial move but didn't think much of it at the time and figured I just gave it wrong priority #. I can see already where if there are many changes at once things get complicated - probably why the tivo.com method screws things up too. I'm a little weary about experimenting too much as I don't want to screw things up.
> 
> Still think season pass backup is worth investing more time. Probably good way to test that initially is setup a dummy season pass, save the data associated with it, delete it, then see if one can re-create it with saved data.


I've got some Season Passes that are now expendable due to Series finales/cancellations (Smallville, V, The Event). Any one of those could "take one for the team". 

A couple posts back, I found that Wish List backups may also be possible, with their keywords being made available. Your earlier mention of "gold mine" may well be an understatement! I'm curious what the TiVo folks think about this thread (if they happen to wander away from the main forums)...


----------



## orangeboy

moyekj said:


> Yes it does seem a little strange. I too was unable to move back to its original place through programming after the initial move but didn't think much of it at the time and figured I just gave it wrong priority #. I can see already where if there are many changes at once things get complicated - probably why the tivo.com method screws things up too. I'm a little weary about experimenting too much as I don't want to screw things up.


This has really got me stumped.

I've found that any reprioritization done through the RPC interface moves the first effected subscriptionId to a priority of 2500 (in my case). Setting a priority value in the request itself appears to have zero effect (at least none that I could observe). The order specified in subscriptionId= is the order of the new priorities, again, starting at 2500 (for me). Based on those observations, I culled all of the subscriptionId values, and placed them into the request, altering the order slightly. The result was the expected order, starting with priority 2500, and running sequentially and consecutively through 2558. OK, no big deal.

The next thing I tried was to specify only 3 subscriptionId values. These were placed at or near the top of the list, and shared the same priority values with the original top 3 of the list. So I had (2) 2500 values, (2) 2501 values, and (2) 2502 values. Well huh. I used my YATR to rearrange the Season Passes back to their original order through the Premiere's GUI. When I rechecked, the priorities reset back to their original values, starting at 2471, and running through 2529!

I noticed in arantius' samples, the priorities were 5008, 5013, and 2502. I'm just wondering if 2500 is some kind of magic number.


----------



## moyekj

moyekj said:


> Still think season pass backup is worth investing more time. Probably good way to test that initially is setup a dummy season pass, save the data associated with it, delete it, then see if one can re-create it with saved data.


I confirmed this worked tonight. I did the following:
1. Obtained all season passes (using levelOfDetail=medium)
2. Wrote the entire JSONArray to a file
3. Read back in the file into a JSONArray
4. On TiVo I manually removed my lowest priority season pass
5. Used "subscribe" to schedule the last item in the array using the fields that were already available (loaded from file):
recordingQuality,maxRecordings,keepBehavior,bodyId,idSetSource,showStatus
I also had to add the following, otherwise if there are any conflicts scheduling fails:
ignoreConflicts=true

Checked Season Pass Manager again and verified it was scheduled in last priority spot and had upcoming recordings showing. Just for grins I also subscribed same one on my other Premiere and verified that worked as well.

So looks like Season Pass backups are finally a reality for Premieres. Obviously this also serves as a way of copying season passes among TiVos.

The fun part of this task is now over - now comes more tedious part to write a GUI around it to save to and load from files and selectively choose which ones you want to schedule on a TiVo.


----------



## orangeboy

moyekj said:


> I confirmed this worked tonight. I did the following:
> 1. Obtained all season passes (using levelOfDetail=medium)
> 2. Wrote the entire JSONArray to a file
> 3. Read back in the file into a JSONArray
> 4. On TiVo I manually removed my lowest priority season pass
> 5. Used "subscribe" to schedule the last item in the array using the fields that were already available (loaded from file):
> recordingQuality,maxRecordings,keepBehavior,bodyId,idSetSource,showStatus
> I also had to add the following, otherwise if there are any conflicts scheduling fails:
> ignoreConflicts=true
> 
> Checked Season Pass Manager again and verified it was scheduled in last priority spot and had upcoming recordings showing. Just for grins I also subscribed same one on my other Premiere and verified that worked as well.
> 
> So looks like Season Pass backups are finally a reality for Premieres. Obviously this also serves as a way of copying season passes among TiVos.
> 
> The fun part of this task is now over - now comes more tedious part to write a GUI around it to save to and load from files and selectively choose which ones you want to schedule on a TiVo.


Cool! Since idSetSource is an array, what fields did you have to include? Just collectionId to identify the Series?

ETA: Coupled with a gridRowSearch, I could see extending this function to include a user search option, with a user typing in a Series title and creating a Season Pass for items found in the (current) Guide Data.


----------



## moyekj

orangeboy said:


> Cool! Since idSetSource is an array, what fields did you have to include? Just collectionId to identify the Series?


 No, I used the entire JSON object as is for idSetSource (as originally read and dumped to file), which in my example was:


Code:


"idSetSource":{"collectionId":"tivo:cl.180357353","type":"seasonPassSource","channel":{"channelNumber":"722","channelId":"tivo:ch.9","isReceived":true,"isBlocked":false,"type":"channel","levelOfDetail":"medium","isHidden":false,"callSign":"USAHD","isDigital":true,"logoIndex":66057,"isKidZone":false,"bitrate":"-1476395008","name":"USA Network HD","sourceType":"cable","isFavorite":false,"stationId":"tivo:st.109673805"}}


----------



## orangeboy

An unexpected nicety: videoPlaybackPositionSet works within the Netflix app! I've been working on incorporating the "scrubber" into YATR*, and happened to try it out with Netflix. I was pleasantly surprised to be able to very quickly move to the end of a stream. MUCH faster than the FF found on the physical remote.

Unrelated, I was wondering about this, found in bodyConfigSearch:



Code:


'hasScheduler' : True

What I was wondering was what hardware _doesn't_ have a scheduler? The TiVo Preview?

*A sneak-peek of YATR with a small scrub bar attached at the bottom:


----------



## moyekj

FYI, you can use following RPC call to get list of all classic UI destinations for "uiNavigate" calls:


Code:


{"levelOfDetail":"high","uiDestinationType":"classicui","noLimit":"true","type":"uiDestinationInstanceSearch","bodyId":"-"}

The summary of uri's for my Premiere:


Code:


"x-tivo:classicui:disablehme"
"x-tivo:classicui:enablehme"
"x-tivo:classicui:livetv"
"x-tivo:classicui:messages"
"x-tivo:classicui:music"
"x-tivo:classicui:photo"
"x-tivo:classicui:timechannel"
"x-tivo:classicui:history"
"x-tivo:classicui:seasonpass"
"x-tivo:classicui:settings"
"x-tivo:classicui:standby"
"x-tivo:classicui:systeminfo"
"x-tivo:classicui:todolist"
"x-tivo:classicui:troubleshooting"
"x-tivo:classicui:wishlist"
"x-tivo:classicui:showcase"
"x-tivo:classicui:playback"
"x-tivo:classicui:showcasegrid"
"x-tivo:classicui:suggestions"

Haven't figured out other valid values for uiDestinationType other than "hme" and "classicuI". There may be a way of getting a list of those values...


----------



## moyekj

To get full list of recordings that will not record:


Code:


{"noLimit":"true","state":["cancelled"],"format":"idSequence","type":"recordingSearch","bodyId":"-"}

The above returns full set of IDs. You can then query individual items with levelOfDetail=medium/high to find out the reason why it won't record:



Code:


{"levelOfDetail":"medium","type":"recordingSearch","objectIdAndType":"509073884986635","bodyId":"-"}

Then look for cancellationReason in returned JSON. For my list there were various cancellationReason values:


Code:


notFirstRun
recordDifferentShowing
programGuideChanged
userCancelledSubscription
expired
explicitlyDeletedFromToDo
gotBetterSuggestion
programSourceConflict

So now one can filter out reasons that usually we don't care about and only list important ones such as "programSourceConflict".
Curiously I do not use/enable suggestions, so not sure why "gotBetterSuggestion" is in there...


----------



## puffdaddy

moyekj said:


> Haven't figured out other valid values for uiDestinationType other than "hme" and "classicuI". There may be a way of getting a list of those values...


Is this was what you were looking for?


Code:


livetv
playback
showcase
showcasegrid
wishlist
todolist
seasonpass
history
timechannel
messages
settings
systeminfo
troubleshooting
networkhelp
testconnection
standby
enablehme
disablehme
music
photo
demoMode
repeatGuidedSetup
externalDrive
networkSetup
connection
watchclip
channelList
channelLineup
restartDvr
setupDialInNumber
connectToService
remoteControl
suggestions
thumbs
myshows
Unsupported screen identifier: %s

*Edit* no, I think you were after whether there are any valid values other than "classicui" or "hme", and I don't see any strings evidence of values beyond those two.


----------



## Fofer

orangeboy said:


> An unexpected nicety: videoPlaybackPositionSet works within the Netflix app! I've been working on incorporating the "scrubber" into YATR*, and happened to try it out with Netflix. I was pleasantly surprised to be able to very quickly move to the end of a stream. MUCH faster than the FF found on the physical remote.
> 
> Unrelated, I was wondering about this, found in bodyConfigSearch:
> 
> 
> 
> Code:
> 
> 
> 'hasScheduler' : True
> 
> What I was wondering was what hardware _doesn't_ have a scheduler? The TiVo Preview?
> 
> *A sneak-peek of YATR with a small scrub bar attached at the bottom:


Wow. Very, very nicely done. :up:


----------



## moyekj

FYI, I've just released a new version of kmttg (v0p8a) which takes advantage of some of these new discoveries, perhaps most notable the ability to backup season passes for Premieres.

Thanks again 'arantius' for the reverse engineering, 'orangeboy' for some help testing and others in this thread that have made contributions. There is yet so much to unlock...


----------



## Fofer

Thank you, moyekj!


----------



## innocentfreak

Nice thanks, downloading now.


----------



## Fofer

moyekj, I have downloaded the new version of kmttg. I entered my MAK and I checked the box next to "Enable iPad style communications with this TiVo" and restarted it. Looks great, and works well, for the most part.

I notice am able to backup my SP's and I am able to see the shows in my Premiere's NP list. 

But when I select a show to initiate playback (and hit the space bar to do so,) I get these errors: "No data available for findRecordingId" or "RPC error response: The active partner ID does not have access to the operation." Basically I am unable to initiate playback on the TiVo of any of these shows, as the release notes indicate I am supposed to be able to do. What am I missing, am I doing something wrong?

Any help appreciated, and thanks for sharing your great work.


----------



## moyekj

Fofer said:


> "RPC error response: The active partner ID does not have access to the operation."


There was a silly bug for playback which is now fixed with new release v0p8b.



Fofer said:


> "No data available for findRecordingId"


This one is tougher issue. In normal kmttg operation to download a show one needs a URL from TiVo which is only available through HMO. Hence the Now Playing list you see is still obtained using HMO method (http based).
However for "iPad playback" functionality one needs a recordingId instead which can only be obtained using RPC calls. i.e. There are 2 different lists of data of the shows, and I have to try and find recordingId in 2nd list based on selection made in the 1st. This is done currently by matching up title, recording date & file size. If a match of all 3 of those is not found then you get above message. My guess would be is the title matching may be failing. Can you post the title of one that is failing? I may just have to re-do logic to not match up titles.


----------



## Fofer

Got it working, thanks for the update. And I haven't seen the "No data available for findRecordingId" response again... I wonder if that was a one-time thing from before I clicked on the "Enable iPad style communications" checkbox.

In any event, it's working now and I like it! Thank you.


----------



## moyekj

ReplayTV had the ability to identify all Season & Series premieres automatically which was a feature I really appreciated and used a lot. There has never been a good equivalent for TiVo. Finally I have been able to implement something for TiVo now using "gridRowSearch".

The algorithm that works is:
Limiting search to 1 channel at a time and then for each JSON entry returned in JSON array looking for:
collectionType="series"
episodeNum=[1]
repeat=false (or no repeat flag present in JSON)
seasonNumber=1 => series premiere
seasonNumber>1 => season premiere

Right now I limit my searches only to certain channels of interest, otherwise the search can take a very long time to complete (I use upcoming 12 days). I plan to release something related to this in next kmttg release, but a GUI for channel filtering is probably going to be needed which will be a pain - I'm really starting to hate GUI programming.


----------



## innocentfreak

Nice, I miss the old wishlist functionality of being able to add watches for Season Premiere, Premiere, and I forget the other terms that TiVo offered as a default. 

Not sure if it would work, but what if you did the channels like you handle the autotransfers? Poll a list of channels with KMTTG and then have the option to highlight channels and add to favorites just the way you add shows to autotransfer with Add Selected Titles. 

I was thinking I wonder if there would be a way to enable season pass manager so you could modify season passes. I need to look at the .SP file KMTTG saves since you have the option to load Season Passes. It makes me think you could possibly move shows around in the file and then reload the file. Of course this could also cause issues so I am hesitant to try it and screw my TiVo up.


----------



## orangeboy

Has anyone tried the discovered interface on the Series 3 family yet? My '648 is still a couple weeks away, packed up and awaiting the movers to come...


----------



## moyekj

moyekj said:


> ReplayTV had the ability to identify all Season & Series premieres automatically which was a feature I really appreciated and used a lot. There has never been a good equivalent for TiVo. Finally I have been able to implement something for TiVo now using "gridRowSearch".
> 
> The algorithm that works is:
> Limiting search to 1 channel at a time and then for each JSON entry returned in JSON array looking for:
> collectionType="series"
> episodeNum=[1]
> repeat=false (or no repeat flag present in JSON)
> seasonNumber=1 => series premiere
> seasonNumber>1 => season premiere
> 
> Right now I limit my searches only to certain channels of interest, otherwise the search can take a very long time to complete (I use upcoming 12 days). I plan to release something related to this in next kmttg release, but a GUI for channel filtering is probably going to be needed which will be a pain - I'm really starting to hate GUI programming.


I have a trial implementation in place as part of kmttg now and innocentfreak took it for a spin and reported a few missing matches. Digging into details the problem is missing episode number information (which is an age old problem with TiVo/zap2it guide listings). i.e. some series don't have episodeNum information.
If anyone has any suggestions on alternate ways of obtaining episode information for such shows via this protocol or other means I'm all ears.


----------



## moyekj

orangeboy said:


> Has anyone tried the discovered interface on the Series 3 family yet? My '648 is still a couple weeks away, packed up and awaiting the movers to come...


 My S3 doesn't even respond to port 1413 at all last I tried, but I will need to check if network remote capability is even enabled on that unit.


----------



## innocentfreak

moyekj said:


> I have a trial implementation in place as part of kmttg now and innocentfreak took it for a spin and reported a few missing matches. Digging into details the problem is missing episode number information (which is an age old problem with TiVo/zap2it guide listings). i.e. some series don't have episodeNum information.
> If anyone has any suggestions on alternate ways of obtaining episode information for such shows via this protocol or other means I'm all ears.


I know sites like TVrage and TheTVDB have APIs you can get access to. I don't think these would really help though unless you did something like Sickbeard in KMTTG for tracking episode guides for shows you specifically added or matched off your season pass listing.


----------



## morac

orangeboy said:


> Has anyone tried the discovered interface on the Series 3 family yet? My '648 is still a couple weeks away, packed up and awaiting the movers to come...


The app is using the older telnet interface for the S3. That's why it only supports basic remote controls. It doesn't support the draggable playback bar. Everything else, like scheduling, is done via the TiVo web site scheduling mechanism.


----------



## tomhorsley

moyekj said:


> If anyone has any suggestions on alternate ways of obtaining episode information for such shows via this protocol or other means I'm all ears.


I (sometimes) see a synopsis of the show when I look at the guide entry (not all shows have a synopsis, or they may have one, but it is the same for every show). For shows that actually do have a synopsis I always wished I could just record the text and use it to remember which shows I've already seen. Seems to me that would be better than an unreliable "new episode" flag since there may have been a power failure or a recording conflict on the first attempt to record it.

That would let me record episodes I haven't seen before (new or not) while avoiding recording the 16 duplicate airing some shows have on some cable channels.


----------



## innocentfreak

tomhorsley said:


> I (sometimes) see a synopsis of the show when I look at the guide entry (not all shows have a synopsis, or they may have one, but it is the same for every show). For shows that actually do have a synopsis I always wished I could just record the text and use it to remember which shows I've already seen. Seems to me that would be better than an unreliable "new episode" flag since there may have been a power failure or a recording conflict on the first attempt to record it.
> 
> That would let me record episodes I haven't seen before (new or not) while avoiding recording the 16 duplicate airing some shows have on some cable channels.


Might want to look into Sickbeard then. This is my big use for it. I simply add shows I want to track that I am recording or watching on Netflix. It pulls episode info from theTVDB.com and then I mark episodes as ignored as I watch them and leave them as skipped until I do.


----------



## moyekj

moyekj said:


> I have a trial implementation in place as part of kmttg now and innocentfreak took it for a spin and reported a few missing matches. Digging into details the problem is missing episode number information (which is an age old problem with TiVo/zap2it guide listings). i.e. some series don't have episodeNum information.
> If anyone has any suggestions on alternate ways of obtaining episode information for such shows via this protocol or other means I'm all ears.


 I improved things just a little by looking at subtitle when episode information is not available and matching "Pilot" or "Series Premiere". But that only applies to series premieres and only those that have those strings in subtitle. Still better than nothing.

For the ones without current season episode information (Warehouse 13 is an example), the HDUI search does not list this season's episode information either which confirms the data is just not available in TiVo listings for such shows.

On unrelated note I found this type of comprehensive grid search really loads the TiVo CPU to the point it becomes pretty sluggish while such a search is happening.


----------



## innocentfreak

Cool let me know and I can test it. I still think it would be cool to tie something like sickbeard into kmttg for an episode guide especially if it could be used to rename TiVo shows with SxxExx info since thetvdb API includes the zap2it series I'd.


----------



## orangeboy

moyekj said:


> On unrelated note I found this type of comprehensive grid search really loads the TiVo CPU to the point it becomes pretty sluggish while such a search is happening.


Would scheduling the search during off hours be possible?


----------



## moyekj

orangeboy said:


> Would scheduling the search during off hours be possible?


 I released a new version of kmttg today with Season Premieres search included. It's up to user when to run it as you need to setup the search and click on Search button to start it. Personally I limit my searches to about 8 channels which takes ~ 3.5 minutes to complete. My Premiere does become sluggish during the search and video playback even breaks up a little at times. The search is very comprehensive as it looks at every show in the guide 12 days ahead for each channel included in the search.


----------



## moyekj

orangeboy said:


> This has really got me stumped.
> 
> I've found that any reprioritization done through the RPC interface moves the first effected subscriptionId to a priority of 2500 (in my case). Setting a priority value in the request itself appears to have zero effect (at least none that I could observe). The order specified in subscriptionId= is the order of the new priorities, again, starting at 2500 (for me). Based on those observations, I culled all of the subscriptionId values, and placed them into the request, altering the order slightly. The result was the expected order, starting with priority 2500, and running sequentially and consecutively through 2558. OK, no big deal.
> 
> The next thing I tried was to specify only 3 subscriptionId values. These were placed at or near the top of the list, and shared the same priority values with the original top 3 of the list. So I had (2) 2500 values, (2) 2501 values, and (2) 2502 values. Well huh. I used my YATR to rearrange the Season Passes back to their original order through the Premiere's GUI. When I rechecked, the priorities reset back to their original values, starting at 2471, and running through 2529!
> 
> I noticed in arantius' samples, the priorities were 5008, 5013, and 2502. I'm just wondering if 2500 is some kind of magic number.


Looking at this again, I think the key then is you must always run the "subscriptionsReprioritize" with JSONArray of all the season pass subscriptionId values, in the order that you want them. Not sure if the "priority" number is even necessary when running the command.

Sort of related, I was looking into how to update existing Season Passes, for example changing number to keep from 10 to 25 or other season pass related properties. AFAICT there is no way to modify an existing SP other than deleting it and adding another one. In order to do that and keep the relative priority to other SPs the "subscriptionsReprioritize" needs to be nailed down to make sure it works as expected.

So I think complete control over SP list is possible. Currently via kmttg one can delete SPs and add new ones with lowest priority (bottom of list). Should be possible to allow re-ordering and updating of individual SP properites as well, but I'm just weary of doing further experimentation beyond that since I don't want to screw things up on either of my Premieres.


----------



## moyekj

FYI, I confirmed tonight that "subscriptionsReprioritize" just needs the full JSON array of subscriptionId's in the order you want them. "priority" parameter is useless/unused. I have a GUI in place in kmttg to change SP order and tested moving a couple of season passes around 5 times and it worked every time. Seems to work a lot more reliably than when I tried a few months ago using tivo.com interface.


----------



## innocentfreak

Very cool, I found the iPad app is much more effective at moving them than using the website. On the website I constantly got errors or the changes didn't go through. 

I would rather move them on the PC though since I usually have the other info in front of me to help me avoid conflicts.


----------



## moyekj

innocentfreak said:


> Very cool, I found the iPad app is much more effective at moving them than using the website. On the website I constantly got errors or the changes didn't go through.
> 
> I would rather move them on the PC though since I usually have the other info in front of me to help me avoid conflicts.


 Yes it's pretty nice that I can now search for series premieres, setup season passes and update the priority of those season passes all from a remote location away from home and without touching my TiVos. Just now scheduled 2 upcoming new show premieres that way.


----------



## Fofer

Now why can't the official TiVo app for iOS do the same? It only lets me manipulate SP's when on my own LAN. They should leverage the functionality of their own web site for this. For mobile users this all ends up feeling like a fractured experience...


----------



## puffdaddy

moyekj said:


> Yes it's pretty nice that I can now search for series premieres, setup season passes and update the priority of those season passes all from a remote location away from home and without touching my TiVos. Just now scheduled 2 upcoming new show premieres that way.


Quite cool. Replaces the lost functionality of the old TivoWebPlus season pass backup/restore. Kudos on taking what was just a tivoapp string and getting it to a useful feature!


----------



## johnd01

Are there other TivoWebPlus features that can be done with this API


----------



## reneg

Here's something that I'd like to see, but I'm not quite sure if it's doable yet with what is known about the MindRPC protocol. I'd like to watch a show on the Tivo and have it use a videoredo .vprj file to automatically skip commercials. Of course this would only work for shows which were transferred off the Tivo but I think it would be cool to have this capability and not have to transfer edited shows back to the Tivo.


----------



## arantius

I've relased my Android app based on this work. See its thread: http://www.tivocommunity.com/tivo-vb/showthread.php?p=8658448

Posting here because the MindRPC related code may be of special interest:
https://github.com/arantius/TiVo-Commander/tree/master/src/com/arantius/tivocommander/rpc


----------



## tomhorsley

Hey! Can this get to the cable card ID numbers? It would be nice if you could cut & paste them somehow into a web interface the cable companies may have someday to avoid typos in number entry.


----------



## arantius

I'm probably wrapping up the last work I will/will need to do on the MindRPC format. The document I wrote on it is probably not 100% up to date, but generally accurate when data is there: https://docs.google.com/document/pub?id=1e4ymm7ROwmW6co2pKENjANGT5xM00VzmzybZ4u8yDE8

Also, I've got a final set of captured data, hosted in many places to hopefully keep it available until (if?) anybody ever wants it:

http://uploadmirrors.com/download/UEIIZMTQ/mindrpc.zip
http://dl.dropbox.com/u/6784911/mindrpc.zip
http://goo.gl/DnUcC
This time, about 4800 requests & responses (including the 900 from my earlier archive). And I think something covering every feature of the (version 1.2) iPad app in there somewhere.

The last thing I plan on doing is playing with the newest version of the iPad app (1.5) and seeing if there are new features to be found.


----------



## moyekj

I just noticed when trying to load a Season Pass I had previously saved to disk that the TiVo won't actually accept it unless the "collectionId" string that is part of "idSetSource" JSON is currently part of guide listings in the destination TiVo. Trying to schedule it yields:
RPC error response: Unknown collection.

So what I thought was true Season Pass backups is not really the case since you can only load Season Passes for shows currently in guide listings. If someone can figure out how to schedule a Season Pass for a show not currently in guide listings I'd love to hear about it. One such show is USA channel "Covert Affairs". I verified doing a search it's not currently in guide listings which is why it's failing.

P.S. I did try scheduling with "collectionId" field removed from "idSetSource" but that failed.


----------



## morac

moyekj said:


> I just noticed when trying to load a Season Pass I had previously saved to disk that the TiVo won't actually accept it unless the "collectionId" string that is part of "idSetSource" JSON is currently part of guide listings in the destination TiVo. Trying to schedule it yields:
> RPC error response: Unknown collection.
> 
> So what I thought was true Season Pass backups is not really the case since you can only load Season Passes for shows currently in guide listings. If someone can figure out how to schedule a Season Pass for a show not currently in guide listings I'd love to hear about it. One such show is USA channel "Covert Affairs". I verified doing a search it's not currently in guide listings which is why it's failing.
> 
> P.S. I did try scheduling with "collectionId" field removed from "idSetSource" but that failed.


The TiVo iPad app itself will only allow scheduling Season Passes (SP) for shows airing in the next 2 weeks (i.e. guide data). That's also true for the Season Pass Manager (SPM) web site. The only method that allows scheduling SPs without the show being in guide data is copying via the SPM and only when both TiVos have the same channel lineup.


----------



## moyekj

morac said:


> The TiVo iPad app itself will only allow scheduling Season Passes (SP) for shows airing in the next 2 weeks (i.e. guide data). That's also true for the Season Pass Manager (SPM) web site. The only method that allows scheduling SPs without the show being in guide data is copying via the SPM and only when both TiVos have the same channel lineup.


Thanks for the info. Since the web site can do it then it means it should be possible, but perhaps not via RPC protocol. Currently the "copy" is really scheduling a new Season Pass which is why collectionId is needed. Ideally there would be a true "copy" operation that does not check against guide data.


----------



## arantius

arantius said:


> The last thing I plan on doing is playing with the newest version of the iPad app (1.5) and seeing if there are new features to be found.


Something about its (iPad app version 1.5) discovery mechanism is different. I used to forge an appropriate mDNS response to get the app to consider my proxy a real TiVo, but that doesn't work anymore. Couldn't notice any new features in the app though, so I'm not super worried.


----------



## jautor

Sorry to revive the old thread, hopefully some of the folks that dove deep into the data are still listening and can answer easily...

I'd like to use this interface to retrieve simple info about what is currently showing (channel, program title, duration, etc.) for use in a 2-way remote control layout. I looked though the first batch of request/responses, but didn't see it. Any pointers?

Thanks,

Jeff


----------



## arantius

jautor said:


> I'd like to use this interface to retrieve simple info about what is currently showing (channel, program title, duration, etc.) for use in a 2-way remote control layout. I looked though the first batch of request/responses, but didn't see it. Any pointers?


Everything I know is written in the doc linked earlier. Sounds like you're interested in tunerStateEventRegister and/or videoPlaybackInfoEventRegister. Might need to dig into the traced requests/responses to find what happens after you call those.


----------



## moyekj

For those that want to investigate this some more with the Spring software update the remote2.py python script needs to be updated to use the new client side PKCS12 certificate & password in order for authentication to work. Example java code for doing it can be found here (createSocketFactory method):
http://code.google.com/p/kmttg/source/browse/trunk/src/com/tivo/kmttg/rpc/Remote.java

If you unzip the official TiVo Android apk file there are some interesting files under there such as a whole description of the MindRPC schema:
com\tivo\trio\schema\schema.conf

All the functions reverse engineered here so far are in there plus many, many more. Looks like many of the those don't actually work through this interface though but there's likely some hidden gems in there that do work.


----------



## swythan

Thanks _so much_ for that moyekj. I was most of the way through writing a Windows Phone app in January when Virgin Media (in the UK) updated their TiVos and blocked the connection.

Your latest version of kmttg works great, and I just got my WP7 app working again, too.


----------



## innocentfreak

So what are the chances someone might be successful with discovering the updated functionality via the Stream?


----------



## moyekj

innocentfreak said:


> So what are the chances someone might be successful with discovering the updated functionality via the Stream?


 Once the Android software is updated to support the Stream that may reveal something as it's much easier to reverse engineer Android java based software (which is how I cracked the PKCS12 certificate & password issue).


----------



## swythan

Incidentally: Virgin Media in the UK updated their TiVo firmware a couple of weeks ago and I can't connect anymore. Given what happened last time (when the need for the client cert was added) I'd expect a similar update to be headed your way (i.e. to the US) in the next few months.

Basically, I can still make the TCP connection and get through the TLS auth phase (which is good!), but get a "not authorised" / "routing error" response to the bodyAuthenticate message (with the correct MAK, of course).

From memory, I think the response was this one:


Code:


{
    "code": "routingError", 
    "text": "The active partner ID does not have access to the operation.",
    "type": "error"
}

I don't expect anyone to be able to work out what needs changing until an updated iOS/Android app comes out. Once that's available I might be on here looking for some guidance!


----------



## Dan203

I was playing with the Stream's system info page yesterday and according to it the video uses encrypted HLS. Not sure if that piece of info is useful to anyone, but someone suggested I post it here for you guys just in case.

Also if you go to the page...

http://<stream ip>:49152/sysinfo

it has some javascript code that calls commands on the stream. Maybe that would help decipher how to communicate with it.

Dan


----------



## bradleys

Dan203 said:


> I was playing with the Stream's system info page yesterday and according to it the video uses encrypted HLS.


Interesting

Protecting Videos on iOS Devices with HLS Encryption


----------



## moyekj

swythan said:


> Incidentally: Virgin Media in the UK updated their TiVo firmware a couple of weeks ago and I can't connect anymore. Given what happened last time (when the need for the client cert was added) I'd expect a similar update to be headed your way (i.e. to the US) in the next few months.
> 
> Basically, I can still make the TCP connection and get through the TLS auth phase (which is good!), but get a "not authorised" / "routing error" response to the bodyAuthenticate message (with the correct MAK, of course).
> 
> From memory, I think the response was this one:
> 
> 
> Code:
> 
> 
> {
> "code": "routingError",
> "text": "The active partner ID does not have access to the operation.",
> "type": "error"
> }
> 
> I don't expect anyone to be able to work out what needs changing until an updated iOS/Android app comes out. Once that's available I might be on here looking for some guidance!


There are many commands that return "The active partner ID does not have access to the operation." messages due to apparently insufficient authorization.

Download the android apk file and unzip it.
Then take a look at:
com/tivo/trio/schema/security.schema

Note that up to this point we've always called "bodyAuthenticate" using type=makCredential, but as you can see in security.schema there are several other credentials that can be used besides "makCredential". For example "temporaryKeyCredential" looks to be interesting. Looks like you can generate a credential using "temporaryKeyCredentialGenerate" and then call "bodyAuthenticate" using that credential. Obviously you would then change type=makCredential to type=temporaryKeyCredential in the bodyAuthenticate json.

I would guess that makCredential is giving us a limited set of operations with which to work and perhaps signing in with different credentials will give us access to a lot more of the operations found in schema.conf. For your situation it may provide an alternate way of getting bodyAuthenticate to work again.

I haven't tried the above yet and too busy with my real job lately to try digging in more, but seems like it's something worth exploring if you have some time.


----------



## swythan

moyekj said:


> Note that up to this point we've always called "bodyAuthenticate" using type=makCredential, but as you can see in security.schema there are several other credentials that can be used besides "makCredential". For example "temporaryKeyCredential" looks to be interesting. Looks like you can generate a credential using "temporaryKeyCredentialGenerate" and then call "bodyAuthenticate" using that credential. Obviously you would then change type=makCredential to type=temporaryKeyCredential in the bodyAuthenticate json.
> 
> I would guess that makCredential is giving us a limited set of operations with which to work and perhaps signing in with different credentials will give us access to a lot more of the operations found in schema.conf. For your situation it may provide an alternate way of getting bodyAuthenticate to work again.


Interesting. I'll definitely take a look at that when I get some time.

Virgin seem to be about to (finally) launch their own iPad app, so I'm guessing that some changes were made to support that (apparently it won't need a direct network connection). I do know the US iPad app currently won't connect to the UK TiVo, so I was expecting to have to wait for new apps to come from TiVo before finding out what had changed.


----------



## moyekj

I played with it a little this morning but didn't really get anywhere:
temporaryKeyCredentialGenerate returns the frustrating "The active partner ID does not have access to the operation." message.
Tried bodyAuthenticate using mmaCredential (with my tivo.com username & password) but TiVo did not like that json and only seems to accept type=makCredential.

It could well that this interface to the "mind" is intentionally limited such that you can't do anything beyond what the current official applications can do.
You'll probably have to wait for the Virgin apps as you mentioned to try reverse engineering what is being done. To me the iOS app is useless as I don't know how to reverse engineer anything there. The Android one to me is a lot easier to reverse engineer.


----------



## Sam Ray

moyekj said:


> What I would really like is a means to access the Recording History, specifically for items that will *NOT* record. As it is now one has to wade through a huge list of useless reasons on the TiVo itself (usually firstRun or 28 day rule) why something will not record to find 1 or 2 that won't record because of tuner conflicts.


I think that is definitely something that Tivo could and should do but if not then someone should. That is, the ability to filter out the lines that are abundant but seldom of interest to see just the exceptions.


----------



## moyekj

Sam Ray said:


> I think that is definitely something that Tivo could and should do but if not then someone should. That is, the ability to filter out the lines that are abundant but seldom of interest to see just the exceptions.


 kmttg has something now which categorizes by reason all shows that won't record.


----------



## moyekj

Also note that there have been some forum posts about the "Fall" update for Series 4 units geting some new HDUI screens including To Do List, so perhaps TiVo took the opportunity to do it right and provide proper filters so you can only see conflicts you really care about. (Of course just like the improved Guide it means you would actually have to use HDUI to get the improvements which means you have to live with HDUI sluggishness).


----------



## Sam Ray

moyekj said:


> kmttg has something now which categorizes by reason all shows that won't record.


Thank you.

I have already downloaded it. It does not start when I open the jar file but I assume it will work; I have not tried to investigate. I certainly should have the association specified but that is the first thing to investigate. This is off-topic for this very useful thread so I don't want to dwell on the off-topic discussion.


----------



## Sam Ray

moyekj said:


> kmttg has something now which categorizes by reason all shows that won't record.


When Java 7 was installed it did not remove Java 6. When I execute kmttg using Java 7 javaw explicitly nothing happens. When I execute the exact some thing using Java 6 it works. So I do not know if it is a problem with the way Java is installed, in which case it is not your problem, or if it is something you would be concerned about. If you want me to post more information, just let me know what and where.


----------



## moyekj

Sam Ray said:


> When Java 7 was installed it did not remove Java 6. When I execute kmttg using Java 7 javaw explicitly nothing happens. When I execute the exact some thing using Java 6 it works. So I do not know if it is a problem with the way Java is installed, in which case it is not your problem, or if it is something you would be concerned about. If you want me to post more information, just let me know what and where.


http://www.tivocommunity.com/tivo-vb/showthread.php?t=387725
I haven't tried Java 7 so could well be issues. I have to keep kmttg Java 5 compatible since some Mac OS's are way behind on Java front.


----------



## moyekj

FYI, I figured out how to get "Recently Deleted" list using "recordingSearch" operation with "state:[deleted]" field. Recordings returned can then be undeleted using "recordingUpdate" operation with "state:complete" field. So next kmttg release will have an additional RPC "Deleted" tab with ability to list and recover deleted shows.


----------



## Fofer

Sweet!


----------



## moyekj

FYI, RPC still works fine as is with the 20.2.2 update being pushed out starting today.
i.e. No changes needed to authentication as was the case last time.
So whatever changes broke it for Virgin and cable company versions do not apply to 20.2.2.


----------



## moyekj

Based on the Android schema.conf file I wrote a script to parse the file and print out a summary of all possible Mind schema v9 operations in a more human readable form. Note that the vast majority of operations are not accessible via RPC, and I've marked each and every operation with a (NO) or (YES) accordingly. In the process of going through that exercise there were a few interesting commands that came up I didn't know worked before, such as:

phoneHomeRequest - initiate a network connection (phone home)
whatsOnSearch - info about what is currently being watched. Only 3 states I've seen so far: "idle", "recording", "liveCache". idle state is if you are currently in a GUI screen or running HME or Flash app, recording if watching a recording, liveCache if viewing live TV. Channel information is given when relevant.

Above 2 I'm making use of in next kmttg release.

Others that are functional via RPC I haven't explored yet that look somewhat interesting:
settingsGet
transportStreamIdMappingsGet
howToWatchSearch
whatToWatchSearch

I uploaded my operation summary file to here in case someone may find it useful:
http://kmttg.googlecode.com/files/rpc_v9_command_summary.txt


----------



## wmcbrine

moyekj said:


> I uploaded my operation summary file to here in case someone may find it useful:
> http://kmttg.googlecode.com/files/rpc_v9_command_summary.txt


Thanks... fascinating stuff.

As you know, the iPad interface isn't the only place where these commands are used -- specifically, the pyTivo "push" system communicates with the TiVo "mind" server using "bodyOfferModify", and a few other commands. Now I finally have a formal spec for bodyOfferModify... and I can see that we're already using all the useful fields. But, I can also see that there are other commands that might let us specify more metadata, if we can figure out how to use them.


----------



## swythan

OK, so Virgin Media in the UK has *finally* released a version of the iPad app that can talk to their version of the TiVo.

Has anyone got a way of capturing the network traffic between the iPad and the TiVo that works (e.g. with the latest US app)? I've got a jailbroken iPad, if that helps.

I've been trying a few different ways to capture the traffic, but this is all new to me, really (I'm a .Net client developer).


----------



## moyekj

swythan said:


> OK, so Virgin Media in the UK has *finally* released a version of the iPad app that can talk to their version of the TiVo.
> 
> Has anyone got a way of capturing the network traffic between the iPad and the TiVo that works (e.g. with the latest US app)? I've got a jailbroken iPad, if that helps.
> 
> I've been trying a few different ways to capture the traffic, but this is all new to me, really (I'm a .Net client developer).


 Android app is much easier to look at for me if there is an Android version that works with Virgin (I would need the apk file).


----------



## badtuned

swythan said:


> OK, so Virgin Media in the UK has *finally* released a version of the iPad app that can talk to their version of the TiVo.
> 
> Has anyone got a way of capturing the network traffic between the iPad and the TiVo that works (e.g. with the latest US app)? I've got a jailbroken iPad, if that helps.
> 
> I've been trying a few different ways to capture the traffic, but this is all new to me, really (I'm a .Net client developer).


Would you be willing to crack the application with your iPad. Plain ipa file is encrypted and there is a small change that the cert password could be found from the binary after it has been decrypted. Process is easy but I don't have any apple devices. Essentially decryption needs to be done on a device which downloaded app from the market.


----------



## swythan

badtuned said:


> Would you be willing to crack the application with your iPad. Plain ipa file is encrypted and there is a small change that the cert password could be found from the binary after it has been decrypted. Process is easy but I don't have any apple devices. Essentially decryption needs to be done on a device which downloaded app from the market.


Yes, I could give that a go if you can point me at a howto.

I've already had a look and found a new cert file, but I checked and the password isn't the same as for the old/US one.

@moyekj : Sorry, no Android app, yet. I would also find that much easier to deal with.

EDIT: Done. I've sent you a link to decrypted binary.


----------



## badtuned

swythan said:


> Yes, I could give that a go if you can point me at a howto.
> 
> I've already had a look and found a new cert file, but I checked and the password isn't the same as for the old/US one.
> 
> @moyekj : Sorry, no Android app, yet. I would also find that much easier to deal with.


"hackulo.us/wiki/IOS_Cracking" (sorry, i need 5 posts to post links) is pretty interesting article and contains links to methods how apps are cracked. It's pretty automated(Clutch and poedCrackMod) these days meaning you don't need to manually play with memory dumps. To cut the story short, vanilla ios will not execute decrypted apps and that's partly the whole point of jailbreaking iDevices.

Yes, I also checked those new certs. There seem to be one cert per tivo application. There's a separate US apps for phone and tablet for android and both of those have different cert(older android tivo app actually had both of those certs in it). I also checked existing cert from US tivo IOS app and that cert is not a same found from android apps.

Every client cert in tivo's basically maps to a partner id. I believe tivo's firmware contains a set of access rules what these partners are allowed to do via mind protocol(Essentially this means that Virgin itself owns a cert which allows admin access to a device). It was pretty natural step for Virgin to change or disable those partner id's which are used in US.


----------



## badtuned

@swythan Can't yet send pm's, but I'll give it a spin.


----------



## badtuned

It's pretty clear that the passphrase is not clear text in ios binary as it's in android app.

Right... going into a world of arm asm language.


----------



## badtuned

Hah, think I got it. Do people generally post hacking methods to this forum?


----------



## moyekj

FYI, you can use this same protocol to talk to middlemind.tivo.com (port 443) instead of a local series 4 TiVo. For bodyAuthenticate you use mmaCredential with your tivo.com username & password instead of the makCredential used for local TiVo authentication, but from that point on everything else is the same.

This opens up some of the functionality provided by this interface to earlier TiVo models such as series 3. I have yet to explore much exactly what does and does not work when using that host. Note that this is the host that is used in "away" mode by the iOS/Android apps. Depending on how much functionality is available I may add RPC remote support (with likely reduced functionality) for earlier TiVo models in next kmttg update.


----------



## morac

moyekj said:


> FYI, you can use this same protocol to talk to middlemind.tivo.com (port 443) instead of a local series 4 TiVo. For bodyAuthenticate you use mmaCredential with your tivo.com username & password instead of the makCredential used for local TiVo authentication, but from that point on everything else is the same.
> 
> This opens up some of the functionality provided by this interface to earlier TiVo models such as series 3. I have yet to explore much exactly what does and does not work when using that host. Note that this is the host that is used in "away" mode by the iOS/Android apps. Depending on how much functionality is available I may add RPC remote support (with likely reduced functionality) for earlier TiVo models in next kmttg update.


Earlier models don't use the same connection to the mind server as the Premiere models. They use the old XMPP/Jabber client/server connection which only supports pushing video downloads, season pass manager updates and not much else. That's why it's not possible to view the Now Playing list for Series 3 models or update the To Do List or Season Passes in real time (the older polling online scheduling method is used).


----------



## moyekj

ToDo lists match up pretty closely to what I see vs local RPC from what I've seen so far with Premiere units. For Season Passes I was able to get Auto-Record wishlists to show up as well which the online SP manager doesn't show, so better functionality possible there. I was also able to delete a season pass on my S3 which look like it worked. I scheduled a season pass and an individual show on my S3 using this interface and it was similar to doing it via tivo.com or one of the other online guides where you get the email confirmations and they worked fine. So it is pretty functional though not real time. I was even able to show Recently Deleted items for my S3 though recovering them or permanently deleting doesn't work through middlemind server. So shows some promise.

I'll have to explore possibility of scheduling season passes for shows not currently in the guide to see if I can get them to work (with local RPC that is not possible), in order to get true Season Pass backups.

P.S. Was also able to update Season Pass options for a SP on my S3 (changed #episodes from 5 to 10) and it worked.


----------



## moyekj

middlemind RPC interface is becoming more and more interesting. Looks like there may be many commands listed in operation summary file I mentioned above that are not available to local RPC but are available via middlemind server. I just started investigating this and 2 operations I've tried so far that I was interested in getting to work previously seem to be available via middlemind:

channelStore
This useful for among other things changing "Channels You Receive" (isReceived field). Somewhat disturbingly it also appears you can change the channel name, logo, number, callsign, etc. if you want to.

categorySearch
This useful for obtaining all categories that can be used for Wishlist creation. kmttg WL buttons omitted category option because I didn't have a way of getting category IDs, but now I do.

I'm sure there will be a ton of other operations that will work via middlemind as well as the above 2 were the only ones I tried so far. For channelStore I need to test it some more with caution (I'm using my older S3 unit for testing that) to see if I can toggle isReceived boolean field which is really the only interest I have in it for now.


----------



## moyekj

Finally figured out how to get HME streaming without the pesky 1.1GB file size limit!
When querying RPC UI destinations, one of the available destinations is the "TiVoCast RSS Video Player HME app". So I figured I'd try to see if I could get it to work with my own URL, and out of convenience fired up pyTivo and tried a pyTivo URL and it worked.

Note that in addition to the pyTivo uri you can set the title, subtitle, description and duration in seconds all of which affect the UI. Here is a sample RPC call to accomplish this:


Code:


{
 "type":"uiNavigate",
 "uiDestinationType":"hme",
 "uri":"x-tivo:hme:uuid:863cb78f-efdd-4106-b572-51733983dc76",
 "parameters":{
  "title":"Title goes here",
  "subtitle":"Subtitle goes here",
  "description":"Description goes here",
  "uri":"http://192.168.10.198:9032/videos/movies/star_wars2.mpg?Format=video%2Fx-tivo-mpeg",
  "duration":8280
 }
}

Note. I tried this on a small video clip as well as a full DVD movie of size 5GB, and the full 5GB movie buffered and I was able to navigate the full movie without interruption.


----------



## innocentfreak

Wow that is very cool. Nice work.


----------



## moyekj

One caveat with pyTivo streaming is it doesn't seem to work when pyTivo needs to transcode (I did comment out ts=on so that transcodes are done to mpeg2 program stream). Works fine with mpeg2 & video/x-tivo-mpeg mime as well as mp4 with video/mp4 mime, but pyTivo transcoding and using video/x-tivo-mpeg doesn't work for some reason...


----------



## wmcbrine

When transcoding, pyTivo uses chunked encoding, because it doesn't know the final length; when not transcoding, it specifies the content-length in the headers. I assume this is the difference.


----------



## davidblackledge

Awesome, you da man, moyekj. I'm looking into adding that to my upcoming app now.

I was hoping that Video Player was the key to watching youtube videos on TiVo, too, but I tried every uri I could come up with for a video on youtube, and it didn't like any of them.
"Video Playback Error | the video has an unknown or incompatible format and cannot be processed by TiVo| Press SELECT to continue"
I now suspect that maybe launching a youtube video uses the youtube app with parameters... I tried using the same parameters as Video Player with no luck.

Could you use the same technique to find parameters available for the youtube app?
uuid=06F36A5F-FE5D-4F03-99A0-64B92896B47F
so 
"uri":"x-tivo:hme:uuid:06F36A5F-FE5D-4F03-99A0-64B92896B47F"

Thanks again, moyekj!


----------



## moyekj

wmcbrine said:


> When transcoding, pyTivo uses chunked encoding, because it doesn't know the final length; when not transcoding, it specifies the content-length in the headers. I assume this is the difference.


 I tried editing plugins/video/video.py to always execute the if compatible section of the headers but it didn't seem to help. Maybe if you can get a chance I would appreciate it if you can monkey with it a little to see if you can get things working for transcoding mode. I've added a crude implementation of starting a stream via RPC using pyTivo as server in the kmttg.jar in this zip. There's a 'pyTivo stream' button under FILES tab. Just make sure your pyTivo.conf is specified in kmttg config and then add a video file in one of your video shares to FILES table, select it and click on the button to start the stream.


----------



## moyekj

davidblackledge said:


> Awesome, you da man, moyekj. I'm looking into adding that to my upcoming app now.
> 
> I was hoping that Video Player was the key to watching youtube videos on TiVo, too, but I tried every uri I could come up with for a video on youtube, and it didn't like any of them.
> "Video Playback Error | the video has an unknown or incompatible format and cannot be processed by TiVo| Press SELECT to continue"
> I now suspect that maybe launching a youtube video uses the youtube app with parameters... I tried using the same parameters as Video Player with no luck.
> 
> Could you use the same technique to find parameters available for the youtube app?
> uuid=06F36A5F-FE5D-4F03-99A0-64B92896B47F
> so
> "uri":"x-tivo:hme:uuid:06F36A5F-FE5D-4F03-99A0-64B92896B47F"
> 
> Thanks again, moyekj!


YouTube on series 4 units is a Flash app, not HME. The HME version doesn't show up in my search. Here's the details on the Flash version:


Code:


{
"instanceName":"YouTube Leanback",
"instanceParameter":[
 {
  "type":"fieldDefinition",
  "valueType":"string",
  "truncate":false,
  "skipInOutput":false,
  "minOccurs":0,
  "description":"Search query, with words separated by the '+' sign. Used to launch app with a specific search query.",
  "copyValueFrom":"Collection.title",
  "ignoreInInput":false,
  "name":"q",
  "levelOfDetail":"high",
  "isNote":false
 },
 {
  "type":"fieldDefinition",
  "valueType":"string",
  "truncate":false,
  "skipInOutput":false,
  "minOccurs":0,
  "description":"YouTube Video ID. Used to launch app with a specific video.",
  "copyValueFrom":"Offer.partnerOfferId",
  "ignoreInInput":false,
  "name":"v",
  "levelOfDetail":"high",
  "isNote":false}
],
"type":"uiDestinationInstance",
"uiDestinationType":"flash",
"levelOfDetail":"high",
"uiDestinationInstanceId":"tivo:di.1000851",
"instanceDescription":"YouTube Leanback",
"uri":"x-tivo:flash:uuid:B8CEA236-0C3D-41DA-9711-ED220480778E",
"serviceGroup":["DG_flash_youtube","DG_hme_youtube_leanback_alpha","DG_hme_youtube_leanback_beta","DG_hme_youtube_leanback_qe"],
"description":"YouTube app","uiDestinationId":"tivo:ud.1000251",
"name":"YouTube",
"parameter":[
 {
  "type":"fieldDefinition",
  "valueType":"string",
  "truncate":false,
  "skipInOutput":false,
  "minOccurs":0,
  "description":"VIDEO or SEARCH",
  "ignoreInInput":false,
  "name":"jumpTo",
  "levelOfDetail":"high",
  "isNote":false
 },
 {
  "type":"fieldDefinition",
  "valueType":"string",
  "truncate":false,
  "skipInOutput":false,
  "minOccurs":0,
  "description":"If jumpto=VIDEO, this should be set to specify which video to play. You can find the video id from the url when watching a youtube video, the v= is the video id",
  "ignoreInInput":false,
  "name":"videoid",
  "levelOfDetail":"high",
  "isNote":false
 },
 {
  "type":"fieldDefinition",
  "valueType":"string",
  "truncate":false,
  "skipInOutput":false,
  "minOccurs":0,
  "description":"If jumpto=SEARCH, this should be set to specify the string to search for.",
  "ignoreInInput":false,
  "name":"query",
  "levelOfDetail":"high","isNote":false
 }
],
"priority":800
}


----------



## davidblackledge

moyekj said:


> YouTube on series 4 units is a Flash app, not HME. The HME version doesn't show up in my search. Here's the details on the Flash version:


YES! that works for the HME version, too! I just have to launch it with parameters "jumpTo=VIDEO" and "videoid=v" where v is the "v" id from a youtube URL!

Thank you!

Now some awesome features of my new app will be thanks to you ;]


----------



## Fofer

davidblackledge said:


> YES! that works for the HME version, too! I just have to launch it with parameters "jumpTo=VIDEO" and "videoid=v" where v is the "v" id from a youtube URL!
> 
> Thank you!
> 
> Now some awesome features of my new app will be thanks to you ;]


Nice, can't wait to see your app!


----------



## swythan

Does anyone know a way of fetching valid TSN's associated with an account using the middlemind gateway?

I can now connect to the Virgin Media middlemind gateway, but to fetch the "My Shows" list I need to include the TSN as the bodyId of the request to avoid getting an error. I know _my _TSN, but I'm wondering how a user that _didn't _have their TiVo accessible on the local network would find theirs out.

FYI: _(The following was worked out by badtuned - I trust he doesn't mind me posting it here)_

The Virgin Media middlemind gateway is at secure-tivo-api.virginmedia.com:443

Instead of using makCredential or mmaCredential, Virgin Media has a new usernameAndPasswordCredential credential type. It uses the user's Virgin Media username & password (i.e. the one used to login to their web account).



Code:


{
    "type":"bodyAuthenticate",
    "credential":
    {
        "type ":"usernameAndPasswordCredential",
        "domain":"virgin",
        "username":"********",
        "password":"********"
    }
}

Note that to connect you will need the new Virgin Media client certificate & password.


----------



## moyekj

Here in the US the TiVos broadcast using Bonjour (as well as old style TiVo Beacon) and advertise information about themselves including their TSN #s, so you can detect TiVos on your LAN and grab information from there.

Also note that when you login using "bodyAuthenticate" you get a "bodyAuthenticateResponse" from middlemind server which includes information about available TiVos on your account in a "deviceId" JSONArray, including tsn #. Here's a sample from mine listing my S3 unit (my other 2 TiVos also were listed but I yanked them out of this posting):


Code:


{
"message": "",
"status": "success",
"type": "bodyAuthenticateResponse"
"deviceId": [
 {
  "friendlyName": "S3",
  "id": "tsn:648000xxxxxxxxx",
  "capabilities": {"type": "bodyCapabilities", "features": ["promptToExtendLive","overlapProtection"]},
  "type": "anyBody",
  "serviceLocation": {"port": 443, "server": "secure-api.tivo.com", "type": "serviceLocationInstruction", "serviceLocationType": "secureApi"}
 },
],
}


----------



## swythan

moyekj said:


> Here in the US the TiVos broadcast using Bonjour (as well as old style TiVo Beacon) and advertise information about themselves including their TSN #s, so you can detect TiVos on your LAN and grab information from there.


At least the mDNS/Bonjour advertising is happening on the VM TiVO, but with slightly different content. Unfortunately there's no way to do mDNS on Windows Phone (my target platform). Anyway, that's a separate question that I'm working around by getting users to enter the IP address themselves, and then getting the TSN from the TLS server certificate.



moyekj said:


> Also note that when you login using "bodyAuthenticate" you get a "bodyAuthenticateResponse" from middlemind server which includes information about available TiVos on your account in a "deviceId" JSONArray, including tsn #.


*That was exactly what I was looking for. Thanks!* I should now be able to support "Away Mode" in my app(s), which will make them useful to loads more people.

Note that the standard configuration in the UK is for the TiVo NOT to be connected to the LAN. It has it's own internal cable modem which it uses for EPG data, VoD, apps, etc. Even if you are using Virgin as your ISP the TiVo has it's own IP address and there is no routing between the TiVo IP and your LAN.

I'm pretty sure that getting this "Away Mode" (i.e. middlemind) working was what made Virgin wait for 12-18 months before releasing an iPad app, as it would have been worthless to the majority of their customers without the "Away Mode" support.


----------



## chirin

Anybody have any advice on running the netflix app and starting content?

It seems like I should be able to just do something like


Code:


MRPC/2 247 179
Type: request
RpcId: 2
SchemaVersion: 7
Content-Type: application/json
RequestType: uiNavigate
ResponseCount: single
BodyId: <tsn>
X-ApplicationName: Quicksilver
X-ApplicationVersion: 1.2
X-ApplicationSessionId: 0x271ced

{"uiDestinationType": "flash", "type": "uiNavigate", "uri": "x-tivo:flash:uuid:F23D193D-D2C2-4D18-9ABE-FA6B8488302F", "parameters": {"jumpTo": "ASIN", "asin": "movies/70208087"}}

Where the asin comes from the partnerOfferId, but that just launches the netflix flash app and doesn't actually play the content.


----------



## moyekj

chirin said:


> Anybody have any advice on running the netflix app and starting content?
> 
> It seems like I should be able to just do something like
> 
> 
> Code:
> 
> 
> MRPC/2 247 179
> Type: request
> RpcId: 2
> SchemaVersion: 7
> Content-Type: application/json
> RequestType: uiNavigate
> ResponseCount: single
> BodyId: <tsn>
> X-ApplicationName: Quicksilver
> X-ApplicationVersion: 1.2
> X-ApplicationSessionId: 0x271ced
> 
> {"uiDestinationType": "flash", "type": "uiNavigate", "uri": "x-tivo:flash:uuid:F23D193D-D2C2-4D18-9ABE-FA6B8488302F", "parameters": {"jumpTo": "ASIN", "asin": "movies/70208087"}}
> 
> Where the asin comes from the partnerOfferId, but that just launches the netflix flash app and doesn't actually play the content.


 From my dump there's also a couple of variables under "instanceParameter" available, so you probably have to set the "partnerOfferId". i.e. something like (obviously replace id with real id):


Code:


"instanceParameter":{"partnerOfferId":"id"}


----------



## chirin

moyekj said:


> From my dump there's also a couple of variables under "instanceParameter" available, so you probably have to set the "partnerOfferId". i.e. something like (obviously replace id with real id):
> 
> 
> Code:
> 
> 
> "instanceParameter":{"partnerOfferId":"id"}


Not having a whole lot of luck, but I'll keep at it. Thanks for the advice.


----------



## chirin

chirin said:


> Not having a whole lot of luck, but I'll keep at it. Thanks for the advice.





Code:


MRPC/2 247 149
Type: request
RpcId: 2
SchemaVersion: 7
Content-Type: application/json
RequestType: uiNavigate
ResponseCount: single
BodyId: tsn:XXXXXXXXXXXXXX
X-ApplicationName: Quicksilver
X-ApplicationVersion: 1.2
X-ApplicationSessionId: 0x26e2b6

{"uiDestinationType": "flash", "type": "uiNavigate", "uri": "x-tivo:flash:uuid:F23D193D-D2C2-4D18-9ABE-FA6B8488302F?partnerOfferId=movies/70208087"}

Success!

Well, it doesn't autoplay, but it opens the selected content, which is good enough.


----------



## moyekj

chirin said:


> Code:
> 
> 
> MRPC/2 247 149
> Type: request
> RpcId: 2
> SchemaVersion: 7
> Content-Type: application/json
> RequestType: uiNavigate
> ResponseCount: single
> BodyId: tsn:XXXXXXXXXXXXXX
> X-ApplicationName: Quicksilver
> X-ApplicationVersion: 1.2
> X-ApplicationSessionId: 0x26e2b6
> 
> {"uiDestinationType": "flash", "type": "uiNavigate", "uri": "x-tivo:flash:uuid:F23D193D-D2C2-4D18-9ABE-FA6B8488302F?partnerOfferId=movies/70208087"}
> 
> Success!
> 
> Well, it doesn't autoplay, but it opens the selected content, which is good enough.


 That's what happens when launching Netflix from HDUI search as well. What's worse is for episodic shows you are taken to the top of the series instead of a particular episode, so then you have to find it again within Netflix among all seasons and episodes. With the older HME Netflix app TiVo would launch you directly into it and start playing it.

BTW, how exactly are you searching for Netflix titles and getting the needed "asin" data?


----------



## moyekj

moyekj said:


> That's what happens when launching Netflix from HDUI search as well. What's worse is for episodic shows you are taken to the top of the series instead of a particular episode, so then you have to find it again within Netflix among all seasons and episodes. With the older HME Netflix app TiVo would launch you directly into it and start playing it.
> 
> BTW, how exactly are you searching for Netflix titles and getting the needed "asin" data?


 Never mind, I found out how. "offerSearch" with Netflix partnerId returns offers with "partnerOfferId" field.


----------



## moyekj

Netflix search via RPC is not very useful, at least for episodic shows. Below is an episode of Star Trek found with "offerSearch" (Star Trek Season 1, episode 9 - Dagger of the Mind). But attempting to use uri with ?partnerOfferId=programs/0000/70109444 doesn't work.


Code:


{"isEpisode":true,"episodic":true,"title":"Star Trek","objectIdAndType":"465102061163341","checksum":"0x3B0DA11791F6F6CB35C17F324DF2B40F","partnerContentId":"epgProvider:ct.EP0039850002","transportType":"partnerStream","seasonNumber":1,"noGiftCardPurchase":false,"type":"offer","contentId":"tivo:ct.252519","purchasableFrom":["dvr"],"levelOfDetail":"low","contentType":"video","hdtv":false,"partnerId":"tivo:pt.3455","collectionId":"tivo:cl.22439","partnerCollectionId":"epgProvider:cl.SH003985","offerId":"tivo:of.ts.52679501","episodeNum":[9],"audioLanguage":["en"],"subscribable":false,"duration":0,"subtitle":"Dagger of the Mind","collectionType":"series","partnerOfferId":"programs/0000/70109444"},

Also as I mentioned searching on TiVo for above episode the episode is found but it only starts Netflix with keyword search "Star Trek" which doesn't help to narrow in much on a specific episode. Can't believe what a lousy mess TiVo allows Netflix integration to be with series 4 platform.


----------



## Fofer

moyekj said:


> Can't believe what a lousy mess TiVo allows Netflix integration to be with series 4 platform.


The HDUI of the base OS _still_ isn't complete.

All of the apps (every one of them) are slow and clunky.

While the addition of Spotify was welcome, character entry doesn't even work with the TiVo Slide remote's keyboard.

TiVo's Amazon app is the single redheaded stepchild that _still_ can't do Instant Prime.

It's all a mess.

I've lost any faith whatsoever that this company can deliver on anything beyond the base DVR functionality we've had for over a decade now.

Any apps above and beyond, I access on my AppleTV or Roku.

It's a sad situation for this purported "One Box" that sadly is anything but.


----------



## philhu

I finally have to second the last comments. The premiere came out YEARS ago now, and still isn't finished. No complete, fast HDUI. The apps are falling apart and are slow. The upgrade cycle is slow and clunky. (How about a tivo app store? )

Every new release fixes 3 items, makes 1 new one and breaks *4*. Every new release does something behind the scenes, like screw up metadata, not allow metadata pieces, display something wrong, break folders. Something!

Do they not regression test the system (Hey, ever hear of ICE)?)? How many developers do they have? Come on, its been 2+ YEARS and HDUI is still broken.

I am starting to think it has all gotten away from them. They have 10+ year old code they are still using and just patching in pieces as they see fit with no rhyme or reason except to kludge it to work. The ultimate sphagetti code, which gets harder and harder to support or upgrade. No long term strategy except to sue someone else over patent infringment.

I was going to upgrade to 2 new tivo premier4 (replacing 3 tivoHD units) and a stream, but instead I will keep one premiere and get 2 FIOS DVR units. Might not be as good as tivo, but they do work and the screens are now hdand do On Demand on fios.


----------



## moyekj

When using middlemind.tivo.com I found this query is useful to get a nice summary of any mind operation:
schemaElementGet

example query:


Code:


{"type":"schemaElementGet", "name":"uiNavigate"}

response:


Code:


{
   "rootElementName": "uiNavigate",
   "operation": [{
      "description": "\n      Causes the UI to navigate to the specified destination.\n    ",
      "name": "uiNavigate",
      "fieldDefinition": [
         {
            "name": "parameters",
            "valueType": "anyDict",
            "type": "fieldDefinition",
            "minOccurs": 0
         },
         {
            "description": "\n        When true, replaces the currently active screen or app on the navigation stack with \n        the new screen or app specified in this request.  A subsequent navigate \n        with a URI of 'x-tivo:return' will not return to the currently active \n        screen or app, but rather the one that was active before it. The default \n        value is false.\n      ",
            "name": "forgetThisScreen",
            "valueType": "boolean",
            "type": "fieldDefinition",
            "minOccurs": 0
         },
         {
            "name": "uri",
            "valueType": "uri",
            "type": "fieldDefinition",
            "minOccurs": 1
         }
      ],
      "isImplemented": false,
      "type": "operation",
      "returnType": ["success"]
   }],
   "levelOfDetail": "low",
   "IsFinal": true,
   "rootElementType": "operation",
   "type": "schemaElement"
}


----------



## moyekj

schemaNumberListGet using middlemind server is also interesting and takes a single parameter "what" whose value is either "allTypes" or "allMembers". Unfortunately for me the "allMembers" query which sounds like the more interesting one to get full set of operation names errors out for some reason. "allTypes" query does work and returns a huge list of all mind object types.


Code:


{"type":"schemaNumberListGet", "what":"allTypes"}


----------



## swythan

Cool. Both those look really handy. Especially for getting up-to-date schema information without having to extract it from one of the official apps.

Schema Parsing
Speaking of which... The other day I wrote a powershell script to pull all the mind operations described in a set of schema files into an Excel spreadsheet, along with their descriptions. If anyone is interested I can post a copy (I'm at work at the moment).

I ran it on the schema extracted from an old copy of the android app that I had lying around. I didn't immediately spot anything particularly interesting that I hadn't seen mentioned here before, but you never know.

Fetching TV Schedule
What I was looking for was a quick way to get "What's on now" information, but I'm none the wiser.

The "Guide" section of the iPad app [1] shows a list of all the available TV channels, with the currently showing program title for each. You can then select a channel and it shows you the schedule for that channel for the rest of the day.

*Does anyone know how to quickly fetch that information (particularly the channel list with "what's on now")? *

I can get it using "gridRowSearch", but it seems very slow compared to how quickly the iPad UI populates itself. Am I just doing it wrong? I tried reducing the time window and level of returned detail for gridRowSearch, which helped, but it still didn't approach how quickly the iPad app does it.

Thanks

[1] The Virgin Media one, but I assume the US one is the same.


----------



## moyekj

I don't know if you've seen kmttg Remote Guide tab? What I do there is there's a "Channels" button you click on once to get full list of all channels using "channelSearch" which I list in the table as folders. That part is not speedy but you only do it once. User can then select a 12 hour time frame to use (can be from now up to about 12 days away and defaults to current time). Then when user clicks on a channel it quickly obtains the listings for that 1 channel in a 12 hour window using "gridRowSearch" and displays the result.
The iPad app is limiting the channel range and the time range to what is displayed in visual range. I believe there has to be individual calls per channel with a very limited time window to do that.
i.e. Speeding it up is all about limiting channel and time range used.


----------



## swythan

moyekj said:


> Speeding it up is all about limiting channel and time range used.


Thanks for that. I'm just glad to know that I've not missed some other way of getting it that I could be using.

BTW: Do you (or anyone else) know if it's still possible to capture the traffic from the iPad (or Android) app? I've been assuming that it's at least _"really quite hard"_ since they changed to authenticating the TLS certificates, but I've never looked to see whether it's actually _impossible_ (given e.g. a jailbroken iPad, some specific network switch, etc).


----------



## moyekj

The TiVo Roamio series that just launched has built in Opera web browser with HTML5 support. After some fussing around I discovered a way to display any arbitrary web page on my Roamio Plus. For example to display the TCF "TiVo Roamio DVRs" web page the RPC command would be:

{
"uiDestinationType":"web",
"type":"uiNavigate",
"bodyId":"tsn:xxxxxxxxxxxxxxx",
"uri":"x-tivo:web:http://www.tivocommunity.com/tivo-vb/forumdisplay.php?f=74"
}

Once displaying a page the remote arrow keys can be used to move around the page and the select key to follow a link. I didn't find a way to go back after clicking a link (the Back button didn't work).

Just for grins I fired up pyTivo and connected to pyTivo server via my Roamio and was able to navigate the web page to select a video to push to the Roamio:
{
"uiDestinationType":"web",
"type":"uiNavigate",
"bodyId":"tsn:xxxxxxxxxxxxxxx",
"uri":"x-tivo:web:http://192.168.10.198:9032"
}

Note that web pages with flash video do not work (missing flash plugin messages).

NOTE: This only works for series 5 TiVos - didn't work for my series 4 units.

Haven't fully explored this as I just discovered this in last hour or so, but perhaps for next kmttg release I'll add another Remote tab where user can enter arbitrary URL to jump to.


----------



## gonzotek

moyekj said:


> Note that web pages with flash video do not work (missing flash plugin messages).


Excellent find! That's funny after all the time they spent playing up how great it was for Flash to be available on the Premiere.

How about pages with html5 tags with tivo-friendly mp4? Maybe try this test page (has ogg theora, webm, and h.264/mp4 examples) http://www.quirksmode.org/html5/tests/video.html


----------



## moyekj

gonzotek said:


> Excellent find! That's funny after all the time they spent playing up how great it was for Flash to be available on the Premiere.
> 
> How about pages with html5 tags with tivo-friendly mp4? Maybe try this test page (has ogg theora, webm, and h.264/mp4 examples) http://www.quirksmode.org/html5/tests/video.html


The H.264/MP4 sample on that page was the only one that worked.


----------



## gonzotek

moyekj said:


> The H.264/MP4 sample on that page was the only one that worked.


Thanks for checking. And that's still good news, as far as I'm concerned.


----------



## moyekj

Setting up a mini web server with basic html5 video tag worked. The TiVo built in web player adds its own basic player with play/pause with "controls" as part of the video tag. As with Flash by default the resolution of the web player is 1280x720, so 720p video fills the screen.
NOTE: mp4 container with ac3 audio resulted in no audio, but aac audio worked.


Code:


<!DOCTYPE HTML>
<html>
<body>
<video width="1280" height="720" controls>
  <source src="h264_aac.mp4" type="video/mp4">
  Your browser does not support the video tag.
</video>
</body>
</html>


----------



## bradleys

Does this mean we can host our own apps directly onto the Roamios?

Outside of an App Store?


----------



## Fofer

Why yes, I believe it does. And there are some amazing HTML5 apps (and games) out there.


----------



## morac

It probably depends on how they are written. Obviously a HTML5 app designed for a tablet won't work on TiVo. I'm guessing TiVo is also using special tags to handle remote presses. It would be interesting to see what the Netflix and YouTube apps look like on a normal web browser.


----------



## moyekj

FYI, the HTML5 YouTube App launches in same manner as above with following URL:
https://www.youtube.com/tv


----------



## bradleys

Pretty cool - this site seems designed for TV, how are the controls?


----------



## morac

bradleys said:


> Pretty cool - this site seems designed for TV, how are the controls?


I can't say what it controls like on a TV, but it works fine on my iPad.


----------



## moyekj

bradleys said:


> Pretty cool - this site seems designed for TV, how are the controls?


 It's the same YouTube App that launches on the Roamios so controls work well and run much faster than the Flash version on the Premieres - but that is mostly because of better hardware not because HTML5 is much faster than Flash.
Note that contrary to what has been said in TCF threads, the Netflix App TiVo is using launches as Flash, not HTML5, which explains why it didn't get user profile support automatically as some expected.


----------



## gonzotek

If possible, can someone with a Roamio try a page with HLS embedded in the video tag? If that works, it's probably possible to implement a pytivo plugin (or other servers like Plex) to transcode to hls on-the-fly.
Here's an Opera pdf that seems to indicate it's a part (or perhaps an option) of the devices sdk that TiVo is using:
http://media.opera.com/media/b2b/tv/201201/Opera_Media_Streaming.pdf

And here are some sample HLS streams embedded in web pages listed at the bottom of Apple's spec page for HLS:
https://developer.apple.com/library...html#//apple_ref/doc/uid/TP40008332-CH102-SW1
They work in Safari on a Mac and iPhone.


----------



## moyekj

gonzotek said:


> If possible, can someone with a Roamio try a page with HLS embedded in the video tag? If that works, it's probably possible to implement a pytivo plugin (or other servers like Plex) to transcode to hls on-the-fly.
> Here's an Opera pdf that seems to indicate it's a part (or perhaps an option) of the devices sdk that TiVo is using:
> http://media.opera.com/media/b2b/tv/201201/Opera_Media_Streaming.pdf
> 
> And here are some sample HLS streams embedded in web pages listed at the bottom of Apple's spec page for HLS:
> https://developer.apple.com/library...html#//apple_ref/doc/uid/TP40008332-CH102-SW1
> They work in Safari on a Mac and iPhone.


Tried the links on the bottom of the page. The page comes up with player controls but there is no video and play time indicated is 0. Pressing Play does nothing.


----------



## gonzotek

Dang, oh well. Thanks for checking.


----------



## moyekj

moyekj said:


> Setting up a mini web server with basic html5 video tag worked. The TiVo built in web player adds its own basic player with play/pause with "controls" as part of the video tag. As with Flash by default the resolution of the web player is 1280x720, so 720p video fills the screen.
> NOTE: mp4 container with ac3 audio resulted in no audio, but aac audio worked.
> 
> 
> Code:
> 
> 
> <!DOCTYPE HTML>
> <html>
> <body>
> <video width="1280" height="720" controls>
> <source src="h264_aac.mp4" type="video/mp4">
> Your browser does not support the video tag.
> </video>
> </body>
> </html>


Playing with this a little more I noticed when I set source H.264 video as 1080p/24 my TV switches to 1080p/24 mode once the video starts playing which looks promising.

The video was > 6GB in size and I monitored total bytes sent to the TiVo and saw it go way beyond the 1.1GB limit that plagues HME streaming, so looks like that is not an issue. Note that likely this is because the transfer is dynamic - i.e. If I pause play then bytes stop transferring to the Roamio so looks like there's no/little buffering happening and this is streaming not transferring. So I think it's very likely that if/when the Mini gets 20.3 software update it will be able to stream video in this manner as well without a problem.

The built in player is very, very basic with only play & pause capabilities and a play bar (I don't see a way to drag/move the play bar) and showing the play bar at all times. But theoretically using HTML5 it should be possible for someone to build a fully custom player that can honor some of the TiVo remote buttons for full trick play functionality.

Of course there's also still the restriction that it seems only mp4 container with h.264 video & AAC audio is supported (and probably requires MOOV atom at start of the file as well though I haven't tested that yet).

I'll release kmttg with Web tab soon so that others that have experience with HTML5 programming can perhaps play with this some more to see what is possible...

EDIT: Actually I did find a way to move the play bar by 1st highlighting it and pressing select and the left/right arrows to move it left/right. However I'm using the very simple/frugal python SimpleHTTPServer as web server and it couldn't handle the scrubbing. Shall have to see if using a real web server solves that issue.

EDIT 2: Switching to use a better web server (I used mongoose) allows scrubbing to be handled properly using the built in TiVo player.


----------



## moyekj

FYI, kmttg v1p0e version just released with new *Web* remote tab for those with Roamio units that want to experiment.


----------



## moyekj

A little progress on a video player with custom controls. This code doesn't display the built in TiVo player controls but instead responds to PAUSE, FF, REW and 1 TiVo remote buttons. I used '1' for toggling info display since I can't seem to get 'Info' button press event (among some other buttons).

I think this is enough for proof of concept that a custom player is possible.


Code:


<!DOCTYPE HTML>
<html>
<body style="overflow:hidden;">
<label id="info" style="color :#FFFFFF; position:absolute; left:30px; top:30px;"></label>
<video id="video" autoplay>
  <source src="h264_aac.mp4" height="auto" type="video/mp4">
  Your browser does not support the video tag.
</video>
<SCRIPT type="text/javascript">
   var sized = 0; // Used by sizeVideo to mark when video sizing is complete
   document.addEventListener( "keypress", doKeyDown, false )
   updateInfo();
   
   function doKeyDown(evt) {
      var video = document.getElementById("video");
      var info = document.getElementById("info");
      // 1 = Toggle info display (Info key doesn't register key event  for some reason...)
      if (evt.which == 49) {
         if (info.style.display == "none") {
            info.style.display = "inline";
         } else {
            info.style.display = "none";
         }
      }
      
      // Pause = toggle pause state
      if (evt.which == 80) {
         if (video.paused == true) {
            video.play();
         } else {
            video.pause();
         }
      }
      
      // REW = skip back 8 seconds
      if (evt.which == 89) {
         video.currentTime -= 8;
      }
      
      // FF = skip forwards 30 seconds
      if (evt.which == 90) {
         video.currentTime += 30;
      }
   };
   
   function pad2(number) {
      return (number < 10 ? '0' : '') + number;
   }
   
   function formattedTime(totalSecs) {
      totalSecs = Math.floor(totalSecs);
      var hours = Math.floor(totalSecs / 3600);
      totalSecs %= 3600;
      var mins = Math.floor(totalSecs / 60);
      var secs = totalSecs % 60;
      
      var display = pad2(mins) + ":" + pad2(secs);
      if (hours > 0) {
         display = pad2(hours) + ":" + display;
      }
      return display;
   }
   
   function sizeVideo() {
      var video = document.getElementById("video");
      if (sized == 0 && video.videoWidth != 0) {
         var w = video.videoWidth;
         var minw = 1280; // Want this min width
         if (w < minw) {
            scale = minw/w;
            w = Math.floor(w * scale);
         }
         video.width = w;
         sized = 1;
      }
   }
      
   function updateInfo() {
      sizeVideo();
      var video = document.getElementById("video");
      var info = document.getElementById("info");
      var duration = video.duration;
      var current = video.currentTime;
      info.innerHTML = formattedTime(current) + " / " + formattedTime(duration); 
      setTimeout("updateInfo()",1000); // Update label text every second
   }
</SCRIPT>
</body>
</html>


----------



## Fofer

This is great stuff! Thanks for sharing the exciting news!


----------



## moyekj

I just realized my Mini does have 20.3 software already (I guess it updated when Roamios released?) and this HTML5 stuff works fine on the Mini as well, including the custom video player.


----------



## AdamNJ

any chance that you are able to navigate to hbogo.com and somehow lightning bolts shoot out of the screen and hbogo works?


----------



## Fofer

Very doubtful as hbogo.com isn't wholly HTML5... it's largely Flash dependent too.


----------



## herbman

Any way to set the user agent? I have found much success in life with certain flash sites by disabling flash (already done in this case) and setting UA to an iPhone. This often pulls down nice h264 mp4 video.


----------



## gonzotek

herbman said:


> Any way to set the user agent? I have found much success in life with certain flash sites by disabling flash (already done in this case) and setting UA to an iPhone. This often pulls down nice h264 mp4 video.


No, not yet anyway. Opera on the desktop allows this (or did the last I checked anyway), but they haven't exposed any browser options on the TiVo so far.


----------



## bradleys

Have you tried putting - opera:config in the url?

It is how you access all the setting in Opera Mobile 6.1


----------



## moyekj

bradleys said:


> Have you tried putting - opera:config in the url?
> 
> It is how you access all the setting in Opera Mobile 6.1


 Interesting suggestion. When doing that the TiVo actually brings up an Opera "Invalid address" box saying that URL is "not currently available" along with some links for opera help. That's the 1st time I've seen that I guess because I didn't try entering invalid URLs previously.

However, the following DID work:
opera:about

Some of the info given:
Version 3.5
Build LSDK5877
Platform Linux
System mips, 3.3.8-2.3
Vega backend DirectFB

Browser identification:
Opera/9.80
...


----------



## moyekj

Some other opera: links that do work:


Code:


opera:plugins
(Reports None found)

opera:cache
(Empty list shown here)

opera:history
(This showed me all of URLs I've entered over last few days)


----------



## bradleys

Just try config: - see if that works...

Others that work with Opera Mini

*o:A-will go to start page
*o:B-will open a page that contains the menu items
*o:C-will open the page that contains the options in the tools menu
*o:X-opens the about page.
*o:Y-add bookmark page
*o:Z-bookmarks
*opera:config-opens the power user settings
*server:sourceURL-shows the source code of the referenced page
*server:refresh-refreshes the page 

Not sure the will work on this browser, but worth a try.


----------



## gonzotek

I wasn't planning on getting a Roamio until early next year or possibly if there was a decent sale before then...but I may just have to put it on a credit card and dive in...  It sounds like there's a lot of fun to be had with it (as a tinkerer), between the hard drive upgrade process and the built-in browser .


----------



## bradleys

Some fun tv apps to try

http://www.theonion.com/video/googletv/

http://vimeo.com/couchmode/staffpicks/sort:date/73790095

http://iptv-app.dailymotion.com/dm-front-googletv/dojoroot/app/pages/webkit/index.jsp?&lang=us

http://redux.com/tv

http://www.clicker.tv/

http://www.crackle.com/gtv/#

Of course porn is available! (NSFW) - don't be shy, tell us how it works!

http://www.vivid.com/


----------



## Fofer

bradleys said:


> Some fun tv apps to try
> 
> http://www.theonion.com/video/googletv/
> 
> http://vimeo.com/couchmode/staffpicks/sort:date/73790095


Wow, if these work, my interest in a Roamio just skyrocketed...


----------



## bradleys

Fofer said:


> Wow, if these work, my interest in a Roamio just skyrocketed...


Yeah, I was going to wait a little bit, but very intrigued! Hoping someone will test these and report on the experience.


----------



## moyekj

Most of the URLs posted above don't work as far as video goes. Unlike normal Opera browsers currently the TiVo one only supports mp4/H264/aac - confirmed this with html5 code and by visiting video test pages. Also there's no flash fallback option either (used by many sites). There is a way to launch to TiVo flash player similar to how to launch to web player, but both are independent of each other.

So I'd say using built in browser for video is fairly worthless right now. If/when TiVo launches an SDK presumably they will provide methods of launching video which cover all the TiVo supported types.


----------



## Fofer

Well, that's bummer. I guess I can leave the Roamio off the shopping list for now. Wish list for the future, maybe.

RE: Flash fallback

Many times when I visit a video site that has flash content, I am able to change the browser agent to iOS:iPad, and then the site gives me an HTML5/mp4 page instead. Not with every site though of course.

On desktop Safari I also use this amazing extension called ClickToFlash that does an excellent job substituting the HTML5/mp4 version of Flash video on many popular sites. It also had an AirPlay button to send that video to an AppleTV.

Just mentioning these to see if such a suggestion may help in this scenario. Since we can't change the Roamio's browser user agent string I am guessing it won't. In that case, we'll have to keep waiting until Flash is declared dead(er) and buried, and these web content creators remove it once and for all and use HTML5/mp4 exclusively instead.


----------



## moyekj

Note that multiple source video failover does work with the TiVo player as it does with most browsers. i.e. If you specify multiple encodings the player will pick one that is supported when possible. For example the TiVo browser doesn't support webm, but failover to mp4 works:


Code:


<video id="video" autoplay>
  <source src="sample.webm" height="auto" type="video/webm">
  <source src="sample.mp4" height="auto" type="video/mp4">
  Your browser does not support the video tag.
</video>

(In above example Firefox doesn't support mp4 but picks the webm version, IE 9 picks the mp4 version, etc.).


----------



## herbman

Can Roamio be configured to use any kind of HTTP proxy?

If so it wouldn't be too hard to write a quick webapp/filter/servlet/etc to simply catch requests and forward them along with a different user agent.


----------



## gonzotek

herbman said:


> Can Roamio be configured to use any kind of HTTP proxy?
> 
> If so it wouldn't be too hard to write a quick webapp/filter/servlet/etc to simply catch requests and forward them along with a different user agent.


Interesting thought..just did some googling around and found old (2006 and earlier) references to TiVos having issues with proxy servers (people trying to get tivos to be transparently proxied, for the purpose of running them in areas where TiVo doesn't provide service support, like South Africa). I've been working with transparent/intercepting proxies for some work-related stuff lately. After I pick up a Roamio, this might be something I can help test.


----------



## onovotny

swythan said:


> At least the mDNS/Bonjour advertising is happening on the VM TiVO, but with slightly different content. Unfortunately there's no way to do mDNS on Windows Phone (my target platform). Anyway, that's a separate question that I'm working around by getting users to enter the IP address themselves, and then getting the TSN from the TLS server certificate.
> 
> *That was exactly what I was looking for. Thanks!* I should now be able to support "Away Mode" in my app(s), which will make them useful to loads more people.
> 
> Note that the standard configuration in the UK is for the TiVo NOT to be connected to the LAN. It has it's own internal cable modem which it uses for EPG data, VoD, apps, etc. Even if you are using Virgin as your ISP the TiVo has it's own IP address and there is no routing between the TiVo IP and your LAN.
> 
> I'm pretty sure that getting this "Away Mode" (i.e. middlemind) working was what made Virgin wait for 12-18 months before releasing an iPad app, as it would have been worthless to the majority of their customers without the "Away Mode" support.


If you're looking for mDNS/Bonjour/Zeroconf on Windows Phone, you can use my Zeroconf NuGet package. Seach for Zeroconf there. Docs and source are on GitHub.

This does require WP8.


----------



## onovotny

It would seem to be currently impossible to build a "real" Win8.1 app for TiVo that uses the MindRPC interfaces.

The real issue is that Win8 and 8.1 do not support sockets with client certificates. Kinda cuts things short very fast 

In Windows 8.1, it is possible to finally ignore SSL errors for HTTP requests, so it'd be possible to use the TTG XML interface. And plain sockets are no problem, so the older remote control API is available too. It'd just be far more limited. 

Bummer though that the "real" API can't be called. It also means TiVo can't write an app themselves for either WP8 or Win8 without a software update on the devices that changes the auth.


----------



## swythan

onovotny said:


> It would seem to be currently impossible to build a "real" Win8.1 app for TiVo that uses the MindRPC interfaces.
> 
> The real issue is that Win8 and 8.1 do not support sockets with client certificates. Kinda cuts things short very fast
> 
> In Windows 8.1, it is possible to finally ignore SSL errors for HTTP requests, so it'd be possible to use the TTG XML interface. And plain sockets are no problem, so the older remote control API is available too. It'd just be far more limited.
> 
> Bummer though that the "real" API can't be called. It also means TiVo can't write an app themselves for either WP8 or Win8 without a software update on the devices that changes the auth.


That did cause me issues for a while, but then I discovered the Bouncy Castle crypto library. It needed a bit of work to get working on the phone, but it made it possible to connect. 

The next problem is that you need a client certificate (which the TiVo _does_ check). I've not come across any obvious method by which TiVo (or Virgin Media) would enable you to get one. So you need to get hold of one of that will work from, err... somewhere else. <cough> ;-)

Anyway, once you've got that (and commented out a check that Bouncy Castle makes on the server certificate that the TiVo is using), then the MindRPC stuff can be used with no problem!

I've not mentioned it on this forum before, but my app is available here. I've only released it in the UK, as that's the only TiVo I've got access to.



onovotny said:


> If you're looking for mDNS/Bonjour/Zeroconf on Windows Phone, you can use my Zeroconf NuGet package. Seach for Zeroconf there. Docs and source are on GitHub.
> 
> This does require WP8.


Thanks for pointing out your mDNS package. I'd already spotted it over the summer, and I was thinking of switching to it for the WP8 version of my app, as the cobbled together solution I'm using at the moment seems to be a bit unreliable. It's nice that you found me, too!


----------



## swythan

swythan said:


> BTW: Do you (or anyone else) know if it's still possible to capture the traffic from the iPad (or Android) app? I've been assuming that it's at least _"really quite hard"_ since they changed to authenticating the TLS certificates, but I've never looked to see whether it's actually _impossible_ (given e.g. a jailbroken iPad, some specific network switch, etc).


Writing above about the client cert reminded me of some work I did on this over the summer...

In an attempt to find out how the new VM iPad app was doing a "Home Mode" connection to a local TiVo without asking for the MAK, I decided to look at capturing the traffic between the iPad app and the "Away Mode" servers. To my surprise, a very simple MitM app worked! I just created some self-signed certificates and opened a TLS socket listening on the relevant port. After editing the hosts file on my (jailbroken) iPad it happily connected to my proxy instead!

Bizarrely it looks like although the TiVo and the Away Mode servers are checking the client certificate; the iPad app itself just completely ignores the server certificate. I didn't even do anything to make my self-signed cert _look_ valid (e.g. editing the trusted root authority lists on the iPad)!

If people are interested I could post the code. It's in C# and probably Windows specific. I imagine Arantius's original script would also probably work if someone who knows Python could make it establish a TLS (server) connection over the socket.

The answer to my orginal question, incidentally, was that the MAK comes back in the "bodyAuthenticateResponse" from the Away Mode servers, but ONLY if you increase the SchemaVersion in your MRPC header to 10. In an amusing quirk, if you send a message with a SchemaVersion higher than *9* to connect directly to a Virgin Media TiVo it will instantly crash and reboot. Luckily my wife was not watching TV when I did that!


----------



## moyekj

Yes please do post the MitM related code. It would be interesting to try and figure out how the iPad app requests an MRS stream from series 4 and 5 TiVos.

Also, for USA the RPC interface connects to TiVo port 1413 as a specific partnerId (3787). That partnerId only has access to very limited Mind operations (I'd say only 5% or less of them). When you try and execute most operations you get message that partnerId doesn't have access to them. It would be really powerful to figure out how to connect to the local Mind with a higher authentication level that is not limited to the tiny set of operations we currently have available.
When connecting to middlemind.tivo.com with username & password for example the # of operations available increases quite significantly, so it implies different authentication methods when connecting to the Mind have different access levels. I'm hoping there is perhaps a different port on the TiVo that once connected to has full Mind operation access.


----------



## onovotny

swythan said:


> That did cause me issues for a while, but then I discovered the Bouncy Castle crypto library. It needed a bit of work to get working on the phone, but it made it possible to connect.
> 
> The next problem is that you need a client certificate (which the TiVo _does_ check). I've not come across any obvious method by which TiVo (or Virgin Media) would enable you to get one. So you need to get hold of one of that will work from, err... somewhere else. <cough> ;-)
> 
> Anyway, once you've got that (and commented out a check that Bouncy Castle makes on the server certificate that the TiVo is using), then the MindRPC stuff can be used with no problem!
> 
> I've not mentioned it on this forum before, but my app is available here. I've only released it in the UK, as that's the only TiVo I've got access to.
> 
> Thanks for pointing out your mDNS package. I'd already spotted it over the summer, and I was thinking of switching to it for the WP8 version of my app, as the cobbled together solution I'm using at the moment seems to be a bit unreliable. It's nice that you found me, too!


Thanks for pointing out that Crypto library. I'll have to see if I can wrap a WinRT plain socket stream around it.

If you have it working in WP8, it must work though as WP8's SocketStream class is the same as Windows 8's.

Any chance you could share some code around getting your connection working?


----------



## bradleys

swythan said:


> I've not mentioned it on this forum before, but my app is available here. I've only released it in the UK, as that's the only TiVo I've got access to.


Good looking app... I wonder if it would work with the retail TiVo's


----------



## swythan

moyekj said:


> Yes please do post the MitM related code. It would be interesting to try and figure out how the iPad app requests an MRS stream from series 4 and 5 TiVos.


No problem. At the moment it's referencing a library of mine that I'm not quite ready to open source (although I'm thinking of it). I'll have a look at making it more self contained tomorrow night and post it up somewhere.



moyekj said:


> Also, for USA the RPC interface connects to TiVo port 1413 as a specific partnerId (3787). That partnerId only has access to very limited Mind operations (I'd say only 5% or less of them).


Interesting. Are you trying to do stuff that the iPad/Android apps can do, or more involved stuff? I've not tried doing anything particularly advanced yet.

From looking at the schema docs that used to be in the Android app I got the impression that they use the same protocol to communicate between the central servers and the TiVo itself, if not also between components on a single TiVo. If that's true then I wouldn't be at all surprised if only the components in the system that they consider "internal" are allowed to do the more advanced operations.

I'd been hoping that TiVo were going to open up a developer program where you could get your app certified, and a client cert & partnerId issued that would give you access to an agreed set of operations, but it looks like that's not going to happen anytime soon.


----------



## swythan

bradleys said:


> Good looking app... I wonder if it would work with the retail TiVo's


Thanks. I imagine it would, with a bit of work to get the configuration right (servers, certificates, etc). The main difficulty is that I'm in the UK and I don't have access to an US retail TiVo.

I'd like to get it working, but the potential for customer support issues if I actually published it to the Store is very off-putting.


----------



## moyekj

swythan said:


> From looking at the schema docs that used to be in the Android app I got the impression that they use the same protocol to communicate between the central servers and the TiVo itself, if not also between components on a single TiVo. If that's true then I wouldn't be at all surprised if only the components in the system that they consider "internal" are allowed to do the more advanced operations.
> 
> I'd been hoping that TiVo were going to open up a developer program where you could get your app certified, and a client cert & partnerId issued that would give you access to an agreed set of operations, but it looks like that's not going to happen anytime soon.


Note that I think arantius with his TiVo Commander app did indeed get his own certificate to use with his app from TiVo, so although there's not an official developer program it looks like you can work something out. Certificate I'm using in kmttg expires in early 2014 so I'll have to figure something out to get a new one. It's much harder now that Android code is obfuscated to get password to go with a new certificate.

Here's just a few examples of operations that work via middlelmind.tivo.com but not port 1314 (i.e. the middlemind server has a bigger scope of operations available):
uiTransitionSearch
schemaNumberListGet
schemaElementGet
operationCountGet

Problem is all the really good operations that are intended to work with local Mind are obviously not available via tivo.com. Things like modifying channel lineup, modifying recorded show properties, etc.


----------



## Philmatic

You guys wanna try Pandora's HTML 5 client? 

https://tv.pandora.com/?model=xbox&...badge=xrniqoryz3kpd46wj3kxto367senmsitdfuli3a


----------



## Dan203

Philmatic said:


> You guys wanna try Pandora's HTML 5 client?
> 
> https://tv.pandora.com/?model=xbox&...badge=xrniqoryz3kpd46wj3kxto367senmsitdfuli3a


Nope. The navigation and signin work, but when you try to play a station it throws an error and then gets stuck in some sort of loop.


----------



## Fofer

Darn. I got excited there for a moment.


----------



## moyekj

If you look at that Pandora URL it is full of Microsoft/Xbox specific options, so looks like Pandora is tailoring HTML5 to specific devices.


----------



## Fofer

Let's hope, then, that they're tailoring one right now, as we type, for TiVo


----------



## bradleys

Is there any reason why the mini platform would not support the Opera App Store?

Can we reasonably expect to see the browser implemented into a future software update?


----------



## moyekj

bradleys said:


> Is there any reason why the mini platform would not support the Opera App Store?
> 
> Can we reasonably expect to see the browser implemented into a future software update?


 The Mini already does run HTML5 - it has same software as the Roamios. You can bring up web pages on the Mini. Also my proof of concept HTML5 video player code I posted in this thread works for true video streaming to the Mini.


----------



## bradleys

Thanks - I will try to play with it tomorrow.


----------



## Philmatic

moyekj said:


> If you look at that Pandora URL it is full of Microsoft/Xbox specific options, so looks like Pandora is tailoring HTML5 to specific devices.


Yes, but it works fine in Chrome on the desktop. It's straight jQuery/Modernizr, which is why I thought it would work on the TiVo, pity.


----------



## moyekj

Philmatic said:


> Yes, but it works fine in Chrome on the desktop. It's straight jQuery/Modernizr, which is why I thought it would work on the TiVo, pity.


 Yes I think the built in browser has pretty minimal support for anything including audio and video tags right now. If there is an html5 version designed to run on TVs it would likely run on the TiVo. That being said I use the current Pandora client on the TiVo every day and it works fine for my needs.


----------



## rocket777

I stumbled onto this thread today and have read the first 30% of it. I'm up to the python scripts.

I installed python in a virtual machine, got a copy of simple json, copied in my ip address and mak and let remote1.py rip. I get an error:

ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:547)

The file remote1.py I got at github submitted by arantius.

Do these scripts still work, and in particular, I've got a *roamio*. I'm a complete novice at python, but I've programmed in other languages. I was hoping to use this script to as a guide to getting moyekj's script going (on post 73 of this thread) to send jumps.

Forgive me if later in this thread there are answers to this, I've spent the day reading this thread and the links trying to get the sample python program going. Since this is my 10th post, perhaps some of the hidden links will give me info I couldn't see before


----------



## moyekj

TiVo caught wind of the reverse engineering and made things harder by introducing PKCS12 authentication with a client side certificate and password. Took quite a bit of detective work to figure out where the certificate was and the associated password, but eventually I figured it out and updated kmttg (Java code) to use. Since then TiVo has made it even harder by further obfuscating the Android app and so it's becoming increasingly more difficult. Certificate I'm currently using expires early next year which is an obstacle I'm not looking forward to overcoming (having to uncover new certificate and associated password will be much harder and likely will have to be done based on the iOS app which is itself pretty obfuscated as well).
Anyway, long story short, the python code would need SSL sockets with PKCS12 authentication which there has been no code posted for in this thread. (Likely is possible but nobody has posted working code for it here).

There's a faint possibility that TiVo may at some point provide an official SDK documenting the mindrpc protocol and providing some official mechanism to get your own certificate + password after registering the app with them. But TiVo has not shown any interest in promoting 3rd party development since they stopped encouraging open source HME development and their Flash-based SDK was a short-lived complete joke. So I don't have high hopes for anything significant to develop but hope I'm wrong.


----------



## arantius

moyekj said:


> TiVo caught wind of the reverse engineering and made things harder by introducing PKCS12 authentication ...


They TiVo-ized it. Even though kmttg and DVR Commander are both open source, the certificates are not and cannot be, so contributing patches is .. not straightforward.



moyekj said:


> Anyway, long story short, the python code would need SSL sockets with PKCS12 authentication which there has been no code posted for in this thread. (Likely is possible but nobody has posted working code for it here).


It absolutely could work. I have a script I wrote for my own purposes which does it. But I can't (effectively) distribute it.



moyekj said:


> There's a faint possibility that TiVo may at some point provide an official SDK documenting the mindrpc protocol ...


Don't hold your breath. A while back I was pointed at http://www.tivo.com/developers -- but now that's just a message saying that program is effectively dead. (And I don't think it ever went anywhere.)


----------



## rocket777

> It absolutely could work. I have a script I wrote for my own purposes which does it. But I can't (effectively) distribute it.


Hmmm, this appears to put a stop on my using the python script in a pipeline so I could write my own little "remote". The idea was this:

myprogram | python-script

Then myprogram could be coded in a language I know (tcl/tk). I figured I could hack the python script to read from stdin turning it into a driver for myprogram.

Sooooo, with that in mind, and the python script not working, I downloaded the latest source of kmttg, and with :up::up::up: on the instructions for building it from source, I was able to get the latest version going with ease (and confirmed the new fractional minutes mod).

I then found in the code where I could add another command line option that would let me use kmttg as a driver. I only need to leverage on the code that sends forward/backward skips, plus pause and unpause.

The idea is to turn a wireless mouse into a remote controller with a scroll wheel and 2 buttons. I have written just such a program for driving the vlc media player.

But before I plunge ahead, however, I was wondering if this seems like a straight forward mod for kmttg. Any suggestions would be appreciated.


----------



## arantius

If you've got the cert to make a connection, the python script (at least the second version) can be rather trivially modified to make an SSL connection with that cert. I.E.



Code:


    self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    self.ssl_socket = ssl.wrap_socket(
        self.socket, certfile=os.path.join(DIR, 'keystore.pem'))
    self.ssl_socket.connect((config.tivo_addr, 1413))

But there's no Python code to make RPCs besides the extremely trivial examples linked above that I know of. If all you want to do is simulate remote presses, I guess it's enough.

I think it's somewhat expensive to set up the SSL connection (for both client and server) so perhaps a daemon holding this connection would be best, if you want transient processes to be sending requests.


----------



## rocket777

arantius said:


> But there's no Python code to make RPCs besides the extremely trivial examples linked above that I know of. If all you want to do is simulate remote presses, I guess it's enough.


Well, not exactly keypresses (of the commands on the existing tivo remote), but rather to be able to skip forward and back quickly by about 5 seconds per mouse wheel click. This amount lets you still see a flow. It's for skipping through a show, not commercials - mouse clicks are for the instant ad skipping. E.G. in a football game, one might scroll from play to play. Much easier than groping a remote for ff and rew or replay etc. More fun! I've tried this out on a game I transferred to my computer and played back in vlc player.



> I think it's somewhat expensive to set up the SSL connection (for both client and server) so perhaps a daemon holding this connection would be best, if you want transient processes to be sending requests.


Yes, from what I could see in wireshark, there's about 20 or so packets per click on the skip button in kmttg. With the latest beta, I set up a 5 second skip and with quick mouse clicks to simulate how quickly I would be twirling the mouse wheel, it seems do-able.

With vlc player, I have to do an html request per skip. It's still fast enough, even with the remote on my laptop and the player on my media computer.

For tivo, Id make right click = pause, left click = 1.9 or 2.9 minute skip forward - perfect for either football or baseball - most of my viewing.

One somewhat tricky item is to bind to any mouse movement and keep the pointer in the center of your own window, so you don't lose focus when the mouse moves. Then you can just put the mouse down on the couch, like it was a regular remote. No need to take your eyes off what your watching. I usually just close the lid on my laptop. When I need to release the mouse, any keyboard key unlocks the mouse from my program.

Since I've already done all this in tcl, that's why I was thinking of a pipeline approach. Add a checkbox and a popup menu selection to my script and an exec call to create the pipe and I'm nearly done on the tcl side.


----------



## alauppe

this stuff blows my mind. keep digging!


----------



## moyekj

FYI, the 20.3.7 software update made things worse for HTML5. Now when you start a web page it no longer grabs focus by default, so I'm unable to scroll through any elements of the page or insert text, etc.


----------



## moyekj

moyekj said:


> FYI, the 20.3.7 software update made things worse for HTML5. Now when you start a web page it no longer grabs focus by default, so I'm unable to scroll through any elements of the page or insert text, etc.


 Series 4 units with 20.3.7 software now support HTML5 as well and have the same bug as with Roamios (not grabbing focus).


----------



## herbman

moyekj said:


> Series 4 units with 20.3.7 software now support HTML5 as well and have the same bug as with Roamios (not grabbing focus).


How does the rendering speed feel?

Without focus could something still run some kind of benchmark with an onload? i.e. sunspider, kraken, etc.

premiere 2 tuner vs premiere 4 tuner vs Roamio basic vs roamio others would be interesting.


----------



## herbman

moyekj said:


> Series 4 units with 20.3.7 software now support HTML5 as well and have the same bug as with Roamios (not grabbing focus).


Other thought, any way to set focus to a known element, or even document element, via javascript on page load?


----------



## moyekj

herbman said:


> How does the rendering speed feel?
> 
> Without focus could something still run some kind of benchmark with an onload? i.e. sunspider, kraken, etc.
> 
> premiere 2 tuner vs premiere 4 tuner vs Roamio basic vs roamio others would be interesting.


 The YouTube html app runs fine on the Premiere now:
http://www.youtube.com/tv
I ran it on my 2 tuner Premiere and while clearly slower than on a Roamio it was actually not too sluggish and very useable. My guess is TiVo is waiting for the general release of 20.3.7 software to be deployed before making it the default YouTube version for series 4 units.


----------



## moyekj

herbman said:


> Other thought, any way to set focus to a known element, or even document element, via javascript on page load?


 For your own apps sure:
document.getElementById('id').focus()

But it used to happen automatically before. Now for example even when you run opera:about there is no way to scroll page down because the main canvas doesn't have focus.


----------



## moyekj

FYI, you can launch html version of netflix on series 4 or 5 TiVos using following RPC command:


Code:


{
"type":"uiNavigate",
"bodyId":"tsn:xxxxxxxxxxxxxxx",
"uri":"x-tivo:netflix:netflix"
}

Similar to YouTube it's a little sluggish on series 4 units but functional. So we might well get html version of Netflix along with YouTube on series 4 units when TiVo decides to flip the switch.
(I was wondering why on the Roamio it has "uiDestinationType":"flash", but that parameter is not needed at all and seems irrelevant given the special netflix uri - x-tivo:netflix).


----------



## moyekj

moyekj said:


> FYI, the 20.3.7 software update made things worse for HTML5. Now when you start a web page it no longer grabs focus by default, so I'm unable to scroll through any elements of the page or insert text, etc.


 20.3.8 software still has this bug and brought back one of the stupid HDUI wishlist issues again - so a step backwards.


----------



## cyphers72

The old Android cert expires 2/14/14. Doing anything interesting here will require extraction of a fresher cert from the Android or iOS app, or finally dig out someone to enable developers at TiVo which will likely never happen at the rate things are going. My understanding is that the last developer relations guy was torched in late 2012 just as the program was finally getting off the ground.


----------



## moyekj

moyekj said:


> FYI, the 20.3.7 software update made things worse for HTML5. Now when you start a web page it no longer grabs focus by default, so I'm unable to scroll through any elements of the page or insert text, etc.





moyekj said:


> 20.3.8 software still has this bug and brought back one of the stupid HDUI wishlist issues again - so a step backwards.


Discovered today using my Slide Pro remote that some buttons do let you navigate around a web page, but not the way the arrow buttons used to work:
Q,A & W,S
Also note that "space" scrolls down if there is a vertical scroll bar.

Looks like Q,A buttons jump up and down between links while W,S jump between other field types (though not input fields).

Better than no control but not as good as the arrow keys used to behave...

NOTE: You can use kmttg Remote tab, Remote sub-tab and then your keyboard buttons to get the above buttons if you don't have a Slide remote.


----------



## swythan

moyekj said:


> Certificate I'm currently using expires early next year which is an obstacle I'm not looking forward to overcoming.


FYI: The client certificate being used by Virgin Media's iPad app (and me) just expired (on 5th Jan 2014). Their entire "TV Anywhere" service essentially went offline for about a day, before coming back to life. You can currently connect through their servers ("Away Mode" / middlemind), but not directly to your own TiVo over the LAN ("Home Mode").

I'm guessing they've (partially?) turned off the client certificate verification on the server. I imagine an updated version of the iPad app will be coming along shortly!

Nice planning Virgin Media! ;-)


----------



## arantius

swythan said:


> I'm guessing they've (partially?) turned off the client certificate verification on the server. I imagine an updated version of the iPad app will be coming along shortly!


If you want to verify this, run DVR Commander (or KMTTG?) against a VM box. Traditionally this fails. I have once in the past verified that all the features work, if a VM cert is used. (But since I don't have a legitimate cert, I've only done so for testing.) If they're really doing no cert validation now, the app would work.

I'd be surprised if they've done this/things will remain this way.


----------



## gonzotek

swythan said:


> FYI: The client certificate being used by Virgin Media's iPad app (and me) just expired (on 5th Jan 2014). Their entire "TV Anywhere" service essentially went offline for about a day, before coming back to life. You can currently connect through their servers ("Away Mode" / middlemind), but not directly to your own TiVo over the LAN ("Home Mode").
> 
> I'm guessing they've (partially?) turned off the client certificate verification on the server. I imagine an updated version of the iPad app will be coming along shortly!
> 
> Nice planning Virgin Media! ;-)


Sounds like a repeat of the hard-coded cookie expiration date TiVos had built-in for session verification of .tivo transfers, which they had to fix last Feb.


----------



## onovotny

@moyekj, Any chance you could put an exe wrapper on Kmttg? Right now it fonts and icons look way too small on Windows Hi DPI displays. On exe's, there are manifest options to specify dpi awareness (enabled or not) and there are compatibility settings that can be set on a shortcut. Neither works on a jar 

I'm happy to help test stuff if you don't have a Hi DPI display.

Thanks!


----------



## moyekj

onovotny said:


> @moyekj, Any chance you could put an exe wrapper on Kmttg? Right now it fonts and icons look way too small on Windows Hi DPI displays. On exe's, there are manifest options to specify dpi awareness (enabled or not) and there are compatibility settings that can be set on a shortcut. Neither works on a jar
> 
> I'm happy to help test stuff if you don't have a Hi DPI display.
> 
> Thanks!


 Not sure why you are posting this in this thread instead of kmttg thread. In kmttg Config->Visual tab you can set font size. For icon you can create a desktop shortcut for kmttg.jar and then change to whatever icon you want by right clicking and selecting Properties->Shortcut->Change Icon.


----------



## onovotny

moyekj said:


> Not sure why you are posting this in this thread instead of kmttg thread. In kmttg Config->Visual tab you can set font size. For icon you can create a desktop shortcut for kmttg.jar and then change to whatever icon you want by right clicking and selecting Properties->Shortcut->Change Icon.


Not sure, just forgot about the other thread? 

That works, though I would hope it could be more automatic. Minor thing, I know, but more ppl are getting high dpi displays. If you can detect you're running on windows, you could do something like this:



Code:


private static int GetZoomFactor()
        {
            try
            {
                using (var regKey = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontDPI"))
                {
                    var pixels = (int)regKey.GetValue("LogPixels");
                    var factor = ((double)pixels / 96) * 100;
                    return (int)factor;
                }
            }
            catch (Exception)
            {
                return 100;
            }
        }


----------



## moyekj

Took some time this morning to see if I could get sample python RPC code working again with the stricter authentication. I did get it working.

* 1st step obviously is obtaining pkcs12 certificate file & associated password. This is the hardest and "top secret" part.
* Generate a cdata.pem file from cdata.p12 file using "openssl" (will prompt you for password ):


Code:


openssl pkcs12 -in cdata.p12 -out cdata.pem -nodes

* Adjust python code to add the pem certificate file:


Code:


self.ssl_socket = ssl.wrap_socket(self.socket, certfile='cdata.pem')

Attached to this post is "remote3.py" which defaults to spitting out MyShows listing with the adjusted authentication and a cdata.pem file that expires 6/1/2015.


----------



## DebiLee

gonzotek said:


> Excellent find! That's funny after all the time they spent playing up how great it was for Flash to be available on the Premiere.
> 
> How about pages with html5 tags with tivo-friendly mp4? Maybe try this test page (has ogg theora, webm, and h.264/mp4 examples)


Ok so now that the Roamio, Mini, and Premiere were redesigned with HAXE instead of Flash, has anything changed with findings from these tests? Has anything been retested recently?


----------



## moyekj

DebiLee said:


> Ok so now that the Roamio, Mini, and Premiere were redesigned with HAXE instead of Flash, has anything changed with findings from these tests? Has anything been retested recently?


 The only development is that since 20.4.1 software update TiVo shutdown ability to launch arbitrary web pages using RPC. See python HME web launcher thread for more on that. Launching Flash pages using uiNavigate still works. If you sign up as Opera TV developer then you get access to their official URL Launcher from paired TiVo(s) from which you can launch web pages.


----------



## HerronScott

moyekj said:


> Confirmation that this is visible to others:
> http://www.tivocommunity.com/tivo-vb/showthread.php?p=10462071#post10462071
> 
> Oops!


Oops indeed! I think TiVo has an authorization issue here with this feature. 

Scott


----------



## Fofer

Wow.


----------



## HerronScott

It will be interesting to see if it really does go away after today based on the end time.

Scott


----------



## morac

I got the TESTING app on my Roamio as well, no HDUI reset was done. It just showed up. That's very scary that anyone can mass put out apps.


----------



## wmcbrine

Oh dear. Well Kevin, it's been nice knowing you.

Maybe you should try actively removing it sooner...


----------



## davidblackledge

moyekj said:


> I'm a little nervous about it because it seems you can't restrict it to a particular TSN, so whatever you do here appears to be universal. Luckily you can specify a start and end time when the UI element you add is valid for, so I used a 24 hour window presuming it will disappear after that.


Holy crap, moyekj.

You didn't think very big though...you could have at least made it an ad for your stuff ;] But I guess you have a reason to be nervous...they'll probably leave you alone BECAUSE it wasn't an ad.

Expect somebody to be working on Easter to patch this rather than lose their job for screwing it up.


----------



## morac

I'd suggest contacting TiVo letting them know what you found and let them know there was no ill intent involved. I'm not sure who you could contact exactly though since they don't have a "security" contact. I guess you could try either Margret or their HQ phone number.


----------



## davidblackledge

moyekj said:


> Net connect + HDUI reset didn't get rid of it, but my guess is it takes several hours for changes to filter through the system just like when adding it.


FYI I finally checked my Roamio and Premiere (without rebooting) for the first time this morning and it wasn't on either of them, so the new expiration may have filtered through before I got there.


----------



## telemark

How About A Nice Game Of Chess?


----------



## Time_Lord

moyekj said:


> Confirmation that this is visible to others:
> http://www.tivocommunity.com/tivo-vb/showthread.php?p=10462071#post10462071
> 
> Oops!


Wow what a security hole. Obviously you didn't push anything beyond an icon but would you have been capable of pushing an app too?


----------



## Fofer

I'm still seeing moyekj's "TESTING" app on my TiVo Mini. I'm guessing it'll go away soon.

Also, still, for quite some time, the Double Spotify's. How do I get rid of the broken Spotify?


----------



## bradleys

This is very interesting - especially if you can figure out how to limit it to a specific box.

But I have a question... Can you use this menu item to actually launch anything? This would be hugely exciting if TiVo hadn't removed ability to launch a web app.


----------



## morac

Moyekj, when did you actually send the command to do this? My Roamio has the "TESTING" icon, but hasn't made a service connection since 2:03 PM Eastern on April 4 (yesterday). 

It sounds like you did it after that which means that neither a service connection nor a HDUI reset is required for it to show up. That matches what I saw when Margret pushed Yahoo Screen to my box. She just emailed me and said it should be there and when I checked it was (no service connection had been done). Pushing seemed to be done through the Mind connection. 

I don't know about having it disappear. In the past I've gotten stuck icons that remain until a reboot. Hopefully that won't be the case now.


----------



## bradleys

I don't have it on either my Roamio or Mini...


----------



## morac

My box just did a connection so I tried resetting the HDUI and the TESTING app is still there.


----------



## Mikeguy

Now, that's a big TiVo "oops."  Hopefully, the Pentagon has patched its similar hole.


----------



## mrizzo80

Holy cow. 

I was linked over here from another thread with a poster explaining how this icon showed up. I thought he was joking that someone from the dev community was responsible...guess not.


----------



## morac

It's still there this morning. Has it been 24 hours? I can't tell since all the original posts about it have been deleted.


----------



## innocentfreak

His post was around 10:27 AM yesterday.

Weird that everything got deleted. I understand removing the original post that had how he did it so it couldn't be recreated, but then anyone subscribed would have received a copy.


----------



## moyekj

If you reset HDUI it will go away.


----------



## Fofer

moyekj said:


> If you reset HDUI it will go away.


How do you reset HDUI on a Roamio or Mini? Is it even possible?


----------



## tatergator1

Thumbs Down, Thumbs Up, Play, Play


----------



## Fofer

I just did that on my Mini -- my screen flashed, and the UI came back -- but it appears to have had no effect on the appearance of moyekj's "TESTING" app... or the double Spotify's.


----------



## moyekj

A reboot of your Mini may fix both those issues.


----------



## morac

After doing a HDUI reset this evening it's gone off my boxes and I have it on good authority that it's no longer possible to do anymore, so everyone can rest easy.

As for the double Spotify apps, if rebooting doesn't remove the duplicate, it's likely the old app is still mapped to your box in which case you'll need contact TiVo.


----------



## moyekj

morac said:


> I have it on good authority that it's no longer possible to do anymore


 Verified.


----------



## davezatz

moyekj said:


> Verified.


Wow, I missed all the fun.


----------



## Fofer

morac said:


> After doing a HDUI reset this evening it's gone off my boxes and I have it on good authority that it's no longer possible to do anymore, so everyone can rest easy. As for the double Spotify apps, if rebooting doesn't remove the duplicate, it's likely the old app is still mapped to your box in which case you'll need contact TiVo.


Thanks! I rebooted and the TESTING app is gone for me. I still see the double Spotify's though, on my Mini (not my Roamio.). I guess I need to call TiVo? Will they have any idea what I am talking about?


----------



## morac

Fofer said:


> Thanks! I rebooted and the TESTING app is gone for me. I still see the double Spotify's though, on my Mini (not my Roamio.). I guess I need to call TiVo? Will they have any idea what I am talking about?


Maybe, maybe not, but if you push it up the chain far enough someone should know.


----------



## Mikeguy

Maybe someone should have added back a functioning Downloads Manager while the door still was open.


----------



## davidblackledge

moyekj said:


> Took some time this morning to see if I could get sample python RPC code working again with the stricter authentication. I did get it working.
> 
> * 1st step obviously is obtaining pkcs12 certificate file & associated password. This is the hardest and "top secret" part.
> * Generate a cdata.pem file from cdata.p12 file using "openssl" (will prompt you for password ):
> 
> 
> Code:
> 
> 
> openssl pkcs12 -in cdata.p12 -out cdata.pem -nodes
> 
> * Adjust python code to add the pem certificate file:
> 
> 
> Code:
> 
> 
> self.ssl_socket = ssl.wrap_socket(self.socket, certfile='cdata.pem')
> 
> Attached to this post is "remote3.py" which defaults to spitting out MyShows listing with the adjusted authentication and a cdata.pem file that expires 6/1/2015.


6/1 is coming up soon... I just now finally am interested in this because YouTube went away from HME (just gives a "device not supported" video now). I realized the best answer I can come up with to replace it for Enter Webz is to launch the latest YouTube app with context. RPC is the only way I know to do that.
THEREFORE my previous enhancement to launch arbitrary HME for a video could be used on YouTube instead to launch a locally-installed HME app that runs an RPC command to launch YouTube with context. *whew*

In other words, I need to learn how to launch mind rpc YouTube. I can then provide an HME app for users to install that does that launching with context Enter Webz passes into the HME app.

Is this worth pursuing at all considering your problems with certificates?

Perhaps I could just tell people "install moyekj's rpc stuff" and I'll provide an HME app that runs your stuff?

Or are you going to just abandon this stuff because of the difficulties with certificates?


----------



## moyekj

TiVo's latest iOS app version has not updated the certificate, and my guess TiVo doesn't even realize that and will have to issue a quick patch release with new certificate once people complain it's not working. Then we'll have to see if it's possible to reverse engineer it again as it's becoming harder and harder. May well turn out to be impossible and all this RPC stuff becomes unusable. Other long, long shot is asking TiVo for an official certificate, but chances of getting one that way I presume are next to none. Perhaps arantius can renew his that expires in July?


----------



## davidblackledge

Thanks. Well, if something works out, I'll offer up a patch for kmttg that will work for what I'm talking about.

I figure just updating the httpserver's rpc command to support YouTube would be an easy enough solution, then I could provide the HME app that activates the rpc command request.

(I guess I could poke around and see if kmttg could also provide what I need to support launching some other video streams since wmcbrine apparently hit a wall with the latest VLC.)


----------



## moyekj

nooneuknow said:


> Could somebody that has done the conversion of the cdata.p12 to cdata.pem, please PM me a link to the .pem version?
> 
> I'd do it myself, but am juggling chainsaws, dealing with a lot of personal matters these days (and my time here is limited, both in how much I can spend here, now, and how long until it is time to say goodbye to TiVo, TCF, and all the great members I've met here). I already have the .p12 version.
> 
> I have a LOT of wireshark captures I've been doing for months, that I intend to use to help contribute to the developers and the community, by distributing them among those I can trust with the data within.
> 
> In order to move forward, I need the .pem version.
> 
> Thanks in advance, to whoever is kind enough to save me the time, by getting the .pem conversion over to me.


The remote3.zip attachment of this post has one, but be advised it expires soon:
http://www.tivocommunity.com/tivo-vb/showthread.php?p=9984124#post9984124


----------



## moyekj

I was very intrigued by Bolt release and the SkipMode feature and started poking around with RPC to see if I could get hold of skip points.

Being able to get exact commercial points has some huge potential uses:
1. Initiate playback via RPC and monitor playback and automatically skip commercials without any user interaction.
2. Use the obtained points as part of kmttg "Ad Cut" step to precisely eliminate commercials without having to use innacurate commercial detection like comskip.

I did manage to get as far as being able to retrieve skip data for any particular show in your My Shows list. Below find data on skip points for a particular show (Scorpion in this case).
However I need some more eyeballs to help decipher exactly how to use the data.

In the data below the "segment" array has what looks to be the non commercial segments of the show (start time and end time in msecs) corresponding to a total of 6 show segments. If you take "endOffset" - "startOffset" for each in msecs and convert to minutes you get:
11.6 mins, 5.6 mins, 5.5 mins, 8.4 mins, 11.6 mins, 0.5 mins for a total of about 43 mins (which jives with typical length of a 1 hour show with commercials removed).

The part I can't quite decipher is what are the time offsets relative to?
And what are the 100s of "clipSyncMark" entries used for?

It seems like one can use the "segment" array along with the "clipMetadata" array together to figure things out. What I need to end up with is offset points in msecs relative to start of the show. i.e. If a commercial point starts at 5 mins then I'd want it to be 300,000 msecs etc.



Code:


   "isTop": true,
   "clipMetadata": [{
      "channelNumber": "1209",
      "contentId": "tivo:ct.335095663",
      "description": "version 1.0  (846634)",
      "contentTitle": "Scorpion",
      "authorId": "4",
      "channelSourceType": "cable",
      "contentSeasonNumber": 2,
      "type": "clipMetadata",
      "syncMark": [
         {
            "type": "clipSyncMark",
            "hash": "4047110867",
            "timestamp": "22924379"
         },
         {
            "type": "clipSyncMark",
            "hash": "1621165014",
            "timestamp": "22926147"
         },
         {
            "type": "clipSyncMark",
            "hash": "1541765360",
            "timestamp": "22927949"
         },
...
100s more clipSyncMark structures here
...
      "offerStartTime": "2015-10-06 01:00:00",
      "contentEpisodeNumber": 3,
      "contentSubtitle": "Fish Filet",
      "segment": [
         {
            "endOffset": "23678765",
            "startOffset": "22979867",
            "description": "Segment 1",
            "keyword": [""],
            "type": "clipSegment"
         },
         {
            "endOffset": "24173626",
            "startOffset": "23839593",
            "description": "Segment 2",
            "keyword": [""],
            "type": "clipSegment"
         },
         {
            "endOffset": "24695314",
            "startOffset": "24364851",
            "description": "Segment 3",
            "keyword": [""],
            "type": "clipSegment"
         },
         {
            "endOffset": "25412664",
            "startOffset": "24910663",
            "description": "Segment 4",
            "keyword": [""],
            "type": "clipSegment"
         },
         {
            "endOffset": "26336554",
            "startOffset": "25638657",
            "description": "Segment 5",
            "keyword": [""],
            "type": "clipSegment"
         },
         {
            "endOffset": "26516467",
            "startOffset": "26489140",
            "description": "Segment 6",
            "keyword": [""],
            "type": "clipSegment"
         }
      ],
      "channelAffiliate": "CBS Affiliate",
      "channelStationId": "tivo:st.1524",
      "clipMetadataId": "tivo:cm.8653",
      "segmentType": "adSkip",
      "createDate": "2015-10-06 01:59:31"
   }],
   "type": "clipMetadataList",
   "IsFinal": true,
   "isBottom": true
}


----------



## reneg

Perhaps the offsets help position to the closest SyncMark. Perhaps the first SyncMark is the start of the hour?

Just playing around in excel, does this make sense? It's close to an hour.


Code:


 	Label	Offset	end-start	(start+1)-end	Seconds	Minutes	Comment
1st SyncMark	22924379		55488	55.49	0.92	Commercial
						
startOffset	22979867	698898		698.90	11.65	Show
endOffset	23678765		160828	160.83	2.68	Commercial
						
startOffset	23839593	334033		334.03	5.57	Show
endOffset	24173626		191225	191.23	3.19	Commercial
						
startOffset	24364851	330463		330.46	5.51	Show
endOffset	24695314		215349	215.35	3.59	Commercial
						
startOffset	24910663	502001		502.00	8.37	Show
endOffset	25412664		225993	225.99	3.77	Commercial
						
startOffset	25638657	697897		697.90	11.63	Show
endOffset	26336554		152586	152.59	2.54	Commercial
						
startOffset	26489140	27327		27.33	0.46	Show
endOffset	26516467					Commercial
				2590619	1001469	3592.09		
						
				3592088	Total Milliseconds


----------



## moyekj

Difficult to see exactly what you did. Can you attach your excel spreadsheet (in a zip)? Thanks.


----------



## moyekj

OK, never mind, I think I see your point. Basically you just subtract out the first SyncMark to get the position relative to start of the show. i.e. The 1st SyncMark is time 0 for the show. What's the point of having 100s of other SyncMarks in the data though?


----------



## reneg

I thought of uploading the spreadsheet or pic from Excel, I should have done that.

I'd guess the 100's of syncs are there to help the Tivo maintain it's precision of jump points within a show. Based on what I've read, they have to be very careful not to miss any of the show during their skips. They can't skip too soon and they can't come into a show late.

Perhaps the sync list is there to resync if there is a video dropout during the broadcast. A video dropout would mess up SkipMode if it wasn't frequently resynced I'd think. I've had instances on my Tivos where an hour show was less than an hour if my cable went out in the middle of recording.

Impressed with the progress you've made so far in RPC.


----------



## moyekj

Yes, so I'm not convinced just using the 1st sync point is the precise thing to do. For now I'll just leave in that assumption to see how it works out so I can move forward to the fun part... Thanks for the help!


----------



## reneg

I'm not convinced it is the precise thing to do either, I was just trying for the simplest explanation which might handle the "normal" case. 

I think they also have to handle the case where a program starts recording early, say a minute or two of padding and the case where a show was clipped for a higher priority recording. For both the cases, they have to somehow align the metadata to the recording.


----------



## innocentfreak

I would look at other shows first. Scorpion for me is one of those shows that always seems to start a minute or so late which might be throwing things off. I think I would avoid most CBS shows because of it.


----------



## reneg

From my Roamio, I downloaded the Scorpion episode and ran it through VideoRedo Ad detective and then manually reviewed the cut points. 

I had trouble with the start of the show being 13 seconds into my recording. The idea that the first SyncMark was the beginning of the show, would have my recording starting at the 55 second mark. The good news is that if I added 42 seconds to the cut points in manual edit, the cut points were within a second or two of the metadata.

I think this shows that first SyncMark is not the start of a relative recording at the top of the hour.


----------



## moyekj

Yes I agree. After looking at a couple of shows the proper start point is not obtained by subtracting the 1st Sync Point. As you said, once the offset problem is resolved it does look like the cut points line up almost exactly with what I would use via VRD manual cuts. So it's just a question of figuring out how to get the proper start point. Assuming the 1st startOffset = time 0 is a better starting point but not accurate either. It doesn't help I don't have a Bolt to see what TiVo playback is using for these test shows.


----------



## moyekj

Well crap. It's painfully obvious TiVo is monitoring this thread now and actively thwarting any further RPC discovery by eliminating access to key functionality. Thanks for nothing TiVo...


----------



## reneg

Well, that's discouraging Tivo.

I was wondering if some more data points about the metadata offsets might be helpful.


Do you get the same offsets if you request the metadata for the same show again?
What are the offsets in the metadata for a show recorded at the same time on a different channel?
What are the offsets in the metadata for a show recorded immediately before or after?
What are the offsets in the metadata for the same show recorded on two different Tivos in the same location or time zone?
What are the offsets in the metadata for the same show recorded in different time zones?

I wonder if things need to be relative to UTC or some other shared sync point/time? They have to deal with different time zones as well as daylight savings.


----------



## davidblackledge

moyekj said:


> Well crap. It's painfully obvious TiVo is monitoring this thread now and actively thwarting any further RPC discovery by eliminating access to key functionality. Thanks for nothing TiVo...


Frustrating, but if you think about it, it was just an open door for other products to steal their hard-fought commercial skipping data. Not completely unreasonable to lock this one down.


----------



## tatergator1

davidblackledge said:


> Frustrating, but if you think about it, it was just an open door for other products to steal their hard-fought commercial skipping data. Not completely unreasonable to lock this one down.


This, and the fact that it is one of the Bolt's exclusive features. For Tivo, the last thing they want is moyekj figuring things out and using the data to improve the experience for Roamio/Premiere users, at least not for the first year or so.

Ira admitted he's been lurking for the last 18 months; it's obvious there are other Tivo employees lurking here as well. I wouldn't be surprised to find out one of them has set up notifications for threads in the Tivo Underground section of TCF to keep up with things like this. You would think that they wouldn't have exposed comskip data to RPC to begin with.


----------



## innocentfreak

It probably doesn't help that Zatz mentioned it on Twitter either.


----------



## L David Matheny

moyekj said:


> Well crap. It's painfully obvious TiVo is monitoring this thread now and actively thwarting any further RPC discovery by eliminating access to key functionality. Thanks for nothing TiVo...


Generally speaking, such a tactic would be counterproductive on their part if they have any desire to retain and expand an enthusiast community. If anything, they should be rewarding you for your work to make their equipment more usable.



davidblackledge said:


> Frustrating, but if you think about it, it was just an open door for other products to steal their hard-fought commercial skipping data. Not completely unreasonable to lock this one down.


Many companies like TV guide services have provisions in their user agreements that prohibit "screen scraping" or other automated methods of capturing proprietary data and using it for unapproved commercial purposes. If other companies try to steal TiVo's commercial breakpoint data to use with other DVRs, surely TiVo could sue them, and the offending company would have to prove that their breakpoint data is independently derived. They might have to copyright it first, but lawsuits are a concept not unknown to TiVo.



tatergator1 said:


> This, and the fact that it is one of the Bolt's exclusive features. For Tivo, the last thing they want is moyekj figuring things out and using the data to improve the experience for Roamio/Premiere users, at least not for the first year or so.


Improving the experience for all TiVo users has been moyekj's mission for years, and TiVo has benefited greatly. If they want to prevent use of their breakpoint data for automatic removal of commercials from transferred programs, maybe they could make that a provision of the TiVo user agreement until they decide to port the feature to Roamio and/or Premiere.



tatergator1 said:


> Ira admitted he's been lurking for the last 18 months; it's obvious there are other Tivo employees lurking here as well. I wouldn't be surprised to find out one of them has set up notifications for threads in the Tivo Underground section of TCF to keep up with things like this. You would think that they wouldn't have exposed comskip data to RPC to begin with.


I should hope that TiVo has people lurking here. They would be insane not to follow these discussions. And is it possible that they need to expose comskip data to RPC for their own purposes?


----------



## windracer

tatergator1 said:


> You would think that they wouldn't have exposed comskip data to RPC to begin with.


Maybe this was so the Minis could eventually take advantage of SkipMode when streaming content from a Bolt? If so, you'd think they'd have to put it back at some point ...


----------



## Fofer

windracer said:


> Maybe this was so the Minis could eventually take advantage of SkipMode when streaming content from a Bolt? If so, you'd think they'd have to put it back at some point ...


...perhaps encrypted, though...?

<sigh>


----------



## tatergator1

L David Matheny said:


> Improving the experience for all TiVo users has been moyekj's mission for years, and TiVo has benefited greatly. If they want to prevent use of their breakpoint data for automatic removal of commercials from transferred programs, maybe they could make that a provision of the TiVo user agreement until they decide to port the feature to Roamio and/or Premiere.


Absolutely, and Ira certainly alluded to moyekj's work as beneficial to the Tivo enthusiasts, and Tivo by extension. He even indicated Tivo should continue to provide for these 3rd party apps.

However, there's a limit to this to protect business interests. This is not the first time moyekj was working to reverse-engineer something, only to have Tivo pull the plug. The Opera browser and launching arbitrary URL's comes immediately to mind. IIRC, I think there are other RPC-related functions that were also obscured after moyekj disclosed he was investigating them in this forum.


----------



## mrizzo80

Too bad TiVo had to pull the plug on this, it looked promising.


----------



## k3rnel

thanks for that , but i cant access to ssl


----------



## cenright

I can't seem to see a way to get a list of programs from another tivo device (similar to My Shows=>Devices=>Other(Tivo)). This could be overcome by connecting to the other tivo...

More importantly, did anyone ever happen to figure out if uiNavigate can be used to play a recording from another Tivo (similar to the same way as described above)?


----------



## cenright

When playing a remote recording, whatsOn has an extra field specifying the remote tivo where content is located:

"remoteSourceBodyId": "tsn:846YYYYXXXXXXX"

Can't seem to figure out where that would go in uiNavigate to play it...



Code:


{
  "whatsOn": [
    {
      "collectionId": "tivo:cl.362463812",
      "contentId": "tivo:ct.387799182",
      "channelIdentifier": {
        "channelNumber": "9-1",
        "sourceType": "terrestrial",
        "stationId": "tivo:st.19939930",
        "type": "channelIdentifier"
      },
      "offerId": "tivo:of.ctd.19939930.9-1.terrestrial.2018-05-08-01-00-00.1800",
      "recordingId": "tivo:rc.1112679",
      "activeAudioStream": {
        "audioStreamFormat": "ac3",
        "audioStreamId": "tivo:aus.148",
        "language": "en",
        "type": "audioStream",
        "streamServiceMode": "completeMain"
      },
      "availableAudioStream": [
        {
          "audioStreamFormat": "ac3",
          "audioStreamId": "tivo:aus.148",
          "language": "en",
          "type": "audioStream",
          "streamServiceMode": "completeMain"
        },
        {
          "audioStreamFormat": "ac3",
          "audioStreamId": "tivo:aus.149",
          "language": "es",
          "type": "audioStream",
          "streamServiceMode": "visuallyImpaired"
        }
      ],
      "playbackType": "recording",
      "remoteSourceBodyId": "tsn:846YYYYXXXXXXX",
      "type": "whatsOn"
    }
  ],
  "type": "whatsOnList"
}


----------



## cenright

tried this:


Code:


{
  "remoteSourceBodyId": "tsn:846YYYYXXXXXXX",
  "type": "uiNavigate",
  "uri": "x-tivo:classicui:playback",
  "parameters": {
   "fUseTrioId": "true",
   "recordingId": "tivo:rc.1112679",
   "fHideBannerOnEnter": "true"
  }
}

and this:


Code:


{
  "type": "uiNavigate",
  "uri": "x-tivo:classicui:playback",
  "parameters": {
   "fUseTrioId": "true",
   "recordingId": "tivo:rc.1112679",
   "remoteSourceBodyId": "tsn:846YYYYXXXXXXX",
   "fHideBannerOnEnter": "true"
  }
}

no workie, anybody with other suggestions?


----------



## cenright

moyekj said:


> When using middlemind.tivo.com I found this query is useful to get a nice summary of any mind operation:
> schemaElementGet
> 
> example query:
> 
> 
> Code:
> 
> 
> {"type":"schemaElementGet", "name":"uiNavigate"}
> 
> response:
> 
> 
> Code:
> 
> 
> {
> "rootElementName": "uiNavigate",
> "operation": [{
> "description": "\n      Causes the UI to navigate to the specified destination.\n    ",
> "name": "uiNavigate",
> "fieldDefinition": [
> {
> "name": "parameters",
> "valueType": "anyDict",
> "type": "fieldDefinition",
> "minOccurs": 0
> },
> {
> "description": "\n        When true, replaces the currently active screen or app on the navigation stack with \n        the new screen or app specified in this request.  A subsequent navigate \n        with a URI of 'x-tivo:return' will not return to the currently active \n        screen or app, but rather the one that was active before it. The default \n        value is false.\n      ",
> "name": "forgetThisScreen",
> "valueType": "boolean",
> "type": "fieldDefinition",
> "minOccurs": 0
> },
> {
> "name": "uri",
> "valueType": "uri",
> "type": "fieldDefinition",
> "minOccurs": 1
> }
> ],
> "isImplemented": false,
> "type": "operation",
> "returnType": ["success"]
> }],
> "levelOfDetail": "low",
> "IsFinal": true,
> "rootElementType": "operation",
> "type": "schemaElement"
> }


A blast from the past....

From KMTTG:
if (type.equals("Help")) { 
// Query middlemind.tivo.com for syntax of a particular RPC command
// Expects RPC command name as "name" in json, such as "keyEventSend"
if (! json.has("levelOfDetail"))
json.put("levelOfDetail", "high");
req = RpcRequest("schemaElementGet", false, json);

Does this call still work? schemaElementGet

don't know if it's user error, but to my bolt keep getting a 500 error using KMTTG -> tried:
{"name": "uiNavigate"}
{"name": "keyEventSend"}


----------



## cenright

cenright said:


> A blast from the past....
> 
> From KMTTG:
> if (type.equals("Help")) {
> // Query middlemind.tivo.com for syntax of a particular RPC command
> // Expects RPC command name as "name" in json, such as "keyEventSend"
> if (! json.has("levelOfDetail"))
> json.put("levelOfDetail", "high");
> req = RpcRequest("schemaElementGet", false, json);
> 
> Does this call still work? schemaElementGet
> 
> don't know if it's user error, but to my bolt keep getting a 500 error using KMTTG -> tried:
> {"name": "uiNavigate"}
> {"name": "keyEventSend"}


===========

error

2021_02_19_16:40:49 ERROR: RPC error response:
{
"code": "routingError",
"text": "PartnerId 3787 does not have access to operation schemaElementGet.",
"type": "error"
}

assume it's no longer offered as part of current authorization?


----------



## moyekj

It stopped working quite a long time ago. Part of TiVo sabotage to prevent me any more discovery of RPC operations. That's when I really started losing interest in anything TiVo related.


----------



## cenright

@moyekj 
Do you even still own tivo hardware? Thank you for all the work on KMTTG. Using it with some customization is about the only thing keeping me around.


----------



## moyekj

cenright said:


> @moyekj
> Do you even still own tivo hardware? Thank you for all the work on KMTTG. Using it with some customization is about the only thing keeping me around.


I have no cable TV service or OTA, but I do keep 1 Roamio running just for purpose of running kmttg "Season Premieres" once a week or so for discovering any series premieres to add to YTTV. That's really the only kmttg RPC function I've used for last several years.


----------

