# Tivo S3 upgraded to 1TB via external RAID



## lightrunner

I was disappointed to find out that the eSata port on the new S3 was not yet active. So like the Glad Press n Seal, I decided to take matters into my own hands and increase the S3 recording capacity the old fashion way. I opened up my SuSE Linux box and expand the drives via an external RAID with an enclosure developed by Sans Digital via a SATA to eSATA cable. I first tested this by taking the original 250GB hard drive and placing it in an external drive enclosure with eSATA support. The test was successful. The following is what I did.

Test

Open the S3
Pull out the hard drive and place the hard drive into eSATA enclosure
Unplug the onboard red SATA cable
Plug in 6ft SATA to eSATA cable to the onboard SATA controller (SATA end of course)
Plug eSATA end to eSATA enclosure 
Power on the eSATA enclosure make sure drive is up and running before powering up S3
Then Power on S3

Here is the link to some picture I took of the test

http://public.fotki.com/lightrunner/tivo-external-hard-/

Then I took the Sans Digital box with five 500 GB drives and configured it for RAID 5 giving me 2TB of storage. I ran through multiple MFSTools back up and restore options including changing the swap files to different sizes but the 2TB would not work. This just kept putting me into a GSOD loop. And yes I even tried swap sizes up to 511. Then I tried the good ole straight copy method in linux (dd) and then tested the tivo with the still original 250GB image since I did not expand the MFS partition yet and this worked. S3 booted up fine. Once I expanded the recording capacity with mfsadd the S3 would then stop working again and go into the same GSOD loop. Also after I expanded the capacity, I noticed I could not run mfsinfo. This command would just take me back to a command prompt. I plugged in the Sans Digital via USB into my MAC and ran pdisk l and noticed that the MFS partitions were there and the expanded partition (/dev/sdd15) was 1.6TB. I even tried manipulating the partitions with pdisk and still it didnt work. Then I decided to recreate the RAID 5 with a smaller volume set. This time 1.5TB and ran the exact same back up / restore /dd procedures and still that didnt work either. Then I shrunk down the volume this time to 1TB and this worked the first time with out a glich. 
So I am wondering maybe the S3 kernel doesnt like seeing the expanded partition beyond a certain size. Or maybe the MFSTools does not create the partitions with the correct settings and numbers beyond a certain drive size. The following was what I did.

Create RAID 5 on Sans Digital with 2TB Volume then let the volume initialize (takes about 2 hours)
Pull out original S3 hard drive and plug into linux box 
Plug Sata to eSata cable into linux box and Sans Digital RAID enclosure
Boot up linux box down load mfstools
Run mfstools backup and restore from /dec/sdc (original s3 drive) to /dev/sdd (RAID)
Unplug the onboard red SATA cable
Plug in 6ft SATA to eSATA cable to the onboard SATA controller (SATA end of course)
Plug eSATA end to eSATA enclosure 
Power on Sans Digital RAID wait till all drives are up and running
Power on S3

Here is the link to the S3 with 1TB upgrade and some other pictures I took during the testing.

http://public.fotki.com/lightrunner/external-raid-test/


----------



## MichaelK

tim allen would be proud. 


grunt grunt more power.

Maybe if you increase the voltage- LOL

very cool indeed.


----------



## MichaelK

there was another post around here someplace about there bing a 1 tera limit but i cant find it


----------



## lightrunner

I did have to lug the RAID enclosure back and forth from the bedroom to the computer quite a bit. I wouldn't be surprised if there was a 1TB limit. I wonder if there is a fix around that in the kernel.


----------



## mindstorm

So how loud in the Sans Digital enclosure with the drives in it? I really like the idea of a raid setup and the enclosure looks to have a nice form factor.


----------



## GoHokies!

Holy smokes, that's cool... :up: :up: :up:


----------



## lightrunner

mindstorm said:


> So how loud in the Sans Digital enclosure with the drives in it? I really like the idea of a raid setup and the enclosure looks to have a nice form factor.


I was very pleased with the low sound levels. During startup as the drives are being initialized you can hear the fan but after that it sounds just like a regular pc. Compared to my ReadyNAS NV+ the Sans Digital is much lower. As a product it self the Sans is a very versatile and easy to manage product. In addition to the eSata there are USB 2.0 and Firewire 800 connections. The LAN and RS232 is for management. And the RAID is true Hot Swap.


----------



## greg_burns

MichaelK said:


> there was another post around here someplace about there bing a 1 tera limit but i cant find it


http://www.tivocommunity.com/tivo-vb/showthread.php?p=4469894#post4469894


JamieP said:


> Keep in mind that the MFS file system can't be larger than 2TiB (512 byte block size with 32 bit unsigned integers for block addresses = 2^41 = 2TiB).
> 
> There are already some display problems when you go above 1TiB, presumably due to a few places in the code where signed integers are being used for size-in-blocks calculations. More details in this thread.


----------



## greg_burns

That Sans Digital MS2UT looks identical to my CoolGear SataVault . Wonder how the two are related?


----------



## mindstorm

lightrunner said:


> I was very pleased with the low sound levels. During startup as the drives are being initialized you can hear the fan but after that it sounds just like a regular pc. Compared to my ReadyNAS NV+ the Sans Digital is much lower. As a product it self the Sans is a very versatile and easy to manage product. In addition to the eSata there are USB 2.0 and Firewire 800 connections. The LAN and RS232 is for management. And the RAID is true Hot Swap.


How's the performance of the S3 with the raid 5 configuration? Have your tried recording two HD streams at the same time? Just wondering if the raid 5 write performance is a factor.


----------



## jlib

Bravo, lightrunner, on your experiment. I and others have been conjecturing on the possibility of RAID 5 on the internal SATA port and your actual proof of concept efforts are much appreciated! It is especially valuable that you encountered the size limitation anomaly as mentioned in the quoted thread. I therefore intend to stay below the 1TiB problem area using a 3 drive (WD 500GB) setup in RAID 5.

My goal is to have the final result be much quieter on seeks than a single Seagate 750 (the current noisy model). The fault-tolerant feature I think is valuable on a virtual disk that large. I don't think the traditional write bottleneck of the RAID 5 configuration will even come close to manifesting with the TiVo application. The only thing I would mention about your system is that technically your cable length is out of spec for the Tivo internal SATA port (the lowest common denominator) which is 1 meter. Might never be a problem but just wanted you to be aware of that if problems develop.

Other threads to look at for those contemplating something similar are Greg_Burns ongoing effort chronicled here and another conjecture thread here.

In any case, you get the prize!


----------



## jlib

greg_burns said:


> That Sans Digital MS2UT looks identical to my CoolGear SataVault . Wonder how the two are related?


I have noticed that many products are just rebranded items from the same manufacturer. For example, the Siig 3 drive internal RAID frame is the same as the Accusys one (just a different color). Most likely that is the same with the two models you idenitified (one true tell is that the manuals are also identical) . The OEM relationship works well for many Taiwanese companies that may have the technology down but lack marketing expertise or contacts in other countries.

Note that those two models you specified both have the magic terms "hardware RAID" and "no drivers needed" so are Tivo ready. I really like the look. I wish they made a 3 drive box like that. The one I have settled on is actually an internal RAID (designed to slip into the 5.25" bays on a PC). So, I have to kluge power to it somehow such as with a surplus PC power supply. Maybe I can keep it all out of sight.


----------



## montivette

Lightrunner,
You sure have enough shoes. Nice work on the upgrade and thanks for posting the link to images.


----------



## jlib

Lightrunner's San Digital MR5CTI RAID box is particularly flexible because you don't have to fully populate it and it has hot spare capability. It is also one of best looking boxes I have seen. The only stumbling block is that it is about $1.5k for the bare enclosure. If price is no object I would go with that one because everything about it reeks of quality. I am going to try a cheaper alternative with the goal of keeping the entire RAID 5 array as close to the price of the TiVo itself as possible.

This is what I have coming in:
Siig SC-000081-S1  SATA RAID 5/0 Bay (also see the Accusys) $263 
Western Digital WD5000YS  Raid Edition 500GB drives ([email protected]$173) $519

Total price for 1TB RAID 5 is $782 (assuming one has a salvageable PC power supply lying around). I should be able to have it together next weekend to verify it actually works in real life.

Of couse, if the fault tolerant aspect is of no concern there are even cheaper alternatives (two disk RAID 0 units) or waiting for quieter 750GB and eventually 1TB single drives.


----------



## 1283

jlib said:


> I don't think the traditional write bottleneck of the RAID 5 configuration will even come close to manifesting with the TiVo application.


I wouldn't count on it unless we can actually test the worst case scenario. Two recordings, 1 playback, multiple MRV and TiVo2Go (if/when we get them), program guide processing, etc. TiVo may have made certain assumptions about what a typical drive can deliver, and then allocates disk bandwidths accordingly. If we dramatically change the disk behavior due to RAID5, we may run into problems.


----------



## slocko

Wow. Tivo going boldly where no Tivo has gone before!


----------



## lightrunner

mindstorm said:


> How's the performance of the S3 with the raid 5 configuration? Have your tried recording two HD streams at the same time? Just wondering if the raid 5 write performance is a factor.


Last night I recorded 2 HD recordings (Raiders vs Broncos and Mets vs Cardinals) while watching a HD playback of the Cowboys vs Texans that I recorded earlier. I didn't notice any slowness or any hiccups. I compared this test with the S3 that I have in my living room with the stock 250GB drive. I scheduled the same recordings and stood in the hallway and controlled both my bedroom S3 with the plasma and my living room S3 with the dlp. I also tested by re running the guided setup to see if there were any delays but there were none. Later in the evening at about midnight I recorded 2 HD shows (An Unfinished Life and Walk the Line). As I was recording both shows I brought up the TV Guide and scrolled through several revolutions of the Guide. This also produced no delays. 
I am sure if one put this setup to a bench with simulated loads and the right instruments they would be able to better guage the delays if there are any. But as a standard user I was satisfied with the performance. 
I am actually surprised at the few that are complaining about RAID 5. Yes depending on the small/large read /write and the stripe size you will notice a degradation in RAID 5 performance if you have a heavy write load. Its no RAID 10 (striped mirrors) but I don't think the write loads that an S3 can produce even when recording (writing) 2 shows and watching (reading) another show can introduce such performance bottle knecks that some of us are talking about. The SANS Digital supports the Native Command Queuing as well as the Western Digital RE2 drives that are populating it.


----------



## lightrunner

montivette said:


> Lightrunner,
> You sure have enough shoes. Nice work on the upgrade and thanks for posting the link to images.


Yeah the wife can't live without her Jimmy Choo's, Manolo Blanick's, etc... not too mention all her hermes,bottega, chanel, louis, gucci, etc.... bags. I really think she has watched one too many episodes of Sex In The City...


----------



## alee

Congratulations lightrunner. Like you, I'm inclined to believe that RAID-5 in a TiVo application is more that sufficient. Consider that a 1hr show requires at most about 6GB or a sustained write of approx. 1.7MB/sec. 2 streams would demand a sustained write speed of 3.4MB/sec give or take. Given most RAID-5 write speeds are at least 2x to 3x that for on the cheap side and approaching 10x that with enterprise class hardware, I doubt we're even approaching the limitations of RAID-5.

However, I'm really curious about performance in "degraded" mode, when 1 of the volumes is rebuilding (yank drive0, replace it with a new HD). Redundancy is extremely important, but in a worst case scenario of recording 2 HD streams + watching 1 stream + 1 fictitious MRV transfer, if disk0 fails, and you replace it, can it still keep up? Sure you could wait until a lower disk demand time, but let's say you do replace the disk during peak.

With a 1TB limit, I'm inclined to stick with RAID-1 from an economy stand point... but either way this was a very interesting exercise. Great job lightrunner!


----------



## greg_burns

lightrunner,

I am curious what approach you are going to take to close up the S3 box. I was considering snipping out a piece of the bottom grate to slide the cable through.

picture of grate


----------



## classicX

I'm sure the wife LOVED seeing THAT monstrosity. (I assume there is a wife, since the boots were right next to the TV stand.)


----------



## lightrunner

greg_burns said:


> lightrunner,
> 
> I am curious what approach you are going to take to close up the S3 box. I was considering snipping out a piece of the bottom grate to slide the cable through.
> 
> picture of grate


This is one I am still trying to decide on. But I think your solution is the best option with out having to make drastic cuts into the box.


----------



## lightrunner

classicX said:


> I'm sure the wife LOVED seeing THAT monstrosity. (I assume there is a wife, since the boots were right next to the TV stand.)


Yes there is a wife and yes she was not happy with the RAID box cluttering the bed room but when I told her it was either the bed room or the living room she reluctantly decided on the bed room. It also helped that she got a new pair of shoes out of it. She's one hell of a negotiator.


----------



## mindstorm

lightrunner said:


> Last night I recorded 2 HD recordings (Raiders vs Broncos and Mets vs Cardinals) while watching a HD playback of the Cowboys vs Texans that I recorded earlier. I didn't notice any slowness or any hiccups.


Excellent news to hear (read)



lightrunner said:


> I am actually surprised at the few that are complaining about RAID 5. Yes depending on the small/large read /write and the stripe size you will notice a degradation in RAID 5 performance if you have a heavy write load.


Not complaining, it's just that I've experienced situations (non Tivo related with some writes being GBs in size) where raid 5 was just too slow for write performance in my line of work ( went with raid10 in the case above). Controllers, the drives all play a part in the overall performance and I just wanted to confirm that raid 5 in your setup could handle the S3 workload and it looks like it can.


----------



## ashu

Kudos, a job well done!
I look forward to seeing how you seal it all off and obtain a higher WAF 
And also to (hopefully) a hack, someday, to extend the 1TB limit!

It's another issue I'm still lookin forward to actually owning an S3, but that is inevitable ...

I'm especially glad that you folks have confirmed the SATA/eSATA equivalency with single drives ... I'm thinking I'll hold off on upgrading until an acceptable 1TB drive is available for the cheap external enclosure I bought ... but if I decide to go RAID, I have 3 excellent threads tor efer to, now 

Oh, and for a lower price, consider the Costco-available 2TB (1.5 in RAID 5, dunno if you can force a lower size ... pull out a couple of disks?) I mentioned in the other thread

And for a hole, instead of the grate on the bottom, why not push out one of the lollipop-shaped tab-like things in the back instead? Heck, jury rig a nice eSATA connected (the kind you get free with eSATA enclosures for the back of regular cases) with it? Although that'll increase the number of in-path cable connections!


----------



## lightrunner

mindstorm said:


> Excellent news to hear (read)
> 
> Not complaining, it's just that I've experienced situations (non Tivo related with some writes being GBs in size) where raid 5 was just too slow for write performance in my line of work ( went with raid10 in the case above). Controllers, the drives all play a part in the overall performance and I just wanted to confirm that raid 5 in your setup could handle the S3 workload and it looks like it can.


Mindstorm,
Thanks for the note. I wasn't referring to you complaining in fact I thought you asked a legitimate question. 
I just see random posts where certain members hymmm and haahh about RAID 5 or other disk technologies. Sometimes it seems that they just want to hear themselves talk or they want to show off their knowledge. You know those type of folks. In fact I have a few network and systems engineers who do that all the time. They are the toughest to manage. 
We have successfully used RAID 5 in 100 and less user environments on Exchange 5.5 and 2000 (now 2003) and SQL based Order Management Systems for Financial Traders. Granted there were only 10 traders actively hitting the OMS platform but these setups ran for about 4 years before we finally moved to RAID 10 however I made that decision once our user and trader base grew by double . Granted we were using SCSI as well as Controllers with a large Cache not to mention the processing power and large memory. I have even successfully used RAID 5 in a Terminal Server environement. 
Of course there were times that I attributed certain latency to RAID 5 performance but it wasn't enough at the time to make such a significant financial investment to implement RAID 10 on a 105 GB Exchange Store at the time (using Email Extender now with short cutting wich shrunk our store to little over half the size) and 40 GB SQL OMS DB. Although we use RAID 10 and a SAN environtment for our heavy processes we still have RAID 5 in our environment.
Now I have seen where a large file moves (10GB) from Direct attached storage with dedicated controller to drives on an onboard controller degraded the performance significantly but we are talking about, as you mentioned, GB. I dont think the HDTV going over the coax from your cable provider is a GB stream in addition to the 'maybe' overhead processing the tivo has for the decoding / encoding which might slow the write a bit more.


----------



## greg_burns

ashu said:


> And for a hole, instead of the grate on the bottom, why not push out one of the lollipop-shaped tab-like things in the back instead? Heck, jury rig a nice eSATA connected (the kind you get free with eSATA enclosures for the back of regular cases) with it? Although that'll increase the number of in-path cable connections!


I'm having a hard time visualizing what lollipop-shaped tab-like thing you are talking about? Any pictures? Megazone's site seems to be down today. 

http://www.tivocommunity.com/tivo-vb/showthread.php?t=315795


----------



## JamieP

greg_burns said:


> http://www.tivocommunity.com/tivo-vb/showthread.php?p=4469894#post4469894


To expand a little further, there is a limit on partition size too. I'm not sure if this is an artifact of mfstools or an inherent MFS limitation, but with -r 2, the partitions can't be larger than 256GiB (274GB), with -r 3 : 512GiB (549GB), with -r 4 1024 GiB (1099GB) .

With a single disk, you can have three partition pairs, but two pairs come from the original tivo disk and only one pair is added by mfsadd. So if you start with a stock 250GB S3 image and expand it, you'll be limited to 250GB +1099GB = 1349 GB. To get more than this, you'd either need to partition the raid into two volumes and add them as separate disks (not sure if the S3 can handle two disks) or find a way to strip off or expand one of the partition pairs in the original stock image so you can add another larger pair. The partition coalesce hack would be one way to do this.

The hard limit on MFS size is 2TiB. I know 2x750GB has been done on the S2 units and the only issue is the display problem previously mentioned.


----------



## lightrunner

JamieP said:


> To expand a little further, there is a limit on partition size too. I'm not sure if this is an artifact of mfstools are an inherent MFS limitation, but with -r 2, the partitions can't be larger than 256GiB (274GB), with -r 3 : 512GiB (549GB), with -r 4 1024 GiB (1099GB) .
> 
> With a single disk, you can have three partition pairs, but two pairs come from the original tivo disk and only one pair is added by mfsadd. So if you start with a stock 250GB S3 image and expand it, you'll be limited to 250GB +1099GB = 1349 GB. To get more than this, you'd either need to partition the raid into two volumes and add them as separate disks (not sure if the S3 can handle two disks) or find a way to strip off or expand one of the partition pairs in the original stock image so you can add another larger pair. The partition coalesce hack would be one way to do this.
> 
> The hard limit on MFS size is 2TiB. I know 2x750GB has been done on the S2 units and the only issue is the display problem previously mentioned.


JamieP, thanks for the note. Its interesting that you mentioned that unfortunately during the testing I did create two 1TB partitions on the SANS to see if the S3 would see each partition as a seperate drive to do just what you are talking about but this didnt work. My feeling on this is that it was more of the SANS issue on the controller backplane. However now I am looking into port multiplier enclosers mentioned here. http://www.sata-io.org/portmultiplier.asp
This basically does exactly what it says. It allows a single SATA port to attach to multiple SATA devices via a port multiplier that would be built into an enclosure of some sort. I will keep you posted of what I find.


----------



## lightrunner

I think this Port Multiplier enclosure might work.

http://www.sonnettech.com/product/fusion500p.html


----------



## JamieP

lightrunner said:


> I think this Port Multiplier enclosure might work.
> 
> http://www.sonnettech.com/product/fusion500p.html


Wouldn't any port multipler solution require port multipler support on the tivo side? Note this fine print in the link you posted:


> * Fusion 500P requires a SATA host controller that supports port multipliers ...


 Do we know the S3 host controller/driver has that support?


----------



## lightrunner

JamieP said:


> Wouldn't any port multipler solution require port multipler support on the tivo side? Note this fine print in the link you posted: Do we know the S3 host controller/driver has that support?


Good catch. Thats a very good question. I guess it wont hurt to find out. Ehhh... the box looks somewhat similar to the SANS box so the wife wont notice its a new box and besides she got new shoes so I can get another box....


----------



## lightrunner

I will also try to get a better look at the main board to see if I can find the SATA controller chipset. If its something like silicon image I can probably find information on their website for mulitplier support


----------



## Thom

Jamie -

I recently tried upgrading a series 1 DSR6000 and a series 2 HDVR2 to dual 750 GB drives, and I could not get MFSTool to successfully expand MFS beyond 1048xxxMiB (as reported by MFSTool). So I wound up using a 750 GB A drive and a 400 GB B drive, with some unused space on the B drive.

Do you know what technique was used to fully utilize the dual 750 GB drives, or can you point me to a post about it?

- Thom



JamieP said:


> To expand a little further, there is a limit on partition size too. I'm not sure if this is an artifact of mfstools are an inherent MFS limitation, but with -r 2, the partitions can't be larger than 256GiB (274GB), with -r 3 : 512GiB (549GB), with -r 4 1024 GiB (1099GB) .
> 
> With a single disk, you can have three partition pairs, but two pairs come from the original tivo disk and only one pair is added by mfsadd. So if you start with a stock 250GB S3 image and expand it, you'll be limited to 250GB +1099GB = 1349 GB. To get more than this, you'd either need to partition the raid into two volumes and add them as separate disks (not sure if the S3 can handle two disks) or find a way to strip off or expand one of the partition pairs in the original stock image so you can add another larger pair. The partition coalesce hack would be one way to do this.
> 
> The hard limit on MFS size is 2TiB. I know 2x750GB has been done on the S2 units and the only issue is the display problem previously mentioned.


----------



## JamieP

lightrunner said:


> I will also try to get a better look at the main board to see if I can find the SATA controller chipset. If its something like silicon image I can probably find information on their website for mulitplier support


I believe the SATA controller is integrated into the broadcom bcm7038.

It's a dual channel controller, and in the boot logs I've seen, it seems to look like a legacy PATA controller to the kernel. It shows two ide channels (ide1 and ide2) with two disks per channel (hd[a-d].) That may just be the way tivo wrote the drivers for it.


----------



## JamieP

Thom said:


> I recently tried upgrading a series 1 DSR6000 and a series 2 HDVR2 to dual 750 GB drives, and I could not get MFSTool to successfully expand MFS beyond 1048xxxMiB (as reported by MFSTool). So I wound up using a 750 GB A drive and a 400 GB B drive, with some unused space on the B drive.


I don't have any specific information. I just know that the upgrade vendors are selling dual 750GB drive systems. I thought people here had posted success stories too.


----------



## ashu

greg_burns said:


> I'm having a hard time visualizing what lollipop-shaped tab-like thing you are talking about? Any pictures? Megazone's site seems to be down today.
> 
> http://www.tivocommunity.com/tivo-vb/showthread.php?t=315795


You shoulda checked in the image you mentioned/attached earlier 

Of course, the existing openings are too sharp/awkward - but nothing a little drilling/dremmeling wouldn't smoothen out enough to add one SATA->eSATA connector cable originating on the onboard SATA port.


----------



## greg_burns

ashu said:


> You shoulda checked in the image you mentioned/attached earlier
> 
> Of course, the existing openings are too sharp/awkward - but nothing a little drilling/dremmeling wouldn't smoothen out enough to add one SATA->eSATA connector cable originating on the onboard SATA port.


Is the link working for you? Megazone's TivoLovers site seems to be down from here.
http://www.tivolovers.com.nyud.net:8080/Photos/Series3-Review/Medium/Inside-12.jpg


----------



## ashu

greg_burns said:


> Is the link working for you? Megazone's TivoLovers site seems to be down from here.
> http://www.tivolovers.com.nyud.net:8080/Photos/Series3-Review/Medium/Inside-12.jpg


Yeah, it works ... sorry, I didn't notice you HAD linked off his site ... although this seems to bea mirror?


----------



## greg_burns

ashu said:


> Yeah, it works ... sorry, I didn't notice you HAD linked off his site ... although this seems to bea mirror?


Working now. Yes, now I see the lollipops.


----------



## ashu

greg_burns said:


> Working now. Yes, now I see the lollipops.


What can I say ... I enjoy colorful, suggestive analogies 
Think that gap is large enough for a cable, or a hooked in eSATA adapter? After some drilling/smoothing, perhaps?


----------



## greg_burns

ashu said:


> What can I say ... I enjoy colorful, suggestive analogies
> Think that gap is large enough for a cable, or a hooked in eSATA adapter? After some drilling/smoothing, perhaps?


AFAI Can Tell, all the lollipops (including sides and back) are blocked when the top cover is replaced.


----------



## greg_burns

I have my Thecus setup now as well...

http://www.tivocommunity.com/tivo-vb/showthread.php?p=4486356&&#post4486356


----------



## MichaelK

i believe the 'lolipops are something of a spring that pushes against the sides of the top when it is slipped on to make the top & bottom/sides a continuous shield/ground/whatever. So breaking off a lollipop does ya nada becasue the top forms a second layer in that area that you would need to drill through.

follow me?

THe holes in the bottom I think are the best bet...


----------



## ChuckyBox

ashu said:


> Of course, the existing openings are too sharp/awkward - but nothing a little drilling/dremmeling wouldn't smoothen out enough to add one SATA->eSATA connector cable originating on the onboard SATA port.


Why not just disconnect TiVo's wires to the the S3's eSATA port, and use your SATA to eSATA cable to wire it up? Then you can plug your RAID box directly into the S3 just like a pro. If TiVo gets around to activating the eSATA port (and you want to use it), you can put it back the way it was, and then cut a hole in your box for the first raid.*

*Or, better, cut a hole in the backplane and install another eSATA socket, and use TiVo's wiring to activate it.

(I'm assuming that the S3's eSATA socket isn't just one component of a larger circuit board, in which case you can't easily re-purpose it. In that case, I'd still cut a hole in the backplane and mount an eSATA socket for the RAID. It's a classy box, let's treat it with some respect.)


----------



## 1283

ChuckyBox said:


> I'm assuming that the S3's eSATA socket isn't just one component of a larger circuit board


It is.


----------



## alee

Congrats lightrunner... you're on Engadget! 
http://www.engadget.com/2006/10/16/tivo-series3-to-1tb-hack/


----------



## 1283

From Engadget: "we've still got to wonder where the guy is going to find the time to watch a working week of high definition TV re-runs."


----------



## sjcbulldog

They just don't understand, it goes to 11.


----------



## bsnelson

I don't think I'm smeeking here, but IIRC, 32 bit Linux has a limitation of 1TB per drive because it's LBA48 implementation is really LBA32. This would suggest a limit of 2TB if one could get the S3 controller to "see" two drives, or the 1TB shown here with a single drive. 

Brad


----------



## JamieP

bsnelson said:


> I don't think I'm smeeking here, but IIRC, 32 bit Linux has a limitation of 1TB per drive because it's LBA48 implementation is really LBA32. This would suggest a limit of 2TB if one could get the S3 controller to "see" two drives, or the 1TB shown here with a single drive.


2^32 * 512 = 2TiB, and the limit is described here as "The 2 TiB limit". Do you have a reference that says it actually clicks in at 1TiB instead? I could imagine that might be the case if the sector numbers are ever treated as signed quantities.


----------



## shootedit5

Cool job lightrunner. It looks like you caused quite a stir over at engadget with the replys. Some people seem to think that you live with your parents from the pics lol... Anyway keep posting the good work. Just so you know my glad press n seal is still working great. thnks..


----------



## greg_burns

lightrunner said:


> This is one I am still trying to decide on. But I think your solution is the best option with out having to make drastic cuts into the box.


----------



## moyekj

So since the 1TB limitation seems to be file system related I would assume that if/once E-Sata port is enabled it will have a 1TB limitation as well?


----------



## 1283

greg_burns, I hope you won't need warranty service.


----------



## greg_burns

c3 said:


> greg_burns, I hope you won't need warranty service.


Me too.


----------



## MichaelK

greg_burns said:


> Me too.


that's mint!


----------



## MichaelK

moyekj said:


> So since the 1TB limitation seems to be file system related I would assume that if/once E-Sata port is enabled it will have a 1TB limitation as well?


one would assume so...


----------



## sjcbulldog

Well, if he does and I were running Tivo, I would service the box just for the interest he is generating in the S3 and assocaited "upgrades". I would keep it very quiet and ask Greg not to talk about it so I don't have to do this all the time, but it would be worth it.

Just my $0.02 worth
scjbulldog


----------



## JDAWG11

Your setup is definetly one to brag about...not only the a/v equipment, but the computers. Are those def tek's in your main room?


----------



## lightrunner

greg_burns, that is way too cool. I was starting to get a bit tired of looking at my bedroom s3 with the covery half way open. Now I know what to do. Thanks for the great pic of the scissor as well so I know what to grab at my local Home Depot. 

c3, I really don't have much time to watch tv. Most of the time it just stays on in the background. Almost all of my S3 RAID build and testing was done in the middle of the night. You'd be surprised at how much energy can be derived from a pot of coffee, some red bull and mountain dew at 3 a.m. on a work day. Yeah I get a little obsessive. 
I mainly did this to contribute to this forum. I had one of these RAID boxes laying around the office so I decided to give it a try. 

Also just a note I upgraded my Living room S3 to 750GB and noticed that my eSata RAID bedroom S3 is much faster when connecting to tivo and loading the data. I did a side by side comparison after a clean delete and repeat guided setup. I did this 3 times just to verify. However, I do not notice any slowness or diferences when recording and watching shows.


----------



## lightrunner

JDAWG11 said:


> Your setup is definetly one to brag about...not only the a/v equipment, but the computers. Are those def tek's in your main room?


Thanks JDAWG11, yes those are Def Tech's powered by a Yamaha RX-Z9. I got the Yamaha a couple years ago and now I need to upgrade since it has no HDMI switching capability.


----------



## alee

How is the RAID performance when in degraded mode (volume rebuilding)?


----------



## lightrunner

alee said:


> How is the RAID performance when in degraded mode (volume rebuilding)?


Have not tried that yet. Will do it this weekend. I will just pull one of the drives and let the hot spare take over. I will report as soon as I am finished.


----------



## greg_burns

lightrunner said:


> greg_burns, that is way too cool. I was starting to get a bit tired of looking at my bedroom s3 with the covery half way open. Now I know what to do. Thanks for the great pic of the scissor as well so I know what to grab at my local Home Depot.


I was lucky and was able to borrow those snips from work. I also used a dremel tool to buff down the sharp edges. I made sure I taped up the holes before starting. Nothing will fry the box faster than little metal shavings lying on the mobo. 

I think c3 was right though, I should have at least waited the 30 days for the warranty to be up.


----------



## FlippedBit

greg_burns said:


>


Might I suggest a pair of nibblers instead.


----------



## ashu

FlippedBit said:


> Might I suggest a pair of nibblers instead.


Excellent idea!

But aren't they a wee bit difficult to control and tame?


----------



## Bai Shen

jlib said:


> This is what I have coming in:
> Siig SC-000081-S1  SATA RAID 5/0 Bay (also see the Accusys) $263
> Western Digital WD5000YS  Raid Edition 500GB drives ([email protected]$173) $519


I'm curious how the Siig works. I saw a similiar one at CompUSA for sale, but I was worried about the heat. My hard drives get really hot.


----------



## greg_burns

Bai Shen said:


> I'm curious how the Siig works. I saw a similiar one at CompUSA for sale, but I was worried about the heat. My hard drives get really hot.


Which drives do you have? The WD 500s?


----------



## greg_burns

FlippedBit said:


> Might I suggest a pair of nibblers instead.


Got any pictures? I think I flunked shop.


----------



## Bai Shen

greg_burns said:


> Which drives do you have? The WD 500s?


Nope. Couple 250s(WD, Maxtor, and Seagate IIRC) and a WD 80. Got a 750 in my myth box, but I haven't checked the temps on it. I'll do that when I put some more memory in it this wknd.


----------



## slocko

never heard of it before, but here it is:

http://www.mytoolstore.com/klein/76011b.html


----------



## greg_burns

greg_burns said:


>


FYI: the hole is still just a hair not wide enough for the head my Sata cable. I may have to snip out down to the next one and make a really big square hole or go at it with the dremel.


----------



## 1283

greg_burns said:


> FYI: the hole is still just a hair not wide enough for the head my Sata cable. I may have to snip out down to the next one and make a really big square hole or go at it with the dremel.


Have you tried the iSATA end of the cable? For the cable in front of me, it's a bit smaller than the eSATA end.


----------



## MichaelK

do they sell the cable and connectors separatly anyplace?

I looked on google but didn't see any...


----------



## greg_burns

MichaelK said:


> do they sell the cable and connectors separatly anyplace?
> 
> I looked on google but didn't see any...


You mean like http://www.satacables.com/. I bought mine there. There is also a http://www.satacable.com/ w/o the "s". Not sure about them.


----------



## greg_burns

c3 said:


> Have you tried the iSATA end of the cable? For the cable in front of me, it's a bit smaller than the eSATA end.


Tried both ways. Also looked at some other non eSata cables I had just for comparision. They are all too tight/thick as well.

Measure twice, cut once they always say.


----------



## Bai Shen

greg_burns said:


> FYI: the hole is still just a hair not wide enough for the head my Sata cable. I may have to snip out down to the next one and make a really big square hole or go at it with the dremel.


Will the eSATA port unplug from the board? Can you convert your cable to use it?


----------



## greg_burns

Bai Shen said:


> Will the eSATA port unplug from the board? Can you convert your cable to use it?


The eSATA port on the S3? No. IIR, it is pretty tightly integrated into the mobo. Funny that none of Megazone's excellent pictures show it.


----------



## Bai Shen

greg_burns said:


> The eSATA port on the S3? No. IIR, it is pretty tightly integrated into the mobo. Funny that none of Megazone's excellent pictures show it.


It was a thought.


----------



## FlippedBit

greg_burns said:


> Got any pictures? I think I flunked shop.


This is the Nibbler I have. $4 from Radio Shack.

http://www.radioshack.com/product/i...=&origkw=nibbler&kw=nibbler&parentPage=search


----------



## greg_burns

FlippedBit said:


> This is the Nibbler I have. $4 from Radio Shack.
> 
> http://www.radioshack.com/product/i...=&origkw=nibbler&kw=nibbler&parentPage=search


Thanks. May have to go pick one up for next cut.


----------



## Mrs.Lee

montivette said:


> Lightrunner,
> You sure have enough shoes. Nice work on the upgrade and thanks for posting the link to images.


 They're mine (wife)...I figure I deserve them since everytime he adds to his home theater or gets a new gadget my home ends up full of boxes and equipment, etc. Basically it gets torn apart for a few days while he obsesses on the newest project. Not to mention he basically works 24/7 sooo...

But he is so smart and talented I am pleased to see others benefit from his "experiments". You should see his NYC office, he did the entire network architecture. His staff adores him because he is hands on and they always tell me they learned more from him than they did in college. He'll probably be annoyed that I posted this but I am very proud of him since he is almost entirely self taught. If you knew where he came from and where he is now you'd never believe his story. Sorry "lightrunner" if I've embarrassed you.


----------



## Mrs.Lee

ClassicX...You are correct, it was unbelivably frustrating for several days...but I'm proud of him and all of his accomplishments. He's incredibly brilliant when it comes to ANYTHING technology wise. He's known as the best on Wall Street, so the firm made him a managing partner I guess fearing he might get snatched up from someone else. He just now found out I registered and replied to a couple of comments so now he's embarrassed and pretty anoyed. But if you knew his story & how he got to where he is you would think it was made up so I'm proud of him and enjoy seeing others benefit from his work. I know nothing about technology but I keep a book of all his publications, etc.


----------



## MichaelK

Damn-

the guy is rich, successfull, has tons of neat toys, AND a cool wife!

grumble grumble...


----------



## optivity

mindstorm said:


> How's the performance of the S3 with the raid 5 configuration? Have your tried recording two HD streams at the same time? Just wondering if the raid 5 write performance is a factor.


My thoughts exactly, have you tried comparing the performance with RAID 0?


----------



## rcompart

Any word on port multipliers working or not? I have an addonics unit that I am dying to try but the tivo isn't here yet. Hurry up UPS!!!


----------



## 1283

rcompart said:


> Any word on port multipliers working or not?


Extremely unlikely X 2. The Broadcom chip has to be port multiplier aware, and the TiVo software has to be able to handle multiple drives through a single SATA port.


----------



## rcompart

Crappy, Well what about the card or backplane in that Sans Digital raid. I'm making my own case and need somethign to make it all work.


----------



## 1283

Not sure what you're trying to do. You're planning to take apart a $700 S3 and a $1,500 Sans Digital box?


----------



## rcompart

Yes, actually I am thinking about doing that. I really only want the controller from it as I have my own case but in one word, yes!


----------



## Flappjakk

With Hitachi Releasing a 1TB harddrive for <$400, do we think that might work in place of the RAID enclosure? The RAID setup you have is sure cool, but i'm guessing it might have cost more than the Tivo.


----------



## jlib

Flappjakk said:


> With Hitachi Releasing a 1TB harddrive for <$400, do we think that might work in place of the RAID enclosure? The RAID setup you have is sure cool, but i'm guessing it might have cost more than the Tivo.


The whole RAID project has to be seen in the historical context. Just nine months ago there were no quiet 750GB drives let alone 1TB. There was no access to the external eSATA port. To accomplish his feat, lightrunner _had_ to use a RAID setup. As exciting as it was at the time it is now pretty much obsolete for most purposes (save someone with a perceived need for data redundancy) and nobody is still advocating the technique. The noise and expense make it impractical for most people. The ultimate way now is to update the internal drive to 1TB with the option for another 1TB via the eSATA port.


----------



## crabbon

True, but I'd like to push the tivo to it's limits.

What about a 5 drive RAID 5 system, holding 1TB drives. That'll be 4TBs. I read somewhere the the 1TB drive limit has been broken using mfslive. I see they have a RAID setup at 2.5TB. Can it be bigger.

Lightrunner, have you done any more testing? I'd like to give this a try on one of my S3s.


----------



## jlib

Well, since it is unknown territory nobody will know for sure. Since it is essentially a database you are dealing with you will probably run up against memory limitations before any hard partition size limitations since it has already been demonstrated at up to 2.5TB. But if you have the resources and curiosity to trudge onward then the only admonition is that the RAID _has_ to be a hardware RAID (no special cards or software drivers needed). The controller chip in the RAID unit will have to make it appear as one drive via one cable with no further intervention needed. There are only a very few such units out there so purchase carefully.


----------



## Carlton Bale

Has anyone looked into using a Silicon Image SiI5723 integrated RAID controller? These are small, cheap, and low-power.

One thing I've learned about hard drives is, if you run them long enough, they will eventually fail. I've suffered through several drive failures. Not that 1TB drive are available, there is not much need for the additional storage space made by combining multiple drives into a RAID set. What I'd like to have is simple way to mirror a drive in a RAID1 configuration.

With one of these SiL5723 boards, I think it would be possible to fit the RAID controller and two 1TB drives in a TiVo enclosure. This would give real-time, continuous protection against drive failures. The controller takes care of all the RAID functions and presents itself as a single SATA drive to the TiVo.

Any thoughts on doing something like this? I don't want to deal with the headache of another TiVo drive failure.


----------



## 1283

5723 is what the THD uses, but not for RAID1. The easiest solution is to route the internal SATA port to the eSATA connector, and then use a RAID enclosure outside.


----------



## jlib

Carlton Bale said:


> ...With one of these SiL5723 boards, I think it would be possible to fit the RAID controller and two 1TB drives in a TiVo enclosure...


You aren't going to find any standalone boards you speak off at the retail level. They would be OEM solutions such as motherboards or external RAID enclosures. Trying to salvage the electronics from an enclosure so to make a more compact and elegant solution by fitting both drives and the electroics in the TiVO would be non trivial in that most OEM solutions now have the drives plugging directly into the card for power and data so may not be friendly to simple extending with cables to the independently mounted drives.

It would just make more sense to use an external enclosure. If you are just looking for simple mirroring, Western Digital has a new two 1TB eSATA drive enclosure that can be configured as RAID 1 for a total of 1TB (Edit: Not a good example; its "triple-interface" does not include eSATA as I thought) . As c3 said you can easily route the TiVo HD internal SATA to the eSATA port (its a little more complicated with the S3 since its eSATA is surface mounted to you would have to do a little cutting for cable access).


----------



## Carlton Bale

I just check and there are no Western Digital enclosures with both eSATA and RAID (they are USB + Firewire only.) However, I did find this enclosure, which should work well. If someone wanted to integrate the board inside the TiVo, this board would probably work, and it supports 4 drives.


----------



## 1283

The AMS system with the 4726 chip would NOT work with TiVo in the RAID 1/10 mode. You need something with the 4723 chip, such as the Thecus system that jlib is using.


----------



## Carlton Bale

c3 said:


> The AMS system with the 4726 chip would NOT work with TiVo in the RAID 1/10 mode. You need something with the 4723 chip, such as the Thecus system that jlib is using.


Why not? Don't both present the drive array as a single eSATA drive? Looking at this page, the 4726 is just a 5 port version of the 4723. But I do think the 3 extra ports would be unnecessary.


----------



## 1283

When an error occurs, the 4726 requires the host software to start the rebuild process. The 4723 firmware would do that by itself.


----------

