# 4K drive problems



## cuda74360 (Jul 12, 2013)

Hi,

I have a HD XL (TCD658000) with a failing hard drive. It started rebooting whenever we watched a certain recording. I ran a kickstart and it failed the SMART test with a Fail 7.

So, I used the MFS Live CD to create a truncated image, and restored it to a WD10EURX. This worked, but videos skip frames every 2-5 minutes. This never happened with the original drive.

I believe that this has to do with the 4K format of the new drive, as the MFS partitions are not aligned (base is not evenly divisible by 8.)

I tried using dd to backup each partition (except for the 1st, since it's the partition table) to an image, corrected the partition alignment with pdisk by deleting and recreating the partitions, and restoring the data back from the images to each new partition. This allowed the TiVo to boot and videos played without issue. There were several problems, though... I didn't get the THX intro, TiVo Central wouldn't display, no background graphics on the Now Playing List, not all remote commands responded, videos continued to play when you hit the TiVo button, etc.

I also have a regular HD (TCD652160.) As a test, I used dd to clone the original drive out of it to the WD10EURX, and used mfsinfo -x to expand the capacity. As was noted in another thread on this forum, this resulted in all MFS partitions being aligned. I didn't have the video playback problems on the HD.

I'm convinced that the 4K sectors of this drive are causing the problems I'm having. Is there a source for the old 512B sector drives? Or am I going at this the wrong way?

Thanks!


----------



## Soapm (May 9, 2007)

Just for reference, I have 2Tb, 4K drives in both my HD and premier with no problems. I don't recall doing anything special beyond jfms.


----------



## cuda74360 (Jul 12, 2013)

Thanks for the response. 

I suspect that the Premiere uses a newer Linux kernel that natively supports the 4K drives,and that it's partitions are already aligned to 4k drives. This would make sense, as it came out around the same time the 4K drives were being phased in. Unfortunately, I don't have access to one to confirm this.

On the HD (not XL), if you DD copy the original 160GB drive (basically what JMFS does) to a 1 TB or 2 TB drive and then expand it with MFS tools, you will end up with all MFS partitions aligned. I tried this with my HD to confirm, and had no video playback problems.

On my HD XL, though, I can't figure out how to get the partitions aligned and am getting skipped frames during playback.

Oddly enough, I ran the WD Diagnostics Extended Test on the factory drive last night and it found no errors. So I erased it and restored the truncated image I took from it last week when it first starting acting up. The drive now passes the Kickstart 54 SMART tests and seems fine.


----------



## jmbach (Jan 1, 2009)

You can consider using your HD image on your HD XL and use a KS 52. If you are adept you can use a hex editing program and change the brev code in block0 of the HD image to that of the HD XL. 
Although you may 4k align the partitions, there is no evidence that the TiVo keeps the alignment internally. Consequently there would be no performance improvement. At the same time I like the idea of 4k align partitions as it will not degrade performance. The only thing that might happen is to lose a few seconds of recording time. 
I have a WD20EURS drive in my S3 and have not those issues yet. I did have those issues with bad capacitors. (with the original drive) After I replaced them had no more problems.


----------



## cuda74360 (Jul 12, 2013)

I tried to use the HD image on my XL. After performing a Kickstart 52, it downloaded the software, rebooted, and hung at the "Just a few more minutes" screen. Is the hex edit required to make this work? I've never used a hex editor before, but would be willing to give it go it you can tell me what to do.

This is the partition map that is being generated by the MFS Live CD when I restore my truncated backup:

Partition map (with 512 byte blocks) on '/dev/sdb'
#: type name length base ( size )
1: Apple_partition_map Apple 63 @ 1 
2: Image Bootstrap 1 1 @ 876054592 
3: Image Kernel 1 8192 @ 876054593 ( 4.0M)
4: Ext2 Root 1 524288 @ 876062785 (256.0M)
5: Image Bootstrap 2 1 @ 876587073 
6: Image Kernel 2 8192 @ 876587074 ( 4.0M)
7: Ext2 Root 2 524288 @ 876595266 (256.0M)
8: Swap Linux swap 262144 @ 877119554 (128.0M)
9: Ext2 /var 524288 @ 877381698 (256.0M)
10: MFS MFS application region 589824 @ 877905986 (288.0M)
11: MFS MFS media region 876054528 @ 64 (417.7G)
12: MFS Second MFS application region 589824 @ 878495810 (288.0M)
13: MFS Second MFS media region 1074438144 @ 879085634 (512.3G)
14: Apple_Free Extra 1390 @ 1953523778

Device block size=512, Number of Blocks=1953525168 (931.5G)
DeviceType=0x0, DeviceId=0x0

As you can see, the Bootstrap partitions are being created with a length of 1, which throws off the alignment of the other partitions. The factory image has the same issue. Is there a way to restore the backup and tell it to use a length of 8 for these partitions?


----------



## jmbach (Jan 1, 2009)

Yes but you will have to do a manual copy of each partition to the new drive and adjust the partition map accordingly. iBored can be used to do it. Would suggest getting used to that program if you want to attempt it. 

The brev code is a backup if for some reason the software cannot poll the hardware to figure out what machine it is. You can use iBored to change the brev code in block0. 

Try another KS 52. Sometimes letting it boot and then do a c&de instead of a KS 52 will work.


----------



## ggieseke (May 30, 2008)

Are you saying that you tried to put a Series 4 image on a Series 3?


----------



## cuda74360 (Jul 12, 2013)

No, I'm not saying that.



ggieseke said:


> Are you saying that you tried to put a Series 4 image on a Series 3?


----------



## nooneuknow (Feb 5, 2011)

As I always do, I'm now going to point out that there is more to misalignment than performance issues, even if they aren't enough to notice on the equipment in question, that the drive is used in (such as a TiVo device).

Misalignment also causes extra read/write/seek operations. This can hypothetically shorten the drive's life, as opposed to the life-span it may have hypothetically had if operated aligned.


----------



## cuda74360 (Jul 12, 2013)

How would this be different than using pdisk and dd?



jmbach said:


> Yes but you will have to do a manual copy of each partition to the new drive and adjust the partition map accordingly. iBored can be used to do it. Would suggest getting used to that program if you want to attempt it.


----------



## jmbach (Jan 1, 2009)

nooneuknow said:


> As I always do, I'm now going to point out that there is more to misalignment than performance issues, even if they aren't enough to notice on the equipment in question, that the drive is used in (such as a TiVo device).
> 
> Misalignment also causes extra read/write/seek operations. This can hypothetically shorten the drive's life, as opposed to the life-span it may have hypothetically had if operated aligned.


That is true. It is those extra operations that can decrease lifespan of the drive and degrade performance. Having the partitions aligned is only the first step. The OS has to cooperate as well or the first step is meaningless.


----------



## jmbach (Jan 1, 2009)

cuda74360 said:


> How would this be different than using pdisk and dd?


You can use pdisk to adjust the partition map. Other than copying a drive to another drive, I have not used dd for anything else. Consequently I do not know the full extent of dd's abilities. If it can copy a certain range of sectors from one drive to a specified area in another drive, then it can be used as well.


----------



## Soapm (May 9, 2007)

jmbach said:


> You can use pdisk to adjust the partition map. Other than copying a drive to another drive, I have not used dd for anything else. Consequently I do not know the full extent of dd's abilities. If it can copy a certain range of sectors from one drive to a specified area in another drive, then it can be used as well.


With dd you would more than likely copy partitions one at a time. So once the new partition map is created you can use dd to fill them with the old data.


----------



## nooneuknow (Feb 5, 2011)

jmbach said:


> That is true. It is those extra operations that can decrease lifespan of the drive and degrade performance. Having the partitions aligned is only the first step. The OS has to cooperate as well or the first step is meaningless.


I don't know enough about all the different OS tech factors to chime in there. I do agree with how you accurately interlinked the performance and lifespan factors as being due to essentially the same reason. On a better day, I'd have included that...

What I think would get people's attention the fastest, is the lowered life-expectancy, which I'd better stick to calling theory, so as to not get jumped.

If you talk to most about a miniscule (in a TiVo) performance drop, which may not be enough to notice, their eyes will just glaze over once you start explaining specifics.

I theorize that the longer an AF/4K/512e drive runs, that the performance should take a bigger and bigger hit. By the time the drive is in use long enough, and/or the data fragments enough, for the same people to notice, the last thing they may think of, is the drive. Everybody is SO FAST to post SUCCESS, post-drive upgrade, when all many have done at that point is to get the drive to boot up, download its updates, and start recording.

Yet, tell them the drive will (in theory) fail sooner, then you may have their full attention...

Unfortunately, it will be quite some time before most people start experiencing any noticeable performance degradation, due to the drive not being aligned, and even longer before complaints of short life spans come rolling in...

I'd like to try and speed these things up, and prove these theories. Any ideas or suggestion, that aren't obvious?


----------



## cuda74360 (Jul 12, 2013)

Soapm said:


> With dd you would more than likely copy partitions one at a time. So once the new partition map is created you can use dd to fill them with the old data.


Yes, that's exactly what I did. I issued the following commands:

dd if=/dev/sdb2 of=sdb2.img bs=1024K
dd if=/dev/sdb3 of=sdb3.img bs=1024K
dd if=/dev/sdb4 of=sdb4.img bs=1024K
dd if=/dev/sdb5 of=sdb5.img bs=1024K
dd if=/dev/sdb6 of=sdb6.img bs=1024K
dd if=/dev/sdb7 of=sdb7.img bs=1024K
dd if=/dev/sdb8 of=sdb8.img bs=1024K
dd if=/dev/sdb9 of=sdb9.img bs=1024K
dd if=/dev/sdb10 of=sdb10.img bs=1024K
dd if=/dev/sdb11 of=sdb11.img bs=1024K
dd if=/dev/sdb12 of=sdb12.img bs=1024K
dd if=/dev/sdb13 of=sdb13.img bs=1024K

Then used pdisk to delete all the partitions, and recreate them. Then issued the following commands:

dd if=sdb2.img of=/dev/sdb2 bs=1024K
dd if=sdb3.img of=/dev/sdb3 bs=1024K
dd if=sdb4.img of=/dev/sdb4 bs=1024K
dd if=sdb5.img of=/dev/sdb5 bs=1024K
dd if=sdb6.img of=/dev/sdb6 bs=1024K
dd if=sdb7.img of=/dev/sdb7 bs=1024K
dd if=sdb8.img of=/dev/sdb8 bs=1024K
dd if=sdb9.img of=/dev/sdb9 bs=1024K
dd if=sdb10.img of=/dev/sdb10 bs=1024K
dd if=sdb11.img of=/dev/sdb11 bs=1024K
dd if=sdb12.img of=/dev/sdb12 bs=1024K
dd if=sdb13.img of=/dev/sdb13 bs=1024K

I skipped over partition 1, since that is the partition map.

No joy. The TiVo did boot, but had the issues I described in the original post.


----------



## cuda74360 (Jul 12, 2013)

nooneuknow said:


> If you talk to most about a miniscule (in a TiVo) performance drop, which may not be enough to notice, their eyes will just glaze over once you start explaining specifics.


I'm experiencing far more than a miniscule performance drop. In addition to the skipped frames, everything else takes longer. It takes at least 2x-3x longer to add/remove a season pass, delete a show, initiate a transfer from another TiVo...


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> Having the partitions aligned is only the first step. The OS has to cooperate as well or the first step is meaningless.


As long as the partitions are aligned, wouldn't the drives 512 byte emulation take care of the rest? At least, that is their "fix" or Windows XP (which has no native support for AF.)

http://www.wdc.com/wdproducts/library/WhitePapers/ENG/2579-771430.pdf


----------



## ggieseke (May 30, 2008)

One thing I'd like to point out is that the linux and MFS application partitions all start with header structures of varying sizes. Getting the actual files aligned to 4K is probably more important in the long run than the start address of the partition itself.

The MFS media partitions use 128K blocks and the individual FSIDs allocated for recordings are usually 1GB, so there's probably less overlap in read/write cycles than you might suspect even if they're not 4K aligned.

Just food for thought...


----------



## jmbach (Jan 1, 2009)

cuda74360 said:


> As long as the partitions are aligned, wouldn't the drives 512 byte emulation take care of the rest? At least, that is their "fix" or Windows XP (which has no native support for AF.)
> 
> http://www.wdc.com/wdproducts/library/WhitePapers/ENG/2579-771430.pdf


Yes and no. The OS still has to cooperate some. The OS groups blocks into clusters. For Windows XP those cluster sizes for the drives we are talking about is 4k or a multiple of 4k. If those clusters were smaller than aligning the partitions are of no help. So in 512 emulation, the drive firmware does the interpolation and converts the 8 - 512 block call (1 cluster) to the drive as one 4k block.


----------



## nooneuknow (Feb 5, 2011)

cuda74360 said:


> I'm experiencing far more than a miniscule performance drop. In addition to the skipped frames, everything else takes longer. It takes at least 2x-3x longer to add/remove a season pass, delete a show, initiate a transfer from another TiVo...


EDIT/ADD: I experienced the same behavior you describe, after the last update rolled out. I don't blame the drive, alignment, or anything else, except TiVo degrading the core TiVo performance with the last update. All four of my Premiere TCD746320 2-Tuner units are doing the same, regardless of if they have a true 512byte sector, or an AF drive in them. The update was comparable to the prior update, if not slightly faster, when first installed. Now, over time, it is an unbearable dog, knowing how fast it was before.

I did post, that people could start blaming the wrong things on the wrong things, with all the AF/4K/512e uncertainty going around. I think this might be an example. That's just an early opinion/theory at this time.
END of EDIT/ADD

I only said it that way, due to being jumped-on by people claiming that there is no performance degradation, or if it is, you'd never see it in TiVo usage scenarios. I got sick of it, so I adjusted.

I'm getting jumped on, no matter what I say. Often, it's when I'm trying to help. Now I see, with even greater clarity, why people like richsadams bailed ship (TCF).

Not that your post is one I consider to be jumping on me. I'm just in the middle of replying to all the actual jumping-upon that I am enduring, right now. Sorry.


----------



## cuda74360 (Jul 12, 2013)

nooneuknow said:


> EDIT/ADD: I experienced the same behavior you describe, after the last update rolled out. I don't blame the drive, alignment, or anything else, except TiVo degrading the core TiVo performance with the last update. All four of my Premiere TCD746320 2-Tuner units are doing the same, regardless of if they have a true 512byte sector, or an AF drive in them. The update was comparable to the prior update, if not slightly faster, when first installed. Now, over time, it is an unbearable dog, knowing how fast it was before.
> 
> I did post, that people could start blaming the wrong things on the wrong things, with all the AF/4K/512e uncertainty going around. I think this might be an example. That's just an early opinion/theory at this time.
> END of EDIT/ADD
> ...


Not trying to jump on you or your post, sorry if it came across that way. I appreciate any help and advice you can provide.

I don't think that an update is causing my issue. After the original drive passed WD diagnostics, I reformatted it and restored my MFS Live backup to it. So both the WD10EURX (AF) and the original WD10EVVS (non-AF) have the same software. There is a huge difference between the two, with the non-AF drive providing much better performance.


----------



## nooneuknow (Feb 5, 2011)

cuda74360 said:


> Not trying to jump on you or your post, sorry if it came across that way. I appreciate any help and advice you can provide.
> 
> I don't think that an update is causing my issue. After the original drive passed WD diagnostics, I reformatted it and restored my MFS Live backup to it. So both the WD10EURX (AF) and the original WD10EVVS (non-AF) have the same software. There is a huge difference between the two, with the non-AF drive providing much better performance.


I didn't take it that way, and it didn't come across that way. Like I said, I was on my umpteenth post, dealing with others which did, and it seeped into my post. Sorry.

I'll try to see if I can figure out why this is happening for you/to you.

All the details, even the most minor ones, could help me get there faster. PM me, if you feel that it would be cluttering up the thread. I really don't think anybody will mind, though. It's what the thread is here for, right?


----------



## unitron (Apr 28, 2006)

Rich is sorely missed.

Let me just chime in here that some 4K drives, mostly the older ones, are 4K physical, but present a 512 "face" to the world. 

Some later ones are straight up 4K.

The version of PartedMagic on the latest (5.24) version of the Ultimate Boot CD has a utility called disk health or something like that that'll show if it's doing the fake 512 thing or not.

It might be that the 4K physical/512 logical drives will work better in S3s.


----------



## Worf (Sep 15, 2000)

Actually, older 4K drives presented themselves as 4K drives, which caused a bunch of issues.

Modern drives are "Advance Format" drives which are 4K internally but present a standard 512 byte interface. Some drives require alignment, while others (Seagate notably) handle misalignment automatically.

http://en.wikipedia.org/wiki/Advanced_format


----------



## nooneuknow (Feb 5, 2011)

Worf said:


> Actually, older 4K drives presented themselves as 4K drives, which caused a bunch of issues.
> 
> Modern drives are "Advance Format" drives which are 4K internally but present a standard 512 byte interface. Some drives require alignment, while others (Seagate notably) handle misalignment automatically.
> 
> http://en.wikipedia.org/wiki/Advanced_format


I was hoping somebody (else) would correct that.

I think the old AF drives put the carriage in front of the horse, by using 4K physical & 4K logical. Then came 4K/512e (4K physical & 512byte logical). 4K for both needs the horse to be better prepared to deal with the carriage.


----------



## cuda74360 (Jul 12, 2013)

unitron said:


> It might be that the 4K physical/512 logical drives will work better in S3s.


The WD10EURX reports having a physical sector size of 4096, with a logical size of 512


----------



## A J Ricaud (Jun 25, 2002)

unitron said:


> Rich is sorely missed.


+1. I have seen him lurking about on the iPad forum, though. He's just as helpful there as he was on this forum. Good guy.


----------



## jmbach (Jan 1, 2009)

cuda74360 said:


> Yes, that's exactly what I did. I issued the following commands:
> 
> dd if=/dev/sdb2 of=sdb2.img bs=1024K
> dd if=/dev/sdb3 of=sdb3.img bs=1024K
> ...


You have to copy block0 and the APM as well as the everything in the other partitions otherwise the tivo will not boot. I am surprised that dd actually copied the partitions individually. As a general rule, the TiVo drive is not a standard anything drive. It is not a standard apple drive, Linux drive, Windows drive. Consequently, unless something is specifically written to access a tivo drive, no standard OS tool will work. In JMFS, the dd command works because it is doing a raw copy that is agnostic to what is on the drive.

Did you mod the actual TiVo drive? What were the sizes of the img files you created for each partition.


----------



## nooneuknow (Feb 5, 2011)

jmbach said:


> You have to copy block0 and the APM as well as the everything in the other partitions otherwise the tivo will not boot. I am surprised that dd actually copied the partitions individually. As a general rule, the TiVo drive is not a standard anything drive. It is not a standard apple drive, Linux drive, Windows drive. Consequently, unless something is specifically written to access a tivo drive, no standard OS tool will work. In JMFS, the dd command works because it is doing a raw copy that is agnostic to what is on the drive.
> 
> Did you mod the actual TiVo drive? What were the sizes of the img files you created for each partition.


Actually, JMFS uses GNU dd_rescue. DD could be on the CD, but it's not what the JMFS front-end uses for it's raw copy operation. I've used dd_rescue from the command line of the JMFS CD to use my own command line options for faster copies, or in the cases where JMFS can't use it's menu system due to it not detecting what it can identify as a TiVo drive (like drives that have been modified by other tools).

It can also copy a TiVo partition. One night while operating while sleepy, I accidentally said SDA1 SDA2, instead of SDA SDB, and copied partition 1 of drive A over the top of partition 2 of drive A. Thankfully, I had made sure to have a backup copy of the drive A I was working with (A was supposed to be source, B was supposed to be target).

Surprisingly, the drive I accidentally copied partition 1 over partition 2 on, only behaved differently, in the sense that it was left unable to do a KS52, or install a software update to partition 2. It basically nuked the alternate partition swapping function. This may be a way to disable future software updates, for those who may want to do that. Or, maybe it will somehow just use the one partition set, and not disable future updates. There are reasons for wanting to do this. However, this was on a TiVo HD drive, which made it hard to see if it could "block" an update. With 11.0m rolling out, perhaps now is my chance to see what happens...

EDIT/ADD: What I found interesting is that while partition 1 is small, and partition 2 is larger, partition 2 retained it's size. I think the accidental over-write just sector-by-sector overwrote the beginning of partition 2, and left the rest of that partition alone. I also tried every kickstart repair function, to try and fix what I had done. Not only did the drive not turn into a brick, but all just remained the same (with the #1 partition overwritten to the beginning of partition #2, and no alternate partition swapping taking place).


----------



## Worf (Sep 15, 2000)

nooneuknow said:


> I was hoping somebody (else) would correct that.
> 
> I think the old AF drives put the carriage in front of the horse, by using 4K physical & 4K logical. Then came 4K/512e (4K physical & 512byte logical). 4K for both needs the horse to be better prepared to deal with the carriage.


Well, the problem was the dominant OS was Windows XP, which lacked all support for 4K, and even worse, always misaligned the partition. If you ran Linux, no problem.

Windows 7 doesn't quite support 4K native, but Windows 8 does. Windows 7 however stuck to a much saner partitioning scheme which didn't misalign partitions (and actually aligns it for SSDs).

Of course, 4K drives are nothing compared to SSDs which require even more special alignment to a block (128-256k).


----------



## nooneuknow (Feb 5, 2011)

Worf said:


> Well, the problem was the dominant OS was Windows XP, which lacked all support for 4K, and even worse, always misaligned the partition. If you ran Linux, no problem.
> 
> Windows 7 doesn't quite support 4K native, but Windows 8 does. Windows 7 however stuck to a much saner partitioning scheme which didn't misalign partitions (and actually aligns it for SSDs).
> 
> Of course, 4K drives are nothing compared to SSDs which require even more special alignment to a block (128-256k).


Well, as soon as MS ends all access to support and Windows/Microsoft update, that stubborn XP might finally take the slide in numbers it has defied. Does XP still have access to updates? I've got a few dormant XP units around that I should update, if it's still an option.

The delicate issue with the blanket statement of "Linux=no problem", is that TiVo is a "Tivoized" Linux OS (take a look on Wikipedia, and you'll find that term and an article all about it). I understand that this whole APM - Apple Partition M(whatever it actually stands for), is yet some other TiVoized part of the mix, which I think is where the problem lies...

I don't want to go Off-Topic... But you brought up SSDs, which are giant flash drives. How does one take a USB (flash memory) stick, and determine what sector size to format it with, other than trial-and-error, by measuring what sector size gives the best read/write times? Is there a tool that can "suss" that information out? Sometimes you just can't go to a website and get that info, especially when many of the drives give no indication of who even made any one part of the whole. I know Windows 8 can somehow figure it out. I just started with Win7, and Win8 isn't for me at all...


----------



## ggieseke (May 30, 2008)

Updates to XP will continue until April 8, 2014. After that it will still run fine but without ongoing security patches I'd probably limit my use of it to running older software and keep it away from the internet.

It's had a hell of a run.


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> You have to copy block0 and the APM as well as the everything in the other partitions otherwise the tivo will not boot. I am surprised that dd actually copied the partitions individually. As a general rule, the TiVo drive is not a standard anything drive. It is not a standard apple drive, Linux drive, Windows drive. Consequently, unless something is specifically written to access a tivo drive, no standard OS tool will work. In JMFS, the dd command works because it is doing a raw copy that is agnostic to what is on the drive.
> 
> Did you mod the actual TiVo drive? What were the sizes of the img files you created for each partition.


Since I restored a truncated image to the drive first, there was no need to copy block 0. I didn't copy the APM, as that would overwrite the corrected partition table I created with the original one.

The image files were the same size as the partitions.


----------



## jmbach (Jan 1, 2009)

Could you post the MFSinfo information on your modified drive.
Just so I understand your process, you first restored the truncated image, backed up each partition, modified the partition structure with pdisk, then restored each partition. 
I like that. :thumbup: Did not even think out using dd on a tivo drive that way. It's a good day when you learn something new.  Much easier to do it that way than with iBored as it is more automated.


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> Could you post the MFSinfo information on your modified drive.


---------------------------------------------------------------------
Super Header
state=0 magic=ebbafeed
devlist=/dev/hda10 /dev/hda11 /dev/hda12 /dev/hda13
zonemap_ptr=1121 total_secs=1951672320
---------------------------------------------------------------------
MFS volume set for /dev/sda
The MFS volume set contains 4 partitions
Partition sectors size
/dev/sda10 589824 288 MiB
/dev/sda11 876054528 427761 MiB
/dev/sda12 589824 288 MiB
/dev/sda13 1074438144 524628 MiB
Total MFS sectors: 1951672320d
Total MFS volume size: 952965 MiB
Estimated hours in a standalone TiVo: 1181
---------------------------------------------------------------------
Zone Maps 
Zone 0: type=0 
map_start=1121 map_size=1 backup_map_start=589822
next_map_start=263266 next_map_size=34 next_backup_map_start=589788
zone_first=1122 zone_last=263265 zone_size=262144 min(chunk)=262144 
free=262144 checksum=1675366169 logstamp=12362757 num_bitmap=1
Zone 1: type=2 
map_start=263266 map_size=34 backup_map_start=589788
next_map_start=263300 next_map_size=34 next_backup_map_start=589754
zone_first=589824 zone_last=876642303 zone_size=876052480 min(chunk)=20480 
free=3911680 checksum=3444533777 logstamp=12363028 num_bitmap=17
Zone 2: type=1 
map_start=263300 map_size=34 backup_map_start=589754
next_map_start=876644352 next_map_size=1 next_backup_map_start=877234175
zone_first=263334 zone_last=589749 zone_size=326416 min(chunk)=8 
free=49840 checksum=4206158201 logstamp=12363028 num_bitmap=17
Zone 3: type=0 
map_start=876644352 map_size=1 backup_map_start=877234175
next_map_start=876906497 next_map_size=34 next_backup_map_start=877234141
zone_first=876644353 zone_last=876906496 zone_size=262144 min(chunk)=262144 
free=262144 checksum=580165030 logstamp=12362757 num_bitmap=1
Zone 4: type=2 
map_start=876906497 map_size=34 backup_map_start=877234141
next_map_start=876906531 next_map_size=34 next_backup_map_start=877234107
zone_first=877234176 zone_last=1951655935 zone_size=1074421760 min(chunk)=20480 
free=6676480 checksum=18646780 logstamp=12363028 num_bitmap=17
Zone 5: type=1 
map_start=876906531 map_size=34 backup_map_start=877234107
next_map_start=0 next_map_size=0 next_backup_map_start=-6148914691236517206
zone_first=876906565 zone_last=877234100 zone_size=327536 min(chunk)=8 
free=260720 checksum=4206404614 logstamp=12362757 num_bitmap=17
Total Inodes = 262144
This MFS volume may be expanded 4 more times
---------------------------------------------------------------------


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> Just so I understand your process, you first restored the truncated image, backed up each partition, modified the partition structure with pdisk, then restored each partition.
> I like that. :up: Did not even think out using dd on a tivo drive that way. It's a good day when you learn something new.  Much easier to do it that way than with iBored as it is more automated.


Yes, that is what I did.


----------



## jmbach (Jan 1, 2009)

Could you post the WinMFS version of MFSInfo output as well


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> Could you post the WinMFS version of MFSInfo output as well


no can do... don't have a windows box


----------



## jmbach (Jan 1, 2009)

No problem. How about the pdisk output.


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> No problem. How about the pdisk output.


Partition map (with 512 byte blocks) on '/dev/sdb'
#: type name length base ( size )
1: Apple_partition_map Apple 63 @ 1 
2: Image Bootstrap 1 8 @ 876054592 
3: Image Kernel 1 8192 @ 876054600 ( 4.0M)
4: Ext2 Root 1 524288 @ 876062792 (256.0M)
5: Image Bootstrap 2 8 @ 876587080 
6: Image Kernel 2 8192 @ 876587088 ( 4.0M)
7: Ext2 Root 2 524288 @ 876595280 (256.0M)
8: Swap Linux swap 262144 @ 877119568 (128.0M)
9: Ext2 /var 524288 @ 877381712 (256.0M)
10: MFS MFS application region 589824 @ 877906000 (288.0M)
11: MFS MFS media region 876054528 @ 64 (417.7G)
12: MFS Second MFS application region 589824 @ 878495824 (288.0M)
13: MFS Second MFS media region 1074438144 @ 879085648 (512.3G)
14: Apple_Free Extra 1376 @ 1953523792

Device block size=512, Number of Blocks=1953525168 (931.5G)
DeviceType=0x0, DeviceId=0x0


----------



## jmbach (Jan 1, 2009)

Since you started out with a 1TB drive and you expanded the bootstrap partitions, what partition(s) did you short to get the extra space to expand the bootstrap partitions.

A couple of things come to mind with your OP problem. One, the truncated backup could be damaged from the rebooting issue. Might try a backup from another source. The other might be capacitors that are going bad and need to be replaced. (That was my problem). 

One last thing. Some drive diagnostics program can/will 'recertify' a failing drive as it is testing it such that it may pass diagnostics again. See if you have a program that will check to see if a drive has been reassigning any sectors. There has to be enough reassigning of sectors for the SMART to trigger but the dynamic reallocation of sectors can also cause some of the same problems.


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> Since you started out with a 1TB drive and you expanded the bootstrap partitions, what partition(s) did you short to get the extra space to expand the bootstrap partitions.


Partition #14, Apple_Free Extra, shrunk from 1390 to 1376.



jmbach said:


> A couple of things come to mind with your OP problem. One, the truncated backup could be damaged from the rebooting issue. Might try a backup from another source. The other might be capacitors that are going bad and need to be replaced. (That was my problem).


I restored the truncated backup to the original drive and it's been running great for the past 2 days. I don't think that backup is the issue.

The capacitors are not bulging or leaking. Since it's working fine with the original drive, I'd guess that the capacitors are probably fine.



jmbach said:


> One last thing. Some drive diagnostics program can/will 'recertify' a failing drive as it is testing it such that it may pass diagnostics again. See if you have a program that will check to see if a drive has been reassigning any sectors. There has to be enough reassigning of sectors for the SMART to trigger but the dynamic reallocation of sectors can also cause some of the same problems.


I'll see what I can come up with.


----------



## unitron (Apr 28, 2006)

cuda74360 said:


> Partition map (with 512 byte blocks) on '/dev/sdb'
> #: type name length base ( size )
> 1: Apple_partition_map Apple 63 @ 1
> 2: Image Bootstrap 1 8 @ 876054592
> ...


Any reason for using the Series 1 "non-optimized" partition layout?


----------



## jmbach (Jan 1, 2009)

Interesting, did not realize that original HD XL drive had an apple_free partition at the end.


----------



## nooneuknow (Feb 5, 2011)

jmbach said:


> Interesting, did not realize that original HD XL drive had an apple_free partition at the end.


DVR_DUDE's 2TB drives for the TCD746320 Premiere, and TCD652160 HD both have apple_free spaces on them as well.

I wonder if there is a legitimate need for them...


----------



## jmbach (Jan 1, 2009)

nooneuknow said:


> DVR_DUDE's 2TB drives for the TCD746320 Premiere, and TCD652160 HD both have apple_free spaces on them as well.
> 
> I wonder if there is a legitimate need for them...


Not really. I read in another post that the extra space came because they used a smaller swap partition when they image a drive. (64 instead of 128) In fact, one person had a replacement drive for their TiVo from one of them and JMFS expansion process was not recognized by the tivo until that apple_free partition was deleted and then jmfs expansion was performed. For TiVos that WinMFS and MFSLive work on, they recognize that the apple_free partition is really just free space and uses it to expand. JMFS does not and creates the expansion partition after the apple_free space (partition).


----------



## nooneuknow (Feb 5, 2011)

jmbach said:


> Not really. I read in another post that the extra space came because they used a smaller swap partition when they image a drive. (64 instead of 128) In fact, one person had a replacement drive for their TiVo from one of them and JMFS expansion process was not recognized by the tivo until that apple_free partition was deleted and then jmfs expansion was performed. For TiVos that WinMFS and MFSLive work on, they recognize that the apple_free partition is really just free space and uses it to expand. JMFS does not and creates the expansion partition after the apple_free space (partition).


Could the apple_free space, if not at the end of the drive, be used as a means to achieve 4K alignment (in partitions after the apple_free space)? Could you have multiple areas of apple_free space used as "spacers" to nudge alignment?

I've been wondering this for a long time, and just now thought to post it.


----------



## Worf (Sep 15, 2000)

ggieseke said:


> Updates to XP will continue until April 8, 2014. After that it will still run fine but without ongoing security patches I'd probably limit my use of it to running older software and keep it away from the internet.
> 
> It's had a hell of a run.


Partially correct. XP left mainstream support on April 14, 2009. This means that XP will no longer get any feature updates at all. Right now, Microsoft is providing 5 extra years of "extended support" which means the only thing XP is getting is security updates.

During mainstream support, it'll get feature improvements and security updates. After that phase passes, it's in extended support where there are no more feature improvements (including things like new IE versions and such) but it will still get security fixes.

And that Apple_Free partition is leftover space that can't be used - it's not terribly big (about 768k or so), just the result of partitioning ending up with free space.

I admit though, when I copied my TiVo drive to a new drive, I didn't bother realigning the partitions - the MFS data partition is written in such huge strips (1MB or so - about a second of video) that any performance loss due to the Read-Modify-Write will be minimal, if at all (because writes are cached, so if another write happens, it'll update the whole sector anyhow).

Of course, the MFS application region is a bigger concern because the updates to it will be slower.

Then again, the TiVo doesn't exactly demand a lot of performance from the hard drive...


----------



## jmbach (Jan 1, 2009)

nooneuknow said:


> Could the apple_free space, if not at the end of the drive, be used as a means to achieve 4K alignment (in partitions after the apple_free space)? Could you have multiple areas of apple_free space used as "spacers" to nudge alignment?
> 
> I've been wondering this for a long time, and just now thought to post it.


That is a good use of the apple_free partition. However you don't need any apple_free partitions between partitions. The best approach is what was described in an earlier post. You can make the partitions bigger without causing problems, just cannot make them smaller. The extra space in the partitions is ignored.


----------



## cuda74360 (Jul 12, 2013)

unitron said:


> Any reason for using the Series 1 "non-optimized" partition layout?


The MFS Live CD's other way puts the MFS partitions at the end of the drive.

The "optimized" way is what I chose, since it split the MFS partitions similarly to the way it was done on the factory drive.


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> Interesting, did not realize that original HD XL drive had an apple_free partition at the end.


It didn't. That only showed up after restoring the truncated image from MFS Live. It worked though, as otherwise there was no free space to do the realignment.


----------



## jmbach (Jan 1, 2009)

cuda74360 said:


> It didn't. That only showed up after restoring the truncated image from MFS Live. It worked though, as otherwise there was no free space to do the realignment.


Okay. Too bad we don't have a pdisk print out of the original partition structure to see what changed with MFSLive restore.


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> Okay. Too bad we don't have a pdisk print out of the original partition structure to see what changed with MFSLive restore.


I made a printout of the original pdisk output, but misplaced it.

I'm not sure it matters though... as I tried a JMFS copy of the original drive to the new 4K drive and still experienced the same sluggish performance.

The problems only occur with the 4K drive.


----------



## jmbach (Jan 1, 2009)

More for self education on how MFSLive works on a drive. 

Have you tried another 4k drive. Might be just the drive.


----------



## cuda74360 (Jul 12, 2013)

nooneuknow said:


> Could the apple_free space, if not at the end of the drive, be used as a means to achieve 4K alignment (in partitions after the apple_free space)? Could you have multiple areas of apple_free space used as "spacers" to nudge alignment?
> 
> I've been wondering this for a long time, and just now thought to post it.


Yes, you can. The Apple_Free partitions are just APM's way of marking free space.


----------



## cuda74360 (Jul 12, 2013)

jmbach said:


> More for self education on how MFSLive works on a drive.
> 
> Have you tried another 4k drive. Might be just the drive.


No, I haven't.


----------



## bmgoodman (Dec 20, 2000)

At the risk of being totally off-base (having not kept up with state-of-the-art), aren't there jumpers on the WD10EURX drive dealing with Advanced Format? Is there any need to consider how the jumpers are set?


----------



## cuda74360 (Jul 12, 2013)

bmgoodman said:


> At the risk of being totally off-base (having not kept up with state-of-the-art), aren't there jumpers on the WD10EURX drive dealing with Advanced Format? Is there any need to consider how the jumpers are set?


From the WD website:

"Early 3.5-inch WD drives with Advanced Format supported jumper pins 7  8. Placing a jumper on these pins adjusted the drives internal alignment for single partition XP installations. Support for this jumper setting is no longer needed on newer drives."

So, this setting won't help since TiVo, and Linux in general, uses multiple partitions.


----------



## unitron (Apr 28, 2006)

cuda74360 said:


> The MFS Live CD's other way puts the MFS partitions at the end of the drive.
> 
> The "optimized" way is what I chose, since it split the MFS partitions similarly to the way it was done on the factory drive.


I realized later that I'd probably not looked at your partition map closely enough (or with enough sleep under my belt) and was probably mistaken in what I said, and sure enough, I was wrong.


----------



## nooneuknow (Feb 5, 2011)

unitron said:


> I realized later that I'd probably not looked at your partition map closely enough (or with enough sleep under my belt) and was probably mistaken in what I said, and sure enough, I was wrong.


There's yet another thing you and I share, or have in common: Posting while sleep deprived.


----------



## cuda74360 (Jul 12, 2013)

Has anyone tried a Seagate ST1000VM002 in a TiVo? I called Seagate, and they confirmed that this drive uses SmartAlign technology, which is supposed to eliminate the need to realign partitions.


----------



## jmbach (Jan 1, 2009)

cuda74360 said:


> Has anyone tried a Seagate ST1000VM002 in a TiVo? I called Seagate, and they confirmed that this drive uses SmartAlign technology, which is supposed to eliminate the need to realign partitions.


Yes. I have this one in my Premiere XL and a WD20EURS in my OLED S3.


----------



## ciper (Nov 4, 2004)

Can someone help me wrap my head around how to verify that the partitions are aligned? It seems to me that the starting point of each partition shown in mfsinfo should be multiplied by 512 and get a number divisible by 4096? Does the drive start counting at block zero, with zero being the bootsector and then block 1 the start of the partition map? Or should the "second" partition start at 63 to be 4k aligned?

Another question about stretching partitions to force alignment. In the example posted previously the partitions are in the following order 1, 11, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13 and the first bootstrap is partition 2. That means if partition 11 is not already aligned there is no way to adjust the size in that way right?

For example 
Partition 11 at 64 blocks long so 64*512/4096=8
Partition 2 876054528*512/4096=109506824
Partition 3 876054600*512/4096=109506825
Partition 13 879085648*512/4096=109885706

Is that right?



cuda74360 said:


> Partition map (with 512 byte blocks) on '/dev/sdb'
> #: type name length base ( size )
> 1: Apple_partition_map Apple 63 @ 1
> 2: Image Bootstrap 1 8 @ 876054592
> ...


----------



## sfhub (Jan 6, 2007)

As long as all the start locations of the swap, var, and the mfs regions are evenly divisible by 4 you are 4k-aligned.

IMO it isn't worth the effort to try and align.

TiVo recording is mostly sequential writes which are unaffected by misalignment.

Disk reads are also unaffected by misalignment.

The only thing you need too worry about is whether your guide indexing can complete in a reasonable amount of time.

I went through this a couple of months ago and was also worried about alignment. On my drives the bootstrap partitions threw off the alignment because they were only 1 block long and there were two of them. I considered whether to make them 4 blocks and preserve alignment but opted instead to just ignore alignment and test out the drive.

I tested recording 2 HD streams, MRV of HD shown to second TiVo, playback of HD show, skipping forward/back, and connected to mother ship to download guide data. It all worked fine.

While indexing was going on, there was maybe a .5 second longer delay bringing up the now playing list, but it wasn't something worth pursuing.

Unitron was right when he said if there were going to be problems people would have complained by now.


----------



## ciper (Nov 4, 2004)

sfhub said:


> IMO it isn't worth the effort to try and align.


You may be right. I was trying to track down a random reboot issue that only showed up after installing a 4k advanced format drive. The original A drive hung a couple times and so I preemptively connected it to my computer to check its smart status. It had not failed yet but there were lots of recovered sectors so I decided to replace it before it died. 
Initially I used dd conv=sync,noerror which should "skip" any bad areas. I was getting a reboot. 
So then I did a straight DD which completed without errors (another 5 damn hours). That also had a random reboot. 
I then used the clean Broflovski image, still had a reboot?!

My theory was that the built in Tivo failsafe that reboots the system when the drive doesn't respond fast enough was kicking in. The reboots mostly happened while two HD streams were buffering/recording and I started to watch a third show (and perhaps guide data was working) which should put the most stress on the drive.

Perhaps its something else? Any ideas?


----------



## sfhub (Jan 6, 2007)

ciper said:


> You may be right. I was trying to track down a random reboot issue that only showed up after installing a 4k advanced format drive. The original A drive hung a couple times and so I preemptively connected it to my computer to check its smart status. It had not failed yet but there were lots of recovered sectors so I decided to replace it before it died.
> Initially I used dd conv=sync,noerror which should "skip" any bad areas. I was getting a reboot.
> So then I did a straight DD which completed without errors (another 5 damn hours). That also had a random reboot.
> I then used the clean Broflovski image, still had a reboot?!
> ...


I'll assume you are sure there is no power supply issue, but if you haven't looked in that direction, it is worth investigating.

There are some drives that are more fickle with TiVo than others.

IMO it is just less effort to get the drive everyone else is having success with, wd20eurs.

As to your current drive, have you tried using a stock image?

Have you confirmed dd will preserve block alignment by writing zeros for the areas it encounters errors? I used dd_rescue to recover my drives which has an option to guarantee the block alignment of the destination matches source because it fills in errors with zeros.

After dd_rescue, I ran kickstart 57 so TiVo could repair any filesystem damage that may have resulted from the writing of zeros for the error sections.


----------



## ciper (Nov 4, 2004)

I have not tried a completely stock image, but the original drive had no bad spots and DD did not encounter any errors on the second run (where it should have stopped copying if it did). 
The command line options of conv=sync,noerror tell it to not stop when an error is encountered and to write zeroes for any error "block" based on the size you specified for the copy. 
I did do a kickstart 57 AND 58 after the second time I copied my original drives and it didn't make a difference.

I just replaced all the caps on the DC side of the power supply a couple hours ago because I'm going nuts trying to figure this out. http://www.tivocommunity.com/tivo-vb/showthread.php?p=9868597#post9868597


----------



## unitron (Apr 28, 2006)

ciper said:


> I have not tried a completely stock image, but the original drive had no bad spots and DD did not encounter any errors on the second run (where it should have stopped copying if it did).
> The command line options of conv=sync,noerror tell it to not stop when an error is encountered and to write zeroes for any error "block" based on the size you specified for the copy.
> I did do a kickstart 57 AND 58 after the second time I copied my original drives and it didn't make a difference.
> 
> I just replaced all the caps on the DC side of the power supply a couple hours ago because I'm going nuts trying to figure this out. http://www.tivocommunity.com/tivo-vb/showthread.php?p=9868597#post9868597


In the interests of keeping that thread confined to hankinsohl's problem(s), what led you to replace your caps?


----------



## ciper (Nov 4, 2004)

unitron said:


> In the interests of keeping that thread confined to hankinsohl's problem(s), what led you to replace your caps?


The rebooting problems got me to check every part of the unit. I've been trying all sorts of wild things to stop it from happening. In checking these things I noticed one cap had what looked like a very slight bulge on top so I decided to replace everything on the DC side.

I also; changed both sata cables. Removed all the connections except coax and component with optical audio (was using hdmi), left the unit running outside of the shelf its normally on (in case of heat), removed it from my UPS and had it running off a surge suppressor directly from the wall. Right now I am running off a single 1tb refurbished 512k block hard drive.

My next step was to run it off an ATX power supply and completely isolate the Tivo power supply from the problem


----------

