# Premiere HD upgrade with jmfs failed



## Oregonian

I tried to upgrade my Premiere HD from a 1T drive to a 2T drive and didn't quite get it right. I'm hoping for a suggestion here to keep from having to go back to an older hard drive.

The version of JMFS that I have been using is 1.04. Both drives are WD AV drives and the new one did not require the parking disabling.

I work with Windows PCs daily and know them fairly well. I know very little about Linux and that has limited my options.

I tried to be clever (my big error!) and used my hard disk duplicator to image the 1T to the 2T drive. My (erroneous) assumption was that I could do the Expand after that copy.

I was short of time (too many household members grumbling about TiVo being down), so I threw the 2T drive in the system after the imaging and figured I would do the Expand later. Another bad choice.

The 2T drive works just fine, but only has 1T total capacity. That's what I expected after the imaging.

Eventually, I removed the 2T drive and tried to do the Expand. The software said that it failed. The numbers that it gave for disk size made it clear that it was only looking at about 1T of disk space.

My next "clever" trick was to try to copy the 2T drive to a 1.5T drive that I had on hand (I didn't have a spare 2T). JMFS objected that the source drive was smaller than the source, but I told it to go ahead. I figured that the 2T drive was only partitioned out to 1T, so it should fit. The copy ended with an error (it ran out of space), but the 1.5T drive seemed to work fine in the TiVo. It booted just fine and I was able to retrieve a random assortment of recordings that I had.

I tried doing the Expand on the 1.5T and got the same sort of error as I did with the 2T drive. Just as a test, I tried running Supersize. It claimed to work without error, but Expand still wouldn't work. I put the 2T back in the TiVo.

As a test, I used JMFS to copy the original 1T drive to the 1.5T. It too 6-8 hours (?), but ended without error. I told it to do the Expand and it completed without error. I never tested this drive, but I am assuming that this process worked properly. If only I had not been "clever" with the disk imager originally!

So..... as I see it, I have two solutions if I want the 2T to work. One is to start over, copy from the 1T to my 2T, and lose the last month or two of recordings. (It has been that long since I started this process.) With my video provider (Frontier), it seems that nearly everything that I record is protected, so I can't just copy these programs over to my other TiVo.

My hope is that there is some clever person here who can tell me some commands to execute on the 2T drive that will allow it to Expand to use the full drive. (Yeah... and I hope to win the Lottery!)

I can recopy to the 1.5T and try any suggestions with little risk. If I can Expand the 1.5T from 1T to the full 1.5T then I would expect to have no problem copying it back to the 2T and expanding it there.

Thanks in advance to anyone who can offer constructive suggestions on this.


----------



## jmbach

Oregonian said:


> I tried to upgrade my Premiere HD from a 1T drive to a 2T drive and didn't quite get it right. I'm hoping for a suggestion here to keep from having to go back to an older hard drive.
> 
> The version of JMFS that I have been using is 1.04. Both drives are WD AV drives and the new one did not require the parking disabling.
> 
> I work with Windows PCs daily and know them fairly well. I know very little about Linux and that has limited my options.
> 
> I tried to be clever (my big error!) and used my hard disk duplicator to image the 1T to the 2T drive. My (erroneous) assumption was that I could do the Expand after that copy.
> 
> I was short of time (too many household members grumbling about TiVo being down), so I threw the 2T drive in the system after the imaging and figured I would do the Expand later. Another bad choice.
> 
> The 2T drive works just fine, but only has 1T total capacity. That's what I expected after the imaging.
> 
> Eventually, I removed the 2T drive and tried to do the Expand. The software said that it failed. The numbers that it gave for disk size made it clear that it was only looking at about 1T of disk space.
> 
> My next "clever" trick was to try to copy the 2T drive to a 1.5T drive that I had on hand (I didn't have a spare 2T). JMFS objected that the source drive was smaller than the source, but I told it to go ahead. I figured that the 2T drive was only partitioned out to 1T, so it should fit. The copy ended with an error (it ran out of space), but the 1.5T drive seemed to work fine in the TiVo. It booted just fine and I was able to retrieve a random assortment of recordings that I had.
> 
> I tried doing the Expand on the 1.5T and got the same sort of error as I did with the 2T drive. Just as a test, I tried running Supersize. It claimed to work without error, but Expand still wouldn't work. I put the 2T back in the TiVo.
> 
> As a test, I used JMFS to copy the original 1T drive to the 1.5T. It too 6-8 hours (?), but ended without error. I told it to do the Expand and it completed without error. I never tested this drive, but I am assuming that this process worked properly. If only I had not been "clever" with the disk imager originally!
> 
> So..... as I see it, I have two solutions if I want the 2T to work. One is to start over, copy from the 1T to my 2T, and lose the last month or two of recordings. (It has been that long since I started this process.) With my video provider (Frontier), it seems that nearly everything that I record is protected, so I can't just copy these programs over to my other TiVo.
> 
> My hope is that there is some clever person here who can tell me some commands to execute on the 2T drive that will allow it to Expand to use the full drive. (Yeah... and I hope to win the Lottery!)
> 
> I can recopy to the 1.5T and try any suggestions with little risk. If I can Expand the 1.5T from 1T to the full 1.5T then I would expect to have no problem copying it back to the 2T and expanding it there.
> 
> Thanks in advance to anyone who can offer constructive suggestions on this.


I think that when you used your disk duplicator it created a partition that fills up the disk so JMFS cannot expand. Run MFSLayout on the JMFS disk on your 2TB drive and post the results. You also might want to have a copy of the MFS Live CD handy because if that is the case, will have to use pdisk on the MFS Live CD to delete that partition then use JMFS to expand.

Jim


----------



## Oregonian

The duplicator should have done a bit-for-bit copy with no interpretation of the data nor changes to partitions.
I will run MFSLayout soon and let you know what it says.

Thanks for the input!


----------



## jmbach

I agree with you. Would not expect it to modify the partition table. I guess it could possibly copy not only the drive data bit for bit but also the drive architecture. I'm reaching now for an explanation. Back with the old drives before ide drives when the controller for the drive was in an adapter card you could modify the drive architecture. In any event MFSlayout will help determine what needs to be done next. 

Sent from my SPH-L710 using Tapatalk 2


----------



## unitron

What is a Premiere HD?

On the back of the TiVo there's a sticker with a model number that starts with TCD.

What is that number?

Did you mean a Premiere XL, by chance, that comes from the factory with a 1TB drive?


----------



## Oregonian

@unitron: The model number is TCD748000. Yes, you are correct that it is a Premier XL.

@jmbach: I have some info, but I think I am missing something in the mfslayout command.

To start with, when I try to do Expand, I get the following report (long numbers eliminated):
Size of Zones
Used 915.68G
Free 11.68G
Total 927.36G
Recordable Space 927.36G

I eventually figured out that to run the mfslayout I had to eXit from the JMFS program to get to the prompt and then type:
sh mfslayout.sh

The result that I get (ignoring the disclaimer at the top) is:
Java.lang.Exception: No root MFS found
at tivo.Mfs.addDisks (Mfs.java:309)
at tivo.Mfs. <init>(Mfs.java:74)
at tivo.Mfs. <init>(Mfs.java:68)
at jmfs.Mfslayout.main(MfsLayout.java:42)

Do I need to type the command differently? In particular, do I need to specify the drive to be looking at?


----------



## unitron

Oregonian said:


> @unitron: The model number is TCD748000. Yes, you are correct that it is a Premier XL.
> 
> @jmbach: I have some info, but I think I am missing something in the mfslayout command.
> 
> To start with, when I try to do Expand, I get the following report (long numbers eliminated):
> Size of Zones
> Used 915.68G
> Free 11.68G
> Total 927.36G
> Recordable Space 927.36G
> 
> I eventually figured out that to run the mfslayout I had to eXit from the JMFS program to get to the prompt and then type:
> sh mfslayout.sh
> 
> The result that I get (ignoring the disclaimer at the top) is:
> Java.lang.Exception: No root MFS found
> at tivo.Mfs.addDisks (Mfs.java:309)
> at tivo.Mfs. <init>(Mfs.java:74)
> at tivo.Mfs. <init>(Mfs.java:68)
> at jmfs.Mfslayout.main(MfsLayout.java:42)
> 
> Do I need to type the command differently? In particular, do I need to specify the drive to be looking at?


I think it's

/root/mfslayout.sh

even though the command prompt already says /root

Are you sure you're copying from the 1TB to the 2TB and then telling it to expand the 2TB and not the 1TB, 'cause it looks like it's not seeing that unused 1TB.

But if you can get mfslayout to work and post that partition map here, that'll give us some info that might help figure out what's going on.


----------



## Oregonian

Progress!
I ran the following command:
sh mfslayout.sh /dev/sda

and I believe it outputted the information that you wanted.

It listed 18 physical zones, one of which is about 400G in size, the last one is over 500G in size. If I read it correctly, it is just telling me that the zones run out to the 1T point and no further.

If you can give me some clues, I can try to capture the output. I tried connecting a USB stick before booting and accessing it with: ls /dev/sdc, but that didn't work (no such file or directory). I was hoping to redirect output of the mfslayout command to a file so I could post it here. Is there an easy way to access either a USB stick or the floppy drive from the JMFS command line?


----------



## unitron

Oregonian said:


> Progress!
> I ran the following command:
> sh mfslayout.sh /dev/sda
> 
> and I believe it outputted the information that you wanted.
> 
> It listed 18 physical zones, one of which is about 400G in size, the last one is over 500G in size. If I read it correctly, it is just telling me that the zones run out to the 1T point and no further.
> 
> If you can give me some clues, I can try to capture the output. I tried connecting a USB stick before booting and accessing it with: ls /dev/sdc, but that didn't work (no such file or directory). I was hoping to redirect output of the mfslayout command to a file so I could post it here. Is there an easy way to access either a USB stick or the floppy drive from the JMFS command line?


You probably have to mount them first.

do

ls -l

and see what you have for virtual directories to use as mount points, but first, what do you get for

sh mfslayout.sh /dev/sdb

?


----------



## Oregonian

ls -l outputs a listing table with lots of information. I'm skipping most of it here (too much to type unless really necessary). All lines have "root root" in them. The ends of each lines are as follows: (I am retyping these, so there may be minor errors.)
.
..
.mc
.bashrc
.Profile
COPYING
README
USAGE
copyDisk.sh
guide.sh
jmfs-rev104.jar
jmfs.sh
log.log
mfsadd.sh
mfslayout.sh


sh mfslayout.sh /dev/sdb gives the "no root MFS found". My presumption from all of this is that /dev/sda is the 2T hard drive.


----------



## unitron

Oregonian said:


> ls -l outputs a listing table with lots of information. I'm skipping most of it here (too much to type unless really necessary). All lines have "root root" in them. The ends of each lines are as follows: (I am retyping these, so there may be minor errors.)
> .
> ..
> .mc
> .bashrc
> .Profile
> COPYING
> README
> USAGE
> copyDisk.sh
> guide.sh
> jmfs-rev104.jar
> jmfs.sh
> log.log
> mfsadd.sh
> mfslayout.sh
> 
> sh mfslayout.sh /dev/sdb gives the "no root MFS found". My presumption from all of this is that /dev/sda is the 2T hard drive.


You might have to create a directory to have a mount point, but let's sort out which drive is which first.

hdparm -i /dev/sda

hdparm -i /dev/sdb

hdparm -i /dev/sdc

etc.

I assume you have both the 1TB and the 2TB drives connected to the PC?


----------



## Oregonian

@unitron re: 1T vs. 2T
The 1T drive is removed from all systems at the moment. The PC I am using to do the JMFS stuff has all drives disconnected except for the 2T WD drive (WD20EURS), a floppy drive, a DVD/CD-RW drive, and a 4G USB stick.


----------



## Oregonian

hdparm -i /dev/sda gives a long listing of information
It starts with identifyin the disk as the WDC WD20EURS
If there is anything in particular you want from that, let me know and I'll type it.

hdparm -i /dev/sdb results in:
HDIO_GET_IDENTITY failed: invalid argument

hdparm -i /dev/sdc results in:
/dev/sdc No such file or directory


----------



## unitron

I was trying to pin down which drive was which 'cause I didn't know you didn't have the source drive connected.

I'm going to have to quit now and come back tomorrow, but do this

hdparm -N /dev/sda

and make sure you have the same number on both sides of the /

Then shut things down and go do something else.


----------



## Oregonian

I think you are on to something here!
When I run hdparm -N /dev/sda, I get: (hand-typed may not be perfect)

Max sectors = 18446744073321613488/3907029168 (18446744073321613488?), HPA setting seems invalid


----------



## Oregonian

I looked further into the HPA issue and concluded that I need to remove it. To that end, I ran the following from the command prompt in JMFS:

hdparm -N p3907029168 --yes-i-know-what-i-am-doing /dev/sda

It comes back with:
Setting maximum sectors permanently
Max sectors = 18446744073321613488/3907029168 (18446744073321613488?), HPA setting seems invalid 

So.... it didn't change anything. Same results after rebooting.

The good news is that I didn't trash the drive!

I looked further into the issue and found HDAT2. When I run it on the drive, it tells me that the Current Native and Current User areas are 2.0T each and the Hidden Area is 0 bytes.

So.... it appears that this is not a simple HDA issue. I am presuming that the real problem is that Max Sectors is set to 18..... which is way too large, although I don't understand why JMFS sees that value and HDAT2 does not appear to. The 3907029168 number seems correct because if I multiply it by 512 bytes/sector I am right at 2Tbytes.

My presumption here is that the Max Sectors value is wrong and needs to be corrected. Does this seem correct and, if so, how can it be done?

As always... many thanks for your input. It is greatly appreciated.


----------



## jmbach

Oregonian said:


> I think you are on to something here!
> When I run hdparm -N /dev/sda, I get: (hand-typed may not be perfect)
> 
> Max sectors = 18446744073321613488/3907029168 (18446744073321613488?), HPA setting seems invalid


Interesting. If I am seeing this straight the max sectors is greater than the native sectors of the drive. The first number should be less than or equal to the second. I guess the duplicator did something funky.

I wonder if copying block 0 of your good 1TB drive to block 0 of the 2TB drive would resolve the issue. Whatever the disk duplicator did is not permanent based on your experiment with the 1.5TB drive. Do you have any utilities that can read / write drive blocks. The only other block that may need to be copied is block 1. Could use your 1.5TB drive as the drive to experiment with. Would copy the first terabyte of your 2TB drive to the 1.5TB. Then try the copy.

Just a thought.

Sent from my SPH-L710 using Tapatalk 2


----------



## jmbach

Another thought. You had hdat2. Have you tried the solution found in the FAQ question 1 on http://www.hdat2.com/ just for fun.....maybe


----------



## unitron

Oregonian said:


> I looked further into the HPA issue and concluded that I need to remove it. To that end, I ran the following from the command prompt in JMFS:
> 
> hdparm -N p3907029168 --yes-i-know-what-i-am-doing /dev/sda
> 
> It comes back with:
> Setting maximum sectors permanently
> Max sectors = 18446744073321613488/3907029168 (18446744073321613488?), HPA setting seems invalid
> 
> So.... it didn't change anything. Same results after rebooting.
> 
> The good news is that I didn't trash the drive!
> 
> I looked further into the issue and found HDAT2. When I run it on the drive, it tells me that the Current Native and Current User areas are 2.0T each and the Hidden Area is 0 bytes.
> 
> So.... it appears that this is not a simple HDA issue. I am presuming that the real problem is that Max Sectors is set to 18..... which is way too large, although I don't understand why JMFS sees that value and HDAT2 does not appear to. The 3907029168 number seems correct because if I multiply it by 512 bytes/sector I am right at 2Tbytes.
> 
> My presumption here is that the Max Sectors value is wrong and needs to be corrected. Does this seem correct and, if so, how can it be done?
> 
> As always... many thanks for your input. It is greatly appreciated.


Did we ever establish if you're using a GigaByte brand motherboard or not?


----------



## jmbach

One other thought. There are newer hdparm versions available. Wondering if you can get one of the latest linux distro on a USB key and run hdparm from there. Might be able to make the changes. I am guessing that JMFS is only seeing the first 10 numbers of the max sector number (1844674407) which is close to 1TB (1950866432) and therfore nothing to expand. So maybe using a newer hdparm version you could reset it or possibly using HDAT2 and going through the reset process eventhough it states everything is fine. My only other idea would be to copy block 0 and 1 of the good drive to the "bad" drive. When you copied the 2TB to the 1.5TB drive with JMFS you could not expand it. Presumably because the the same sector reporting issue. Would of been interesting to see hdparm information on the 1.5TB after you copied it to see if it actually was the case. When you copied the good 1TB to the 1.5TB you were able to expand it. So whatever happened is not permanent. Presumably if you copy the 1TB to 2TB with JMFS you could expand it. This is assuming what I have already said is correct. Experimenting is going to help. Wish there was some way to flash copy instead of taking hours.


----------



## Oregonian

" If I am seeing this straight the max sectors is greater than the native sectors of the drive."
Yes... it is greater. More significantly though, is the fact that it is 20 digits long vs. 10 digits for the correct number of sectors.

"Did we ever establish if you're using a GigaByte brand motherboard or not?"
Never discussed that here, but it is a Biostar MCP6P M2+.
I have another test bench computer here with an ASUS board in it. I may try it for some of the next tests.

In order to be more cautious (and to not disrupt the working Tivo), I'm going to re-copy the 1T drive to the 1.5T drive using the duplicator. Before and after the copy I am going to see what hdparm -N tells me. I'm betting that the 1T (original Tivo drive) will have proper numbers. With this process I will have a 1.5T drive on which I don't have to worry about trashing data.

I like the idea of copying blocks 0 and 1 to the new drive, though I'm not sure how to accomplish that. I have Windows-based utilities that can do that, but I've seen many warnings about accessing a TiVo Premiere drive under Windows. In any case, if I use the 1.5T copy, I won't waste anything except time.

"When you copied the good 1TB to the 1.5TB you were able to expand it."
I may not have been clear on that one. I used JMFS to copy the 1T drive to the 1.5T drive and all went well.

"Presumably if you copy the 1TB to 2TB with JMFS you could expand it"
I expect that you are correct. This is what I should have done originally instead of using the disk duplicator. If it weren't for the fact that the 2T drive has been in service for over a month now (and has lots of new recordings), I would just erase it and recopy from the 1T using JMFS as I should have in the first place!

I'm going to recopy the 1T to the 1.5T with the disk duplicator, see if the problem is recreated, then try the suggestions above (different hdparm, HDAT2 FAQ, block copying) and see what I find.


----------



## unitron

Oregonian said:


> " If I am seeing this straight the max sectors is greater than the native sectors of the drive."
> Yes... it is greater. More significantly though, is the fact that it is 20 digits long vs. 10 digits for the correct number of sectors.
> 
> "Did we ever establish if you're using a GigaByte brand motherboard or not?"
> Never discussed that here, but it is a Biostar MCP6P M2+.
> I have another test bench computer here with an ASUS board in it. I may try it for some of the next tests.
> 
> In order to be more cautious (and to not disrupt the working Tivo), I'm going to re-copy the 1T drive to the 1.5T drive using the duplicator. Before and after the copy I am going to see what hdparm -N tells me. I'm betting that the 1T (original Tivo drive) will have proper numbers. With this process I will have a 1.5T drive on which I don't have to worry about trashing data.
> 
> I like the idea of copying blocks 0 and 1 to the new drive, though I'm not sure how to accomplish that. I have Windows-based utilities that can do that, but I've seen many warnings about accessing a TiVo Premiere drive under Windows. In any case, if I use the 1.5T copy, I won't waste anything except time.
> 
> "When you copied the good 1TB to the 1.5TB you were able to expand it."
> I may not have been clear on that one. I used JMFS to copy the 1T drive to the 1.5T drive and all went well.
> 
> "Presumably if you copy the 1TB to 2TB with JMFS you could expand it"
> I expect that you are correct. This is what I should have done originally instead of using the disk duplicator. If it weren't for the fact that the 2T drive has been in service for over a month now (and has lots of new recordings), I would just erase it and recopy from the 1T using JMFS as I should have in the first place!
> 
> I'm going to recopy the 1T to the 1.5T with the disk duplicator, see if the problem is recreated, then try the suggestions above (different hdparm, HDAT2 FAQ, block copying) and see what I find.


Before you do anything tell me about this disk duplicator so that I might google it and find out exactly how it does what it does, and whether that's preferable to running

dd

or one of its descendants.

The 1.5 is where you have all the old recordings and a month or so of newer ones as well, correct?


----------



## unitron

Apparently if a drive has an HPA (intentionally), then

hdparm -N 

returns something like 

max sectors = 586070255/586072368, HPA is enabled

so that the 

xxx/yyy

part has the 

xxx 

as what the drive reports to OS'es and such, and the 

yyy

part as what it really is.

So for some reason your 2TB is reporting what it really is correctly, but the "here's the lie we tell the OS" part is way screwy.

If I don't pass out I'll try to run 

hdparm -N

on a 20EURS I haven't quite finished installing and see what I get.


----------



## jmbach

Maybe we are over thinking this whole thing. Even though it is not a Gigabyte motherboard, some Googling indicates other motherboards may have problems as well. May be when you are doing this hook a "scrap" drive up to the first Sata port. If the HPA information looks similar with max sectors > native sectors we have our answer. 

Sent from my SPH-L710 using Tapatalk 2


----------



## jmbach

Also I would disable any RAID settings in the BIOS. 

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

No RAID settings in the BIOS.

I looked at my original 1T hard drive in the same system, same cable, same JMFS disk and it does not show the symptom. That is, hdparm -N shows the same number of sectors before and after the /.

I now have the 1.5T as a copy of the original 1T with the same symptom (huge number before the / in hdparm -N). I tried to Expand it with JMFS and it made it to 1.36T, but gave an error and won't use the whole drive. The good news is that this means that I have a drive I can mess with with no fear of losing data.

I tried the 'Restore' command in HDAT2 and got an error:
Aborted Command
Vendor Specific: 0002h Device is now in Security Locked Mode
Word Location: 00h Invalid Data Structure revision
Byte Location: 0000...00b

The 1T drive is my original TiVo drive that was removed from the system over a month ago.
The 1.5T drive is a copy of the 1T drive, so it doesn't have current data.
The 2T drive is in the TiVo and has current data.

I booted Knoppix 6.7.1 (I know, not the latest, but I had the CD already), ran the Terminal Emulator and ran:
sudo hdparm -N /dev/sdb
and got
Max Sectors = 2930277168/2930277168 HPA is disabled

This was with the 1.5T drive. It moved to /sdb because there is now another drive in the system.

I rebooted to the JMFS disk and ran:
hdparm -N /dev/sdb
and got
Max sectors = 18446744073321613488/2930277168 (18446744073321613488?), HPA setting seems invalid

So... it seems to be an issue with the HDPARM command on the JMFS disk.

Just to rule out hardware issues, I did the same test with JMFS and the 1.5T drive in a system with an Intel D945GCCR motherboard. HDPARM reports the same results on this board as on the Biostar motherboard. I didn't test with Knoppix on the Intel, but I am assuming that it will give the correct results.

The Biostar motherboard DOES support RAID, but I have that disabled.

So.... could I move the hdparm command from Knoppix to JMFS? Do you think that might do the trick? Is there some way I can run JMFS from within Knoppix?


----------



## jmbach

Maybe compiling jmfs from source on newer Linux kernel. I guess the other thing would be to copy the 1TB to 1.5TB via jmfs and see if we still get the same HPA problem. If not then the duplicator I'd doing something. Might still be a block 0 issue. Here is a link that has some HPA information http://blog.atola.com/restoring-factory-hard-drive-capacity/ which might be of some help.

Also it blows my idea that it was reading the first 10 digits for drive size. Because if it did then it would not have expanded at all. 
Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

More info.....

On the Biostar motherboard I ran JMFS hdparm -N on a 1.5T hard drive that has never known Linux or TiVo at all. I get similar bad results as with the other 1.5T drive.

I downloaded a new copy of JMFS 1.0.4 and burned it to a CD and got the same results.

(more to follow shortly)


----------



## jmbach

The other interesting thing is that the 20 digit number looks the same for both the 1.5TB and 2TB drive. Curious. 

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

I tried JMFS hdparm on a 120G drive on the Intel motherboard and got:
Max Sectors: 234441648/16337840(234441648?) HPA setting seems invalid (buggy kernel device driver?)

The 234441648 is likely correct for the 120G drive. It is not at all clear to me why it is reporting the second number (implying an 8G drive size).

I'm thinking that this has nothing to do with my disk duplicator as it was never used with the 120G drive or the second 1.5T drive. This looks more like an issue with JMFS hdparm not reading drives correctly in my two computers.

I must correct an earlier statement that I made. I said that I had successfully re-copied the 1T to the 1.5T using JMFS. To be honest, at this point I'm not sure it was successful. I'm going to start a copy tonight with JMFS to see what happens. Considering that it sees the size incorrectly right off the bat (even with the drive that has never seen Linux or TiVo), I'm not optimistic.


----------



## jmbach

Oregonian said:


> More info.....
> 
> On the Biostar motherboard I ran JMFS hdparm -N on a 1.5T hard drive that has never known Linux or TiVo at all. I get similar bad results as with the other 1.5T drive.
> 
> I downloaded a new copy of JMFS 1.0.4 and burned it to a CD and got the same results.
> 
> I tried JMFS hdparm on an 80G drive on the Intel motherboard and got:
> Max Sectors: 234441648/16337840(234441648?) HPA setting seems invalid (buggy kernel device driver?)
> 
> (The 16337840 is likely correct for the 80G drive. The 23... number is more than 10x larger!)
> I'm thinking that this has nothing to do with my disk duplicator as it was never used with the 80G drive or the second 1.5T drive. This looks more like an issue with JMFS hdparm not reading drives correctly in my two computers.


I guess I would have to agree. The only hesitation I have is that I would have expected your original 1TB drive to give similar results but it does not. I also doubt that advanced format is a problem as your 80GB drive is probably not advanced format. Might want to check to be sure and not assume.

I'll see if I can compile jmfs with a newer kernel and utilities. Got to find my Linux computer first and update it.

Only other thing would be the block 0 copy. iBored is a fairly easy utility that works under Windows. It opens drives up as read only and you have to ok to make a drive writable. The only issue I have with it is figuring out which drive is which. It gives you drive size and not the name in its choice menu. Only a problem when I have several drives of the same size attached to my computer. You can copy block 0 to a file and then remove your good drive and attach the 1.5TB drive and copy the files to block 0. I would use the program with a non essential drive first to get used to it. There are other similar utilities bit I find this the easier to use especially for this purpose. I use other ones to scan through large drives as they do the job quicker. I have used iBored to successfully copy an expanded 2TB back to a 1TB drive so I can have a working back up. But as unitron stated we could be entering dangerous waters here.

Sent from my SPH-L710 using Tapatalk 2


----------



## unitron

Oregonian said:


> I tried JMFS hdparm on a 120G drive on the Intel motherboard and got:
> Max Sectors: 234441648/16337840(234441648?) HPA setting seems invalid (buggy kernel device driver?)
> 
> The 234441648 is likely correct for the 120G drive. It is not at all clear to me why it is reporting the second number (implying an 8G drive size).
> 
> I'm thinking that this has nothing to do with my disk duplicator as it was never used with the 120G drive or the second 1.5T drive. This looks more like an issue with JMFS hdparm not reading drives correctly in my two computers.
> 
> I must correct an earlier statement that I made. I said that I had successfully re-copied the 1T to the 1.5T using JMFS. To be honest, at this point I'm not sure it was successful. I'm going to start a copy tonight with JMFS to see what happens. Considering that it sees the size incorrectly right off the bat (even with the drive that has never seen Linux or TiVo), I'm not optimistic.


Since I'm sick and you're not, do my experiment for me.

Burn yourself a copy of the MFS Live cd v1.4 (mfslive.org) and see if the version of hdparm on it returns the same goofy output.


----------



## jmbach

Following the main JMFS thread, MapleLeaf had the same HPA issue. He just did the copy and expand anyway (albeit very paranoid that something would go wrong) with JMFS and was successful. So I am no longer sure that the HPA problem we see is a true issue but more of a red herring. Forget the Block 0 stuff for now. I am working on a live CD with JMFS with an updated Linux kernel. That may work. So lets go back to basics. When you ran MFSLayout on your 2TB drive, how many partitions were present? You could use the MFSLive CD and use pdisk to get a similar result. There should should be 14 partitions with the last labeled " Ext2 SQLite" partition. Any partition after that may be giving you your problems. Would be nice if you could post the results. Sometimes it is easier to take a picture with your phone and use that.


----------



## retiredqwest

1. Have a HPFS or FAT disk attached together with your Tivo disk. Harddrive or USB.
2. Boot jmfs
3. Exit to shell (x)
4. Run fdisk -l. Check the HPFS/FAT partition names, it has to have a number after disk name, like:


Code:


   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1      194522  1562497933+  HPFS

5. Create directory: mkdir mydisk
6. Mount the partition on which you want to copy the log (akin to "assign drive letter" in Windows). If it is HPFS, then you need to specify the driver to use (the default one is read-only). No need to do it for FAT. For HPFS: mount -t ntfs-3g /dev/<partiton name> mydisk (for example: "mount -t ntfs-3g /dev/sda1 mydisk")
For FAT: mount /dev/<partiton name> mydisk (for example: "mount /dev/sda1 mydisk")
6. Run "/root/mfslayout.sh /dev/sda1" and see if you get an output on the screen. If so, then run "/root/mfslayout.sh /dev/sda1 > 1tb.log" This pipes the screen output to that file. Then run "cp 1tb.log mydisk/" (it will copy the log to the root directory of the given partition). 
7. Unmount partition (just a precaution, so all the buffers are flushed and file system is clean): umount mydisk

This is the partition map from a stock 320G TP, I don't have a 1TB TPXL map. If you want to post a copy that would be nice.

1 : start= 1, size= 63 ( 31.50K), type='Apple_partition_map', name='Apple'
13: start= 64, size= 343828320 (163.95G), type='MFS' , name='MFS media region 2'
2 : start= 343828384, size= 1 (512.00b), type='Image' , name='Bootstrap 1'
3 : start= 343828385, size= 16384 ( 8.00M), type='Image' , name='Kernel 1'
4 : start= 343844769, size= 524288 (256.00M), type='Ext2' , name='Root 1'
5 : start= 344369057, size= 1 (512.00b), type='Image' , name='Bootstrap 2'
6 : start= 344369058, size= 16384 ( 8.00M), type='Image' , name='Kernel 2'
7 : start= 344385442, size= 524288 (256.00M), type='Ext2' , name='Root 2'
8 : start= 344909730, size= 262144 (128.00M), type='Swap' , name='Linux swap'
9 : start= 345171874, size= 1048576 (512.00M), type='Ext2' , name='/var'
14: start= 346220450, size= 6291456 ( 3.00G), type='Ext2' , name='SQLite'
10: start= 352511906, size= 1638400 (800.00M), type='MFS' , name='MFS application region'
12: start= 354150306, size= 1638400 (800.00M), type='MFS' , name='MFS application region 2'
11: start= 355788706, size= 269353742 (128.44G), type='MFS' , name='MFS media region'

ALL S4 have the same partition map and number of partitions - 14...... just different numbers in some of the partitions because of different HDD sizes.


----------



## Oregonian

The 1T to 1.5T (second 1.5T drive, never seen Linux or TiVo before) copy (with JMFS) succeeded overnight and expansion was successful.

The 1.5T drive still has the weird info with hdparm, but I can ignore that if the rest works.

The Disk Duplicator is a StarTech.com SATDOCK22RE:
http://www.startech.com/HDD/Duplicators/SATA-Hard-Drive-HDD-Duplicator-Dock-eSATA-USB~SATDOCK22RE

I am going to try using it to copy the original 1T to the second 1.5T (the one that worked with JMFS) and see what happens. I'm wondering if there was some odd problem that happened in the original copy process.

I suspect that the odd result with hdparm (under JMFS) is a red herring. The process worked correctly when JMFS was used to copy the drive, despite the odd parameters.


----------



## Oregonian

@jmbach: The post you quoted about my 80G was incorrect, which is why I edited it. I suspect that my editing was after you quoted it. I was mistaken about the size of the drive; it is actually 120G. Otherwise, I think the comments are appropriate.

I'm running another disk duplication now (original 1T to newer 1.5T) just to see if this was a one-off event.

As soon as my Intel box is done with an unrelated task (it's actually work related!), I'll get some dumps of the mfslayout on different drives and will also run hdparm from MFSLive and see what that gives.

@retiredqwest: thanks for the details on outputting to a drive with which I can actually work. That will get the real data posted here.


----------



## unitron

Oregonian said:


> I tried JMFS hdparm on a 120G drive on the Intel motherboard and got:
> Max Sectors: 234441648/16337840(234441648?) HPA setting seems invalid (buggy kernel device driver?)
> 
> The 234441648 is likely correct for the 120G drive. It is not at all clear to me why it is reporting the second number (implying an 8G drive size).
> 
> I'm thinking that this has nothing to do with my disk duplicator as it was never used with the 120G drive or the second 1.5T drive. This looks more like an issue with JMFS hdparm not reading drives correctly in my two computers.
> 
> I must correct an earlier statement that I made. I said that I had successfully re-copied the 1T to the 1.5T using JMFS. To be honest, at this point I'm not sure it was successful. I'm going to start a copy tonight with JMFS to see what happens. Considering that it sees the size incorrectly right off the bat (even with the drive that has never seen Linux or TiVo), I'm not optimistic.


You could use WinMFS to copy the 1TB to the 1.5 and let it expand into all of it, which should give you only 15 partitions with no Apple Free on the end, and then use jmfs to copy that to the 2TB and expand into the rest with a single 16th MFS Media partition.


----------



## Oregonian

I took the original 1T TiVo drive and used my disk duplicator to copy it to the second 1.5T drive. I then ran JMFS on it (on the Intel system) and tried to expand it. The first time I ran it, no error was reported, BUT it only used 1T total of the drive.

I ran mfslayout and fdisk and outputted the results to the two files that are attached.

I edited the files by adding Return at the end of what seemed the appropriate end of each line.

Please note that the mfslayout output shows 500G of Unallocated space. This tells me that it knows that the drive is 1.5T in size, but it won't use the last 500G.

Note also the 15th partition, 64.68M in size, type='Apple_Free'. Could this be the problem? If so, how do I delete it? Keep in mind that the data on the 1.5T is not live and I can mess with it as I wish (short of ruining the physical drive, that is).


----------



## Oregonian

I ran mfslayout on the original 1T TiVo drive and have attached the edited output here.

I notice that it has the 'Apple_Free' partition #15.

I will try JMFS to copy the 1T to the 1.5T again and see if that partition is copied.


----------



## Oregonian

@unitron:
I don't doubt (now) that I could use JMFS to copy my original 1T (out of service for a month or so) to the 2T (that is in use now) and successfully expand it. The problem is that I want to keep what is presently on the 2T.

Is there an easy way to delete the 'Apple_Free' partition? Do you think that this would resolve the issue on the 2T drive?


----------



## unitron

That Apple Free is what's buggering things up, but I'm going to have to go back over this whole thread to see where it came from.

Tomorrow.

When I can stop coughing for more than a minute or two at a time.

I hope.


----------



## jmbach

You can use the MFS Live CD and use pdisk to delete / manipulate partitions on a TiVo disk. 

And I do believe that is your problem. The apple free partition is blocking expansion. JMFS is probably not copying the the apple free partition allowing it to expand. 

So if you delete the apple free partition then you could use JMFS to expand. 

Sent from my SPH-L710 using Tapatalk 2


----------



## unitron

jmbach said:


> You can use the MFS Live CD and use pdisk to delete / manipulate partitions on a TiVo disk.
> 
> And I do believe that is your problem. The apple free partition is blocking expansion. JMFS is probably not copying the the apple free partition allowing it to expand.
> 
> So if you delete the apple free partition then you could use JMFS to expand.
> 
> Sent from my SPH-L710 using Tapatalk 2


Glad I took one more look before quiting for the night.

I'm not so certain I'd want to jump right into using pdisk on a Premiere drive.

First we need to find out if it's standard for the XL to have an Apple Free partition at the end, it seems rather sloppy of TiVo to do it that way.

Maybe we could persuade comer to comment on how to go about partition editing on S4 drives safely.

Something about this whole thing seems screwy.


----------



## jmbach

I do not have an apple partition on my 1TB TPXL. Also I have seen the 2TB TP4XL and it does not have one either. 

And I agree it seems screwy. 

I have used pdisk to edit partitions on a TPXL. 

Sent from my SPH-L710 using Tapatalk 2


----------



## unitron

jmbach said:


> I do not have an apple partition on my 1TB TPXL. Also I have seen the 2TB TP4XL and it does not have one either.
> 
> And I agree it seems screwy.
> 
> Sent from my SPH-L710 using Tapatalk 2


You guys aren't gonna let off long enough to let me get any sleep, are you?

Any chance you could post the partition map of that TCD748000?


----------



## jmbach

Good night 

Sent from my SPH-L710 using Tapatalk 2


----------



## jmbach

Here is the MFSLayout information for my TPXL 1TB drive.

1 : start= 1, size= 63 ( 31.50K), type='Apple_partition_map', name='Apple'
13: start= 64, size=1074438805 (512.33G), type='MFS' , name='MFS media region 2'
2 : start=1074438869, size= 1 (512.00b), type='Image' , name='Bootstrap 1'
3 : start=1074438870, size= 16384 ( 8.00M), type='Image' , name='Kernel 1'
4 : start=1074455254, size= 524288 (256.00M), type='Ext2' , name='Root 1'
5 : start=1074979542, size= 1 (512.00b), type='Image' , name='Bootstrap 2'
6 : start=1074979543, size= 16384 ( 8.00M), type='Image' , name='Kernel 2'
7 : start=1074995927, size= 524288 (256.00M), type='Ext2' , name='Root 2'
8 : start=1075520215, size= 262144 (128.00M), type='Swap' , name='Linux swap'
9 : start=1075782359, size= 1048576 (512.00M), type='Ext2' , name='/var'
14: start=1076830935, size= 6291456 ( 3.00G), type='Ext2' , name='SQLite'
10: start=1083122391, size= 1638400 (800.00M), type='MFS' , name='MFS application region'
12: start=1084760791, size= 1638400 (800.00M), type='MFS' , name='MFS application region 2'
11: start=1086399191, size= 867125977 (413.48G), type='MFS' , name='MFS media region'

The attached file has the complete MFSLayout information.
View attachment TPXL1TB.txt


----------



## jmbach

The other method to erase the partition is to use the iBored method described in the other forum doing the S3 upgrade to 2TB. It would be modified because you are deleting partition 15 and not 16.

Either way once that Apple Free partition is erased you can expand the drive. 

Also if you noticed the only difference between the 320GB TP and 1TB TPXL drive is the MFS media partitions are larger. In a TP4XL 2TB drive the bootloader partitions are larger as well as the MFS media partitions. In all the drives the non media partitions are placed in the center of the drive. 

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

Wonderful input!

The 1tb_edited.txt file that I posted earlier is from my 1TB hard drive that came installed in my Premiere XL and was used for months. Unless JTMS added something to it (highly unlikely, IMO), the partitions are as it was received from TiVo. Actually... that's not exactly true. The system came from Weaknees and the drive was reformatted once by them under warranty. I just remembered that fact and it may be highly relevant.

jmbach's layout doesn't have the partition 15 (Apple_Free).

I have a copy of (my wife's) 320M Premiere drive that I've not done anything with. I'll take a look at it when the Intel computer is available to see if it has the Apple_Free partition on it.

Since the 1.5T drive is just for testing, I'm quite happy to try deleting the Apple_Free partition. I'll look up the iBored method.

Another interesting point is that when I copied the 2T drive to the 1.5T drive using JMFS, I beleive that the Apple_Free partition remained, though I've not confirmed that. The copy DID generate an error as the destination was smaller, so that may have been the issue.

I'm going to suggest the following (uncertain) conclusions:

1) The issue with hdparm -N was a red herring. Probably a good idea for JMFS to update this utility (or whatever it takes), but not the issue here.
2) A standard TiVo drive does not have the Apple_Free partition, but it got added by Weaknees (more speculation than anything else here)
3) When JMFS copies a drive, it does NOT copy the Apple_Free partition, as long as it completes the copy successfully. (I will confirm this tomorrow morning once the 1T to 1.5T JMFS copy is done)
4) Deleting the Apple_Free partition will allow the drive to be expanded (once I figure out how to do this, I can test this conclusion)

I'm betting that if I had another WD20EURS drive I could resolve this by using JMFS to copy from my current 2T drive to the new one and then expand it. I have quite a few drives laying around, but 2T is a bit out of my normal range.


----------



## Oregonian

Just another note to add here....

I found this thread:
http://www.tivocommunity.com/tivo-vb/showthread.php?t=496554&highlight=ibored
that sure implies that Weaknees is adding (intentionally or otherwise) the Apple_Free partition.


----------



## Oregonian

More info.....

It took about 5 hours to copy my original 1T to the 1.5T using JMFS.

I ran mfslayout and have attached the output as 15T_AfterCopy_edited.text.
Note that it DOES have the Apple_Free partition.

Then I ran Expand on the drive. I ran mfslayout and have attached the output as 15T_AfterExpand_edited.txt

I've just disproved some of my tentative conclusions. The Apple_Free partition doesn't seem to be an obstacle in this case.

I'm not seeing any significant differences between what I posted as 15tb_edited.txt (the 1.5T drive copied with the duplicator that wouldn't expand beyond 1T) and 15T_AfterCopy_edited.txt (the 1.5T drive copied with JMFS, before Expand, but which subsequently Expanded without problem).


----------



## Oregonian

Last post (for the night).....

I am using the duplicator to copy the 1T (original) to one of the 1.5T drives. I'm presuming that it will NOT Expand afterwards. If this is the case, I'll try deleting the Apple_Free partition with ibored and see if that resolves it. I'm really uncertain on this one given that JMFS was able to deal with the Apple_Free partition on the last test.

I grabbed the 320G drive that I made as a copy of the Premiere (non-XT) TiVo that I bought directly from TiVo. I've attached the mfslayout output as 320G_original_edited.txt. I note that it does NOT have the Apple_Free partition.


----------



## jmbach

Oregonian said:


> Just another note to add here....
> 
> I found this thread:
> http://www.tivocommunity.com/tivo-vb/showthread.php?t=496554&highlight=ibored
> that sure implies that Weaknees is adding (intentionally or otherwise) the Apple_Free partition.


That does seem to be the case. No one knows why it is there and as far as I know no one has asked or got an answer from weaknees. It is assumed it is a byproduct of their copy process.

Sent from my SPH-L710 using Tapatalk 2


----------



## jmbach

Oregonian said:


> More info.....
> 
> It took about 5 hours to copy my original 1T to the 1.5T using JMFS.
> 
> I ran mfslayout and have attached the output as 15T_AfterCopy_edited.text.
> Note that it DOES have the Apple_Free partition.
> 
> Then I ran Expand on the drive. I ran mfslayout and have attached the output as 15T_AfterExpand_edited.txt
> 
> I've just disproved some of my tentative conclusions. The Apple_Free partition doesn't seem to be an obstacle in this case.
> 
> I'm not seeing any significant differences between what I posted as 15tb_edited.txt (the 1.5T drive copied with the duplicator that wouldn't expand beyond 1T) and 15T_AfterCopy_edited.txt (the 1.5T drive copied with JMFS, before Expand, but which subsequently Expanded without problem).


Interesting, your final 1.5TB drive has 16 partitions. I was reading some forum discussions when comer was developing JMFS and when there was 16 partitions TP would delete the 16th partition. Could you boot that drive once in your Tivo and see if you still have more recording space as anticipated. Then recheck it with MFSlayout to see if the partition structure has changed.

Also I am wondering what sequence you used to expand the drive. Was the original drive still plugged in when you expanded the copy? To look at the partition table you had to exit the guided setup so did you do the expand manualIy or automated through the guided setup. I have not looked deep into the source code if it handles two tivo drives being hooked up different than just one being hooked up when expanding or if the guided setup does things differently than manually expanding.

Sent from my SPH-L710 using Tapatalk 2


----------



## retiredqwest

jmbach said:


> Interesting, your final 1.5TB drive has 16 partitions. I was reading some forum discussions when comer was developing JMFS and when there was 16 partitions TP would delete the 16th partition. Could you boot that drive once in your Tivo and see if you still have more recording space as anticipated. Then recheck it with MFSlayout to see if the partition structure has changed.


That would be over 16 partitions. Over 16 the TIVO sees that as an external drive and will ask to divorce it.

Only to prove a point....Here's the partition map for a THD upgraded by Weaknees:

Partition Maps
#: type name length base ( size )
1 Apple_partition_map Apple [email protected] ( 31.5K)
2 Image Bootstrap 1 [email protected] ( 512.0 )
3 Image Kernel 1 [email protected] ( 4.0M)
4 Ext2 Root 1 [email protected] ( 256.0M)
5 Image Bootstrap 2 [email protected] ( 512.0 )
6 Image Kernel 2 [email protected] ( 4.0M)
7 Ext2 Root 2 [email protected] ( 256.0M)
8 Swap Linux swap [email protected] ( 127.0M)
9 Ext2 /var [email protected] ( 256.0M)
10 MFS MFS application region [email protected] ( 288.0M)
11 MFS MFS media region [email protected] ( 65.6G)
12 MFS Second MFS application region [email protected] ( 288.0M)
13 MFS Second MFS media region [email protected] ( 82.0G)
14 MFS New MFS Application [email protected] ( 512.0K)
15 MFS New MFS Media [email protected] ( 782.5G)
16 Apple_Free Extra [email protected]( 11.2M)

Total SA SD Hours: 1040 Total DTV SD Hours: 908 0 % Free
Software: 11.0k-01-2-652 Tivo Model: TCD652160

The stock THD has 13 partitions, Weaknees and WINmfs always add 2 partitions when they expand a drive. In the above, deleting partition 16 would allow JMFS to copy and expand that drive.

So Oregonian, I think you have figured you don't really have a stock TPXL. Like was said no one knows why Weaknees adds the apple free partition. Nor will they admit doing so.....

And I seriously doubt deleting the apple free will allow JMFS to expand the 2TB.


----------



## Oregonian

@jmbach:
In all cases when I have done the expand it has been with the drive to be expanded as the only HD connected and used the guided setup. That is, I booted JMFS, eXited to a prompt, mounted the USB drive, ran mfslayout redirected to the USB, rebooted, then tried Expand through the JMFS menu system.


----------



## unitron

Here's the deal:

Before the S4s (Premiere) came out a few early models with small drives only had one MFS pair, but for the most part it was

10-11=first MFS pair

12-13=second MFS pair

MFS Live (and the old MFS Tools on which it was based) and WinMFS would both "upgrade" by copying the original drive to a bigger one and then filling the extra space with a 3rd MFS pair.

Total 15 partitions (out of the maximum allowable 16--that's not Apple Partition Map imposed, that's TiVo imposed).

WinMFS was better at being able to do that without leaving any unused space.

Unused space generally gets labeled an Apple Free partition.


Along came Premieres, with that extra SQLite partition and who knows what other changes, and the old software couldn't handle the S4 drives.

comer came up with something I still don't understand having to do with coalescing partitions--turning an MFS pair into a single MFS Media partition, or at least I think that's what it is--and created jmfs.

Here's how jmfs works:

It uses 

ddrescue

to do a byte for byte copy of the original drive (or whatever other drive you provide as the source drive if you can get it to recognize it as a TiVo drive) to the (supposedly larger) target drive.

Then it fills the extra space with that single MFS media partition.

If you start with a genuine installed at the factory stock TiVo Series 4 drive, then it copies 14 partitions and adds a 15th.

Some of this is more pertinent to the Series 3 HD and Series 3 HD XL, which for some reason jmfs will also work on, even though it won't recognize the original S3 as a TiVo drive.

When jmfs copies, it copies everything.

If that includes an Apple Free partition, it'll copy that as well.

ddrescue

just copies bytes, and doesn't worry about what they are--it just "Xeroxes" the source onto the target.


(what some drive duplicator docking thingie does, who knows?)


Once the copying is done, then it adds that single MFS Media partition.

With a stock Premiere or Premiere XL drive, that will be partition 15.


I would think that even with that teeny little Apple Free partition Weaknees added to the OP's 1TB drive, that it would be able to "Xerox" it to a larger drive and add the single MFS Media partition, which would be partition 16, and it should still work, since you haven't exceeded the 16 partition per drive limit.

And on a non-Premiere maybe it would, but I don't know if anyone knows enough about how TiVo changed things on the S4s to know what you can and can't get away with.


Also, Weaknees used the non-optimized partition layout instead of the one the stock Premiere and Premiere XL drives use, and I don't know what, if any, difference that might make.


I'm curioius if the OP's original original 1TB drive was sent to Weaknees so they could get settings and shows and stuff off of it to put on the replacement and if that's why they didn't just "Xerox" a virgin Premiere XL drive with the "optimized" layout and no free space.


We need to know which of the OP's drives has the stuff he wants to save before we go into which drive he should be hex editing, if he should be hex editing.


==========================================================


All this stuff down here is about jmfs and the S3 HD and HD XL.

I include it to show where some concepts that popped up in this thread come from.


I do not know why, but in addition to recogizing Premiere drives as TiVo drives, according to whatever criteria comer gave jmfs to work with to make that determination, it'll also recognize the TCD652160 (S3 HD) and TCD658000 (S3 HD XL) as TiVo drives and work with them, but for reasons unkown (and frustrating) to me, it won't recognize or work with the original S3 (TCD648250) drive.


You can take an HD drive and use MFS Live or WinMFS to copy it to a larger drive and then add a third MFS pair, which would be partitions 14 and 15, but if you don't fill the entire drive for whatever reason, you'll have a 16th Apple Free partition, and if you try to use jmfs to go to an even larger drive, it'll copy all 16 partitions and then add a 17th, which breaks the only 16 partitions per drive rule.

When you boot that in the TiVo, it thinks that 17th partition is the first partition on a screwed up external drive, and makes you divorce said phantom external, which means it deletes that single MFS media partition that jmfs installed.


----------



## Oregonian

OK.. more data.. and more confusion!

I took my original (Weaknees) 1T drive and used my hardware disk duplicator to copy it to a 1.5T drive. I've attached an output from mfslayout as 15t_after_hardware_copy_edited.txt

I then rebooted JMFS and ran the Expand function from its menu. It said that it succeeded at expanding it to 1.36G. I've attached an output from mfslayout as 15t_hardware_copy_expanded_edited.txt

I did not expect the expand to work as this is the same sort of procedure that I used with my original 2T drive (that is now working but with only 1T capacity). I also note that there is an Apple_Free partition on the drive that didn't seem to get in the way.

So..... this doesn't seem to be an issue specifically with my disk duplicator. I'm wondering if something odd happened when I originally copied the 1T to the 2T and that has been my problem all along.

My next step will be to grab the 2T out of the working TiVo, run mfslayout on it, and copy it (using JMFS) to one of the 1.5T drives. I'm assuming that this will give me a drive that will not expand that I can mess with.


----------



## unitron

I fugured out where that extra space came from on the OP's Weaknees drive.

They made the swap partition smaller!?! 

They cut it in half! 

We can argue about whether the swap partition needs to be bigger than the size TiVo chose, but this is the first I've ever seen of somebody thinking they made it too large.

And they basically just threw away the space.

Maybe it's got to do with some weird what partition boundary falls on what cylinder boundary alignment voodoo or something, but I don't get it.


----------



## unitron

Oregonian said:


> OK.. more data.. and more confusion!
> 
> I took my original (Weaknees) 1T drive and used my hardware disk duplicator to copy it to a 1.5T drive. I've attached an output from mfslayout as 15t_after_hardware_copy_edited.txt
> 
> I then rebooted JMFS and ran the Expand function from its menu. It said that it succeeded at expanding it to 1.36G. I've attached an output from mfslayout as 15t_hardware_copy_expanded_edited.txt
> 
> I did not expect the expand to work as this is the same sort of procedure that I used with my original 2T drive (that is now working but with only 1T capacity). I also note that there is an Apple_Free partition on the drive that didn't seem to get in the way.
> 
> So..... this doesn't seem to be an issue specifically with my disk duplicator. I'm wondering if something odd happened when I originally copied the 1T to the 2T and that has been my problem all along.
> 
> My next step will be to grab the 2T out of the working TiVo, run mfslayout on it, and copy it (using JMFS) to one of the 1.5T drives. I'm assuming that this will give me a drive that will not expand that I can mess with.


Does the 2TB out of the working TiVo have fewer than 2TB worth of partitions on it?

Otherwise, how does all of it get onto a smaller drive?

jmfs's copy method won't adjust anything and it'll copy over a bootpage and partition map that says "I'm a 2TB drive", unless what's on the 2TB was "Xeroxed" from a smaller drive in the first place and it was never expanded.


----------



## Oregonian

@unitron



> I'm curioius if the OP's original original 1TB drive was sent to Weaknees so they could get settings and shows and stuff off of it to put on the replacement and if that's why they didn't just "Xerox" a virgin Premiere XL drive with the "optimized" layout and no free space.
> 
> We need to know which of the OP's drives has the stuff he wants to save before we go into which drive he should be hex editing, if he should be hex editing.


The XL was purchased new from Weaknees. After a while, it kept rebooting. I pulled the drive and ran the Western Digital factory tests on it and it failed. I tried to talk Weaknees in letting me just send the drive back, but they wanted the whole system. I sent it, they wiped the drive, reinstalled the TiVo software, and sent it back to me. There was none of my data on it when it was returned. My presumption is that they used some sort of stock image or drive to copy to it.

I am also wondering if Weaknees' 1T image is not that of a TiVo 1T drive but of a 320G drive that has been expanded.

I have four drives that I have been messing with:

The 1T drive from Weaknees that works properly but has been out of use for over a month

Two 1.5T drives that are being used for this testing. They can be (and have been) erased, copied, edited, or whatever.

The 2T drive that I installed in my XL that has the problem of not being able to be expanded. I originally copied it with my disk duplicator, which I have assumed created the problem, but I am not at all convinced of that now (because of last night/today's experience with the 1T to 1.5T copying).

The goal here is to fix whatever it is on the 2T drive that is preventing expanding it, expand it, preserve the data on it, and continue to use it.


----------



## Oregonian

@unitron:



> Does the 2TB out of the working TiVo have fewer than 2TB worth of partitions on it?
> 
> Otherwise, how does all of it get onto a smaller drive?
> 
> jmfs's copy method won't adjust anything and it'll copy over a bootpage and partition map that says "I'm a 2TB drive", unless what's on the 2TB was "Xeroxed" from a smaller drive in the first place and it was never expanded


I will run a mfslayout on the 2T drive today and give you an accurate answer. My recollection (not to be trusted) is that it only goes out to 1T of partitions.

If I had another of the same 2T drive I would be copying to it to do the testing. I don't. My assumption is that if I copy the 2T to a 1.5T drive it will copy the first 1.5T of the drive and then exit with an error. Given that the drive is only partitioned to 1T (to be confirmed), I hope this will work. In any case, I'm not changing the 2T drive so the worst that I do is waste my time.

The 2T drive was copied with my hardware duplicator from the 1T that I purchased in the XL from Weaknees. I have presumed that this was the source of my problems, but am no longer sure because I was able to successfully copy the 1T to a 1.5T with the duplicator and then expand it.

I attempted to expand the 2T, but it was not successful. I should be clearer on that. I can't honestly say if it passed or failed the first expansion attempt as it was my first time ever to do it and I could have missed an error message. When I reconnected it to the TiVo, I noted that it didn't have additional space on it. I put it back in the PC and tried Expand again and definitely got an error.


----------



## unitron

Oregonian said:


> @unitron
> 
> The XL was purchased new from Weaknees. After a while, it kept rebooting. I pulled the drive and ran the Western Digital factory tests on it and it failed. I tried to talk Weaknees in letting me just send the drive back, but they wanted the whole system. I sent it, they wiped the drive, reinstalled the TiVo software, and sent it back to me. There was none of my data on it when it was returned. My presumption is that they used some sort of stock image or drive to copy to it.
> 
> I am also wondering if Weaknees' 1T image is not that of a TiVo 1T drive but of a 320G drive that has been expanded.
> 
> I have four drives that I have been messing with:
> 
> The 1T drive from Weaknees that works properly but has been out of use for over a month
> 
> Two 1.5T drives that are being used for this testing. They can be (and have been) erased, copied, edited, or whatever.
> 
> The 2T drive that I installed in my XL that has the problem of not being able to be expanded. I originally copied it with my disk duplicator, which I have assumed created the problem, but I am not at all convinced of that now (because of last night/today's experience with the 1T to 1.5T copying).
> 
> The goal here is to fix whatever it is on the 2T drive that is preventing expanding it, expand it, preserve the data on it, and continue to use it.


You make it sound like Weaknees did not actually replace the hard drive with a new hard drive, but re-used the old one.

If the WD diagnostic found errors on the old one, those weren't TiVo software errors, those were hardware--this drive's no good anymore-- errors.


----------



## jmbach

Oregonian said:


> I am also wondering if Weaknees' 1T image is not that of a TiVo 1T drive but of a 320G drive that has been expanded.


It will be difficult to tell completely since all the non media partitions are the same and they used a "non optimize" partition structure. But I think not since the MFS media partitions are the size one would see in a 1TB TP drive natively. Unless Weaknees has figured out how to change size of the MFS media partitions which I doubt. If they did they could of increased the size of one of the media partitions to make up for the extra space that occurred when they decreased the swap partition.

Still would like to see if having the tivo booting off of the 1.5TB drive keeps all 16 partitions. From what I have read, it seems the reason comer used a coalased partion to expand the TP was that it would delete a partition when the tivo booted up keeping only 15 partitions. Now that was when a partition pair was being created to expand. It could be that since the 16th partition is a coalased partition the tivo would be happy. Nevertheless I would like to see what would happen.


----------



## unitron

jmbach said:


> It will be difficult to tell completely since all the non media partitions are the same and they used a "non optimize" partition structure. But I think not since the MFS media partitions are the size one would see in a 1TB TP drive natively. Unless Weaknees has figured out how to change size of the MFS media partitions which I doubt. If they did they could of increased the size of one of the media partitions to make up for the extra space that occurred when they decreased the swap partition.
> 
> Still would like to see if having the tivo booting off of the 1.5TB drive keeps all 16 partitions. From what I have read, it seems the reason comer used a coalased partion to expand the TP was that it would delete a partition when the tivo booted up keeping only 15 partitions. Now that was when a partition pair was being created to expand. It could be that since the 16th partition is a coalased partition the tivo would be happy. Nevertheless I would like to see what would happen.


WinMFS is actually able to change the size of MFS Media partitions.

But only for the various S3s and earlier.

comer's discussion of how he developed jmfs is scattered over at least 3 different web sites and who knows how many private emails, PMs, etc., to which we are not privy.

Although even if I had access to all of it, it'd probably still be above my head.

When jmfs is used on S3 HD and HD XL drives that already have 3 MFS pairs, it can add a single MFS Media partition which is the 16th partition, and it works just fine.

With the S4s, who knows?


----------



## Oregonian

> You make it sound like Weaknees did not actually replace the hard drive with a new hard drive, but re-used the old one.
> 
> If the WD diagnostic found errors on the old one, those weren't TiVo software errors, those were hardware--this drive's no good anymore-- errors.


The Weaknees tech made it clear that they were reformatting the drive and NOT replacing it. I discussed/argued with him about this, complaining that the drive failed the manufacturer's tests and should be replaced, but I lost the discussion. So... yes... I have the original drive that was reformatted. I ran it for about 8 months without problems after it came back, then did the copy to the 2T drive.


----------



## Oregonian

> When you boot that in the TiVo, it thinks that 17th partition is the first partition on a screwed up external drive, and makes you divorce said phantom external, which means it deletes that single MFS media partition that jmfs installed.


It has been too long since I did the original copy (or my memory is too flawed), but this sounds very familiar. I'd have mentioned it had I remembered it.

Somewhere along the line of installing the 2T drive I got this sort of message. That is, TiVo thought that there was an external drive missing and wanted to delete any references to it. I am not certain whether this was right after the 1T to 2T duplication (not likely) or after the first attempt at expanding the 2T drive (more likely). I allowed it to "forget" about the external drive (I've never had an external drive on the TiVo) and all seemed well afterwards.

I'm expecting that the mfslayout of the 2T will be informative. I'll try to post that today.

I will also boot the 1.5T drive and see what happens to the partitions.


----------



## unitron

Oregonian said:


> The Weaknees tech made it clear that they were reformatting the drive and NOT replacing it. I discussed/argued with him about this, complaining that the drive failed the manufacturer's tests and should be replaced, but I lost the discussion. So... yes... I have the original drive that was reformatted. I ran it for about 8 months without problems after it came back, then did the copy to the 2T drive.


Well, now I know from whom not to buy a TiVo.

It sounds like they couldn't make a claim against WD, since WD sold it OEM to TiVo, Inc., and for some reason being a TiVo authorized re-seller wouldn't help them get a replacement from TiVo, either.

How much time had passed between the initial purchase and the start of the problems?


----------



## jmbach

unitron said:


> WinMFS is actually able to change the size of MFS Media partitions.
> 
> But only for the various S3s and earlier.


Was not aware of that. Then I wonder why when expanding instead of adding a couple of partitions just increase the size of the original MFS media partitions. I guess maybe that might lose shows.



unitron said:


> comer's discussion of how he developed jmfs is scattered over at least 3 different web sites and who knows how many private emails, PMs, etc., to which we are not privy.


Yes it is. Reviewing some of the information, apparently S4 tivos remove /dev/sda16 from the superheader and left only /dev/sda15 after a pair was added in the usual fashion (app and media partition). The actual partition still existed in the table but the tivo did not have access to the media partition only the app partition part of the pair. It is unclear if TiVo showed increase recording capablitiy from my reading but the actual recording time never went beyond the original drive capacity. This is what the coalased partition solved.

This is why I would like to see what happens when the drive boots up with the 16 partitions.


----------



## ggieseke

Just to stir the pot. 

Here's a WinMFS printout of my ancient dual-drive SVR-2000.
-------------------------------------------------------
Mfsinfo (Drive A: 5, Drive B: 6)

Boot Page (Byte Swapped)
Boot Page: root=/dev/hda4 
Active Boot Partition: 3 Active Root Partition: 4
Backup Boot Partition: 6 Backup Root Partition: 7

MFS Super Header
state=0 magic=abbafeed
devlist=/dev/hda10 /dev/hda11 /dev/hda12 /dev/hda13 /dev/hdb2 /dev/hdb3
zonemap_ptr=1121 total_secs=479171584

Zone Maps
Z0:	type=0
map_start=1121 map_size=1 backup_map_start=1048574
next_map_start=263266 next_map_size=18 next_backup_map_start=1048556
zone_first=1122 zone_last=263265 zone_size=262144 min(chunk)=262144
free=262144 checksum=eeae17f9 logstamp=39638027 num_bitmap=1
Z1:	type=2
map_start=263266 map_size=18 backup_map_start=1048556
next_map_start=263284 next_map_size=66 next_backup_map_start=1048490
zone_first=1048576 zone_last=57698303 zone_size=56649728 min(chunk)=2048
free=33437696 checksum=93bfd983 logstamp=39638053 num_bitmap=16
Z2:	type=1
map_start=263284 map_size=66 backup_map_start=1048490
next_map_start=57699329 next_map_size=17 next_backup_map_start=57700334
zone_first=263350 zone_last=1048485 zone_size=785136 min(chunk)=8
free=748736 checksum=b249a240 logstamp=39638053 num_bitmap=18
Z3:	type=2
map_start=57699329 map_size=17 backup_map_start=57700334
next_map_start=239054849 next_map_size=17 next_backup_map_start=239055854
zone_first=57700352 zone_last=239054847 zone_size=181354496 min(chunk)=8192
free=68534272 checksum=207df462 logstamp=39638027 num_bitmap=16
Z4:	type=2
map_start=239054849 map_size=17 backup_map_start=239055854
next_map_start=0 next_map_size=0 next_backup_map_start=3735928559
zone_first=239055872 zone_last=479171583 zone_size=240115712 min(chunk)=8192
free=111058944 checksum=a723e86c logstamp=39638053 num_bitmap=16

Partition Maps
#: type name length base ( size )
1 Apple_partition_map Apple [email protected] ( 31.5K)
2 Image Bootstrap 1 [email protected] ( 2.0M)
3 Image Kernel 1 [email protected] ( 2.0M)
4 Ext2 Root 1 [email protected] ( 128.0M)
5 Image Bootstrap 2 [email protected] ( 2.0M)
6 Image Kernel 2 [email protected] ( 2.0M)
7 Ext2 Root 2 [email protected] ( 128.0M)
8 Swap Linux swap [email protected] ( 127.0M)
9 Ext2 /var [email protected] ( 128.0M)
10 MFS MFS application region [email protected] ( 512.0M)
11 MFS MFS media region [email protected] ( 27.0G)
12 MFS New MFS Application [email protected] ( 512.0K)
13 MFS New MFS Media [email protected] ( 86.5G)
14 Apple_Free Extra [email protected] ( 1.9M)

Tivo B drive 
1 Apple_partition_map Apple [email protected] ( 31.5K)
2 MFS New MFS Application [email protected] ( 512.0K)
3 MFS New MFS Media [email protected] ( 114.5G)
4 Apple_Free Extra [email protected] ( 2.4M)

Total SA SD Hours: 255	Total DTV SD Hours: 222 45 % Free
Software: 3.0-01-1-010	Tivo Model: not set in MFS
-------------------------------------------------------
Nothing but an early 80s PC and the original mfstools ever touched those drives, so the Apple_Free Extra concept has been around for a while.

One side note - every drive that I've seen so far seems to have the original partition table intact, and wiping the extra partitions with iBored or whatever SHOULD return it to factory normal. In the example I posted the original 30GB drive was replaced with a 60GB drive, then an 80GB drive, then upgraded to dual 120GB drives. I think that if I wiped partitions 12-14 on drive A and removed the references to /dev/hdbX in the MFS master header it would happily revert to its original config.

I'm finding a surprising amount of consistency in the MFS file system from old S1s to new Premieres, but it's hard to say anything definitive yet. The entire file system and any assumptions about it so far should be taken with a big grain of salt. Giants like Spike and the MFSLive coders figured out most of it and I'm just standing on their shoulders, but there are still gaps in the code where nobody knows what the heck is going on or tiny details are just wrong. With rotsa ruck I can add something.

Keep posting those mfslayout dumps. When you stare at screens of hex dumps for days looking for a pattern every data point is important.


----------



## unitron

Before anyone rushes off to start hex editing Series 1 drives, remember everything's upside down and backwards because of byte-swapping.


As far as I know, making unpartitioned space into a partition of the type Apple Free was part of the Apple Partition Map scheme of doing things since long before TiVo "borrowed" it.

Just like making the partition map itself a partition.


----------



## ggieseke

unitron said:


> Before anyone rushes off to start hex editing Series 1 drives, remember everything's upside down and backwards because of byte-swapping.
> 
> As far as I know, making unpartitioned space into a partition of the type Apple Free was part of the Apple Partition Map scheme of doing things since long before TiVo "borrowed" it.
> 
> Just like making the partition map itself a partition.


Yeah, the byte-swapped format is a pain. I can analyze all of the Ext2 and MFS partitions on a S2-Premiere in seconds, but testing the same code on those S1 drives takes forever because I have to "swab" every sector first.

I don't think Weaknees invented the Apple Free stuff, I just wanted to make that very point. They're just doing what everyone before them has done with the APM scheme. It probably isn't a devious plan to prevent us from expanding their drives.

FWIW, I don't think TiVos care if the entire drive is partitioned in some way or not. Copy a factory 320GB drive to a 2TB drive byte-for-byte and it could care less if the remaining space is unpartitioned. It still works, it just doesn't recognize the unused space on the drive.


----------



## jmbach

I have experimentally proved that last statement with a TPXL. 

Sent from my SPH-L710 using Tapatalk 2


----------



## retiredqwest

Using JMFS I copied the TP 320G to a 1TB, it showed 294G used. I did not expand nor supersize the 1TB.

I then copied the 1TB to a 500G. JMFS lets you know that the target drive is not large enough BUT, will allow you to do the copy anyway. Going over 300G I knew JMFS would try to copy the entire 1TB to the 500G. Finally it failed, ran mfslayout on the 500G and it showed 294G used and 170G or so unallocated. SO, I tried the expand on the 500G and it reported successful. Ran mfslayout again and it showed 0 unallocated and 470G or so used. 

I haven't tried the 500G in the TP, but I see no reason why it won't work.


----------



## jmbach

I agree. It should work. Any apple free partitions on the drive? I think I am going to try an experiment and use a 320GB image and copy it to a larger drive. Place a small apple free partition right at the end of the last partition and see if I can expand it. If it does expand see how the tivo responds. If it does not expand, remove the apple free partition and then try to expand. 

Sent from my SPH-L710 using Tapatalk 2


----------



## unitron

retiredqwest said:


> Using JMFS I copied the TP 320G to a 1TB, it showed 294G used. I did not expand nor supersize the 1TB.
> 
> I then copied the 1TB to a 500G. JMFS lets you know that the target drive is not large enough BUT, will allow you to do the copy anyway. Going over 300G I knew JMFS would try to copy the entire 1TB to the 500G. Finally it failed, ran mfslayout on the 500G and it showed 294G used and 170G or so unallocated. SO, I tried the expand on the 500G and it reported successful. Ran mfslayout again and it showed 0 unallocated and 470G or so used.
> 
> I haven't tried the 500G in the TP, but I see no reason why it won't work.


Well, 320GB decimal probably is 294GB binary.

Still don't see the attraction of copying part of a larger drive (including the soon to be inaccurate partition map) to a smaller one.


----------



## retiredqwest

jmbach said:


> I agree. It should work. Any apple free partitions on the drive? I think I am going to try an experiment and use a 320GB image and copy it to a larger drive. Place a small apple free partition right at the end of the last partition and see if I can expand it. If it does expand see how the tivo responds. If it does not expand, remove the apple free partition and then try to expand.


Apple free partition, no.....I've never bought anything from Weaknees nor ever will.

And it booted up in the TP just fine.


----------



## jmbach

retiredqwest said:


> Apple free partition, no.....
> 
> And it booted up in the TP just fine.


Excellent.

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

More info.....

I pulled the 2T drive from the TiVo XL (this is the one that I copied from the 1T a month or so ago, tried to expand later, was unsuccessful, got the message about the damaged external drive and let TiVo clean it up) and ran mfslayout on it. It is attached as 2T_edited.txt.

BTW... when I do /root/mfslayout.sh /dev/sda > 2T_edited.txt the output file doesn't have CR/LF (or whatever) at the end of each line. I'm having to manually add them. I'm happy to do that to get help here, but is there a way to avoid that?

What I find interesting about the mfslayout.sh is that it appears to have about 1T of space in the first 15 partitions, then there's partition 16, type='MFS' that is 931.50G. My assumption is that this is the partition that I'd like the TiVo to use but it won't. I'm wondering if I should delete it and try to expand again.

I compared this output to 15T_AfterExpand.txt (a copy from the 1T to the 1.5T that appeared to successfully expand) and I don't see any significant differences other than the size of the 16th partition. I'm going to fire up the 1.5T drive and see if it really works.

In the meantime, I'm going to use JMFS to copy this drive to a 1.5T drive, hoping to get the first 15 partitions intact and who knows what with the 16th. At that point I'll probably use ibored to delete whatever is present in the 16th partition and change the last partition in all of the preceding ones. If that works as I am hoping, I should be able to use JMFS to expand it.

Or.... I'm completely wrong and will have wasted more time!


----------



## Oregonian

I took that 1.5T drive that appeared to expand successfully (copied with JMFS from the 1T drive) and installed it in my TiVo and booted it. It gave me the error about a corrupted external drive and I am letting it delete it. I'll post the mfslayout when I can.


----------



## jmbach

Well that was enlightening. It looks like what you will have to do is delete the 15th and 16th partition information in the partition table and then use JMFS to expand. You can test this by copying the 2TB to your 1.5TB drive and then do the delete process on the 1.5TB followed by expand. 

It looks like JMFS has done its job to expand the drive but because the TP OS apparently only likes a certain number of partitions in the superheader, divorces the "drive" because the information located in the 15th partition is not the coalased partition (the 16th partition is but the superheader does not see it) 

Sent from my SPH-L710 using Tapatalk 2


----------



## jmbach

Also in Windows I recommend notepad++ to view the files. It will display the captured MFSlayout output correctly where as regular notepad will display it as one line. (no cr/lf in the captured output ) 

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

I let the TiVo delete the "corrupted external drive" on the 1.5T drive. It appeared to work afterwards, with a reported 157 hours of recording space. This seems about right for 1T of disk space, not for the full 1.5T.

I have attached the mfslayout as 1.5T_after_Tivo_edited.txt. It looks as if it can't get to the 465.75G MFS partition at the end. I'm also not seeing the difference between the drive before and after the TiVo "corrected" the corrupted drive.


----------



## Oregonian

I took the same 1.5T and ran pdisk from MFS and deleted partition 16. The mfslayout is attached as 15T_after_TiVo_and_Partition_Delete_edited.txt

I am going to boot this on the TiVo, see what it says, then try expanding it.


----------



## jmbach

Oregonian said:


> I took the same 1.5T and ran pdisk from MFS and deleted partition 16. The mfslayout is attached as 15T_after_TiVo_and_Partition_Delete_edited.txt
> 
> I am going to boot this on the TiVo, see what it says, then try expanding it.


You will need to delete the 15th and 16th partitions. The Apple_Free and MFS expanded partitions otherwise you will have the same problem as before.

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

I booted the 1.5T after deleting partition 16 and it ran without error, but showed 157 hours (1T worth) of recording time.

I booted MFS and ran fdisk on the 1.5T drive and have posted the results as 15fdisk_after_TiVo_and_Delete_edited.txt

I think that the significant thing here is that it thinks that /dev/sdb (the 1.5T drive) doesn't have a valid partition table. JMFS gives a similar error.

I'm trying to run ibored to edit the partition table, but my Linux shortcomings are getting in the way. I copied the folder to a USB stick and tried to run it, but I get an error starting with:

error while loading shared libraries: libgtx.....

This happens whether I start in the iBored folder or in the iBored Libs folder.

Any suggestions on how to run iBored?


----------



## Oregonian

I have tried to delete the 15th partition, Apple_Free, but it doesn't seem to go away. That is, I run the delete command (which I have figured out correctly as I was able to delete the 16th partition) and don't get errors, but the partition persists. I'll try again.


----------



## jmbach

What OS are you running iBored on. If using windows version have to run as administrator. 

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

I ran pdisk -i and redirected output to a text file. I have attached it as 15T_pdisk_edited.txt

It didn't capture my typing so I have added that in {}.

I didn't capture another l /dev/sdb, but it gives the same results as the first one. That is, the Apple_Free partition doesn't go away.

I'm suspecting that the ".. doesn't have a valid partition table" that I saw earlier is the most significant symptom.


----------



## Oregonian

I tried running the Linux version of iBored on JMFS. I have been warned online to not try to access a Premiere XL drive under Windows. Is it safe to access it with iBored under Windows XP?


----------



## Oregonian

I used the Windows version of iBored and changed the partition numbers from 15 to 14 and zeroed out the 15th block. MFSLayout output is attached as 15T_After_Pdisk_Delete_edited.txt

I ran JMFS and did Expand and it said that it was successful at expanding to 1.36T

I booted in the TiVo and it gave no errors. It indicated that I had (drum roll...) 238 hours of recording space! I did a quick run through my recordings and they appeared to be present.

So.... it appears that if I use iBored to manually remove the last two partitions, I'm good.

I will wait for the 2T to 1.5T copy to finish, do the same partition editing on that drive, confirm it works, then do the same on the 2T and I'll be done!


----------



## jmbach

Congratulations. 
The main thing when working with Windows and TiVo drives is to make sure windows does not mount the drive. If it does then it screws up block 0 and the Tivo will not boot up. 

Sent from my SPH-L710 using Tapatalk 2


----------



## jmbach

Oregonian said:


> I'm suspecting that the ".. doesn't have a valid partition table" that I saw earlier is the most significant symptom.


That is okay as fdisk will always say this for TiVo drives.

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

Latest work:
I copied the 2T (with latest data) to a 1.5T drive using JMFS. It warned me about the destination being smaller, of course. Since the 2T was only working to 1T of space, I figured this was safe.

I've attached the fdisk and mfslayout on that as 15_from_20_jmfs_fdisk_edited.txt and 15t_from_20_mfslayout_edited.txt.

I used iBored (Windows) to zero out partitions 15 and 16 and changed the last partition number on all the other partitions to 14 (checking afterwards to confirm that I didn't make any errors).

I used JMFS to expand the drive. I booted it on the TiVo and got no errors, it showed over 200 hours (I forget the exact number), and my recordings were there! I played a few minutes of one from yesterday and it appeared to play properly. I think I've got it!

So... I took the 2T drive and did the same partition editing and expanding. I have included the mfslayout on it as 2T_after_good_expand_edited.txt

I booted the 2T drive in the TiVo, no errors, it shows 317 hours of total recording time, and my recordings appear to be intact. Success!!!!!


----------



## jmbach

:thumbup:

Sent from my SPH-L710 using Tapatalk 2


----------



## Oregonian

Some conclusions (please comment if any seem incorrect, this is the important part!)

1) The hdparm -N issue is a quirk about JMFS and doesn't matter.
2) The "does not contain a valid partition" error is standard for TiVo drives and doesn't matter (thanks to jmbach for offering this)
3) Copying with my disk duplicator had nothing to do with the problem
4) The problem started with the Apple_Free partition being present on my original drive.
5) Pdisk (at least the version that I used) won't delete that partition
6) iBored for Windows WILL delete the partition and appears to be safe as long as you don't do anything else to the drive in Windows


I can't say with certainty why the Apple_Free partition was on the drive. My best guess is that it has to do with the system coming from WeakNees, possibly related to their reload of it.

It would be informative if someone could post the mfslayout for a WeakNees drive (preferably 1T) that has never been repaired by WeakNees (or been expanded or anything else unusual).

It would also be informative if anyone has seen a drive properly functioning after expanding that has an Apple_Free partition (as the second to last one). I am suspecting that this is not possible.

If my suspicions are correct about WeakNees, their drives can't be expanded with JMFS without the manual partition editing. To work around this, either of these changes to JMFS would be in order:

a) when JMFS copies from one drive to another, if there is an Apple_Free partition as the last one, don't copy it
b) when JMFS expands a drive and finds the Apple_Free partition, it should prompt the user and delete it if approved. A warning mentioning that this is safe if the drive was just copied would probably be in order.

I would think that option "a" is preferable as it is "safe". That is, if not copying the partition causes a problem, it will (I hope) show up right away and the user can recopy from the original.

Option "b" seems much riskier. Unless we know for sure that nothing will be lost (now or in future releases) by deleting that partition, it could be a problem. This option shouldn't be needed for drives copied with option "a" active, but would be needed if one was trying to fix a drive that was copied by a disk duplicator (my situation, probably rare) or one that was copied with an older version (such as the current one) of JMFS.

I wonder if JMFS could look at the data inside the Apple_Free partition and note that it is all blank (is it? or is it just random data?) to advise how "safe" it is to delete it under option "b".


----------



## Oregonian

So... if someone has a problem where they have copied a Premiere drive and it won't expand correctly, I'd recommend the following:

1) Confirm that it has an Apple_Free partition as the last or second-to-last partition
2) Use iBored to zero the Apple_Free partition and the one that follows (if there is one)
3) Change the Last Partition number in ALL of the other partitions to the correct number
4) Expand with JMFS


----------



## Oregonian

Again.... thanks to all for your input here. I'd not have gotten through this without your help. I've learned more about Linux and TiVo drives in the last few days than I really cared to, but it's probably good for me (more so with the Linux stuff).


----------



## unitron

We don't know if what they did to your stock drive to avoid having to replace it reflects what would be on a new drive from them or not.

I really don't understand why they chopped your swap partition in half or put the partitions on there in non-"optimized" order, but I sure am curious.

I think your case is enough of an outlier that comer's not going to make any changes to jmfs because of it.

I'm not sure that it's a case of pdisk not being able to delete the Apple Free partition so much as if you don't replace it with something else it's going to call any unused space an Apple Free partition when it looks at it. But I could be wrong about that.

Dude, you hex-edited a Series 4 drive and things are better now than before you started! You rock!


----------



## Oregonian

> I'm not sure that it's a case of pdisk not being able to delete the Apple Free partition so much as if you don't replace it with something else it's going to call any unused space an Apple Free partition when it looks at it. But I could be wrong about that.


I'd agree except for the fact that iBored was able to delete it and it didn't reappear.

As far as the "outlier" goes, that really depends on whether this is an issue with all Weaknees-supplied systems or if it relates to the "repair" that was done. If it is all of their systems, a fix might be in order (or step-by-step documentation of what I described). If it is just the "repaired" ones, this will likely be rare.

Nonetheless, I may write up thorough directions for the partition deleting. It is pretty easy once you know the steps and get past the quirks of iBored. Do you think that would be useful?



> You rock!


Yeah.. and I likely could have replaced the TiVo once or twice with all the time I spent! I keep reminding myself that these things are part of "education".


----------



## unitron

Oregonian said:


> I'd agree except for the fact that iBored was able to delete it and it didn't reappear.
> 
> As far as the "outlier" goes, that really depends on whether this is an issue with all Weaknees-supplied systems or if it relates to the "repair" that was done. If it is all of their systems, a fix might be in order (or step-by-step documentation of what I described). If it is just the "repaired" ones, this will likely be rare.
> 
> Nonetheless, I may write up thorough directions for the partition deleting. It is pretty easy once you know the steps and get past the quirks of iBored. Do you think that would be useful?
> 
> Yeah.. and I likely could have replaced the TiVo once or twice with all the time I spent! I keep reminding myself that these things are part of "education".


By all means write it up.

When you deleted it with iBored, did you try looking at the partition table with pdisk again?


----------



## Oregonian

I don't have a pdisk output for the 1T drive, but if you look 11 posts back, you can find the 15T_After_Pdisk_Delete_edited.txt file that has pdisk for the 1.5T drive after I did the delete with pdisk.

If it were really important, I could pull the 2T drive back out and run any maps on it, but I'd prefer not.


----------



## jmbach

unitron said:


> When you deleted it with iBored, did you try looking at the partition table with pdisk again?


Good question. The answer is that it should not show up. The Apple_Free partition is an actual entry into the partition table that describes the size of the empty partition. By deleting the partition all you are doing is erasing the entry in the partition table by zeroing out the entry in the table and adjusting the data in all the other partition entries to the maximum number of partitions present on the disk.


----------



## unitron

Oregonian said:


> I don't have a pdisk output for the 1T drive, but if you look 11 posts back, you can find the 15T_After_Pdisk_Delete_edited.txt file that has pdisk for the 1.5T drive after I did the delete with pdisk.
> 
> If it were really important, I could pull the 2T drive back out and run any maps on it, but I'd prefer not.


Let's not fix it so much that it quits working. 

I'll experiment with pdisk on my own sometime.


----------



## unitron

jmbach said:


> Good question. The answer is that it should not show up. The Apple_Free partition is an actual entry into the partition table that describes the size of the empty partition. By deleting the partition all you are doing is erasing the entry in the partition table by zeroing out the entry in the table and adjusting the data in all the other partition entries to the maximum number of partitions present on the disk.


It's been a few years, but somewhere along the way I'd been through the "pdisk appears to delete but really doesn't" dance myself, and was curious if any light could be shed and also it's nice to know if something is going to be in read/write mode when you're expecting it to be read-only unless you specifically tell it to write something.


----------



## jmbach

Oregonian said:


> I don't have a pdisk output for the 1T drive, but if you look 11 posts back, you can find the 15T_After_Pdisk_Delete_edited.txt file that has pdisk for the 1.5T drive after I did the delete with pdisk.


The interesting thing would have been to see what JMFS would do with the drive. When you deleted the 16th partition, pdisk combined the original Apple_Free partition with the rest of free space into one big Apple_Free partition. Either it will say no room to expand, or convert the Apple_Free partition to a coalased MFS partition since there is no other space available to use. I'll try an experiment with that tonight. Should have some free time I hope.



Oregonian said:


> If it were really important, I could pull the 2T drive back out and run any maps on it, but I'd prefer not.


No sense in doing that.


----------



## Oregonian

I have written up a procedure for all of this. I'd appreciate any feedback on it. I have no problem with it being posted anywhere, but I would think it better to wait until I've gotten some comments on it in case there are errors to correct or improvements to be made.


----------



## jmbach

Good write up. A real test of it would be to find a 6th grader and see if he or she can follow the directions successfully.


----------



## unitron

jmbach said:


> Good write up. A real test of it would be to find a 6th grader and see if he or she can follow the directions successfully.


And then we get the 6th grader to explain it so that even we can understand.


----------



## philhu

A good test at this point would also be to take the expanded drive and try marrying an expander to it, then give the before and after partition maps.

I thiunk what the apple_free is is weaknees stacking up the drive for expansion.

The tivo assumes an expander drive if it sees more than 16 partitions.

So I think weaknees adds the applefree partition so there are exactly 16 partitions.

Then if you expande it with an expander, the tivo adds the 17th partition on the expander drive.

The applefree does confuse jmfs, which is why removing it works.

I think


----------



## jmbach

philhu said:


> A good test at this point would also be to take the expanded drive and try marrying an expander to it, then give the before and after partition maps.
> 
> I thiunk what the apple_free is is weaknees stacking up the drive for expansion.
> 
> The tivo assumes an expander drive if it sees more than 16 partitions.
> 
> So I think weaknees adds the applefree partition so there are exactly 16 partitions.
> 
> Then if you expande it with an expander, the tivo adds the 17th partition on the expander drive.
> 
> The applefree does confuse jmfs, which is why removing it works.
> 
> I think


Interesting idea. Will need to see how an original TiVo drive marries the extender. An original TiVo drive just has 14 partitions and no extra space for a Apple_Free partition. So it appears not to be needed in this situation. Unless when it marries the extender it shrinks the swap partition as Weaknees does and adds the Apple_Free partition. However in this case the Apple_Free partition is not at the end of the drive but right next to the swap partition. I doubt TiVo rewrites half the drive to shift the partition to the physical end of the drive. Nevertheless, when I get an expander to play with we can try what you suggest.

Sent from my SPH-L710 using Tapatalk 2


----------



## philhu

ok, cool

I do know that the 17th partition is what triggers the tivo to see an expander. Because when i was playing and added a 17th partition, i started getting unconnected expander drive errors until I recopied the original drive back

So, a standard drive partition map before and after expander marrying would answer alot of questions.

The apple_free is probably half the swap space, which is why it is right after it.

Maybe tivo adds another swapspace on the expander drive too, each half the original size.


----------



## Oregonian

> So I think weaknees adds the applefree partition so there are exactly 16 partitions.





> I do know that the 17th partition is what triggers the tivo to see an expander.


Looking at my original 1T drive (purchased new from Weaknees, but also "repaired" and probably reimaged by them), it does have an Apple_Free partition of 64.68M, but it is the 15th partition. I have attached that map as 1tb_edited.txt

When I copied it to the 1.5T drive (using JMFS) it kept the same partitions. When I expanded it with JMFS, it added a 16th partition, type=MFS. I have included this map as 15T_AfterExpanded_edited.txt.

When I booted that drive in my TiVo XL, it gave me the message about the corrupted expansion drive. Keep in mind this is with exactly 16 partitions.

I resolved it by deleting the 15th partition (Apple_Free) and the 16th partition (MFS). It then expanded properly with JMFS (no Apple_Free) and worked to full capacity in the TiVo.

I am not questioning if more than 16 partitions causes the expansion drive error. At least in my case, the symptom appeared with 16 partitions. It appears to be the presence of Apple_Free before expansion that caused the trouble.

If you look at post #55 on this thread, there is a map of a drive that Weaknees had expanded. It has 16 partitions with the last one being Apple_Free. While it is not certain, it appears that somewhere along the line (all drives/expanded drives/reimaged drives?) Weaknees adds the Apple_Free partition that prevents expansion. I am making no assumption that this is intentional.

I've never used an external drive with the TiVo, so I have no input on how they affect the whole process. I am just trying to clarify some facts about partition counts.

The good news is that the Apple_Free problem is not too difficult to identify nor resolve, once you know to look for it.


----------



## philhu

hmmm

More food for the testers.

I was sure it was the 17th that caused the expansion error. Maybe it was the 16th


----------



## jmbach

What would have been interesting is what the superheader looked like with the Apple_Free partition after expanding with JMFS but before the Tivo "divorced" it. When tivo adds an extender drive it adds a partition pair to the drive and then adds the partition pair to the superheader. /dev/sdb2 and /dev/sdb3. This gives TiVo 3 pairs of partitions in the superheader. This may be the limit that can exist in the superheader. What is interesting in the development of JMFS, when a pair of partitions were added, the 16th was removed from the superheader by divorcing the drive. Apparently TiVo is particular about where the 16th partition resides or it is somehow different than a normal MFS media partition. However a coalased 15th partition can be snuck in and somehow fool the Tivo to add space. 

Sent from my SPH-L710 using Tapatalk 2


----------



## philhu

I just read all the info over at mfslive.org forum and how they learned and developed the whole premiere drive expansion stuff.

From what they talked about the partition pairs might be checked by tivo during a fsfix or mfsfix run

There is also talk there about exapnders but no answers yet. they did say that if tivo sees a partition 16, it assumes it is an expander and then fails trying to read the expander.


----------



## unitron

If you have a stock S3, S3 HD or S3 HD XL, it comes from the factory with a full drive, no Apple Free, and only 13 partitions*, 4 of which are 2 MFS pairs.


Apparently they detect an external drive by detecting an external drive, not by counting partitions.

But when a 17th partition is encountered on boot, and it's not on a properly configured external, the error condidition in the TiV0's errror condidition library which most closely resembles that circumstance is a bad external which needs divorcing, so that's what it gives you as your only choice.


Perhaps if one could find a way to expand the second MFS Media partition on an S3 instead of adding a 3rd MFS pair, one could have a bigger than stock internal and an external.


*(which is why you can use WinMFS to add another MFS pair and then use jmfs to add a final MFS media partition without going over 16, as long as you use a middleman drive or temporarily make a big drive look smaller to WinMFS)


----------



## Oregonian

> What would have been interesting is what the superheader looked like with the Apple_Free partition after expanding with JMFS but before the Tivo "divorced" it


If what you are looking for is the output of mfslayout, I believe that you have it. The 15T_AfterExpand_edited.txt file that I posted in #113 is a 1.5T drive that was copied with JMFS from a 1T Weaknees drive, then expanded, before putting it in the TiVo and getting the error.

I've attached 1.5T_after_Tivo_edited.txt which is after the TiVo "divorce". There is a chance that this is the other 1.5T drive (the one that was copied with the hardware duplicator instead of JMFS), but I don't think so.

The only significant differences that I notice are the deletions of three Logical and Physical Zones at the end of the tables. I would presume that these are the zones that JMFS added in the expansion.


----------



## Oregonian

> There is also talk there about exapnders but no answers yet. they did say that if tivo sees a partition 16, it assumes it is an expander and then fails trying to read the expander.


My 1.5T drive with the Apple_Free partition had only 16 partitions and got that error about the expander. The map for that is 15T_AfterExpand_edited.txt

As I mentioned earlier, having more than 16 partitions may cause this error, but that is not the only scenario that does.


----------



## jmbach

Looking back on the 1.5TB after expand and before tivo and the one after tivo and if the zones on MFSlayout represent the superheader information then only a single partition was added after the expand to the superheader and the Tivo did not like it because it was labeled sda16 and not sda15. Interesting. 

Sent from my SPH-L710 using Tapatalk 2


----------

