Time Machine "cleaning up"

[Waters75]

Hello it's been quite a long time . I am not pretty sure if this is the right section, having a problem with time machine (software?) on a time capsule (hardware?)

Anyway: I have a macbook air 13' (early 2015) running Mac OS High Sierra 10.13.4, and a Time Capsule A1409.
The time capsule is not full by far, it has actually a Terabyte of free space out of 2.

I had noted, in the past, that backups of dates that I tought I should have had were greyed-out, as if they were too old (the hard drive has never been far from full in the past, either) but I wasn't sure and I still not am, so it is a moot point now.

What seems weird now, and the reason why I'm writing, is that Time Machine is performing a "cleaning up". Why is that, having so much free space? Is it compressing old files maybe, or deleting them? And if it is deleting, why?

I could not find any option to limit the space dedicated to time machine, so I have always assumed that this kind of operation should have been performed when there was not space enough, which is not the case. Can you help me to figure this out?

Thank you in advance for any help!

 

[chscag]

Time Machine stores its backups on the Time Capsule in a different format than it does on an ordinary external hard drive. I can only guess that Time Machine has found some problems with the "sparsebundle" on your TC and is cleaning it up (whatever that means). Our advice to you is make sure you have another means of backup available to you. We recommend using Carbon Copy Cloner or SuperDuper to create a cloned backup of your hard drive.

 

[Waters75]

I most definitely will then, my backups are way more precious to me than my Mac itself. Thanks for your help!

 

[Slydude]

I haven't used this trick in a while but it might work for you after you've made a backup with something like Carbon Copy Cloner

1. Go to your Time Capsule and find the sparsebundle that Time Machine has created. It usually has the same name as your computer unless you've given ti a new name.
2. Double click the sparsebundle to open it. If the Finder has been set to show hard drives on the desktop the sparsebundle should appear as a hard drive.
3. Launch Disk Utility and find your Time Machine drive in the list.
4. Select the drive and rund Disk First Aid.

 

[chscag]

Hey Sly:

That's a good suggestion. I saw that hint given on a site that I was viewing while trying to understand what Time Machine was doing by "cleaning up". I'm not really keen on making backups to a network or network drive (Time Capsule) because of the many things that can go wrong.

Which is why I use CCC rotating with two different external hard drives. I probably should consider off site backup but am not sure that with the amount of data I have that it's worth the expense and time spent doing the backup.

 

[Slydude]

I dont remember if I discovered that tip or read it somewhere. Most likely I read it. Either way it has saved my bacon once or twice.

I need to take a serious look at iff-site backup as well.

 

[Waters75]

Thanks, Slydude. I'll try, after having made a backup as you advise with one of the suggested application

I'm not really keen on making backups to a network or network drive (Time Capsule) because of the many things that can go wrong.

I've never heard of such problems. What can possibly go wrong on a network drive, as opposed to another external hard drive?

 

[MacInWin]

Network backups use what is called a sparsebundle, that is, a file with the data compressed to reduce the bandwidth needed to transport the data. That compression makes the backup faster, which is probably why Apple designed it that way. Unfortunately, that compression means that the backup no longer has the same structure as the original, as the data is written in "bands" of data. To assemble the backup, you have to use Time Machine, or mount the drive, in which case the OS will reassemble the drive from the bands. But if one byte of any band goes bad, the entire backup can be lost because the reassembly fails. That failure makes network backups more troublesome than backups to a drive directly attached. In a directly attached drive, the structure of the drive is in the same folder/file structure of the "normal" drive, with the exception that instead of copying the unchanged files/folders, a hard link pointer is created that points to the location of the original file/folder. But no reassembly is required, just a tracking down through the links.

EDIT: correct terminology. It's Bands, not Stripes as I used earlier.

 

[Waters75]

Network backups use what is called a sparsebundle, that is, a file with the data compressed to reduce the bandwidth needed to transport the data. That compression makes the backup faster, which is probably why Apple designed it that way. Unfortunately, that compression means that the backup no longer has the same structure as the original, as the data is written in "bands" of data. To assemble the backup, you have to use Time Machine, or mount the drive, in which case the OS will reassemble the drive from the bands. But if one byte of any band goes bad, the entire backup can be lost because the reassembly fails. That failure makes network backups more troublesome than backups to a drive directly attached. In a directly attached drive, the structure of the drive is in the same folder/file structure of the "normal" drive, with the exception that instead of copying the unchanged files/folders, a hard link pointer is created that points to the location of the original file/folder. But no reassembly is required, just a tracking down through the links.

EDIT: correct terminology. It's Bands, not Stripes as I used earlier.

Thank you for the clarification As per chscag advise I'm trying with a trial version of a supplementary app. During the setup I'm asked whether I want to make a fixed, sparse or sparsebundle. The image will be made on the same Time Capsule A1409, which is unfortunately the only one I own since another one broke.
Shuld I choose "fixed" mode to avoid the danger bound with the "sparse bundle" that you were saying?

 

[Slydude]

Do you have another hard drive that you can use to put the recovered files onto? It's generally not a good idea to recover a file back onto the drive that you are trying to recover files from. In fact, some programs will not let you do that due to the potential for overwriting part of the very file you are trying to recover.

 

[MacInWin]

I agree with Sly, you need to leave the TC drive alone for now until it gets sorted out. If you want to make a different backup, or recover files, get a different drive to hold them.

 

[Slydude]

To answer your question about the sparse, fixed, and sparsebundle files you may find this useful. A fixed sparsebundle is a disk image that is constrained to a specific size. If the sparsebundle is not fixed the size of that file can change as the contents of the disk being copied changes.

 

[Waters75]

Thanks for your answers; being my technical abilities limited, I still am a bit confused however.
(In this moment, I am unfortunately unemployed and, not doubting of the advantages of having more than one hard drive, I cannot afford one at the moment)

Network backups use what is called a sparsebundle, that is, a file with the data compressed to reduce the bandwidth needed to transport the data. That compression makes the backup faster, which is probably why Apple designed it that way. Unfortunately, that compression means that the backup no longer has the same structure as the original, as the data is written in "bands" of data. To assemble the backup, you have to use Time Machine, or mount the drive, in which case the OS will reassemble the drive from the bands. But if one byte of any band goes bad, the entire backup can be lost because the reassembly fails. That failure makes network backups more troublesome than backups to a drive directly attached. In a directly attached drive, the structure of the drive is in the same folder/file structure of the "normal" drive, with the exception that instead of copying the unchanged files/folders, a hard link pointer is created that points to the location of the original file/folder. But no reassembly is required, just a tracking down through the links.

I have made a web search; the app that gave me the option to make a fixed, sparse or sparsebundle copy is Chronosync.
You are saying that the downside of a backup with time capsule is that it would make a compressed copy of the backup, with the danger of seeing it unuseful if a bit of information is corrupted. However, as far as I understand, all of the options (included "fixed", which I hoped to be a "structure of the drive is in the same folder/file structure of the "normal" drive, with the exception that instead of copying the unchanged files/folders, a hard link pointer is created that points to the location of the original file/folder" that I hoped to be, but is in fact a sparsebundle itself. So no change.
Maybe Carbon Copy Cloner or SuperDuper culd do this, on a time capsule? Or is the app suggesting me just this option due to the nature of the external hard drive like other similar applications?

Or will these apps, maybe Chronosync, like Time Machine itself, allow me to do this on a directly attached hard drive? And which one would do the trick, once I manage to get one?

I would like this, something that makes a copy with the same structure of the original drive, but incrementally, keeping somewhere the older versions of files or deleted files apart (lncrementally)

Do you have another hard drive that you can use to put the recovered files onto? It's generally not a good idea to recover a file back onto the drive that you are trying to recover files from. In fact, some programs will not let you do that due to the potential for overwriting part of the very file you are trying to recover.

Chronosync is actually allowing me to do this, even if it is not very useful as far as I understand..

 

[Slydude]

When Time Machine creates a backup to a network device, such as your Time Capsule, it creates a sparsebundle. The files to be backed up are then placed inside that sparsebundle file. The sparsebundle is essentially just a container that holds the files being backed up. The size of that sparsebundle file can expand as the amo0unt of data to be stored increases. A fixed sparsebundle file would always remain the same size. If you were backing up your internal hard drive, for example, the sturcture/organization of the files inside the sparsebundle would be the same as on your internala drive,

Recent versions of MacOS can boot from a Time Machine backup in an emergenmcy but I'll have to do some digging to see if Macs can boot from Time Machine backups inside a sparsebundle.

 

[chscag]

Recent versions of MacOS can boot from a Time Machine backup in an emergenmcy but I'll have to do some digging to see if Macs can boot from Time Machine backups inside a sparsebundle.

As far as I'm aware, Sparsebundle backups are not bootable. Regular Time Machine backups are bootable and will boot the machine using EFI.

 

[Slydude]

As far as I'm aware, Sparsebundle backups are not bootable. Regular Time Machine backups are bootable and will boot the machine using EFI.

That's what I thought as well but I wasn't sure. Didn't want to give out the wrong information.

 

[MacInWin]

I use Chronosync to keep two systems synchronized for certain folders, but don't use it for backups so I cannot answer the question about sparsebundles and exactly how it does work. I do use Carbon Copy Cloner to copy files onto a networked drive structure and it does put them there in a "useable" format just like it would to a directly attached drive. I presume SuperDuper would work in a similar fashion. What I have set up is a scheduled clone of certain folders from my internal drive to a folder on the networked drive. I have it set to run what CCC calls "safety net" that keeps the changed files in a separate folder so that if I want/need to go back to an older version I can get it from those historical files. In my case, it's NOT bootable because I don't clone the entire drive, just the specific folders I want to store on the network drive. The one thing you need to be careful about with CCC, if you should decide to use it, is that if you let it, CCC will eliminate anything on the destination that is not on the source drive. I turned that function OFF because I just wanted to clone folders, not the entire drive, so I wanted the rest of the stuff on the destination to be left alone. It's not difficult, but something to think about as you start with CCC.

 

[Waters75]

I'll dig into CCC then. Thank you all for the clear and very exhaustive answers