Blog Article 8-17-21: Do You Have A Backup - #4: A Small Interlude

[MacInWin]

Blog Article Author:

MacInWin (Jake)

Blog Article link:

Do You Have A Backup - #4: A Small Interlude - Mac-Forums Blog

Originally, this topic was going to be the final one in the series, but after some contemplation, I think it needs to be addressed sooner. There are some major changes in macOS and in the hardware that seem to be making big changes in the backup/restore processes and options. Basically,… Read More

www.mac-forums.com

Originally, this topic was going to be the final one in the series, but after some contemplation, I think it needs to be addressed sooner. There are some major changes in macOS and in the hardware that seem to be making big changes in the backup/restore processes and options.

Basically, two major shifts have occurred: 1) Apple introduced the T2 Security chip and 2) Apple released the M1 System-on-a-Chip (SoC) machines using Apple Silicon instead of the Intel CPU family it had been using. Those two changes have brought some major and minor changes in how the system operates. I suspect that the soon-to-come M1/Mx machines and macOS Monterey may well make even more, but until those products are actually out for consumer use, we can only speculate.

Let me start by giving credit to the website, The Eclectic Light Company for summarizing the issues well. The depth of analysis done by the owner of that site is nothing short of amazing! (And, his articles on art are astounding as well!) If you want detailed technical discussions of the impact of T2 and M1 SoC on booting a Mac, you can spend days reading at Eclectic Light.

The T2 Chip
According to Apple, these Mac computers have the Apple T2 Security Chip:

• iMac (Retina 5K, 27-inch, 2020)
• iMac Pro
• Mac Pro (2019)
• Mac Pro (Rack, 2019)
• Mac mini (2018)
• MacBook Air (Retina, 13-inch, 2020)
• MacBook Air (Retina, 13-inch, 2019)
• MacBook Air (Retina, 13-inch, 2018)
• MacBook Pro (13-inch, 2020, Two Thunderbolt 3 ports)
• MacBook Pro (13-inch, 2020, Four Thunderbolt 3 ports)
• MacBook Pro (16-inch, 2019)
• MacBook Pro (13-inch, 2019, Two Thunderbolt 3 ports)
• MacBook Pro (15-inch, 2019)
• MacBook Pro (13-inch, 2019, Four Thunderbolt 3 ports)
• MacBook Pro (15-inch, 2018)
• MacBook Pro (13-inch, 2018, Four Thunderbolt 3 ports)

In those systems, to be able to boot from an external disk, you have to change the startup security settings explicitly to allow that to happen. So, if you haven’t enabled it in advance, to boot from any external recovery disk you have to boot into the recoveryOS on the internal storage and use Startup Security Utility located there to change the boot security policy. But the challenge is that the recoveryOS is, in fact, stored in a separate volume within the startup Container, and thus is on the same internal drive as the System and Data Volumes. As a result, if the internal drive fails with a hardware failure and cannot be used at all, then you cannot boot into recoveryOS to make the changes to be able to boot from the external backup drive. You could reduce the default security on your T2 machine to make the external recovery drive bootable, but that’s not recommended as the T2 security chip is integral to the basic security setup of the machine and you would open your data up to much greater vulnerability than you may wish to do. So, a clone backup on a T2 system is only useful if the failure is not hardware related but only some corruption of the System and/or Data Volumes. The idea that one could have a hardware failure of the internal drive and boot from the external while waiting for a replacement drive to arrive to be installed is no longer practical in the T2 systems, even those where the drive is, in fact, replaceable.

The M1 Systems
The M1 systems get even more complex. The SoC approach means that there is not really any dedicated hardware to be called a “drive” as such. All of the storage is dynamically allocated as needed to whatever process needs it. There is a structure of Containers and Volumes, as in any APFS drive, but unlike on the T2 systems, the recoveryOS is not a Volume in the same Container as System and Data, but in a completely separate Container, not sharing any disk space with the macOS volumes. The impact is that a recovery on an M1 system is more robust, as it comes with a full toolset when booted from the internal drive. Apple calls that “1 True Recovery.” But it also gets more tricky if it's a hardware issue.

What about our external clone backup? Well, the clones cannot currently copy that separate recovery Container because of the security of the system. And that fact means that any backup disk that could boot won’t have the key tools, including Startup Security Utility that is needed to allow an M1 system to boot from an external drive. The value of a “bootable” clone is much, much less than in a pre-T2 Intel system, or even than a T2 Intel machine. Bottom line? Getting an M1 system to boot from an external drive is difficult to nearly impossible if the internal drive storage holding the recovery Container is damaged at all.

Closing
Why this interlude? Because the changes thus far created by the T2 chip and by the M1 SoC are likely just the tip of the iceberg as Apple moves further and further away from the old Intel architecture. And that move means that it is unlikely that a full clone, even if it were possible to make, isn’t bootable now and most likely won’t be bootable on that new Mx system in a year (or maybe even less).

And that then means that the overall strategy of backups will change. Clones will be of lower value than they are in the pre-T2 Intel systems. Couple that with the fact that with an M1 SoC chipset any hardware failure means the entire SoC has failed, so a hardware failure of the storage being used for internal drives is not repairable and will require replacement of the motherboard. AppleCare+ is going to be much more attractive!

If the storage being used by APFS for the System and Data Volumes is corrupted, not by a hardware failure but by some data corruption, then the recovery process will be to boot into the recoveryOS (1 True Recovery) and reinitialize the storage being used for the Container that holds the System and Data Volumes, then the user can restore the user data to the Data Volume from a backup. Basically, all the backup needs to hold is your data, not the system files at all. And that is a factor in deciding what to backup!

Now, back to your regular programming…

 

[chscag]

Thanks Jake. Excellent overview.

 

[MacInWin]

Thanks, Charlie. The situation is going to get even more complex with Monterey. Anybody contemplating upgrading to Monterey needs to plan NOW for the safety of a rollback to Big Sur. Once Monterey is installed, going back to BS is going to be very challenging. And if folks are thinking a clone will solve that for them, they are going to be very disappointed. Have a read of this for some background: Retaining access to Big Sur when upgrading your M1 Mac to Monterey

 

[lclev]

I guess my concern is - and maybe someone can clarify this for me - in the case of a hardware failure or meltdown that forces a reinstall of the OS does this also mean a reinstall of any apps that were not part of the OS? For example - Affinity Photo, Brave browser, etc. Granted I backup all my data weekly and the really important stuff goes to the cloud immediately. But the "fun" of reinstalling all my apps....yuck! Or are they part of the backed up data ( I assume I will not be that lucky.)

Lisa

Oh, and excellent article Jake - thanks!

 

[pm-r]

The situation is going to get even more complex with Monterey. Anybody contemplating upgrading to Monterey needs to plan NOW for the safety of a rollback to Big Sur.

I dare say with all the involvements and various processes Apple is creating for the user to contend with for the sake of backing up and protecting their data etc, that many users may just say to heck with it and disregard the Microsoft virus problems as it doesn't seem to have so many obstacles in the way and just make the switch to using a Windows OS based computer to bypass all the complexities.

Apple's old adage of "It just works" has long gone into the Sunset and past to the Twilight Zone for many a user. It really isn't going to help to make it worse for many.

Just a thought eh... ???

BTW: Great Blog articles.

- Patrick
=======

 

[MacInWin]

I guess my concern is - and maybe someone can clarify this for me - in the case of a hardware failure or meltdown that forces a reinstall of the OS does this also mean a reinstall of any apps that were not part of the OS? For example - Affinity Photo, Brave browser, etc. Granted I backup all my data weekly and the really important stuff goes to the cloud immediately. But the "fun" of reinstalling all my apps....yuck! Or are they part of the backed up data ( I assume I will not be that lucky.)

Lisa

Oh, and excellent article Jake - thanks!

Lisa, nothing has really changed, as far as I can tell, with applications in the backups. Apps which don't need tight registration will just restore. Apps with tight registration (Adobe, Microsoft, etc), may require either re-registration or re-installation. Things like Affinity, Brave, etc., should just restore and work. I've not had to do a restoration to an M1 or Big Sur machine yet, so I have no personal experience, but from what I know about how the backup is made, apps should restore.

 

[MacInWin]

I dare say with all the involvements and various processes Apple is creating for the user to contend with for the sake of backing up and protecting their data etc, that many users may just say to heck with it and disregard the Microsoft virus problems as it doesn't seem to have so many obstacles in the way and just make the switch to using a Windows OS based computer to bypass all the complexities.

Apple's old adage of "It just works" has long gone into the Sunset and past to the Twilight Zone for many a user. It really isn't going to help to make it worse for many.

Just a thought eh... ???

BTW: Great Blog articles.

- Patrick
=======

No need to be so pessimistic, Patrick. "It just works" still applies. It just works differently. What is changing is that in the "olden" days, one could do a simple clone backup and if the internal drive died, run from the clone (albeit with some performance hits) while waiting for the internal drive replacement to come in and be installed. Then clone back and voila! all is well!

Now, because of the System-on-a-chip approach and the added security to Big Sur and Monterey, Apple has made the logical assumption that if the storage fails, the entire logic board is dead. Basically, if storage has a hardware failure the entire SoC has failed. With that assumption (and I think it's valid, by the way), they added security by using the storage area for various boot processes that ensure that the SoC is not compromised by any malware hackers may have tried to create. Again, not a bad decision, given the premise that failed storage means failed system. That logical decision and design leads to the dilemma for clone developers: They cannot get to those other Containers (by design and for security) so they cannot make a bootable clone that can boot even if the entire storage of the SoC has failed. But even if they could get to it, it would do no good to have it on the backup drive because the boot process looks for it ONLY in the on-board storage, before it looks to any external drive.

Does that mean clones are no longer of any value? No. Clones will still be a valid backup process, and they may well be valid as boot drives as long as the failure of the internal storage is software and not hardware. So a corrupted internal "drive" can still be reformatted and reinstalled from a clone. The boot process will look to the hidden Containers in Storage, initialize the boot, then look to the external drive for the rest of the boot and operations. Disk Utility will be able to reinitialize the visible portion of the internal storage "drive" and then the system can be reinstalled, even cloned back.

So, not really a huge change. But for me what that means is that I'll probably stop backing up the entire storage and just concentrate on the Data volume. If the internal storage gets corrupted, I'll do an Internet recovery and reinstallation, then restore from the clone. That will be, I think, faster given that I have good internet speeds. For those not so lucky to have good internet, there may be a way to make a bootable installer USB thumb drive that can be used to reinstall to the internal. I've not seen a process for that, yet, but if I do, I'll post it here.

But the days of having a fully bootable external drive just in case the internal storage dies are no longer.

 

[lclev]

Well I loved the ability to create a bootable clone. Also the ability to move the entire drive content to a new Apple computer and be up and running in short time.

Lisa

 

[MacInWin]

Lisa, the second is still there. You can restore from or use Migration Assistant from a clone to be up and running quickly. And you can create a bootable clone, and it will boot as long as the internal storage on an Mx system is still functioning. The only thing "lost" is the ability to boot from an external clone if the storage is totally failed. But that's inherent in a System-on-a-chip design. If the storage fails, the chip is failed and the system is failed.

 

[harryb2448]

Used Migration Assistant to move everything to the new iMac and using Apple's Thunderbolt/Thunderbolt 2 cable on my Silicon Power TB1 external drive and all was sweet and fast. TB1 still runs twice as fast as USB3.

Thanks heaps Jake.

 

[pm-r]

Lisa, the second is still there.

Sort of it seems, and if one doesn't mind doing the whole bunch of extra work, but I agree with Lisa and prefer a clone Drive that one can just connect and boot from and everything works as expected and hardly any interruption in data loss or workflow.

But I may never end up using one of those super newfangled Mac models anyway so my old methods will still be working just the way I like.

- Patrick
=======

 

[lclev]

Well I am contemplating getting an M2 or Mx 14" MBP when they come out. Not totally sold yet. I have a 2020 13" MBP that I really like and I will most likely keep it even if I get an M-model. I do like the promise of adding more ports to the next M series. I miss having a USB port. But what ever will I do with all of my adapters and dongles???

Lisa

 

[MacInWin]

Sort of it seems, and if one doesn't mind doing the whole bunch of extra work, but I agree with Lisa and prefer a clone Drive that one can just connect and boot from and everything works as expected and hardly any interruption in data loss or workflow.

But I may never end up using one of those super newfangled Mac models anyway so my old methods will still be working just the way I like.

- Patrick
=======

Patrick, I don't know what the "whole bunch of extra work" you mentioned may be, but consider some scenarios:

1. Old Intel Mac. Logic board dies because CPU fails. Clone backup is useless because it won't boot. Solution is to get a new Mac and restore using Migration Assistant (can't do a full clone because new Mac won't run old macOS version anyway). Reboot from internal.

2. New M1 Mac. System-on-a-chip fails. Clone backup is useless because it won't boot. Solution is to get a new Mac and restore using Migration Assistant. (This is why AppleCare+ is super valuable now.)

I don't see a "whole bunch of extra work" in there.

Ah, you argue, what if on an M1 Mac just the storage fails, like a drive failure? Well, in modern Macs the storage is soldered on so pretty much it's the same scenario in that you have to get a new Mac and restore from the clone as in Scenario 1. But let's work with a really old Mac, with a removal drive, either rotating, or blade, or in a box. OK, in those scenarios if the drive itself dies, you can boot from an external while you wait for a replacement to arrive. So, that is scenario 3.

3. Old Mac. Replaceable internal drive dies. User can boot from external while ordering a new drive. Drive arrives, open case, take out old one, put in replacement, boot with Option key to get to the external, reformat the new internal drive (they always come formatted for Windows), clone the external to the internal, reboot. You may or may not need to reinstall the OS before the use of Migration Assistant, depending on what version of macOS.

All I'm really trying to say is that with an integrated SoC, when the SoC fails, the logic board has to be replaced and while that is happening, a "bootable" clone is pretty much useless. In making a backup plan for a new Mx system, then, you just need to know that the circumstances where a bootable clone has value are limited to those where the internal storage is messed up, but not failed. Just like a bootable clone in an older machine is limited to failures where it's the drive and not the CPU or logic board.

The move to SoC is a step forward in technology. Just like moving from tubes to transistors, from individual components to integrated circuits on chips to integrated systems. Now the entire system is on a chip (sort of, it's actually three in a matrix) so the integration is tighter. The advantage is performance, the disadvantage is repairability. You can have performance, efficiency, repairability, pick any two.

 

[timehawk]

While I recognize the business-focused security improvements of needing an Apple tool to allow an external backup drive to be bootable, I am sorry that Big Sur is confounding Carbon Copy Cloner (and other 3rd party backup providers) from being able to create fully bootable backups. It puts computer consultants into a pickle, and for most Mac users (who only use their Mac in the security of their home) it takes away their ability to quickly get running again. It's a computing decision that spans both home and business users, and Apple is putting more weight on the business security side than the simplicity for the home user.

 

[MacInWin]

I don't think Apple is "business-focused" on this. They are just security conscious and focused on using their own silicon instead of relying on Intel (who have proven to be somewhat unreliable lately). Basically, a bootable backup drive is no longer really needed. it's unfortunate that companies like Bombich and ShirtPocket are caught in the change, but so were the buggy whip makers when Ford started making automobiles. They will need to pivot to the new reality or face the consequences of being inflexible. Both could, for example, simply acknowledge that with the new Apple Silicon full clones have lower value and focus on bringing a better backup option to folks who have Apple silicon. Up to now, I think ShirtPocket has done a better job of it. Bombich seems to me to be more in the "how dare they do that?" camp. We'll see how they react.

 

[timehawk]

Yes, security is vital, and will be at the forefront of many changes in computing. I was not ranking issues, just stating a new backup restriction faced by Mac users when they have total SSD failure. They will have to get hold of a running Mac to be able to move forward and access their files: they can't boot from their backup drive.

 

[Rod]

just concentrate on the Data volume. If the internal storage gets corrupted, I'll do an Internet recovery and reinstallation, then restore from the clone.

Great article Jake, the above quote pretty much sums up my current and future backup strategy.
I wonder about the value of a bootable installer on a USB thumb drive as a part of macOS recovery into the future? Assuming it will still work in the M1 and Mx devices might making a USB/USB-C bootable installer still be a handy way to restore the macOS or do you prefer the online restore?

 

[MacInWin]

Great article Jake, the above quote pretty much sums up my current and future backup strategy.
I wonder about the value of a bootable installer on a USB thumb drive as a part of macOS recovery into the future? Assuming it will still work in the M1 and Mx devices might making a USB/USB-C bootable installer still be a handy way to restore the macOS or do you prefer the online restore?

The Mx systems will NOT boot if the internal storage has a failure. So an external drive can only be used as a boot drive if the internal drive is still available. For now, that includes all external drives--they can ONLY boot if the internal storage is still functioning. And if the internal storage is functioning, there is already a full system of the current OS stored in the secure enclave known as the Sealed System Volume that can be reinstalled. Because of that, I see no use-case for a bootable external thumb drive installer for Mx systems (or for that matter, systems running Monterey).

If you want to know why, keep reading, otherwise that's it.

On an Mx system running either Big Sur or Monterey (the only two version supported at this time), the system files are kept in a secure enclave where they are encrypted, write protected and from which an SHA-256 hash is calculated. That hash is then stored in the enclave with the metadata. Every file has a hash, the hashes are then hashed as groups and a final master hash generated, verified by Apple as correct, encrypted, and stored. Apple calls that the Seal, or Sealed System Volume. Only Apple can open the Seal with their security codes. (Somebody may crack that, but probably not until quantum computers come along, I suspect.) Then a snapshot of that system is made, and it is that snapshot that is actually booted. Part of the boot process checks to see if the hash of the snapshot matches the hash of the Seal, and if it does not, the system boots into recovery mode. If the hash does match, the boot process continues.

Now consider trying to boot from an external drive. The process will calculate the hash of the boot system on the external and if it matches the Seal, let the boot continue (there is a LOT more going on, but just go along with this for a minute). But if the internal storage has failed, the hash cannot be retrieved and the system boots to Recovery Mode, not to the USB drive. For that reason alone, there is not much use for a bootable external installer. I can't think of a situation where one would be needed, if it could be booted.

So what do we do when the system gets corrupted by something? (Either malware or our own incompetence, or just random luck.) Well, as I said if the hash doesn't match, the system boots to recovery mode. The Seal version is then used to "reinstall" the OS, in which case it goes through the hashing process to generate a new master hash, check that against the Apple signature, and only if it matches create a new snapshot of the system for the boot process.

There are ways to disable some of the security Apple built in. SIP can still be turned off, for example, but if any changes are made that affect the Seal value, when it boots it won't match the Apple signature for that OS and stop. So, in a practical sense, if the system boots, the system files are in perfect condition. If it won't, reinstalling probably won't fix it. If the internal "drive" fails, the files can't be checked and the system won't boot at all.

But you CAN boot from an external drive if the internal storage is functional. How is that possible with all of this Seal stuff? Well, to get to the bootable drives on an Mx system you press and hold the power button until you see the "Loading startup options" appear. Then select the boot disk you want and click Continue. On the next restart, that disk will be the boot disk. Note that you can only set it if the internal drive is functional because the low-level boot (LLB) process checks the Seal before you get to the options. But when you point to some other drive as the desired boot drive, the Seal value for THAT drive is calculated and stored. In BS, that value was stored in the metadata on the internal storage, but I have read that with Monterey it is now stored on the external drive in the metadata. If that's true, what that means is that you can move an external boot drive from one Mx Mac to another, set it as the boot drive as described, and boot from it and the hash on the external won't need to be recalculated twice (once to certify, again in the LLB process). In either case, on the next boot the external will be the boot drive. Again, the LLB will check the internal hash, then the external hash and then boot, so the internal still needs to be functional to boot the external.

How might one use that external boot process? How about for beta testing? You get the beta version, install to an external, set it as a boot drive and then boot from it. Play around, test, do whatever you want/need to do, then set the internal drive as the boot drive again and reboot to exit the beta. You can use the System Preferences "Startup Disk" option to see the boot drive to get back easier.

OK, lots of words so I'll stop there.

 

[Rod]

Okay, that's a bit to get my head around but I think I get the gist of it enough to see the answer to my question, as least as how it relates to recovery.
There is another scenario though. What if one installs Monterey and like we often see on this forum decide that for some reason they must revert to the previous macOS? Would it be possible to boot from an earlier bootable installer or bootable clone to erase the internal drive and reinstall the previous macOS along with the Data volume in the case of a bootable clone?

 

[IWT]

@Rod,

I don't have a definitive answer to your excellent question - but from my reading of articles from various sources, the impression I got was, going back to a previous version is becoming extremely difficult and may be near impossible with the M1/X models.

Jake and other wise counsel may have a more precise answer.

Ian