From Mac to Freedom Laptop 3: Data Recovery

Leaving Apple’s Nursing Home

This series is about replacing a MacBook Air with an equally beautiful Freedom Software laptop. It is also about setting up a Freedom Software laptop for the kind of user who wants it to Just Work with the least possible involvement and no interest in how it works.

Part 1 of this series was about the rationale and the hardware.

Part 2 of this series was about choosing and configuring the software.

Part 3: Data Recovery

First I would recover all the generic user files. Most of the documents are in portable formats such a text or PDF or Open Document Format (used with NeoOffice on the Mac, LibreOffice or OpenOffice on the new computer)

After that, I wanted to recover some important data from specific Mac apps:

  • Photos from iPhoto / Apple Photos
  • email from Apple Mail
  • references from Zotero
  • bookmarks from Safari

(Ordered from most to least important.)

Restore Data from TimeMachine or from SSD?

  • from a TimeMachine backup: put together a software solution, copy data off it
  • from the Mac’s SSD: extract the SSD from the Mac, buy a special adapter, copy data off it

From a TimeMachine backup

There was a recent TimeMachine backup, stored on our NAS. As I don’t have a working Mac to run TimeMachine on, I searched for Linux software able to read it. My searches led to using sparsebundlefs plus tmfs to mount the backup as a directory tree. It took me quite some attempts at fighting the permissions system, especially the way that FUSE filesystems deal with permissions, until I could see a list of top level folders, one per snapshot, with one named “Latest” pointing at the latest snapshot. Inside there was apparently a complete snapshot of the Mac’s disk filesystem.

The TimeMachine backup was not encrypted. While the connection from the TimeMachine app to its storage folder on the NAS had required a password, the data inside was not encrypted. (By comparison, some backup systems such as Borg encrypt data on the client side before sending it to the server, so that only the data’s owner can decrypt and read it.)

The sparsebundlefs + tmfs software appeared to have given access to all the files. When I started copying them, however, two issues arose. First, the extraction speed was initially terrible, 0.2 MB/s, when the extractor was running on one machine, with sparsebundlefs remotely accessing the NAS TimeMachine storage through SSHFS, and rsync’ing its output over to the new machine. I suspected the main problem was random access to the NAS’s moderately slow spinning disk, and the secondary problem may be the SSHFS access to it.

Rather than measure and diagnose the exact cause, I copied the TM backup folder over to a faster disk (still a spinning disk) on the extractor machine. This copying went much faster, presumably because it was mostly sequential reading. Using that copy directly on the extractor machine, so cutting out SSHFS too, the extraction process was then much faster.

Then the second problem struck. The extracted data was much larger than expected, too large for the disks on the extractor machine and the target machine. It turned out the extractor was not preserving symlinks. It was presenting every symlinked directory as a separate copy of the directory. I did not know what directories (and perhaps some files too) had originally been symlinked on the Mac, and I could no longer boot it to find out.

I could guess some of the symlinks, partly from prior knowledge, partly from using ‘du’ to spot directory trees of identical huge sizes, and partly from only sources where I found some lists of symlinks others have catalogued, especially those created by migrations through successive versions of iPhoto to Photos. I confirmed the guesses by using an ‘rsync –dry-run’ to verify whether the content of one directory was identical to the other for each of my guesses. (‘diff -r’ works too but is slower because it always reads the full file content whereas rsync takes a shortcut if the file size and timestamp match.)

I ended up manually adding ‘exclude’ rules to my ‘rsync’ invocation. I excluded (in the home dir):

  • Applications/
  • Library/
    • except for “Library/Mail” and “Library/Mail Downloads”
  • Pictures/iPhoto Library*.migratedphotolibrary/
    • an old pre-migration folder that should have contained symlinks
  • Pictures/Photos Library*.photoslibrary/Originals/
    • which should have been a symlink to ‘Masters’

I also excluded a few other files and folders that held nothing interesting and would clutter or confuse the target. Here is the exclude list I used (not mentioning the ‘Library/Mail’ and ‘Library/Mail Downloads’ exceptions).

Pictures/iPhoto Library Test.migratedphotolibrary
Pictures/Photos Library Test.photoslibrary/Originals
Pictures/Photos Library Test.photoslibrary/resources

(Your photo library would not have the word ‘Test’ in its name, by default. Mine did, caused by some manual repair by an Apple shop technician years ago.)

For additional speed in transferring a large amount of data to the new laptop, I copied a couple of chunks of it over on a USB memory stick, as rsync over the WiFi connection was going at only 5 MB/s (~50 Mpbs) even when near the WiFi access point. It would have been a good idea to buy a USB-to-Ethernet adaptor for a task like this, which could have gone much faster.

More details on TimeMachine storage format and manually accessing it: Deep Dive or here, by Glenn ‘devalias’ Grant.

From the Mac’s SSD

Reading data directly from the Mac’s SSD would have saved me time in fiddling with the sparsebundlefs + tmfs software, and in dealing with the data that should have been symlinks but wasn’t.

Apple used a non-standard SSD connector on some MacBook Air (and Pro) models. We can buy an adapter for the particular Mac model, to connect the SSD to a standard SATA connector, or to a USB-to-SATA adapter.

I ordered an SSD adapter. When it arrived, I got out my collection of security screwdriver bits (various sizes and odd shapes) and found I didn’t have the required tiny 5-pointed star shape. Dang.

I will order the special screwdriver because, even though I completed the data transfer, I do not want to sell or dispose of the broken Mac with the private data still on it. (It’s not encrypted. Next time it should be. And indeed I have set up the new computer with disk encryption.)

I have also heard that one can get low level access to an internal drive through the Thunderbird port. I have not investigated whether this is possible in my case.

Recover Photos from iPhoto / Apple Photos

The plain JPEG (etc.) files are found in the ‘Pictures/Photos Library.photoslibrary/Masters’ folder.

TODO: Find out if there were also metadata stored separately, e.g. photo album names and comments.

Recover Email from Apple Mail

For mail accounts using IMAP: the mail should be on the mail server. Don’t bother trying to recover anything from the local data.

For mail accounts using POP: the mail is stored only locally and we will want to recover it.

We find an “Apple Mail to dovecot mailbox converter” at

I installed Erlang (as required) and ran it… and it did not work. Here is the output from a test run on a single message:

$ escript emlx_to_mbox.escript --single ~/tm-home/Library/Mail/V4/863E1A15-*/INBOX.mbox/233CA490-*/Data/0/0/1/Messages/100638.emlx 
emlx_to_mbox.escript:13: Warning: erlang:get_stacktrace/0 is deprecated and will be removed in OTP 24; use use the new try/catch syntax for retrieving the stack backtrace
escript: exception error: no case clause matching 
                     <<"Received: from [] (HELO SERVER)\n  by SERVER (CommuniGate Pro SMTP 6.0.11)\n  with ESMTP id 399065899 for EMAIL@DOMAIN; Tue, "...>>}
  in function  emlx_to_mbox_escript__escript__1634__919879__991744__2:get_header_value/2 (emlx_to_mbox.escript, line 286)
  in call from emlx_to_mbox_escript__escript__1634__919879__991744__2:process_emlx_file/4 (emlx_to_mbox.escript, line 71)
  in call from escript:run/2 (escript.erl, line 758)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1 
  in call from init:do_boot/3

I have not programmed in Erlang before. Maybe now would be a good time to start?

Recover References from Zotero

Copying the Zotero folder to the Linux laptop Just Worked. Hooray!

Recover Bookmarks from Safari


From Mac to Freedom Laptop 2: Software

Leaving Apple’s Nursing Home

This series is about replacing a MacBook Air with an equally beautiful Freedom Software laptop. It is also about setting up a Freedom Software laptop for the kind of user who wants it to Just Work with the least possible involvement and no interest in how it works.

Part 1 of this series was about the rationale and the hardware.

Part 2: Software

Our leaving Apple is, fortunately for us, much easier than for someone who has both feet firmly planted in Apple’s walled garden.

This particular laptop was being used more like an old-fashioned stand-alone computer than a portal to Apple services. Finding and switching to alternative Freedom Software apps will not be too much of a hurdle.

  1. What Operating System?
  2. What Replacement Apps?
  3. What to Configure
  4. Practise the Configuration

What Operating System?

We’re going for something mainstream and stable and familiar. We’re not going for 100% hard-core freedom such as LibreBoot and Trisquel, important though they are. While there are several good options, for me (long time Ubuntu fan) it’s probably going to be one of:

  • Ubuntu, in its default Gnome form — the most common, generic Linux; stable and widely known and supported.
  • Ubuntu, with customisations (e.g. a MacOS-like launcher).
  • Elementary OS — slick, MacOS-inspired style; based on Ubuntu; generally great for beginners, especially from MacOS; but perhaps too quirky and too niche (compared with Ubuntu) to be the best choice for this situation. (Here’s a review.)

Also considered:

  • PureOS — slick; made with great dedication to software and phone freedom; likely too new, quirky and niche (compared with Ubuntu) to be the best choice for this situation.
  • Ubuntu Budgie — MacOS-like style.
  • Ubuntu MATE — supports MacOS-like, Windows-like and Ubuntu-like layouts; but I’m wary it may not offer the best modern simplicity and future, being based on older Gnome2.

For Ubuntu customisations, we might be looking at just installing a MacOS-like dock such as Dash to Dock. My philosophy is not to care much about the non-functional visual elements of a design, such as its colour theme and icon style and type fonts. Where to find things (apps, settings, files) is more important. Aesthetics do matter too, and the MacOS dock with its icons standing on a reflective surface and unfolding and bouncing is undoubtedly beautiful. However if placed at the bottom of the screen it has a serious flaw which is that vertical screen space is tight on a laptop screen, and for tasks like writing a document or programming, even just reading a web page, windows should use the maximum height available. Even with options to maximize a window or auto-hide the dock, side docking is a better arrangement.

What Replacement Apps?

Replacement user apps:

  • email (Apple Mail)
    • –> MailSpring (FOSS desktop UI, like web mail UI, for IMAP mail accounts)
    • –> Thunderbird? (for recovering local-only (POP) mail from Apple Mail)
  • web browser (Safari)
    • –> Firefox
  • photos (Apple Photos)
    • –> Shotwell?
  • Zotero reference manager
    • –> Zotero also works in Linux

Configuration, Admin:

  • access to network-shared documents folder
    • –> SMB share: also works in Linux
  • backup (TimeMachine)
    • –> Borgmatic (efficient external backup)
    • –> Cronopete? (friendly UI but limited, internal?)
    • –> TimeShift? (for system configuration)

What to Configure

User Apps

  • MailSpring
    • gmail account
    • other account
  • Thunderbird?
  • Firefox
    • create Firefox account
    • plugin: Bitwarden
    • plugin: Adblock Plus
    • plugin: Zotero
  • Shotwell
  • Zotero


  • machine name, user name, …
  • remote admin
    • ssh
    • xdmcp or rdp or xspice?
  • backup: Borgmatic
  • backup: TimeShift?
  • online accounts (Ubuntu config)
    • Ubuntu One (needed for Livepatch)
    • Google account?
    • our self-hosted Nextcloud (calendar, address book, …)
  • external accounts (manually)
    • our shared documents folder (NFS/SMB share): open & bookmark
  • printer
    • install hplip (for HP printers), hplip-gui (esp. useful for scanner)
    • connect to our local and network printer(s)
  • preferences
    • lock / power
    • appearance, …
  • remove unwanted apps
    • games, favourites, …

Additional Apps Wanted

  • Bitwarden
  • syncthing?
    • share with phone
    • share among users
  • KDEConnect?

Practise the Configuration

The goal is to configure a laptop that is ready to use, not one where the user will need to configure everything bit by bit as they start using it.

We can practice setting everything up, before doing it for real. The practice run is going to involve lots of manual work and trial and error. We can document it, and try to automate parts of it, to help make the real configuration quick and smooth.

For the practice run, we can use dummy accounts wherever we connect to external systems. For example, create an external email account that is not using the user’s real email address or password. We don’t want the user’s real email to be spammed with all our configuration attempts, certainly not the trial and error.

When it comes to the final real configuration, the user will have to be involved. Their real email account will receive various configuration emails and they will have to create and store some new passwords. The user will have to choose whether they want to be involved hands-on in this, choosing their own passwords etc., or whether they want us to do it all for them, depending on their attitude. This is that part that needs to go quickly and smoothly.

What to use for a test environment?

  • additional user account on own system
  • virtual machine (VM)

We could practise some of the configuration inside an additional user account on our own laptop, assuming we use a similar enough operating system and don’t mind installing the required apps on it. That is a good start. Better would be to create a VM and then we would be able to practice every part of the set up, except for any hardware that is not available on the VM such as perhaps a fingerprint reader or a printer.

We will surely need to fix and adjust the configuration afterwards. Remote admin access will be useful, if the user is willing. There are broadly speaking two levels to choose from, depending on the user’s relationship to the admin:

  • remote desktop (via e.g. VNC):
    • the user needs to be already logged in and accepts our connection at their discretion;
    • the user can watch what the admin is doing, though they might well not understand it;
    • the admin can thereby access the user’s data and account settings in the same way the user would;
    • the admin may access system configuration (via sudo).
  • remote log-in, by command-line (via SSH) and/or graphical (via e.g. XDMCP):
    • the user need not be involved nor see it;
    • the admin has the same control as if they had the machine physically, which means full access to the system configuration, and, depending on how the system was set up (encryption), likely also access to the user’s own data and account settings, to be used with the user’s consent, of course.

Switching from a MacBook Air to a Freedom Software Laptop

Leaving Apple’s Nursing Home

This article tells how we replaced a MacBook Air with a freedom-software laptop, aiming to keep it delightful to use and to carry about, while standing up in support of the principles of freedom of the users, freedom from the control and lock-in that Apple wields over its users, its subjects.

The Opportunity: the old MacBook Air dies.

The MacBook Air showing a “panic” message at switch-on and dotted lines across the screen

It’s terminal. The diagnosis is the soldered-on RAM has failed. Technically speaking it could be repaired, but it’s not worth it. We need a new laptop.

This is the opportunity. We have to make an effort to replace this and set everything up again, one way or the other, so can we make the effort to switch to freedom software at the same time? Why should we?

The Choice: Apple or Freedom?

While we should choose our direction according to our values and principles, we all find it hard to see and evaluate the big picture.

Apple promises to sell us a world in which “our” computer systems do what we want and what we need, easily and quickly and beautifully. At first sight, that is indeed what their products look like. Only when we dive deeper into their ecosystem, that is when we begin to learn how controlling they are. Devices we buy from Apple are not “ours”, they are tightly controlled by Apple. Apple restrict both what we are allowed to do (legal controls) and what we are able to do (practical enablement). Let’s see an example of how this works out.

As long as we play along inside Apple’s walled garden, everything smells of roses. Now let’s try to message a friend who has not bought Apple, or share photos with them. Suddenly we hit the wall. Our friend is Outside, and Apple has locked the doors. But it’s OK, we say, they’re not blocking us, look, we just need to install and sign up to Facebook’s WhatsApp or Google’s Photos because that’s what our friend is using. That seems to work. Why? Because Apple chooses to unlock the door for us to install those particular apps, according to agreements with those particular vendors. Apple only lets us install software from their own store, and they only let in software that conforms to strict Apple-centric rules. That’s very strongly enforced on iPhones, with MacOS moving swiftly in the same direction. The marketing message that says this is all to protect us from nefarious cyber threats. Who could deny that there is a grain of truth behind that? Yet the unspoken reality is they are mainly protecting their control over our digital life.

Besides, installing another app to meet a friend outside this garden only “works” in a crude way: it still does not allow us to invite our friend to meet us in our current messaging system. Instead we have to go and visit them in one of those separate, equally proprietary walled gardens, where we can’t share our photos and contacts and messages directly.

It’s not only Apple. Google and Microsoft are doing it too, while Apple and Amazon wield the tightest restrictions over their users. If you were not aware how bad it is, try reading up about how the vendors can remotely install and uninstall software on what they like to call “our” device.

The Future of Computers

Two of the most readable short articles illuminating this sad state of affairs are Your Phone Is Your Castle and The Future of Computers: The Neighborhood and The Nursing Home by Kyle Rankin. The author is the chief security officer of Purism, one of several small companies that are passionately contending to change the landscape by offering a digital life characterised by principles of freedom. Freedom in the sense that we the users are in ultimate control of our digital data systems, not the other way around. “As a social purpose company, Purism can prioritize its principles over profit. The mission to provide freedom, privacy, and security will always come first.”

Another player is /e/ Foundation (“Your data is YOUR data!”), bringing us de-Googled Android phones. These phones can run without any dependence on or control by Google: instead the user is in ultimate control. The irony of Android being marketed as an “open source” operating system is that only parts of it are open source and people have had to expend a huge amount of effort to build replacements for Google’s proprietary parts. But now the huge efforts of many volunteers over many years, now beginning to be augmented by some small companies including /e/, are paying off and these alternatives exist. Read more in, for example, a late-2020 interview in The Register.

These companies are formed from small groups of people following their beliefs. Together they are building the next wave of the freedom software movement that is perhaps most widely known as the Linux world. Taking the idea far beyond freedom to re-use and re-mix just individual software programs, they are bringing freedom now to the world of connected digital services that we use to store our family memories and to communicate with one another.

Freedom Software Laptops

Back to laptops.

A few big-name manufacturers make a few of their models available to buy with Linux pre-installed. Sadly they hide rather than promote this option, seeming to consider it merely a necessity to satisfy certain business customers, and offering little beyond a basic default installation which could easily be done at home.

The best way to support freedom software, and to get a machine that is already properly set up for it, is to buy from one of the small companies that specialise in it.

A DuckDuckGo web search for “Linux laptops” found plenty of starting points, some articles listing the favourite mainstream laptops that people like to run Linux on, others listing the specialist companies that sell Linux laptops.

I ended up looking at both alternatives: buying a mainstream laptop, likely second-hand, or buying a new laptop from a specialist. The category I am looking for this time is slim, ultra-light or “ultrabook”, around 14″ screen size, to replace the feel of a MacBook Air.

Best liked mainstream laptops this year seem to be first Dell’s XPS 13 series, and second Lenovo’s ThinkPad X1 Carbon series. Each range covers a wide range of specs.

Specialist linux laptop vendors include System76 (such as their Lemur pro), Purism (e.g. Librem 14), and Pine64 (e.g. PineBook pro), along with several more. Some make their own hardware, and others buy mainstream or OEM hardware and customise it. Most offer a choice of operating system, all based on well known open source OS’s (the GNU/Linux or *BSD families), sometimes customised or own-branded.

Then I found Laptop with, a trading name of Comexr B.V. in the Netherlands. They sell a range of laptop styles, all based on the OEM brand “Clevo”, and have a lovely set of customisation options ranging from hardware components to setting up disk encryption, choosing installed applications and choosing my user login name. None of that is anything I couldn’t do at home, but it shows they go further than a basic default installation of the OS and it genuinely will save me some time and effort. For me, they offer the extra advantage of shipping with UK tax and duties already included.

Second-hand? Tempting. New? Sensible.

To begin with, I could not accept the cost of buying new, as machines I considered decent spec were available for hundreds of pounds less. Eventually, I re-balanced my assessment in favour of buying something that is intended to last for years, and I mean ten years. The hassle of changing from one computer to another, setting everything up and getting used to the differences, can be realistically valued at tens of hours. From that point of view, it made sense to buy something new and high spec so that it doesn’t seem too terrible after many years.

So it is that I am ordering the Clevo L141MU 14-inch Magnesium Laptop. I will go for a mid-to-high hardware spec, particularly focusing on speed because I want it to be pleasant to use, and mid-level RAM and SSD capacity because this is an upgradeable computer and the prices of those will come down. RAM in particular can be upgraded later with no hassle. Upgrading the SSD later would require externally copying its contents to the new one which might be an evening’s work.

It is even lighter than the MacBook Air it replaces, and just fractionally less thin.