APFS is a copy-on-write file system, consisting of a set of immutable objects that are the fundamental building blocks of the file system’s design. APFS objects are made up of one or more fixed-size blocks. Block sizes are configurable at the time of formatting a new container. Valid block sizes are any power-of-two sized value between 4 KiB and 64 KiB of data, and must always be an integer multiple of the block size of the underlying storage device. At the time of this writing, the default (and thus most common) block size is 4 KiB.

Object Headers

While some objects are headerless, most begin with an obj_phys_t structure as their header. Like all APFS on-disk objects, this structure is stored with little-endian values.

#define MAX_CKSUM_SIZE 8

typedef uint64_t oid_t;
typedef uint64_t xid_t;

typedef struct obj_phys {
    uint8_t o_cksum[MAX_CKSUM_SIZE]; // 0x00
    oid_t o_oid;                     // 0x08
    xid_t o_xid;                     // 0x10
    uint32_t o_type;                 // 0x18
    uint32_t o_subtype;              // 0x1C
} obj_phys_t;                        // 0x20

The object headers are immediately followed by type-specific data, and any remaining space between the object’s data and the end of its last block is always zeroed and is reserved for future use.

Checksum

The integrity of an on-disk APFS object’s data can be verified by calculating a Fletcher-64 checksum of all the object’s data after the first 8 bytes. This checksum can be compared with the value of the o_cksum field in the object’s header. If these values do not match, then the object is either only partially flushed to disk or is otherwise corrupted. Note that like most uses of checksums, this is not a security feature, but is only used to detect unintentionally corrupted data.

uint64_t fletcher64(const void* data, size_t size) {
    uint64_t sum1 = 0;
    uint64_t sum2 = 0;

    // Calculate the number of 32-bit words
    size_t words_left = size / sizeof(uint32_t);

    // Interpret the data as a set of 32-bit words
    const uint32_t* words = static_cast<const uint32_t*>(data);

    while (words_left > 0) {
        // Truncate sums after a maximum of 1024 words
        const n = std::min(words_left, 1024);

        // Compute the checksums
        for (size_t i = 0; i < n; i++) {
            sum1 += words[i];
            sum2 += sum1;
        }

        // Calculate the modulo of the sums
        sum1 %= UINT32_MAX;
        sum2 %= UINT32_MAX;

        words_left -= n;
        words += n;
    }

    // Calculate the value needed to be able to get a checksum of zero
    const uint64_t ck_low = UINT32_MAX - ((sum1 + sum2) % UINT32_MAX);
    const uint64_t ck_high = UINT32_MAX - ((sum1 + ck_low) % UINT32_MAX);

    // Combine the sums
    return ck_low | (ck_high << 32);
}

Object and Transaction IDs

Each object has a unique 8-byte object identifier (oid), which is stored in the header’s o_oid field, along with an 8-byte transaction identifier (xid). Most APFS objects are immutable. When a change is made and flushed to disk, an entirely new object is created elsewhere on disk and is assigned the same oid as the original object, but with a higher xid.

Once the updated object has been fully flushed to disk, and all other objects that reference the original object have been updated to reference the newer object, the transaction is considered complete and the original object’s blocks are free to be reused by APFS. While these blocks are not immediately wiped for reuse, the lifetime of unreferenced objects is relatively short.

Types and Subtypes

The remaining two fields in the header encode the object’s type and (optional) subtype identifiers. Each distinct APFS object type is assigned a unique type identifier. With few exceptions, this identifier is stored in the 16 least-significant bits of the o_type field in the header, with the 16 most-significant bits being used for type flags.

The following is a list of all currently-known object types and their identifiers. We will discuss the details of many of them throughout the course of this blog series.

There are three additional known object types that use all 32-bits of the o_type header field and do not contain type flags.

B-Tree objects also contain subtypes, which help identify the specific purpose of the tree. These subtype identifiers are stored in the header’s o_subtype field. The following is a list of known b-tree subtypes and the structures that they map.

Type Flags

As previously mentioned, when object types are indicated in object headers and other APFS structures, they are usually combined with up to 16 bits of flags that give extra information. The currently defined flags are as follows:

// Object Kind Flags
#define OBJ_VIRTUAL 0x00000000
#define OBJ_EPHEMERAL 0x80000000
#define OBJ_PHYSICAL 0x40000000

// Other Flags
#define OBJ_NOHEADER 0x20000000
#define OBJ_ENCRYPTED 0x10000000
#define OBJ_NONPERSISTENT 0x08000000

The two most-significant bits are used to denote the kind of APFS object: virtual, ephemeral, or physical. All APFS objects fit into one of these three categories. The difference between these will be the subject of tomorrow’s post.

If the OBJ_NOHEADER flag is set, then the object type in question does not start with an obj_phys_t header. These types of objects are rare, and so far I’ve only seen it used for space manager bitmap objects. Note that these objects are different than the headerless objects that are used in sealed volumes, which we will discuss in a future posts.

The OBJ_ENCRYPTED flag denotes an object that is always encrypted on disk, and the OBJ_NONPERSISTENT flag denotes an object that is never written to disk at all (this flag will only be set for ephemeral objects in memory that do not require persistence).