Overview

On eligible volumes (Data, Enterprise, or User role), APFS records residency state transitions for speculatively downloaded files: when they are materialized (downloaded), accessed, evicted (data removed but inode retained), or purged (fully deleted). This information helps the system optimize its prefetch decisions by understanding which speculative downloads were actually useful.

Tracking is active only when bit 0 of the spec_telem_enablement boot-arg (or sysctl) is set, and only on volumes that are not snapshot-mounted.

Inode Flags

Two flags in j_inode_val_t.internal_flags control participation:

#define INODE_MAINTAIN_SPECULATIVE_TELEMETRY 0x20000000  // bit 29
#define INODE_SPECULATIVE_TELEMETRY_ACTIVE   0x40000000  // bit 30

INODE_MAINTAIN_SPECULATIVE_TELEMETRY is inherited from parent directories (part of the inherited flags mask). It marks an inode or directory as eligible for telemetry tracking. INODE_SPECULATIVE_TELEMETRY_ACTIVE is set when the inode qualifies for active tracking and causes creation or update of the telemetry extended attribute.

Extended Attribute

Telemetry data is stored in a file-system-owned, embedded extended attribute:

#define SPECULATIVE_TELEMETRY_EA_NAME "com.apple.system.fs.speculative_telemetry"

typedef struct spec_telemetry_xattr {
    uint8_t version;      // 0x00 (must be 0)
    uint8_t use_state;    // 0x01
    uint16_t flags;       // 0x02
    uint64_t timestamp;   // 0x04
} spec_telemetry_xattr_t; // 0x0C (12 bytes)

• version: Must be 0 (future versions are rejected)
• use_state: The current residency/usage state
• flags: Bit 0 = dirty (state changed but fsevent not yet sent), bits 2-5 = residency reason
• timestamp: APFS timestamp (nanoseconds since epoch) of the last state transition

This attribute is always 12 bytes and cannot be set from userland.

Use States

Residency Reasons

The flags field (bits 2-5) encodes why the file was speculatively downloaded. This allows the system to distinguish between different prefetch strategies and measure their effectiveness. APFS validates the range (0-6) but does not interpret the value; the semantic meanings are defined by the FileProvider framework:

Lifecycle

1. Eligibility: An inode qualifies for telemetry if it is a regular file or directory, INODE_MAINTAIN_SPECULATIVE_TELEMETRY is set, the volume has an eligible role, and spec_telem_enablement bit 0 is set.

2. Creation: When a qualifying inode is created in a tracked directory, INODE_SPECULATIVE_TELEMETRY_ACTIVE is set.

3. Materialization: When speculative content is downloaded, the extended attribute is created with use_state = 1 (Materialized) and the current timestamp.

4. Access: When the file is read by a user or process, the state transitions to 4 (Accessed). This is the key metric: speculative downloads that are accessed were useful.

5. Eviction: When the system reclaims space by removing the file’s data (while keeping the inode), the state transitions to 2 (Evicted).

6. Purge: When the file is fully deleted, the state transitions to 3 (Purged).

7. Event Reporting: When the telemetry state has changed but the corresponding event has not yet been reported, the dirty bit (SPEC_TELEM_FLAG_DIRTY, bit 0 of flags) is set. It is cleared before applying the next state change.

Directory Integration

For directories with INODE_MAINTAIN_DIR_STATS, telemetry participation is tracked at the directory level through the telemetry_count field of the expanded directory-statistics record (j_dir_stats_expanded_val_t), updated during reconciliation. This enables aggregate reporting of speculative download effectiveness per directory hierarchy. (Note that 0x100 in the directory-statistics flag set is DIR_STATS_INITIALIZED, which is unrelated to telemetry.)

Inode Extended Fields

Two legacy extended field types support telemetry:

Forensic Considerations

• The com.apple.system.fs.speculative_telemetry extended attribute reveals which files were speculatively downloaded and whether they were ever accessed.
• The timestamp field provides precise timing of state transitions.
• The residency reason identifies the prefetch strategy that triggered the download.
• Files in the “Materialized” state (never accessed) represent wasted bandwidth and storage.
• The INODE_MAINTAIN_SPECULATIVE_TELEMETRY flag on directories identifies which directory hierarchies are managed by cloud sync services.

Conclusion

Speculative telemetry gives APFS visibility into the effectiveness of speculative downloads. By tracking the complete lifecycle from materialization through access or eviction, the system can make informed decisions about which content to prefetch. For forensic analysis, these records reveal cloud sync activity, file access patterns, and storage management decisions.

Overview

A graft takes an APFS-formatted disk image file that resides on a host volume and mounts its file system tree under a designated directory on that same volume. To applications, the grafted content appears as ordinary files and directories within the host volume’s namespace. Up to 255 grafts can be active on a single volume simultaneously.

The kernel builds a blockmap LUT (lookup table) that maps logical block addresses within the graft image to physical blocks on the host volume’s storage. This allows the grafted file system to be read using the same block I/O path as the host volume.

Graft Constraints

• Maximum 255 grafts per volume
• The graft file must be a regular file with nonzero size
• The graft file must not have hard links (nlink must be less than 2)
• The graft file must not be compressed
• Grafts cannot be nested: the graft file must not reside inside another graft
• The graft directory must be an existing, non-deleted directory that is not already a graft point
• The host volume must not itself be a graft
• On encrypted volumes, the file must use protection class C or D; cloned files and empty files are rejected
• Volumes undergoing crypto transformation cannot graft

Graft Extended Attributes

Three extended attributes on the graft file’s inode record the J-object ID reservation:

These attributes are file-system-owned (XATTR_FILE_SYSTEM_OWNED) and persist after ungraft, allowing subsequent grafts of the same file to reclaim the same ID range without conflicts.

Graft Lifecycle

Phase 1: Validation

The kernel verifies that all constraints are met: the file exists, is not compressed, has no hard links, the directory is valid, and the volume has capacity for another graft.

Phase 2: Blockmap LUT Construction

The kernel iterates all data extents of the graft file and builds an in-memory B-Tree (subtype OBJECT_TYPE_GRAFT_BLOCKMAP_LUT_TREE) that maps logical blocks within the image to physical blocks on the host volume. If the file is a clone of an already-grafted file, the existing blockmap is shared.

Phase 3: Encryption

On encrypted volumes, the graft file’s encryption key is unwrapped and retained for later I/O translation.

Phase 4: Container and Volume Mount

The APFS container embedded in the graft image is mounted using the blockmap LUT for I/O translation. The first volume within the grafted container is then mounted.

Phase 5: Metadata LUT Enhancement

The blockmap is augmented with metadata block mappings (container superblock, checkpoint areas, space manager, object maps, B-Tree nodes). Metadata blocks are distinguished from data blocks by having bit 31 set in the LUT key.

Phase 6: Image4 Authentication

For sealed graft images (Cryptexes), the volume’s root hash is verified against an Image4 payload and manifest. Authentication volume types include:

Phase 7: J-Object ID Range Reservation

A range of object identifiers is reserved from the host volume’s ID space for the grafted content. This ensures grafted inodes do not collide with host volume inodes. The reservation is persisted in the graft extended attributes.

Phase 8: State Registration

The graft state is registered, a synthetic device ID is generated, and grafted file-system vnodes are loaded. An IOKit AppleAPFSGraft service node is published.

Ungraft

The ungraft operation reverses the graft:

1. Remove graft state and decrement the volume’s graft count
2. Wait for all concurrent readers to drain
3. Revoke vnodes belonging to the graft
4. Detach the IOKit service node
5. Unmount the grafted volume and container
6. Clear the graft inode flags (INODE_IS_GRAFT_DIR on the directory, INODE_IS_GRAFT_FILE on the file)
7. Release crypto state if present

The ungraft ioctl supports flags for ungrafting all grafts on a volume (bit 0) and forcing ungraft even when vnodes are in use (bit 1).

Forensic Considerations

• The principal on-disk artifact of an active graft is a set of inode flags stored in j_inode_val_t.internal_flags (not bsd_flags): INODE_IS_GRAFT_FILE (bit 49, 0x2000000000000) on the host file, INODE_IS_GRAFT_DIR (bit 47, 0x800000000000) on the graft directory, and INODE_INSIDE_GRAFT (bit 46, 0x400000000000) on every inode belonging to the grafted tree. Bits 46 and 47 together identify any inode participating in a graft.
• The graft extended attributes (com.apple.fs.graft-*) on a file indicate it was used as a graft image, even if no graft is currently active.
• The graft-vol-uuid must match the current volume UUID, providing a way to verify the graft’s provenance.
• The reserved J-object ID range reveals which inode numbers belong to grafted content.
• On sealed system volumes, grafts (Cryptexes) provide the mechanism for Rapid Security Responses: security patches can be applied without modifying the sealed system volume itself.
• The blockmap LUT is memory-only; it is not persisted on disk. Forensic analysis of grafted content requires parsing the graft image file’s extents to reconstruct the logical-to-physical mapping.

Conclusion

Volume grafting extends APFS’s capabilities by allowing sealed disk images to be mounted as subdirectories. Combined with Image4 authentication, this provides a secure mechanism for distributing system extensions and security updates (Cryptexes) without breaking the sealed system volume’s integrity guarantees.

Overview

Encryption rolling is triggered when a volume transitions between encryption states: from unencrypted to encrypted, from encrypted to unencrypted, or from one key to another. Because a volume may contain terabytes of data, this operation cannot complete in a single transaction. Instead, APFS maintains an er_state_phys_t object that tracks progress across transactions, allowing the operation to resume after crashes or reboots.

The encryption rolling state object is referenced by the apfs_er_state_oid field of the Volume Superblock.

Phases

Encryption rolling proceeds through three phases, tracked in the ersb_flags field. The er_phase_t values below are not stored directly: they are shifted into bits 12-13 of ersb_flags (extract via ERSB_FLAG_ER_PHASE_MASK 0x3000 and ERSB_FLAG_ER_PHASE_SHIFT 12), so ER_PHASE_DATA_ROLL (2) appears on disk as 0x2000.

Each phase processes its objects in windows, encrypting or decrypting a chunk of data at a time (typically 1 MiB per window).

er_state_phys_t

The on-disk encryption rolling state (version 2, 128 bytes total).

#define ER_MAGIC 0x464C4142 // 'FLAB'

typedef struct er_state_phys_header {
    obj_phys_t ersb_o;     // 0x00
    uint32_t ersb_magic;   // 0x20
    uint32_t ersb_version; // 0x24
} er_state_phys_header_t;  // 0x28

typedef struct er_state_phys {
    er_state_phys_header_t ersb_header;        // 0x00
    uint64_t ersb_flags;                       // 0x28
    uint64_t ersb_snap_xid;                    // 0x30
    uint64_t ersb_current_fext_obj_id;         // 0x38
    uint64_t ersb_file_offset;                 // 0x40
    uint64_t ersb_progress;                    // 0x48
    uint64_t ersb_total_blk_to_encrypt;        // 0x50
    oid_t ersb_blockmap_oid;                   // 0x58
    uint64_t ersb_tidemark_obj_id;             // 0x60
    uint64_t ersb_recovery_extents_count;      // 0x68
    oid_t ersb_recovery_list_oid;              // 0x70
    uint64_t ersb_recovery_length;             // 0x78
} er_state_phys_t;                             // 0x80

• ersb_header: Object header with magic ('FLAB') and version (1 or 2)
• ersb_flags: Operation type, phase, checksum block size, and status flags
• ersb_snap_xid: Transaction identifier of the snapshot used as the rolling baseline
• ersb_current_fext_obj_id: Object identifier of the file extent currently being processed
• ersb_file_offset: Byte offset within the current file
• ersb_progress: Number of blocks processed so far
• ersb_total_blk_to_encrypt: Total blocks that need processing
• ersb_blockmap_oid: Object identifier of the rolling block map
• ersb_tidemark_obj_id: Tracks the boundary between processed and unprocessed data
• ersb_recovery_extents_count: Number of recovery extents for crash recovery
• ersb_recovery_list_oid: Object identifier of the recovery extent list
• ersb_recovery_length: Total length of recovery data in blocks

Encryption Rolling Flags

#define ERSB_FLAG_ENCRYPTING       0x00000001
#define ERSB_FLAG_DECRYPTING       0x00000002
#define ERSB_FLAG_KEYROLLING       0x00000004
#define ERSB_FLAG_PAUSED           0x00000008
#define ERSB_FLAG_FAILED           0x00000010
#define ERSB_FLAG_CID_IS_TWEAK     0x00000020
#define ERSB_FLAG_CM_BLOCK_SIZE_MASK  0x00000F00
#define ERSB_FLAG_CM_BLOCK_SIZE_SHIFT 8
#define ERSB_FLAG_ER_PHASE_MASK    0x00003000
#define ERSB_FLAG_ER_PHASE_SHIFT   12
#define ERSB_FLAG_FROM_ONEKEY      0x00004000

The operation type is one of ENCRYPTING, DECRYPTING, or KEYROLLING (key rolling is not supported in the current implementation). The current phase is extracted from bits 12-13. The checksum block size (bits 8-11) encodes the hardware encryption block size used for integrity checks.

Rolling Window Algorithm

The rolling process operates on a window of file extents at a time:

Pre-Roll Phase

1. Enter a transaction.
2. Lock file extents for the current window.
3. For each extent, compute AES-XTS tweaks and read the unrolled data.
4. Compute SHA-256 checksums for each checksum-block-sized chunk and store truncated hashes in a recovery buffer.
5. Write recovery data to disk so the operation can resume after a crash.
6. Commit the transaction.

Data Roll Phase

1. Encrypt (or decrypt) each extent’s data in-place using XTS-AES.
2. Write the rolled data back to its physical location.
3. If a write fails, retry up to 30 times with a 1-second sleep between attempts. After exhausting retries, mark the operation as failed.

Post-Roll Phase

1. Enter a new transaction.
2. Update the block map to record the new encryption state.
3. For decryption, update file extent records to remove the crypto association.
4. Delete recovery data.
5. Update progress counters.
6. Commit.

Recovery Blocks

Recovery blocks store data needed for crash recovery during the data roll phase. If the system crashes after writing encrypted data but before committing the post-roll transaction, the recovery data allows the operation to be verified and resumed.

typedef struct er_recovery_block_phys {
    obj_phys_t erb_o;       // 0x00
    uint64_t erb_offset;    // 0x20
    oid_t erb_next_oid;     // 0x28
    uint8_t erb_data[];     // 0x30
} er_recovery_block_phys_t;

• erb_o: The object’s header
• erb_offset: Byte offset into the recovery data stream
• erb_next_oid: Object identifier of the next recovery block, or zero
• erb_data: Recovery data payload (checksums of the pre-roll data)

General-Purpose Bitmaps

A general-purpose bitmap tracks per-block rolling state, indicating which blocks have been processed and which remain.

typedef struct gbitmap_phys {
    obj_phys_t bm_o;       // 0x00
    oid_t bm_tree_oid;     // 0x20
    uint64_t bm_bit_count; // 0x28
    uint64_t bm_flags;     // 0x30
} gbitmap_phys_t;          // 0x38

• bm_o: The object’s header
• bm_tree_oid: Object identifier of the B-Tree storing bitmap blocks
• bm_bit_count: Total number of bits in the bitmap
• bm_flags: Reserved (zero)

The bitmap blocks themselves are stored as gbitmap_block_phys_t objects containing the raw bitmap data. Each bit represents one block in the volume; a set bit indicates the block has been rolled.

Forensic Considerations

Encryption rolling state reveals important information:

• A non-zero apfs_er_state_oid in the Volume Superblock indicates an encryption transition was in progress (or interrupted).
• The ersb_progress and ersb_total_blk_to_encrypt fields reveal how far the operation had progressed.
• The ERSB_FLAG_FAILED or ERSB_FLAG_PAUSED flags indicate an interrupted or failed transition.
• During a partial roll, some blocks are encrypted and others are not. The general-purpose bitmap identifies which blocks have been processed, enabling correct decryption of mixed-state volumes.
• The ERSB_FLAG_FROM_ONEKEY flag indicates the volume was previously encrypted with a per-volume key, which affects tweak computation for unrolled blocks.

Conclusion

Encryption rolling provides crash-safe, incremental encryption state transitions for APFS volumes. Its multi-phase design (OMAP, data, snapshots) ensures all volume data is processed, while recovery blocks and general-purpose bitmaps enable correct resumption after interruptions. Understanding this mechanism is essential for forensic analysis of volumes in transitional encryption states.

Overview

The clonegroup tree tracks groups of files that share physical data extents through cloning (e.g., cp --clone or the clonefile syscall). It is a B-Tree with subtype OBJECT_TYPE_CLONEGROUP_TREE, referenced by the apfs_clonegroup_tree_oid field in the Volume Superblock.

Within each clone group, exactly one inode is designated the full clone: it owns the physical data extents shared by the group. All other members are partial clones that reference the full clone’s extents via copy-on-write. When an inode has a INO_EXT_TYPE_CLONEGROUP_ID (type 21) extended field set, it belongs to the clone group identified by that field’s value.

Record Types

The clonegroup tree contains two types of records, distinguished by a record_type field in the key:

On-Disk Structures

Mapping Records (record_type = 1)

Mapping records track which inodes belong to a clone group.

typedef struct clonegroup_mapping_key {
    uint64_t group_id;     // 0x00
    uint8_t record_type;   // 0x08 (always 1)
    uint64_t inode_id;     // 0x09
    uint64_t private_id;   // 0x11
} clonegroup_mapping_key_t; // 0x19 (25 bytes, packed)

• group_id: The clone group identifier
• record_type: Always 1 for mapping records
• inode_id: The inode number of the group member
• private_id: The inode’s data stream identifier (private_id from j_inode_val_t)

Keys are sorted by group_id, then record_type, then inode_id, then private_id.

#define CLONEGROUP_FLAG_FULL_CLONE     0x10
#define CLONEGROUP_FLAG_PURGEABLE_MASK 0x0F

typedef struct clonegroup_val {
    uint64_t physical_size; // 0x00
    uint32_t flags;         // 0x08
    uint8_t xfields[];      // 0x0C
} clonegroup_val_t;

• physical_size: The total physical size in bytes of extents this inode contributes to the group. For the full clone, this equals the on-disk size of all shared extents. For partial clones, this is 0.
• flags: Bit 4 (CLONEGROUP_FLAG_FULL_CLONE) indicates this inode owns the physical extents. Bits 0-3 encode purgeable urgency.
• xfields: Optional extended fields (same format as inode extended fields)

Cookie Records (record_type = 2)

Cookie records signal that a clone group has been reduced to a single member and can be cleaned up.

typedef struct clonegroup_cookie_key {
    uint64_t group_id;    // 0x00
    uint8_t record_type;  // 0x08 (always 2)
    uint64_t cookie;      // 0x09
} clonegroup_cookie_key_t; // 0x11 (17 bytes, packed)

Note that the cookie field in the key is a uint64_t. The cookie record’s value is separate: it is a single byte set to 0. The record’s presence triggers the solo-group cleanup path.

Lifecycle

Group Creation

When a file is first cloned and the clone group does not yet exist:

1. A mapping record is inserted for the source inode with CLONEGROUP_FLAG_FULL_CLONE set and physical_size reflecting its data extent size.
2. INO_EXT_TYPE_CLONEGROUP_ID is set on the source inode.
3. A mapping record is inserted for the clone with physical_size = 0 (partial clone).
4. INO_EXT_TYPE_CLONEGROUP_ID is set on the clone.

Adding Members

Each subsequent clone of any group member gets its own mapping record as a partial clone. The group grows without any data being physically copied.

Full Clone Promotion and Demotion

As clones diverge through copy-on-write, an inode’s relationship to the shared extents changes:

• When an inode that was a partial clone has fully diverged (all its extents are unique), it becomes a full clone of its own data.
• When a full clone is deleted, ownership of the shared physical extents must transfer to another group member.

These transitions are tracked by setting or clearing CLONEGROUP_FLAG_FULL_CLONE and updating physical_size.

Deletion

When a group member is deleted:

1. Its mapping record is removed from the clonegroup tree.
2. If the deleted inode was the full clone, ownership transfers to another member.
3. If only one member remains, a cookie record is inserted to mark the group for cleanup.

Solo Group Cleanup

When a group is reduced to a single member, the clone group tracking overhead is no longer needed. The cleanup process removes the remaining mapping record, the cookie record, and the INO_EXT_TYPE_CLONEGROUP_ID extended field from the surviving inode.

Forensic Considerations

The clonegroup tree provides insight into file relationships that cannot be derived from extent records alone:

• It reveals which files were created by cloning, even after copy-on-write has caused their extents to partially or fully diverge.
• The physical_size field on the full clone indicates how much shared data exists, which is important for accurate disk space accounting.
• Cookie records reveal clone groups that are in the process of being dissolved.
• The group_id links related files that may be spread across different directories, enabling reconstruction of clone relationships.

Conclusion

Clonegroups provide the bookkeeping layer above APFS’s extent-level reference counting. While physical extents track shared blocks, clonegroups track shared relationships between files. This enables efficient space accounting, orderly ownership transfer during deletion, and cleanup when clone groups dissolve.

Overview

A compressed file is identified by the UF_COMPRESSED BSD flag set in its inode record. When this flag is present, the file’s actual data is stored in either an extended attribute named com.apple.decmpfs (for small files) or in the file’s resource fork (for larger files). The kernel transparently decompresses data on read, so applications never see the compressed form.

The decmpfs_disk_header

The com.apple.decmpfs extended attribute begins with a fixed header:

#define DECMPFS_MAGIC 0x636d7066 // 'cmpf'

typedef struct {
    uint32_t compression_magic;  // 0x00
    uint32_t compression_type;   // 0x04
    uint64_t uncompressed_size;  // 0x08
    uint8_t attr_bytes[];        // 0x10
} decmpfs_disk_header;

• compression_magic: Must equal DECMPFS_MAGIC (0x636d7066). All fields are little-endian. The literal 0x636d7066 spells 'cmpf' as a big-endian integer, but stored little-endian it appears on disk as the byte sequence 66 70 6d 63 ('f','p','m','c').
• compression_type: Identifies the compression algorithm and data location (see below)
• uncompressed_size: The original uncompressed file size in bytes (for DATALESS_PKG_CMPFS_TYPE this field is reinterpreted: the low 40 bits hold the package size and the upper bits hold a child-entry count)
• attr_bytes: Inline compressed data (for xattr-stored types), or empty for resource fork types

The maximum size of the entire com.apple.decmpfs extended attribute is 3802 bytes. If the compressed data exceeds this limit, it must be stored in the resource fork.

Compression Types

Odd-numbered types (3, 7, 9, 11, 13) store data inline in the extended attribute. Even-numbered types (4, 8, 10, 12, 14) store data in the resource fork.

Dataless Files

Special compression types represent files whose content is not stored locally:

#define DATALESS_CMPFS_TYPE     0x80000001
#define DATALESS_PKG_CMPFS_TYPE 0x80000002

These are placeholders for iCloud-synced or network-mounted content. The metadata (size, permissions) exists locally, but the data is fetched on demand.

Parsing a Compressed File

1. Check the UF_COMPRESSED flag (bit 5 of bsd_flags in j_inode_val_t).
2. Read the com.apple.decmpfs extended attribute from the File System Tree.
3. Verify compression_magic equals DECMPFS_MAGIC.
4. Read compression_type to determine the algorithm and data location.
5. Locate the compressed data:
   • Inline (types 1, 3, 7, 9, 11, 13): Data follows the header in attr_bytes.
   • Resource fork (types 4, 8, 10, 12, 14): Data is in the com.apple.ResourceFork extended attribute.
6. Decompress using the appropriate algorithm.

Resource Fork Chunking

Resource fork compression types split data into 65,536-byte (64 KB) chunks, each compressed independently. The number of chunks is ceil(uncompressed_size / 65536). Two chunking schemes exist, selected by compression_type alone (not by any field in the resource fork):

Fixed-offset scheme (Type 4, zlib)

The resource fork begins with a 256-byte classic HFS+ resource-fork header whose big-endian data_offset equals 0x104 (struct decmpfs_rsrc_chunk_table_fixed). At fixed resource-fork offset 0x104 a little-endian uint32_t chunk count is stored, and at 0x108 a chunk table follows: one 8-byte entry per chunk, each a [offset, length] pair of little-endian uint32_ts. Each chunk’s offset is relative to 0x104, so chunk i’s compressed bytes start at resource-fork byte 0x104 + offset[i] and run for length[i] bytes. The resource fork ends with a constant 50-byte HFS+ resource-map trailer.

Absolute-offset scheme (Types 8, 10, 12, 14)

There is no resource-fork header, no stored chunk count, and no cmpf resource map. At resource-fork byte 0, an array of num_chunks + 1 little-endian 4-byte absolute offsets begins (struct decmpfs_rsrc_chunk_table_abs), where num_chunks = ceil(uncompressed_size / 65536). Read offsets[i] at byte 4 * i. Chunk i spans [offsets[i], offsets[i+1]), so its compressed length is offsets[i+1] - offsets[i]. The final entry offsets[num_chunks] equals the total resource-fork size.

Stored chunks

In both schemes a chunk that would not shrink under compression is stored uncompressed, flagged by a marker as its first byte. When a chunk’s first byte equals the algorithm’s marker, skip that byte and copy the remaining length - 1 bytes verbatim; otherwise decompress with the type’s algorithm. The markers are zlib/LZFSE/LZBITMAP = 0xFF, LZVN = 0x06, and none (type 10) = 0xCC.

Interaction with APFS

When the kernel hides extended attributes from userland for compressed files:

• com.apple.decmpfs is always hidden
• com.apple.ResourceFork is hidden when it contains compression data

This means forensic tools accessing raw APFS structures will see these attributes, but tools going through the VFS layer will not. The INODE_HAS_UNCOMPRESSED_SIZE flag (0x40000) in internal_flags indicates the inode’s uncompressed_size field is valid.

On sealed volumes, compressed data integrity is verified through the sealed volume’s hash tree. The apfs_verify_uncompressed_data function checks decompressed blocks against expected hashes.

Forensic Considerations

• Transparent compression is extremely common on macOS system volumes. Most files in /System and /usr are compressed.
• The reported file size (in the inode) is the compressed size (allocated extents). The actual size is in uncompressed_size from the decmpfs header or the inode’s extended field.
• Tools that read raw disk data must handle decompression to access file contents.
• The compression type reveals which macOS version created the file: LZVN (10.9+), LZFSE (10.11+), LZBITMAP (macOS 11+).
• Dataless files (types 0x80000001, 0x80000002) indicate cloud-synced content whose data was never stored locally or has been evicted.

Conclusion

DECMPFS provides transparent, per-file compression that is deeply integrated into APFS through extended attributes and resource forks. Understanding the compression types and chunking schemes is essential for any tool that needs to access file contents on APFS volumes, particularly system volumes where compression is the default.

Why Siblings Exist

Traditional Unix file systems track hard links implicitly: an inode has a link count (nlink), and each directory entry pointing to it constitutes a link. There is no built-in way to enumerate all the names of a hard-linked file without scanning the entire file system.

APFS tracks hard links explicitly. Each hard link to an inode is called a sibling and is assigned its own unique identifier. This enables:

• Efficient enumeration of all names for a file
• Bidirectional mapping between sibling identifiers and inodes
• Support for macOS Carbon APIs that require distinguishing between links to the same file
• Proper Spotlight indexing and file coordination across multiple names

The sibling with the lowest identifier is the primary link. The inode’s parent_id and INO_EXT_TYPE_NAME extended field always reflect the primary link’s parent directory and name.

Sibling Link Records

Sibling link records (type APFS_TYPE_SIBLING_LINK) map from an inode to each of its hard links. They are stored in the File System Tree.

typedef struct j_sibling_key {
    j_key_t hdr;          // 0x00
    uint64_t sibling_id;  // 0x08
} j_sibling_key_t;        // 0x10

• hdr: The record’s header. The object identifier is the inode number.
• sibling_id: The sibling’s unique identifier

typedef struct j_sibling_val {
    uint64_t parent_id;  // 0x00
    uint16_t name_len;   // 0x08
    uint8_t name[0];     // 0x0A
} j_sibling_val_t;

• parent_id: The inode number of the parent directory containing this link
• name_len: The length of the name including the null terminator
• name: The null-terminated UTF-8 name of the directory entry

For a file with three hard links, there will be three sibling link records, all sharing the same inode number in their key header but each with a unique sibling_id. Each record stores the parent directory and name for that particular link.

Sibling Map Records

Sibling map records (type APFS_TYPE_SIBLING_MAP) provide the reverse mapping: given a sibling identifier, find the inode.

typedef struct j_sibling_map_key {
    j_key_t hdr; // 0x00
} j_sibling_map_key_t; // 0x08

• hdr: The record’s header. The object identifier is the sibling’s unique identifier.

typedef struct j_sibling_map_val {
    uint64_t file_id; // 0x00
} j_sibling_map_val_t; // 0x08

• file_id: The inode number of the underlying file

This bidirectional mapping (sibling link: inode -> sibling ID + location; sibling map: sibling ID -> inode) allows efficient traversal in either direction.

Sibling Identifier Allocation

Sibling identifiers are allocated from the same object identifier space as inode numbers (from the volume’s next_obj_id counter). Each directory record for a hard-linked file stores its sibling identifier in the DREC_EXT_TYPE_SIBLING_ID extended field, linking the directory entry to its corresponding sibling records.

Operations

When the first hard link is created (the target’s nlink is still 1 and its existing directory entry has no DREC_EXT_TYPE_SIBLING_ID field), the original entry is first promoted to a sibling: a sibling identifier is allocated for it, a DREC_EXT_TYPE_SIBLING_ID field is added to that existing directory entry, and sibling link and map records are created for the original link. The steps below then run for the new link.

When a hard link is created:

1. A new sibling identifier is allocated from next_obj_id for the new link (on the first hard link, a second identifier is also allocated to promote the original entry; see above).
2. A sibling link record is inserted into the File System Tree, keyed by the target inode number and the new sibling ID.
3. A sibling map record is inserted, keyed by the sibling ID, with the target inode as the value.
4. The directory record receives a DREC_EXT_TYPE_SIBLING_ID extended field with the sibling ID.
5. Because sibling identifiers are handed out in increasing order from next_obj_id, a newly created link always has a higher identifier than every existing sibling, so creating a link never changes which sibling is the primary link.

When a hard link is removed:

1. Both the sibling link record and sibling map record are deleted.
2. If the removed link was the primary link, the inode’s metadata is updated to reflect the next-lowest sibling as the new primary.

Hard-Link Fixup at Mount

On volumes where the APFS_FEATURE_HARDLINK_MAP_RECORDS feature flag (bit 1 of apfs_features) is not set, the implementation runs a fixup pass at mount time. This pass iterates all APFS_TYPE_SIBLING_LINK records and ensures a corresponding APFS_TYPE_SIBLING_MAP record exists for each one. Progress is tracked via the fixup-hardlink-progress extended attribute on the root directory (inode 2), which stores the last processed object identifier.

Once fixup completes, APFS_FEATURE_HARDLINK_MAP_RECORDS is set and the progress attribute is removed. This mechanism handles the transition from older APFS implementations that did not maintain sibling map records.

Forensic Considerations

Sibling records are valuable for forensic analysis:

• They allow complete enumeration of all paths to a file without scanning every directory entry on the volume.
• The parent_id in sibling link records reveals which directories contain links to a file, even if some of those directory entries have been deleted or are in snapshots.
• Inconsistencies between sibling records and directory entries (or between sibling link and sibling map records) may indicate tampering or corruption.
• The DREC_EXT_TYPE_SIBLING_ID extended field in directory records provides a cross-reference that can validate the integrity of the sibling mapping.

Conclusion

APFS’s explicit hard link tracking through sibling records distinguishes it from traditional Unix file systems. The bidirectional mapping between inodes and sibling identifiers enables efficient enumeration, correct primary link tracking, and robust support for macOS APIs that distinguish between names of the same file.

Overview

The EFI jumpstart mechanism is intentionally simple. The driver can be located by reading a few data structures starting from physical block zero, without any B-Tree traversal or complex APFS parsing. This minimal dependency means that UEFI firmware (or virtualization software) can load the APFS driver with only basic block-read capability.

The nx_efi_jumpstart field of the NX Superblock stores the physical block address of the jumpstart structure. This field is written during container creation and is not used by the kernel APFS driver during normal operation.

Boot Procedure

To boot from an APFS partition using the embedded EFI driver:

1. Read physical block zero (the container superblock). Verify the Fletcher-64 checksum and confirm nx_magic equals NX_MAGIC ('BSXN').

2. Read the physical block at the address in nx_efi_jumpstart.

3. Verify nej_magic equals NX_EFI_JUMPSTART_MAGIC ('RDSJ'), verify the Fletcher-64 checksum, and confirm nej_version is 1.

4. Allocate a contiguous memory buffer of at least nej_efi_file_len bytes.

5. Read the nej_num_extents extent records from nej_rec_extents and load each extent’s blocks sequentially into the memory buffer.

6. Execute the loaded EFI driver.

nx_efi_jumpstart_t

#define NX_EFI_JUMPSTART_MAGIC 'RDSJ'
#define NX_EFI_JUMPSTART_VERSION 1

typedef struct nx_efi_jumpstart {
    obj_phys_t nej_o;             // 0x00
    uint32_t nej_magic;           // 0x20
    uint32_t nej_version;         // 0x24
    uint32_t nej_efi_file_len;    // 0x28
    uint32_t nej_num_extents;     // 0x2C
    uint64_t nej_reserved[16];    // 0x30
    prange_t nej_rec_extents[];   // 0xB0
} nx_efi_jumpstart_t;

• nej_o: The object header (type OBJECT_TYPE_EFI_JUMPSTART, physical)
• nej_magic: Must equal NX_EFI_JUMPSTART_MAGIC ('RDSJ', on-disk bytes 4A 53 44 52)
• nej_version: Must equal 1
• nej_efi_file_len: The total size of the embedded EFI driver in bytes
• nej_num_extents: The number of physical extent records that follow
• nej_reserved: Reserved (128 bytes, set to zero)
• nej_rec_extents: A variable-length array of prange_t records describing where the EFI driver blocks are stored on disk

Each prange_t in the extent array specifies a starting physical address and a block count:

typedef struct prange {
    paddr_t pr_start_paddr; // 0x00
    uint64_t pr_block_count; // 0x08
} prange_t;                  // 0x10

The extents must be read sequentially and concatenated to assemble the complete driver image.

GPT Partition Type

APFS partitions are identified in the GUID Partition Table by the following type UUID:

7C3457EF-0000-11AA-AA11-00306543ECAC

UEFI firmware uses this UUID to identify partitions that may contain an APFS container with an embedded EFI driver.

Forensic Considerations

The EFI jumpstart structure is useful for forensic validation:

• Its presence and validity confirm that the partition was formatted as APFS (as opposed to being partially overwritten).
• The driver extents reference physical blocks that should be within the container’s bounds. Out-of-range addresses indicate corruption.
• The jumpstart structure is independent of checkpoints. Since it is only written during container creation, it provides a stable reference point that survives checkpoint-level corruption.

Conclusion

The EFI jumpstart mechanism provides a minimal, self-contained boot path for APFS containers. Its simplicity (a single physical object with direct extent references) ensures that UEFI firmware can load the APFS driver without implementing any of the complex B-Tree or checkpoint logic that the rest of APFS requires.

Overview

Each APFS container has exactly one Reaper, stored as an ephemeral object in the checkpoint data area. Its object identifier is recorded in the nx_reaper_oid field of the NX Superblock. The Reaper runs in a dedicated kernel thread with throttled I/O priority, processing entries from a linked list of reap list blocks. When a handler cannot complete its work within a single transaction, it saves its progress in a state buffer and resumes in a new transaction.

nx_reaper_phys_t

The top-level Reaper structure.

typedef struct nx_reaper_phys {
    obj_phys_t nr_o;              // 0x00
    uint64_t nr_next_reap_id;    // 0x20
    uint64_t nr_completed_id;    // 0x28
    oid_t nr_head;               // 0x30
    oid_t nr_tail;               // 0x38
    uint32_t nr_flags;           // 0x40
    uint32_t nr_rlcount;         // 0x44
    uint32_t nr_type;            // 0x48
    uint32_t nr_size;            // 0x4C
    oid_t nr_fs_oid;             // 0x50
    oid_t nr_oid;                // 0x58
    xid_t nr_xid;                // 0x60
    uint32_t nr_nrle_flags;      // 0x68
    uint32_t nr_state_buffer_size; // 0x6C
    uint8_t nr_state_buffer[];   // 0x70
} nx_reaper_phys_t;

• nr_o: The object header (type OBJECT_TYPE_NX_REAPER, ephemeral)
• nr_next_reap_id: The identifier to assign to the next reap operation (initialized to 1)
• nr_completed_id: The identifier of the most recently completed reap operation
• nr_head: Object identifier of the first reap list block (zero if empty)
• nr_tail: Object identifier of the last reap list block (zero if empty)
• nr_flags: Reaper state flags (see below)
• nr_rlcount: Number of reap list blocks in the chain
• nr_type: The object type currently being reaped
• nr_size: Size parameter for the current reap operation
• nr_fs_oid: The volume object identifier associated with the current reap
• nr_oid: Object identifier of the object being reaped (zero when idle)
• nr_xid: Transaction identifier for the current operation
• nr_nrle_flags: Flags from the reap list entry being processed
• nr_state_buffer_size: Size of the state buffer in bytes
• nr_state_buffer: Variable-length buffer for handler progress state

The state buffer allows reap handlers to save their position across transaction boundaries. For a 4096-byte block, this buffer is 3984 bytes.

Reaper Flags

Reap Lists

Reap list blocks form a singly linked list from nr_head to nr_tail. Each block contains an array of entries describing objects to be reaped.

typedef struct nx_reap_list_phys {
    obj_phys_t nrl_o;                // 0x00
    oid_t nrl_next;                  // 0x20
    uint32_t nrl_flags;              // 0x28
    uint32_t nrl_max;                // 0x2C
    uint32_t nrl_count;              // 0x30
    uint32_t nrl_first;             // 0x34
    uint32_t nrl_last;              // 0x38
    uint32_t nrl_free;              // 0x3C
    nx_reap_list_entry_t nrl_entries[]; // 0x40
} nx_reap_list_phys_t;

• nrl_o: The object header (type OBJECT_TYPE_NX_REAP_LIST, ephemeral)
• nrl_next: Object identifier of the next reap list block in the chain, or zero
• nrl_flags: Reserved
• nrl_max: Maximum number of entries (calculated as (block_size - 64) / 40)
• nrl_count: Number of active entries
• nrl_first: Index of the first active entry, or 0xFFFFFFFF if empty
• nrl_last: Index of the last active entry, or 0xFFFFFFFF if empty
• nrl_free: Index of the first free entry slot, or 0xFFFFFFFF if full

Within each block, two linked lists are threaded through the entry array using index chains: an active list of entries awaiting processing and a free list of available slots.

nx_reap_list_entry_t

Each entry describes a single object to be reaped.

typedef struct nx_reap_list_entry {
    uint32_t nrle_next;   // 0x00
    uint32_t nrle_flags;  // 0x04
    uint32_t nrle_type;   // 0x08
    uint32_t nrle_size;   // 0x0C
    oid_t nrle_fs_oid;    // 0x10
    oid_t nrle_oid;       // 0x18
    xid_t nrle_xid;       // 0x20
} nx_reap_list_entry_t;   // 0x28

• nrle_next: Index of the next entry in the chain, or 0xFFFFFFFF
• nrle_flags: Entry flags (see below)
• nrle_type: The object type to reap
• nrle_size: Size parameter for the handler
• nrle_fs_oid: Volume object identifier (zero for container-level objects)
• nrle_oid: Object identifier of the object to reap
• nrle_xid: Transaction or reap identifier

Reap List Entry Flags

When an object is added to the Reaper, two entries are typically appended: a call entry (NRLE_VALID | NRLE_CALL) that triggers the type-specific handler, and a completion entry (NRLE_VALID | NRLE_REAP_ID_RECORD) that updates nr_completed_id when processed. Sub-object entries (such as a volume’s object map during volume deletion) are inserted at the head so they are processed before their parent.

Volume Deletion Phases

The most complex reap operation is deleting an entire volume. This proceeds through a sequence of phases (beginning at APFS_REAP_PHASE_START = 0, which transitions immediately to the snapshot phase), tracked in an apfs_reap_state_t stored in nr_state_buffer:

typedef struct apfs_reap_state {
    uint64_t last_pbn;    // 0x00
    xid_t cur_snap_xid;   // 0x08
    uint32_t phase;       // 0x10
} __attribute__((packed)) apfs_reap_state_t;     // 0x14 (packed, no padding)

• last_pbn: Physical block number where extent reaping last paused
• cur_snap_xid: Transaction identifier of the snapshot currently being reaped
• phase: Current deletion phase (0-4)

Phase 1: APFS_REAP_PHASE_SNAPSHOTS

All snapshots belonging to the volume are reaped. The Reaper iterates through each snapshot’s extent reference tree, freeing physical extents. Progress is tracked by cur_snap_xid. Each snapshot’s extentref tree is then deleted, exactly as in normal snapshot deletion.

Phase 2: APFS_REAP_PHASE_ACTIVE_FS

After all snapshots are gone, the active file system’s extents are freed. The Reaper walks the volume’s extent reference tree and frees all data extents owned by the volume. Progress is tracked by last_pbn. Supplemental trees are also destroyed: the sealed volume’s file extent tree (apfs_fext_tree_oid, present when APFS_INCOMPAT_SEALED_VOLUME is set) and the per-file key upgrade/rotation tree (apfs_pfkur_tree_oid, present when APFS_INCOMPAT_PFK_UPGRADE_ROTATION is set).

Phase 3: APFS_REAP_PHASE_DESTROY_OMAP

The volume’s Object Map is destroyed. This is added to the Reaper as a sub-object, using its own state tracking (omap_reap_state_t). After the OMAP is fully reaped, crypto state, key caches, and the volume’s superblock metadata are cleared.

typedef struct omap_reap_state {
    uint32_t omr_phase;  // 0x00
    uint32_t omr_pad;    // 0x04
    omap_key_t omr_ok;   // 0x08
} omap_reap_state_t;     // 0x18

• omr_phase: Current phase (OMAP_REAP_PHASE_MAP_TREE = 1, OMAP_REAP_PHASE_SNAPSHOT_TREE = 2)
• omr_ok: The last freed key, used to resume iteration after a transaction boundary

Phase 4: APFS_REAP_PHASE_DONE

All volume structures have been freed. The reap operation is complete.

Crash Recovery

The Reaper’s design guarantees crash-safe resumption. If the system crashes mid-reap:

1. The Reaper’s ephemeral object is restored from the checkpoint. Since nr_oid is nonzero and NR_CONTINUE is set in nr_flags, the Reaper knows to resume.
2. On the next mount, the Reaper thread starts and enters a transaction.
3. Since nr_oid is already set, it skips entry dequeue and goes directly to handler dispatch.
4. The handler reads its saved state from nr_state_buffer and resumes where it left off.

This ensures that even multi-transaction deletions spanning many checkpoints will always complete, regardless of how many crashes occur during the process.

Forensic Considerations

The Reaper is forensically significant because:

• Partially reaped volumes may still contain recoverable data. The phase field in the reap state indicates how far deletion has progressed. Data in phases not yet reached may be fully intact.
• The reap list reveals which objects are pending deletion. A volume that appears missing from the container’s nx_fs_oid array may still exist in the Reaper’s queue.
• The nr_completed_id and nr_next_reap_id fields provide a history of how many reap operations have occurred, giving insight into container activity.
• Free queue entries from reaper-freed blocks retain their transaction identifiers, indicating when deletion occurred.

Conclusion

The Reaper provides crash-safe, multi-transaction garbage collection for APFS. Its state machine design allows arbitrarily large deletions (entire volumes, snapshot cleanup, object map destruction) to proceed incrementally across as many transactions as needed, with guaranteed resumption after crashes. Combined with the Space Manager’s free queues, it ensures that block deallocation is always consistent and recoverable.

What’s new

• QSPICE .prot decryption. SpiceCrypt now decrypts QSPICE protected sub-circuits: randomized base-16 encoding, a seed-keyed dual stream cipher, DEFLATE decompression, and Windows-1252 detokenization. Surrounding plaintext lines pass through untouched. The full reverse-engineered scheme is documented in SPECIFICATIONS/qspice.md.
• Unified auto-detection. decrypt_stream() and decrypt() now auto-detect across Binary File, PSpice, QSPICE, and LTspice formats. Point SpiceCrypt at a file and it picks the right scheme.
• New public API. QSpiceFileParser and QSpiceCipher are now exported for callers that want to work with QSPICE directly.
• Block-count reporting. Since a single file can hold many protected sub-circuits, decrypt_stream() now returns the block count for QSPICE inputs.
• Graceful degradation. A protected block that fails to decode now passes through unchanged with a warning instead of aborting the whole file.

Breaking changes

The deprecated v2.0.0 backward-compatibility shims have been removed: the top-level des.py, binary_file.py, and crypto_state.py modules are gone. Import the LTspice internals directly instead:

from spice_crypt.ltspice import ...

The CLI and the primary decrypt / decrypt_stream entry points are unchanged.

Upgrading

pip install --upgrade spice-crypt

Or with uv:

uv tool install --upgrade spice-crypt

Links

• Repository: github.com/jtsylve/spice-crypt
• PyPI: pypi.org/project/spice-crypt
• QSPICE specification: SPECIFICATIONS/qspice.md

If you run into issues or have feature requests, please open an issue on GitHub.

Disclaimer: SpiceCrypt is intended solely for enabling simulator interoperability with lawfully obtained models. Using it to violate intellectual property rights is immoral and is not an acceptable use of the tool.

Overview

Each APFS container has exactly one Space Manager, stored as an ephemeral object in the checkpoint data area. Its object identifier is recorded in the nx_spaceman_oid field of the NX Superblock. The Space Manager tracks block allocation using a three-tier hierarchy: the top-level spaceman_phys_t structure contains per-device metadata, which references Chunk Address Blocks (CABs) or Chunk Info Blocks (CIBs) directly, which in turn reference individual allocation bitmaps.

Chunks and Bitmaps

The Space Manager divides each storage device into fixed-size chunks. Each chunk is a contiguous range of blocks tracked by a single allocation bitmap. The number of blocks per chunk is stored in sm_blocks_per_chunk.

typedef struct chunk_info {
    uint64_t ci_xid;         // 0x00
    uint64_t ci_addr;        // 0x08
    uint32_t ci_block_count; // 0x10
    uint32_t ci_free_count;  // 0x14
    paddr_t ci_bitmap_addr;  // 0x18
} chunk_info_t;              // 0x20

• ci_xid: The transaction identifier of the last transaction that modified this chunk’s bitmap
• ci_addr: The first block address of this chunk
• ci_block_count: The number of blocks in this chunk. The count occupies the lower 20 bits (CI_COUNT_MASK, 0x000fffff); the upper 12 bits (CI_COUNT_RESERVED_MASK, 0xfff00000) are reserved and zero on disk.
• ci_free_count: The number of free blocks in this chunk. As with ci_block_count, the count is the lower 20 bits (CI_COUNT_MASK) and the upper 12 bits (CI_COUNT_RESERVED_MASK) are reserved and zero on disk.
• ci_bitmap_addr: The physical address of the allocation bitmap for this chunk, or zero if no bitmap has been allocated

The allocator does track two per-chunk markers, one for chunks reserved to the metazone (pinned-to-main) and one for chunks currently assigned to an allocation zone, but these are runtime in-memory bookkeeping. They have no on-disk representation in chunk_info_t and never appear in the count fields.

Chunk Info Blocks and Chunk Address Blocks

Chunk info structures are grouped into Chunk Info Blocks (CIBs), physical objects that each hold an array of chunk_info_t entries.

typedef struct chunk_info_block {
    obj_phys_t cib_o;              // 0x00
    uint32_t cib_index;            // 0x20
    uint32_t cib_chunk_info_count; // 0x24
    chunk_info_t cib_chunk_info[]; // 0x28
} chunk_info_block_t;

• cib_o: The object header (type OBJECT_TYPE_SPACEMAN_CIB)
• cib_index: The index of this CIB within its device’s array
• cib_chunk_info_count: The number of chunk info entries in this block
• cib_chunk_info: A variable-length array of chunk info structures

For large containers where the number of CIBs exceeds what can be stored directly in the Space Manager, a second level of indirection is used: Chunk Address Blocks (CABs).

typedef struct cib_addr_block {
    obj_phys_t cab_o;       // 0x00
    uint32_t cab_index;     // 0x20
    uint32_t cab_cib_count; // 0x24
    paddr_t cab_cib_addr[]; // 0x28
} cib_addr_block_t;

• cab_o: The object header (type OBJECT_TYPE_SPACEMAN_CAB)
• cab_index: The index of this CAB within its device’s array
• cab_cib_count: The number of CIB addresses stored in this block
• cab_cib_addr: A variable-length array of physical CIB addresses

When sm_cab_count in the device structure is zero, CIB addresses are stored directly in the Space Manager. When nonzero, the CAB indirection layer is present.

Free Queues

When blocks are freed, they are not immediately returned to the allocation bitmaps. Instead, they are placed into free queues: B-Trees that hold recently freed extents until all transactions that might reference them have been checkpointed. This ensures crash-safe deallocation.

APFS maintains three free queues:

typedef struct spaceman_free_queue {
    uint64_t sfq_count;           // 0x00
    oid_t sfq_tree_oid;           // 0x08
    xid_t sfq_oldest_xid;         // 0x10
    uint16_t sfq_tree_node_limit; // 0x18
    uint16_t sfq_pad16;           // 0x1A
    uint32_t sfq_pad32;           // 0x1C
    uint64_t sfq_reserved;        // 0x20
} spaceman_free_queue_t;          // 0x28

• sfq_count: The number of entries in this free queue
• sfq_tree_oid: The object identifier of the B-Tree that stores the entries, or zero if not yet created
• sfq_oldest_xid: The oldest transaction identifier among all entries
• sfq_tree_node_limit: When the B-Tree node count exceeds this limit, the queue is drained more aggressively

Free Queue Entries

Free queue entries use a key that sorts first by transaction identifier, then by physical address:

typedef struct spaceman_free_queue_key {
    xid_t sfqk_xid;          // 0x00
    paddr_t sfqk_paddr;      // 0x08
} spaceman_free_queue_key_t; // 0x10

The value is a uint64_t block count. Single-block extents store a zero-length value in the B-Tree to save space (the count of 1 is implied).

When inserting entries, the implementation coalesces adjacent extents that share the same transaction identifier, reducing B-Tree size and improving drain efficiency.

Internal Pool

The Internal Pool (IP) is a dedicated set of blocks used for allocating B-Tree nodes and other metadata structures. It provides a reserved area that guarantees metadata allocations can succeed even when the container is nearly full. The IP has its own allocation bitmaps, separate from the per-chunk bitmaps used for data.

Key fields in spaceman_phys_t:

• sm_ip_base: The physical base address of the internal pool blocks
• sm_ip_block_count: The total number of blocks in the pool (bit 63 is a fragmentation flag)
• sm_ip_bm_base: The physical base address of the IP bitmap blocks
• sm_ip_bm_block_count: The number of IP bitmap blocks (bit 31 is a fragmentation flag)
• sm_ip_bm_size_in_blocks: The number of bitmap blocks needed to cover the pool
• sm_ip_bm_tx_multiplier: The number of bitmaps per transaction (at least 4)

When the fragmentation flag is set (bit 63 of sm_ip_block_count or bit 31 of sm_ip_bm_block_count), the pool blocks or bitmaps are not contiguous. Their physical addresses must be looked up through a Metadata Fragmented Extent List Tree rather than computed from the base address.

Allocation Zones

APFS uses allocation zones to group related allocations together on disk, reducing fragmentation and improving sequential read performance. Each device has up to 8 allocation zones (SM_DATAZONE_ALLOCZONE_COUNT), with zone IDs 1 through 4 corresponding to minimum allocation sizes in blocks.

typedef struct spaceman_allocation_zone_info_phys {
    spaceman_allocation_zone_boundaries_t saz_current_boundaries;
    spaceman_allocation_zone_boundaries_t saz_previous_boundaries[7];
    uint16_t saz_zone_id;
    uint16_t saz_previous_boundary_index;
    uint32_t saz_reserved;
} spaceman_allocation_zone_info_phys_t;

• saz_current_boundaries: The current start and end block addresses of this zone
• saz_previous_boundaries: A circular buffer of the 7 most recent previous chunk assignments
• saz_zone_id: The allocation size class (1-4 blocks, or 0 for unused)
• saz_previous_boundary_index: Index into the circular buffer for the next rotation

Each allocation zone boundary is a simple range:

typedef struct spaceman_allocation_zone_boundaries {
    uint64_t saz_zone_start; // 0x00
    uint64_t saz_zone_end;   // 0x08
} spaceman_allocation_zone_boundaries_t;

When an allocation zone’s current chunk becomes full, the allocator scans for a new chunk with sufficient free space, rotates the old boundaries into the circular buffer, and updates the current boundaries. Which chunk is currently assigned to a zone is tracked by a runtime in-memory allocation-zone marker with no on-disk representation; the persistent zone state lives in saz_current_boundaries and saz_previous_boundaries.

Metazone

The metazone is a contiguous region at the beginning of each device reserved exclusively for metadata allocation. Data allocations must not use metazone blocks. This separation ensures that metadata structures (B-Tree nodes, Space Manager bitmaps) are clustered together near the start of the device for efficient access.

The metazone size scales with device capacity:

• Devices smaller than approximately 6 GB have no metazone
• Devices smaller than 16 GB use a 512 MB metazone
• Larger devices use a tiered formula that allocates progressively smaller fractions as device size increases, capped at one-quarter of the total device size

Metazone membership is a contiguous region derived from device geometry, not a per-chunk on-disk flag. At runtime the allocator marks chunks reserved to the metazone as pinned-to-main and excludes them from data allocation zones, but that marker is in-memory state with no on-disk representation in chunk_info_t.

spaceman_phys_t

The top-level structure tying everything together:

typedef struct spaceman_phys {
    obj_phys_t sm_o;
    uint32_t sm_block_size;
    uint32_t sm_blocks_per_chunk;
    uint32_t sm_chunks_per_cib;
    uint32_t sm_cibs_per_cab;
    spaceman_device_t sm_dev[SD_COUNT];
    uint32_t sm_flags;
    uint32_t sm_ip_bm_tx_multiplier;
    uint64_t sm_ip_block_count;
    uint32_t sm_ip_bm_size_in_blocks;
    uint32_t sm_ip_bm_block_count;
    paddr_t sm_ip_bm_base;
    paddr_t sm_ip_base;
    uint64_t sm_fs_reserve_block_count;
    uint64_t sm_fs_reserve_alloc_count;
    spaceman_free_queue_t sm_fq[SFQ_COUNT];
    uint16_t sm_ip_bm_free_head;
    uint16_t sm_ip_bm_free_tail;
    uint32_t sm_ip_bm_xid_offset;
    uint32_t sm_ip_bitmap_offset;
    uint32_t sm_ip_bm_free_next_offset;
    uint32_t sm_version;
    uint32_t sm_struct_size;
    spaceman_datazone_info_phys_t sm_datazone;
    // Variable-length arrays follow...
} spaceman_phys_t;

The structure is followed by variable-length arrays: IP bitmap XID arrays, IP bitmap offset arrays, IP bitmap free-next arrays, and CIB/CAB address arrays for each device. The total on-disk size must fit within one block.

Each device is described by a spaceman_device_t:

typedef struct spaceman_device {
    uint64_t sm_block_count;  // 0x00
    uint64_t sm_chunk_count;  // 0x08
    uint32_t sm_cib_count;    // 0x10
    uint32_t sm_cab_count;    // 0x14
    uint64_t sm_free_count;   // 0x18
    uint32_t sm_addr_offset;  // 0x20
    uint32_t sm_reserved;     // 0x24
    uint64_t sm_reserved2;    // 0x28
} spaceman_device_t;          // 0x30

• sm_block_count: Total blocks on this device
• sm_chunk_count: Number of chunks
• sm_cib_count: Number of CIBs
• sm_cab_count: Number of CABs (zero if CIBs are stored directly)
• sm_free_count: Total free blocks on this device
• sm_addr_offset: Byte offset within spaceman_phys_t where the CIB/CAB address array begins

Forensic Considerations

The Space Manager is particularly valuable for forensic analysis:

• Free queue entries identify blocks that were recently freed but may still contain recoverable data. The transaction identifier on each entry indicates when the block was freed.
• Allocation bitmaps reveal which blocks are currently in use versus free, which can be cross-referenced against file extent records to find orphaned data.
• Chunk info transaction identifiers (ci_xid) indicate when each chunk’s allocation state last changed, providing a coarse timeline of write activity across the disk.
• Allocation zones reveal where the file system tends to place related data, which can help reconstruct file system activity patterns.

Conclusion

The Space Manager implements a sophisticated hierarchical allocation system that balances performance, fragmentation avoidance, and crash safety. Its three-tier structure (CABs, CIBs, bitmaps) scales from tiny containers to multi-terabyte devices. Free queues ensure safe deallocation across transactions, while allocation zones and the metazone organize blocks for optimal access patterns.