Overview

The clonegroup tree tracks groups of files that share physical data extents through cloning (e.g., cp --clone or the clonefile syscall). It is a B-Tree with subtype OBJECT_TYPE_CLONEGROUP_TREE, referenced by the apfs_clonegroup_tree_oid field in the Volume Superblock.

Within each clone group, exactly one inode is designated the full clone: it owns the physical data extents shared by the group. All other members are partial clones that reference the full clone’s extents via copy-on-write. When an inode has a INO_EXT_TYPE_CLONEGROUP_ID (type 21) extended field set, it belongs to the clone group identified by that field’s value.

Record Types

The clonegroup tree contains two types of records, distinguished by a record_type field in the key:

On-Disk Structures

Mapping Records (record_type = 1)

Mapping records track which inodes belong to a clone group.

typedef struct clonegroup_mapping_key {
    uint64_t group_id;     // 0x00
    uint8_t record_type;   // 0x08 (always 1)
    uint64_t inode_id;     // 0x09
    uint64_t private_id;   // 0x11
} clonegroup_mapping_key_t; // 0x19 (25 bytes, packed)

• group_id: The clone group identifier
• record_type: Always 1 for mapping records
• inode_id: The inode number of the group member
• private_id: The inode’s data stream identifier (private_id from j_inode_val_t)

Keys are sorted by group_id, then record_type, then inode_id, then private_id.

#define CLONEGROUP_FLAG_FULL_CLONE     0x10
#define CLONEGROUP_FLAG_PURGEABLE_MASK 0x0F

typedef struct clonegroup_val {
    uint64_t physical_size; // 0x00
    uint32_t flags;         // 0x08
    uint8_t xfields[];      // 0x0C
} clonegroup_val_t;

• physical_size: The total physical size in bytes of extents this inode contributes to the group. For the full clone, this equals the on-disk size of all shared extents. For partial clones, this is 0.
• flags: Bit 4 (CLONEGROUP_FLAG_FULL_CLONE) indicates this inode owns the physical extents. Bits 0-3 encode purgeable urgency.
• xfields: Optional extended fields (same format as inode extended fields)

Cookie Records (record_type = 2)

Cookie records signal that a clone group has been reduced to a single member and can be cleaned up.

typedef struct clonegroup_cookie_key {
    uint64_t group_id;    // 0x00
    uint8_t record_type;  // 0x08 (always 2)
    uint64_t cookie;      // 0x09
} clonegroup_cookie_key_t; // 0x11 (17 bytes, packed)

The cookie value is a single byte set to 0. Its presence triggers the solo-group cleanup path.

Lifecycle

Group Creation

When a file is first cloned and the clone group does not yet exist:

1. A mapping record is inserted for the source inode with CLONEGROUP_FLAG_FULL_CLONE set and physical_size reflecting its data extent size.
2. INO_EXT_TYPE_CLONEGROUP_ID is set on the source inode.
3. A mapping record is inserted for the clone with physical_size = 0 (partial clone).
4. INO_EXT_TYPE_CLONEGROUP_ID is set on the clone.

Adding Members

Each subsequent clone of any group member gets its own mapping record as a partial clone. The group grows without any data being physically copied.

Full Clone Promotion and Demotion

As clones diverge through copy-on-write, an inode’s relationship to the shared extents changes:

• When an inode that was a partial clone has fully diverged (all its extents are unique), it becomes a full clone of its own data.
• When a full clone is deleted, ownership of the shared physical extents must transfer to another group member.

These transitions are tracked by setting or clearing CLONEGROUP_FLAG_FULL_CLONE and updating physical_size.

Deletion

When a group member is deleted:

1. Its mapping record is removed from the clonegroup tree.
2. If the deleted inode was the full clone, ownership transfers to another member.
3. If only one member remains, a cookie record is inserted to mark the group for cleanup.

Solo Group Cleanup

When a group is reduced to a single member, the clone group tracking overhead is no longer needed. The cleanup process removes the remaining mapping record, the cookie record, and the INO_EXT_TYPE_CLONEGROUP_ID extended field from the surviving inode.

Forensic Considerations

The clonegroup tree provides insight into file relationships that cannot be derived from extent records alone:

• It reveals which files were created by cloning, even after copy-on-write has caused their extents to partially or fully diverge.
• The physical_size field on the full clone indicates how much shared data exists, which is important for accurate disk space accounting.
• Cookie records reveal clone groups that are in the process of being dissolved.
• The group_id links related files that may be spread across different directories, enabling reconstruction of clone relationships.

Conclusion

Clonegroups provide the bookkeeping layer above APFS’s extent-level reference counting. While physical extents track shared blocks, clonegroups track shared relationships between files. This enables efficient space accounting, orderly ownership transfer during deletion, and cleanup when clone groups dissolve.

Overview

A compressed file is identified by the UF_COMPRESSED BSD flag set in its inode record. When this flag is present, the file’s actual data is stored in either an extended attribute named com.apple.decmpfs (for small files) or in the file’s resource fork (for larger files). The kernel transparently decompresses data on read, so applications never see the compressed form.

The decmpfs_disk_header

The com.apple.decmpfs extended attribute begins with a fixed header:

#define DECMPFS_MAGIC 0x636d7066 // 'cmpf'

typedef struct {
    uint32_t compression_magic;  // 0x00
    uint32_t compression_type;   // 0x04
    uint64_t uncompressed_size;  // 0x08
    uint8_t attr_bytes[];        // 0x10
} decmpfs_disk_header;

• compression_magic: Must equal DECMPFS_MAGIC (0x636d7066). All fields are little-endian.
• compression_type: Identifies the compression algorithm and data location (see below)
• uncompressed_size: The original uncompressed file size in bytes (for DATALESS_PKG_CMPFS_TYPE this field is reinterpreted: the low 40 bits hold the package size and the upper bits hold a child-entry count)
• attr_bytes: Inline compressed data (for xattr-stored types), or empty for resource fork types

The maximum size of the entire com.apple.decmpfs extended attribute is 3802 bytes. If the compressed data exceeds this limit, it must be stored in the resource fork.

Compression Types

Odd-numbered types (3, 7, 9, 11, 13) store data inline in the extended attribute. Even-numbered types (4, 8, 10, 12, 14) store data in the resource fork.

Dataless Files

Special compression types represent files whose content is not stored locally:

#define DATALESS_CMPFS_TYPE     0x80000001
#define DATALESS_PKG_CMPFS_TYPE 0x80000002

These are placeholders for iCloud-synced or network-mounted content. The metadata (size, permissions) exists locally, but the data is fetched on demand.

Parsing a Compressed File

1. Check the UF_COMPRESSED flag (bit 5 of bsd_flags in j_inode_val_t).
2. Read the com.apple.decmpfs extended attribute from the File System Tree.
3. Verify compression_magic equals DECMPFS_MAGIC.
4. Read compression_type to determine the algorithm and data location.
5. Locate the compressed data:
   • Inline (types 1, 3, 7, 9, 11, 13): Data follows the header in attr_bytes.
   • Resource fork (types 4, 8, 10, 12, 14): Data is in the com.apple.ResourceFork extended attribute.
6. Decompress using the appropriate algorithm.

Resource Fork Chunking

Resource fork compression types split data into 65,536-byte (64 KB) chunks. Two chunking schemes exist:

Scheme v1 (Type 4, zlib)

The resource fork data section begins with a chunk table: an array of uint32_t offsets, one per chunk plus a trailing entry. The compressed size of chunk i is offsets[i+1] - offsets[i].

Scheme v2 (Types 8, 10, 12, 14)

The resource fork contains a resource map with type 'cmpf' (0x636D7066). At offset 260, a uint32_t chunk count is stored. Starting at offset 264, each chunk is described by an 8-byte entry containing a uint32_t offset and uint32_t size.

Interaction with APFS

When the kernel hides extended attributes from userland for compressed files:

• com.apple.decmpfs is always hidden
• com.apple.ResourceFork is hidden when it contains compression data

This means forensic tools accessing raw APFS structures will see these attributes, but tools going through the VFS layer will not. The INODE_HAS_UNCOMPRESSED_SIZE flag (0x40000) in internal_flags indicates the inode’s uncompressed_size field is valid.

On sealed volumes, compressed data integrity is verified through the sealed volume’s hash tree. The apfs_verify_uncompressed_data function checks decompressed blocks against expected hashes.

Forensic Considerations

• Transparent compression is extremely common on macOS system volumes. Most files in /System and /usr are compressed.
• The reported file size (in the inode) is the compressed size (allocated extents). The actual size is in uncompressed_size from the decmpfs header or the inode’s extended field.
• Tools that read raw disk data must handle decompression to access file contents.
• The compression type reveals which macOS version created the file: LZVN (10.9+), LZFSE (10.11+), LZBITMAP (macOS 11+).
• Dataless files (types 0x80000001, 0x80000002) indicate cloud-synced content whose data was never stored locally or has been evicted.

Conclusion

DECMPFS provides transparent, per-file compression that is deeply integrated into APFS through extended attributes and resource forks. Understanding the compression types and chunking schemes is essential for any tool that needs to access file contents on APFS volumes, particularly system volumes where compression is the default.

Why Siblings Exist

Traditional Unix file systems track hard links implicitly: an inode has a link count (nlink), and each directory entry pointing to it constitutes a link. There is no built-in way to enumerate all the names of a hard-linked file without scanning the entire file system.

APFS tracks hard links explicitly. Each hard link to an inode is called a sibling and is assigned its own unique identifier. This enables:

• Efficient enumeration of all names for a file
• Bidirectional mapping between sibling identifiers and inodes
• Support for macOS Carbon APIs that require distinguishing between links to the same file
• Proper Spotlight indexing and file coordination across multiple names

The sibling with the lowest identifier is the primary link. The inode’s parent_id and INO_EXT_TYPE_NAME extended field always reflect the primary link’s parent directory and name.

Sibling Link Records

Sibling link records (type APFS_TYPE_SIBLING_LINK) map from an inode to each of its hard links. They are stored in the File System Tree.

typedef struct j_sibling_key {
    j_key_t hdr;          // 0x00
    uint64_t sibling_id;  // 0x08
} j_sibling_key_t;        // 0x10

• hdr: The record’s header. The object identifier is the inode number.
• sibling_id: The sibling’s unique identifier

typedef struct j_sibling_val {
    uint64_t parent_id;  // 0x00
    uint16_t name_len;   // 0x08
    uint8_t name[0];     // 0x0A
} j_sibling_val_t;

• parent_id: The inode number of the parent directory containing this link
• name_len: The length of the name including the null terminator
• name: The null-terminated UTF-8 name of the directory entry

For a file with three hard links, there will be three sibling link records, all sharing the same inode number in their key header but each with a unique sibling_id. Each record stores the parent directory and name for that particular link.

Sibling Map Records

Sibling map records (type APFS_TYPE_SIBLING_MAP) provide the reverse mapping: given a sibling identifier, find the inode.

typedef struct j_sibling_map_key {
    j_key_t hdr; // 0x00
} j_sibling_map_key_t; // 0x08

• hdr: The record’s header. The object identifier is the sibling’s unique identifier.

typedef struct j_sibling_map_val {
    uint64_t file_id; // 0x00
} j_sibling_map_val_t; // 0x08

• file_id: The inode number of the underlying file

This bidirectional mapping (sibling link: inode -> sibling ID + location; sibling map: sibling ID -> inode) allows efficient traversal in either direction.

Sibling Identifier Allocation

Sibling identifiers are allocated from the same object identifier space as inode numbers (from the volume’s next_obj_id counter). Each directory record for a hard-linked file stores its sibling identifier in the DREC_EXT_TYPE_SIBLING_ID extended field, linking the directory entry to its corresponding sibling records.

Operations

When the first hard link is created (the target’s nlink is still 1 and its existing directory entry has no DREC_EXT_TYPE_SIBLING_ID field), the original entry is first promoted to a sibling: a sibling identifier is allocated for it, a DREC_EXT_TYPE_SIBLING_ID field is added to that existing directory entry, and sibling link and map records are created for the original link. The steps below then run for the new link.

When a hard link is created:

1. A new sibling identifier is allocated from next_obj_id for the new link (on the first hard link, a second identifier is also allocated to promote the original entry; see above).
2. A sibling link record is inserted into the File System Tree, keyed by the target inode number and the new sibling ID.
3. A sibling map record is inserted, keyed by the sibling ID, with the target inode as the value.
4. The directory record receives a DREC_EXT_TYPE_SIBLING_ID extended field with the sibling ID.
5. Because sibling identifiers are handed out in increasing order from next_obj_id, a newly created link always has a higher identifier than every existing sibling, so creating a link never changes which sibling is the primary link.

When a hard link is removed:

1. Both the sibling link record and sibling map record are deleted.
2. If the removed link was the primary link, the inode’s metadata is updated to reflect the next-lowest sibling as the new primary.

Hard-Link Fixup at Mount

On volumes where the APFS_FEATURE_HARDLINK_MAP_RECORDS feature flag (bit 1 of apfs_features) is not set, the implementation runs a fixup pass at mount time. This pass iterates all APFS_TYPE_SIBLING_LINK records and ensures a corresponding APFS_TYPE_SIBLING_MAP record exists for each one. Progress is tracked via the fixup-hardlink-progress extended attribute on the root directory (inode 2), which stores the last processed object identifier.

Once fixup completes, APFS_FEATURE_HARDLINK_MAP_RECORDS is set and the progress attribute is removed. This mechanism handles the transition from older APFS implementations that did not maintain sibling map records.

Forensic Considerations

Sibling records are valuable for forensic analysis:

• They allow complete enumeration of all paths to a file without scanning every directory entry on the volume.
• The parent_id in sibling link records reveals which directories contain links to a file, even if some of those directory entries have been deleted or are in snapshots.
• Inconsistencies between sibling records and directory entries (or between sibling link and sibling map records) may indicate tampering or corruption.
• The DREC_EXT_TYPE_SIBLING_ID extended field in directory records provides a cross-reference that can validate the integrity of the sibling mapping.

Conclusion

APFS’s explicit hard link tracking through sibling records distinguishes it from traditional Unix file systems. The bidirectional mapping between inodes and sibling identifiers enables efficient enumeration, correct primary link tracking, and robust support for macOS APIs that distinguish between names of the same file.

Overview

The EFI jumpstart mechanism is intentionally simple. The driver can be located by reading a few data structures starting from physical block zero, without any B-Tree traversal or complex APFS parsing. This minimal dependency means that UEFI firmware (or virtualization software) can load the APFS driver with only basic block-read capability.

The nx_efi_jumpstart field of the NX Superblock stores the physical block address of the jumpstart structure. This field is written during container creation and is not used by the kernel APFS driver during normal operation.

Boot Procedure

To boot from an APFS partition using the embedded EFI driver:

1. Read physical block zero (the container superblock). Verify the Fletcher-64 checksum and confirm nx_magic equals NX_MAGIC ('BSXN').

2. Read the physical block at the address in nx_efi_jumpstart.

3. Verify nej_magic equals NX_EFI_JUMPSTART_MAGIC ('RDSJ'), verify the Fletcher-64 checksum, and confirm nej_version is 1.

4. Allocate a contiguous memory buffer of at least nej_efi_file_len bytes.

5. Read the nej_num_extents extent records from nej_rec_extents and load each extent’s blocks sequentially into the memory buffer.

6. Execute the loaded EFI driver.

nx_efi_jumpstart_t

#define NX_EFI_JUMPSTART_MAGIC 'RDSJ'
#define NX_EFI_JUMPSTART_VERSION 1

typedef struct nx_efi_jumpstart {
    obj_phys_t nej_o;             // 0x00
    uint32_t nej_magic;           // 0x20
    uint32_t nej_version;         // 0x24
    uint32_t nej_efi_file_len;    // 0x28
    uint32_t nej_num_extents;     // 0x2C
    uint64_t nej_reserved[16];    // 0x30
    prange_t nej_rec_extents[];   // 0xB0
} nx_efi_jumpstart_t;

• nej_o: The object header (type OBJECT_TYPE_EFI_JUMPSTART, physical)
• nej_magic: Must equal NX_EFI_JUMPSTART_MAGIC ('RDSJ', on-disk bytes 4A 53 44 52)
• nej_version: Must equal 1
• nej_efi_file_len: The total size of the embedded EFI driver in bytes
• nej_num_extents: The number of physical extent records that follow
• nej_reserved: Reserved (128 bytes, set to zero)
• nej_rec_extents: A variable-length array of prange_t records describing where the EFI driver blocks are stored on disk

Each prange_t in the extent array specifies a starting physical address and a block count:

typedef struct prange {
    paddr_t pr_start_paddr; // 0x00
    uint64_t pr_block_count; // 0x08
} prange_t;                  // 0x10

The extents must be read sequentially and concatenated to assemble the complete driver image.

GPT Partition Type

APFS partitions are identified in the GUID Partition Table by the following type UUID:

7C3457EF-0000-11AA-AA11-00306543ECAC

UEFI firmware uses this UUID to identify partitions that may contain an APFS container with an embedded EFI driver.

Forensic Considerations

The EFI jumpstart structure is useful for forensic validation:

• Its presence and validity confirm that the partition was formatted as APFS (as opposed to being partially overwritten).
• The driver extents reference physical blocks that should be within the container’s bounds. Out-of-range addresses indicate corruption.
• The jumpstart structure is independent of checkpoints. Since it is only written during container creation, it provides a stable reference point that survives checkpoint-level corruption.

Conclusion

The EFI jumpstart mechanism provides a minimal, self-contained boot path for APFS containers. Its simplicity (a single physical object with direct extent references) ensures that UEFI firmware can load the APFS driver without implementing any of the complex B-Tree or checkpoint logic that the rest of APFS requires.

Overview

Each APFS container has exactly one Reaper, stored as an ephemeral object in the checkpoint data area. Its object identifier is recorded in the nx_reaper_oid field of the NX Superblock. The Reaper runs in a dedicated kernel thread with throttled I/O priority, processing entries from a linked list of reap list blocks. When a handler cannot complete its work within a single transaction, it saves its progress in a state buffer and resumes in a new transaction.

nx_reaper_phys_t

The top-level Reaper structure.

typedef struct nx_reaper_phys {
    obj_phys_t nr_o;              // 0x00
    uint64_t nr_next_reap_id;    // 0x20
    uint64_t nr_completed_id;    // 0x28
    oid_t nr_head;               // 0x30
    oid_t nr_tail;               // 0x38
    uint32_t nr_flags;           // 0x40
    uint32_t nr_rlcount;         // 0x44
    uint32_t nr_type;            // 0x48
    uint32_t nr_size;            // 0x4C
    oid_t nr_fs_oid;             // 0x50
    oid_t nr_oid;                // 0x58
    xid_t nr_xid;                // 0x60
    uint32_t nr_nrle_flags;      // 0x68
    uint32_t nr_state_buffer_size; // 0x6C
    uint8_t nr_state_buffer[];   // 0x70
} nx_reaper_phys_t;

• nr_o: The object header (type OBJECT_TYPE_NX_REAPER, ephemeral)
• nr_next_reap_id: The identifier to assign to the next reap operation (initialized to 1)
• nr_completed_id: The identifier of the most recently completed reap operation
• nr_head: Object identifier of the first reap list block (zero if empty)
• nr_tail: Object identifier of the last reap list block (zero if empty)
• nr_flags: Reaper state flags (see below)
• nr_rlcount: Number of reap list blocks in the chain
• nr_type: The object type currently being reaped
• nr_size: Size parameter for the current reap operation
• nr_fs_oid: The volume object identifier associated with the current reap
• nr_oid: Object identifier of the object being reaped (zero when idle)
• nr_xid: Transaction identifier for the current operation
• nr_nrle_flags: Flags from the reap list entry being processed
• nr_state_buffer_size: Size of the state buffer in bytes
• nr_state_buffer: Variable-length buffer for handler progress state

The state buffer allows reap handlers to save their position across transaction boundaries. For a 4096-byte block, this buffer is 3984 bytes.

Reaper Flags

Reap Lists

Reap list blocks form a singly linked list from nr_head to nr_tail. Each block contains an array of entries describing objects to be reaped.

typedef struct nx_reap_list_phys {
    obj_phys_t nrl_o;                // 0x00
    oid_t nrl_next;                  // 0x20
    uint32_t nrl_flags;              // 0x28
    uint32_t nrl_max;                // 0x2C
    uint32_t nrl_count;              // 0x30
    uint32_t nrl_first;             // 0x34
    uint32_t nrl_last;              // 0x38
    uint32_t nrl_free;              // 0x3C
    nx_reap_list_entry_t nrl_entries[]; // 0x40
} nx_reap_list_phys_t;

• nrl_o: The object header (type OBJECT_TYPE_NX_REAP_LIST, ephemeral)
• nrl_next: Object identifier of the next reap list block in the chain, or zero
• nrl_flags: Reserved
• nrl_max: Maximum number of entries (calculated as (block_size - 64) / 40)
• nrl_count: Number of active entries
• nrl_first: Index of the first active entry, or 0xFFFFFFFF if empty
• nrl_last: Index of the last active entry, or 0xFFFFFFFF if empty
• nrl_free: Index of the first free entry slot, or 0xFFFFFFFF if full

Within each block, two linked lists are threaded through the entry array using index chains: an active list of entries awaiting processing and a free list of available slots.

nx_reap_list_entry_t

Each entry describes a single object to be reaped.

typedef struct nx_reap_list_entry {
    uint32_t nrle_next;   // 0x00
    uint32_t nrle_flags;  // 0x04
    uint32_t nrle_type;   // 0x08
    uint32_t nrle_size;   // 0x0C
    oid_t nrle_fs_oid;    // 0x10
    oid_t nrle_oid;       // 0x18
    xid_t nrle_xid;       // 0x20
} nx_reap_list_entry_t;   // 0x28

• nrle_next: Index of the next entry in the chain, or 0xFFFFFFFF
• nrle_flags: Entry flags (see below)
• nrle_type: The object type to reap
• nrle_size: Size parameter for the handler
• nrle_fs_oid: Volume object identifier (zero for container-level objects)
• nrle_oid: Object identifier of the object to reap
• nrle_xid: Transaction or reap identifier

Reap List Entry Flags

When an object is added to the Reaper, two entries are typically appended: a call entry (NRLE_VALID | NRLE_CALL) that triggers the type-specific handler, and a completion entry (NRLE_VALID | NRLE_REAP_ID_RECORD) that updates nr_completed_id when processed. Sub-object entries (such as a volume’s object map during volume deletion) are inserted at the head so they are processed before their parent.

Volume Deletion Phases

The most complex reap operation is deleting an entire volume. This proceeds through a sequence of phases (beginning at APFS_REAP_PHASE_START = 0, which transitions immediately to the snapshot phase), tracked in an apfs_reap_state_t stored in nr_state_buffer:

typedef struct apfs_reap_state {
    uint64_t last_pbn;    // 0x00
    xid_t cur_snap_xid;   // 0x08
    uint32_t phase;       // 0x10
} apfs_reap_state_t;     // 0x14

• last_pbn: Physical block number where extent reaping last paused
• cur_snap_xid: Transaction identifier of the snapshot currently being reaped
• phase: Current deletion phase (0-4)

Phase 1: APFS_REAP_PHASE_SNAPSHOTS

All snapshots belonging to the volume are reaped. The Reaper iterates through each snapshot’s extent reference tree, freeing physical extents. Progress is tracked by cur_snap_xid. Each snapshot’s extentref tree is then deleted, exactly as in normal snapshot deletion.

Phase 2: APFS_REAP_PHASE_ACTIVE_FS

After all snapshots are gone, the active file system’s extents are freed. The Reaper walks the volume’s extent reference tree and frees all data extents owned by the volume. Progress is tracked by last_pbn. Supplemental trees are also destroyed: the sealed volume’s file extent tree (apfs_fext_tree_oid, present when APFS_INCOMPAT_SEALED_VOLUME is set) and the per-file key upgrade/rotation tree (apfs_pfkur_tree_oid, present when APFS_INCOMPAT_PFK_UPGRADE_ROTATION is set).

Phase 3: APFS_REAP_PHASE_DESTROY_OMAP

The volume’s Object Map is destroyed. This is added to the Reaper as a sub-object, using its own state tracking (omap_reap_state_t). After the OMAP is fully reaped, crypto state, key caches, and the volume’s superblock metadata are cleared.

typedef struct omap_reap_state {
    uint32_t omr_phase;  // 0x00
    uint32_t omr_pad;    // 0x04
    omap_key_t omr_ok;   // 0x08
} omap_reap_state_t;     // 0x18

• omr_phase: Current phase (OMAP_REAP_PHASE_MAP_TREE = 1, OMAP_REAP_PHASE_SNAPSHOT_TREE = 2)
• omr_ok: The last freed key, used to resume iteration after a transaction boundary

Phase 4: APFS_REAP_PHASE_DONE

All volume structures have been freed. The reap operation is complete.

Crash Recovery

The Reaper’s design guarantees crash-safe resumption. If the system crashes mid-reap:

1. The Reaper’s ephemeral object is restored from the checkpoint. Since nr_oid is nonzero and NR_CONTINUE is set in nr_flags, the Reaper knows to resume.
2. On the next mount, the Reaper thread starts and enters a transaction.
3. Since nr_oid is already set, it skips entry dequeue and goes directly to handler dispatch.
4. The handler reads its saved state from nr_state_buffer and resumes where it left off.

This ensures that even multi-transaction deletions spanning many checkpoints will always complete, regardless of how many crashes occur during the process.

Forensic Considerations

The Reaper is forensically significant because:

• Partially reaped volumes may still contain recoverable data. The phase field in the reap state indicates how far deletion has progressed. Data in phases not yet reached may be fully intact.
• The reap list reveals which objects are pending deletion. A volume that appears missing from the container’s nx_fs_oid array may still exist in the Reaper’s queue.
• The nr_completed_id and nr_next_reap_id fields provide a history of how many reap operations have occurred, giving insight into container activity.
• Free queue entries from reaper-freed blocks retain their transaction identifiers, indicating when deletion occurred.

Conclusion

The Reaper provides crash-safe, multi-transaction garbage collection for APFS. Its state machine design allows arbitrarily large deletions (entire volumes, snapshot cleanup, object map destruction) to proceed incrementally across as many transactions as needed, with guaranteed resumption after crashes. Combined with the Space Manager’s free queues, it ensures that block deallocation is always consistent and recoverable.

What’s new

• QSPICE .prot decryption. SpiceCrypt now decrypts QSPICE protected sub-circuits: randomized base-16 encoding, a seed-keyed dual stream cipher, DEFLATE decompression, and Windows-1252 detokenization. Surrounding plaintext lines pass through untouched. The full reverse-engineered scheme is documented in SPECIFICATIONS/qspice.md.
• Unified auto-detection. decrypt_stream() and decrypt() now auto-detect across Binary File, PSpice, QSPICE, and LTspice formats. Point SpiceCrypt at a file and it picks the right scheme.
• New public API. QSpiceFileParser and QSpiceCipher are now exported for callers that want to work with QSPICE directly.
• Block-count reporting. Since a single file can hold many protected sub-circuits, decrypt_stream() now returns the block count for QSPICE inputs.
• Graceful degradation. A protected block that fails to decode now passes through unchanged with a warning instead of aborting the whole file.

Breaking changes

The deprecated v2.0.0 backward-compatibility shims have been removed: the top-level des.py, binary_file.py, and crypto_state.py modules are gone. Import the LTspice internals directly instead:

from spice_crypt.ltspice import ...

The CLI and the primary decrypt / decrypt_stream entry points are unchanged.

Upgrading

pip install --upgrade spice-crypt

Or with uv:

uv tool install --upgrade spice-crypt

Links

• Repository: github.com/jtsylve/spice-crypt
• PyPI: pypi.org/project/spice-crypt
• QSPICE specification: SPECIFICATIONS/qspice.md

If you run into issues or have feature requests, please open an issue on GitHub.

Disclaimer: SpiceCrypt is intended solely for enabling simulator interoperability with lawfully obtained models. Using it to violate intellectual property rights is immoral and is not an acceptable use of the tool.

Overview

Each APFS container has exactly one Space Manager, stored as an ephemeral object in the checkpoint data area. Its object identifier is recorded in the nx_spaceman_oid field of the NX Superblock. The Space Manager tracks block allocation using a three-tier hierarchy: the top-level spaceman_phys_t structure contains per-device metadata, which references Chunk Address Blocks (CABs) or Chunk Info Blocks (CIBs) directly, which in turn reference individual allocation bitmaps.

Chunks and Bitmaps

The Space Manager divides each storage device into fixed-size chunks. Each chunk is a contiguous range of blocks tracked by a single allocation bitmap. The number of blocks per chunk is stored in sm_blocks_per_chunk.

typedef struct chunk_info {
    uint64_t ci_xid;         // 0x00
    uint64_t ci_addr;        // 0x08
    uint32_t ci_block_count; // 0x10
    uint32_t ci_free_count;  // 0x14
    paddr_t ci_bitmap_addr;  // 0x18
} chunk_info_t;              // 0x20

• ci_xid: The transaction identifier of the last transaction that modified this chunk’s bitmap
• ci_addr: The first block address of this chunk
• ci_block_count: The number of blocks in this chunk (lower 20 bits). Upper 12 bits hold flags (see below).
• ci_free_count: The number of free blocks in this chunk (lower 20 bits)
• ci_bitmap_addr: The physical address of the allocation bitmap for this chunk, or zero if no bitmap has been allocated

Chunk Info Flags

Chunk Info Blocks and Chunk Address Blocks

Chunk info structures are grouped into Chunk Info Blocks (CIBs), physical objects that each hold an array of chunk_info_t entries.

typedef struct chunk_info_block {
    obj_phys_t cib_o;              // 0x00
    uint32_t cib_index;            // 0x20
    uint32_t cib_chunk_info_count; // 0x24
    chunk_info_t cib_chunk_info[]; // 0x28
} chunk_info_block_t;

• cib_o: The object header (type OBJECT_TYPE_SPACEMAN_CIB)
• cib_index: The index of this CIB within its device’s array
• cib_chunk_info_count: The number of chunk info entries in this block
• cib_chunk_info: A variable-length array of chunk info structures

For large containers where the number of CIBs exceeds what can be stored directly in the Space Manager, a second level of indirection is used: Chunk Address Blocks (CABs).

typedef struct cib_addr_block {
    obj_phys_t cab_o;       // 0x00
    uint32_t cab_index;     // 0x20
    uint32_t cab_cib_count; // 0x24
    paddr_t cab_cib_addr[]; // 0x28
} cib_addr_block_t;

• cab_o: The object header (type OBJECT_TYPE_SPACEMAN_CAB)
• cab_index: The index of this CAB within its device’s array
• cab_cib_count: The number of CIB addresses stored in this block
• cab_cib_addr: A variable-length array of physical CIB addresses

When sm_cab_count in the device structure is zero, CIB addresses are stored directly in the Space Manager. When nonzero, the CAB indirection layer is present.

Free Queues

When blocks are freed, they are not immediately returned to the allocation bitmaps. Instead, they are placed into free queues: B-Trees that hold recently freed extents until all transactions that might reference them have been checkpointed. This ensures crash-safe deallocation.

APFS maintains three free queues:

typedef struct spaceman_free_queue {
    uint64_t sfq_count;           // 0x00
    oid_t sfq_tree_oid;           // 0x08
    xid_t sfq_oldest_xid;         // 0x10
    uint16_t sfq_tree_node_limit; // 0x18
    uint16_t sfq_pad16;           // 0x1A
    uint32_t sfq_pad32;           // 0x1C
    uint64_t sfq_reserved;        // 0x20
} spaceman_free_queue_t;          // 0x28

• sfq_count: The number of entries in this free queue
• sfq_tree_oid: The object identifier of the B-Tree that stores the entries, or zero if not yet created
• sfq_oldest_xid: The oldest transaction identifier among all entries
• sfq_tree_node_limit: When the B-Tree node count exceeds this limit, the queue is drained more aggressively

Free Queue Entries

Free queue entries use a key that sorts first by transaction identifier, then by physical address:

typedef struct spaceman_free_queue_key {
    xid_t sfqk_xid;          // 0x00
    paddr_t sfqk_paddr;      // 0x08
} spaceman_free_queue_key_t; // 0x10

The value is a uint64_t block count. Single-block extents store a zero-length value in the B-Tree to save space (the count of 1 is implied).

When inserting entries, the implementation coalesces adjacent extents that share the same transaction identifier, reducing B-Tree size and improving drain efficiency.

Internal Pool

The Internal Pool (IP) is a dedicated set of blocks used for allocating B-Tree nodes and other metadata structures. It provides a reserved area that guarantees metadata allocations can succeed even when the container is nearly full. The IP has its own allocation bitmaps, separate from the per-chunk bitmaps used for data.

Key fields in spaceman_phys_t:

• sm_ip_base: The physical base address of the internal pool blocks
• sm_ip_block_count: The total number of blocks in the pool (bit 63 is a fragmentation flag)
• sm_ip_bm_base: The physical base address of the IP bitmap blocks
• sm_ip_bm_block_count: The number of IP bitmap blocks (bit 31 is a fragmentation flag)
• sm_ip_bm_size_in_blocks: The number of bitmap blocks needed to cover the pool
• sm_ip_bm_tx_multiplier: The number of bitmaps per transaction (at least 4)

When the fragmentation flag is set (bit 63 of sm_ip_block_count or bit 31 of sm_ip_bm_block_count), the pool blocks or bitmaps are not contiguous. Their physical addresses must be looked up through a Metadata Fragmented Extent List Tree rather than computed from the base address.

Allocation Zones

APFS uses allocation zones to group related allocations together on disk, reducing fragmentation and improving sequential read performance. Each device has up to 8 allocation zones (SM_DATAZONE_ALLOCZONE_COUNT), with zone IDs 1 through 4 corresponding to minimum allocation sizes in blocks.

typedef struct spaceman_allocation_zone_info_phys {
    spaceman_allocation_zone_boundaries_t saz_current_boundaries;
    spaceman_allocation_zone_boundaries_t saz_previous_boundaries[7];
    uint16_t saz_zone_id;
    uint16_t saz_previous_boundary_index;
    uint32_t saz_reserved;
} spaceman_allocation_zone_info_phys_t;

• saz_current_boundaries: The current start and end block addresses of this zone
• saz_previous_boundaries: A circular buffer of the 7 most recent previous chunk assignments
• saz_zone_id: The allocation size class (1-4 blocks, or 0 for unused)
• saz_previous_boundary_index: Index into the circular buffer for the next rotation

Each allocation zone boundary is a simple range:

typedef struct spaceman_allocation_zone_boundaries {
    uint64_t saz_zone_start; // 0x00
    uint64_t saz_zone_end;   // 0x08
} spaceman_allocation_zone_boundaries_t;

When an allocation zone’s current chunk becomes full, the allocator scans for a new chunk with sufficient free space, rotates the old boundaries into the circular buffer, and updates the current boundaries. The CI_ALLOC_ZONE_HINT flag on chunks tracks which chunk is currently assigned to a zone.

Metazone

The metazone is a contiguous region at the beginning of each device reserved exclusively for metadata allocation. Data allocations must not use metazone blocks. This separation ensures that metadata structures (B-Tree nodes, Space Manager bitmaps) are clustered together near the start of the device for efficient access.

The metazone size scales with device capacity:

• Devices smaller than approximately 6 GB have no metazone
• Devices smaller than 16 GB use a 512 MB metazone
• Larger devices use a tiered formula that allocates progressively smaller fractions as device size increases, capped at one-quarter of the total device size

Chunks within the metazone are marked with the CI_PINNED_TO_MAIN flag and are excluded from data allocation zones.

spaceman_phys_t

The top-level structure tying everything together:

typedef struct spaceman_phys {
    obj_phys_t sm_o;
    uint32_t sm_block_size;
    uint32_t sm_blocks_per_chunk;
    uint32_t sm_chunks_per_cib;
    uint32_t sm_cibs_per_cab;
    spaceman_device_t sm_dev[SD_COUNT];
    uint32_t sm_flags;
    uint32_t sm_ip_bm_tx_multiplier;
    uint64_t sm_ip_block_count;
    uint32_t sm_ip_bm_size_in_blocks;
    uint32_t sm_ip_bm_block_count;
    paddr_t sm_ip_bm_base;
    paddr_t sm_ip_base;
    uint64_t sm_fs_reserve_block_count;
    uint64_t sm_fs_reserve_alloc_count;
    spaceman_free_queue_t sm_fq[SFQ_COUNT];
    uint16_t sm_ip_bm_free_head;
    uint16_t sm_ip_bm_free_tail;
    uint32_t sm_ip_bm_xid_offset;
    uint32_t sm_ip_bitmap_offset;
    uint32_t sm_ip_bm_free_next_offset;
    uint32_t sm_version;
    uint32_t sm_struct_size;
    spaceman_datazone_info_phys_t sm_datazone;
    // Variable-length arrays follow...
} spaceman_phys_t;

The structure is followed by variable-length arrays: IP bitmap XID arrays, IP bitmap offset arrays, IP bitmap free-next arrays, and CIB/CAB address arrays for each device. The total on-disk size must fit within one block.

Each device is described by a spaceman_device_t:

typedef struct spaceman_device {
    uint64_t sm_block_count;  // 0x00
    uint64_t sm_chunk_count;  // 0x08
    uint32_t sm_cib_count;    // 0x10
    uint32_t sm_cab_count;    // 0x14
    uint64_t sm_free_count;   // 0x18
    uint32_t sm_addr_offset;  // 0x20
    uint32_t sm_reserved;     // 0x24
    uint64_t sm_reserved2;    // 0x28
} spaceman_device_t;          // 0x30

• sm_block_count: Total blocks on this device
• sm_chunk_count: Number of chunks
• sm_cib_count: Number of CIBs
• sm_cab_count: Number of CABs (zero if CIBs are stored directly)
• sm_free_count: Total free blocks on this device
• sm_addr_offset: Byte offset within spaceman_phys_t where the CIB/CAB address array begins

Forensic Considerations

The Space Manager is particularly valuable for forensic analysis:

• Free queue entries identify blocks that were recently freed but may still contain recoverable data. The transaction identifier on each entry indicates when the block was freed.
• Allocation bitmaps reveal which blocks are currently in use versus free, which can be cross-referenced against file extent records to find orphaned data.
• Chunk info transaction identifiers (ci_xid) indicate when each chunk’s allocation state last changed, providing a coarse timeline of write activity across the disk.
• Allocation zones reveal where the file system tends to place related data, which can help reconstruct file system activity patterns.

Conclusion

The Space Manager implements a sophisticated hierarchical allocation system that balances performance, fragmentation avoidance, and crash safety. Its three-tier structure (CABs, CIBs, bitmaps) scales from tiny containers to multi-terabyte devices. Free queues ensure safe deallocation across transactions, while allocation zones and the metazone organize blocks for optimal access patterns.

What’s changed since 2022

APFS is not a frozen target. It has continued to evolve across macOS releases, picking up new on-disk features and quietly refining old ones. The structures I documented in December 2022 were accurate then, but a lot has shifted underneath them since.

My own approach has changed too. The original posts were written largely day-by-day, under the self-imposed pressure of an advent calendar. Since then I’ve built far better tooling for this kind of work, and I now validate each structure directly against the current implementation rather than against memory and notes. That has let me correct a few details, sharpen explanations that were fuzzier than I’d like, and add depth in places where the original posts only scratched the surface.

Living references, not snapshots

The most important change is one of intent. I want this series to be something you can actually rely on, not a museum piece dated December 2022.

So rather than leaving the original posts untouched and bolting corrections onto the end, I’ve revised them in place. Each one now reflects current APFS instead of a four-year-old snapshot. Posts that were updated carry an “Updated” date in their byline so you can see at a glance what has been touched. The permalinks haven’t moved, so any links or bookmarks you already have will keep working.

New parts, coming over the next two weeks

The original run also left real gaps. Some of the container’s internal machinery never got covered, and several features either postdate the 2022 series or simply didn’t make the cut when I ran out of December. Over the next two weeks I’ll be publishing new entries to round the series out:

• Space Manager: how APFS tracks free and allocated blocks
• The Reaper: crash-safe, multi-transaction garbage collection
• EFI Jumpstart: booting from an APFS container
• Hard Links and Siblings: the sibling-link records behind hard links
• Transparent Compression (DECMPFS): inline and resource-fork compression
• Clonegroups: tracking copy-on-write clones
• Encryption Rolling: re-encrypting a volume in place
• Volume Grafting: overlaying volumes for system updates
• Speculative Telemetry: tracking speculatively downloaded content

That brings the series to 27 parts spanning the container layer, B-Trees, the volume and file-system layer, integrity and encryption, and APFS’s more advanced features.

Read the series

The series index lays out the full planned structure. Parts that haven’t published yet are marked “Coming Soon” and will light up as they go live over the coming days. If you read the series back in 2022, it’s worth a second look. If you’re coming to it fresh, there’s never been a better time.

2.0 added a supervisor/worker architecture for analyzing multiple binaries simultaneously. 2.1 introduced progressive tool discovery so the LLM could find specialized tools on demand instead of loading ~195 schemas at startup. 2.2 added meta-tools that let the LLM write multi-step analysis scripts, issue bulk operations, and persist state across sessions through a daemon.

Each release solved a real friction point. But that progression revealed something about the interface itself. The tools the LLM actually calls (decompile this function, get cross-references to that address, rename this symbol, search for strings matching this pattern) described reverse engineering in the abstract, not IDA in particular. IDA was the engine behind those tools, but the tool surface itself was generic. An LLM asking to decompile main doesn’t care whether the answer comes from Hex-Rays or Ghidra’s decompiler. It cares about the pseudocode.

That realization is why ida-mcp is now re-mcp (reverse engineering MCP). Version 3.0 ships with a full Ghidra backend alongside the existing IDA Pro backend, with a shared tool interface that makes LLM workflows portable across both.

Why Ghidra matters here

The most common response I heard after publishing ida-mcp was some variation of “this looks great, but I don’t have an IDA license.” IDA Pro is the industry standard for binary analysis, but it costs thousands of dollars per seat. For students, independent researchers, CTF players, and hobbyists, that puts LLM-driven reverse engineering out of reach before it even starts.

Ghidra, released by the NSA as open source in 2019, has become the primary free alternative. It supports dozens of processor architectures, its decompiler is capable, and it has an active community building extensions and loaders. By adding Ghidra as a backend, re-mcp makes everything from 2.0 through 2.2 (multi-database analysis, progressive tool discovery, execute scripts, batch operations) available to anyone willing to install a free tool and a JDK.

Getting started with Ghidra

The Ghidra backend requires Python 3.12+, Ghidra 12+, and JDK 21+. Ghidra’s install path is found automatically from the GHIDRA_INSTALL_DIR environment variable or platform-specific default locations.

uv tool install re-mcp-ghidra

Then configure your MCP client:

{
  "mcpServers": {
    "ghidra": {
      "command": "uvx",
      "args": ["re-mcp-ghidra"]
    }
  }
}

From there, everything works the way it did with IDA. Open a binary, wait for analysis to complete, and start asking questions.

The meta-tools from 2.2 work on the Ghidra backend too. Here’s an execute script that finds functions referencing error strings and summarizes them:

strings = await invoke("find_code_by_string", {
    "pattern": "invalid|error|fail", "limit": 50
})
seen = set()
results = []
for hit in strings["items"]:
    fn = hit.get("function_name", "")
    if not fn or fn in seen:
        continue
    seen.add(fn)
    decomp = await invoke("decompile_function", {
        "address": hit["function_address"]
    })
    results.append({
        "function": decomp["function_name"],
        "address": decomp["address"],
        "matched_string": hit["string_value"],
        "lines": len(decomp["decompiled_code"].splitlines())
    })
return {"functions_with_error_strings": results}

One tool call. The LLM gets back every function that references an error string, with its decompiled size, ready for triage. The same workflow pattern from the 2.2 post applies here (the only difference being response field names like decompiled_code vs. pseudocode).

Comparing engines

There’s a practical reason to support both backends even if you already have an IDA license. IDA and Ghidra have different analysis engines, different heuristics for function boundary detection, different type propagation strategies. Running the same binary through both and comparing the output is a common practice in professional reverse engineering; each tool catches things the other misses.

With re-mcp, you configure both servers, and the LLM can open the same binary in each and compare function lists, decompiler output, and cross-references across the two.

One interface, two engines

Both backends implement the same core tool interface: identical tool names, identical parameters, and the same categories of information in responses (though individual field names in responses may differ slightly between engines). From a user’s perspective, it doesn’t matter which engine is running: the LLM issues the same tool calls and returns comparable results either way.

The shared surface covers the operations that define a reverse engineering session:

• Functions: list, decompile, disassemble, rename, set prototypes
• Navigation: cross-references (to and from), imports, exports, entry points, names
• Search: strings with regex filtering, byte patterns, immediate values
• Types: local type libraries, structures, enums, type application
• Annotation: comments, names, bookmarks
• Patching: byte-level modification, segment operations
• Meta-tools: search_tools, get_schema, call, execute, batch

An execute script that crawls error strings, decompiles referencing functions, and renames them follows the same logic on either engine; scripts only need to adjust for the field name differences noted above.

Each backend also retains capabilities specific to its engine. The IDA backend keeps everything from the 2.x releases: IDAPython scripting via run_script, file region mapping, executable rebuilding, IDC evaluation, and the eight guided prompts for structured analysis workflows. The Ghidra backend brings its own strengths: Function ID for automatic library function identification and data type archive support.

Architecture and transport

re-mcp is a monorepo with three packages: re-mcp-core (supervisor, transport, meta-tools), re-mcp-ida (IDA Pro backend wrapping idalib), and re-mcp-ghidra (Ghidra backend wrapping pyghidra). The core package doesn’t depend on IDA or Ghidra. Backends are discovered through Python entry points, so you install only what you need:

# IDA users
uv tool install re-mcp-ida

# Ghidra users
uv tool install re-mcp-ghidra

# Both
uv tool install re-mcp --with re-mcp-ida --with re-mcp-ghidra

Future backends (Binary Ninja, radare2, or something that doesn’t exist yet) would slot in as additional packages implementing the same worker interface, with no changes to the core or any existing backend.

re-mcp 3.0 switches the default transport to direct stdio: one session, workers terminate on disconnect. This is simpler to set up than the HTTP daemon that ida-mcp 2.2 defaulted to, and it works universally with every MCP client. For workflows that need persistence, the daemon is still available via proxy or serve subcommands (e.g., re-mcp-ghidra serve, re-mcp-ida serve). The transport mode is independent of the backend; all options work the same for re-mcp-ida, re-mcp-ghidra, and the unified re-mcp --backend <name> command.

Migrating from ida-mcp

The legacy ida-mcp PyPI package now redirects to re-mcp-ida. Existing installations continue to work after upgrading:

uv tool install --upgrade ida-mcp
# or install directly
uv tool install re-mcp-ida

The MCP tool interface is backward compatible. Existing execute scripts, batch operations, and direct tool calls work without changes. Requirements are unchanged: IDA Pro 9+ with Python 3.12+. The main visible difference is the entry point name (ida-mcp becomes re-mcp-ida), though the old name continues to work as an alias.

Environment variables follow the same pattern as before, prefixed per backend. IDA_MCP_ variables carry over unchanged for the IDA backend; the Ghidra backend uses GHIDRA_MCP_ with the same suffixes.

Links

• Repository: github.com/jtsylve/re-mcp
• PyPI: re-mcp-ida · re-mcp-ghidra · re-mcp

If you run into issues or have feature requests, please open an issue on GitHub.

IDA Pro and Hex-Rays are trademarks of Hex-Rays SA. Ghidra is developed by the National Security Agency. re-mcp is an independent project and is not affiliated with or endorsed by Hex-Rays or the NSA.

In 2.1, each action was a discrete tool call: decompile this function, get cross-references to that address, rename this symbol. Every step was a full MCP round trip. Every intermediate result landed in the context window. An analysis workflow that a human would express as a ten-line IDAPython script became thirty sequential tool calls, each waiting for the previous one to return before the LLM could decide what to do next. The LLM knew what it wanted to do, but it couldn’t say it all at once.

2.2 introduces meta-tools that let the LLM operate at a higher level of abstraction: writing multi-step analysis scripts, issuing bulk operations, and calling tools it discovers at runtime. It also makes the server persistent, so analysis state survives across sessions. And for the first time, ida-mcp can analyze firmware and raw binaries directly.

Meta-tools

execute: sandboxed analysis scripts

execute accepts Python code that calls IDA tools through await invoke(name, params), with full control flow: loops, conditionals, regex, struct unpacking, list comprehensions. Individual tools are still the right choice for simple operations, but for multi-step analysis, the LLM becomes a script writer.

Consider a common reverse engineering task: finding every function that references an error string and understanding how each one handles the error. In 2.1, this was a multi-step conversation:

1. Call get_strings with a filter → get back 40 matching strings
2. Call get_xrefs_to for the first string address → get back 3 cross-references
3. Call decompile_function for each referencing function → get back pseudocode
4. Repeat steps 2–3 for each of the remaining 39 strings

That’s potentially 160+ tool calls, each a full round trip, with the LLM holding intermediate addresses in context between calls. If the context window fills up mid-workflow, earlier results get compacted and the LLM loses track of where it was.

With execute, the same workflow is a single tool call:

strings = await invoke("get_strings", {"filter": "error|fail|panic"})
results = []
for s in strings["strings"]:
    xrefs = await invoke("get_xrefs_to", {"address": s["address"]})
    for xref in xrefs["xrefs"]:
        decomp = await invoke("decompile_function", {"address": xref["from"]})
        results.append({
            "string": s["value"],
            "function": decomp["name"],
            "pseudocode": decomp["pseudocode"]
        })
return results

One round trip. The LLM gets back a structured result containing every error-handling function with its decompiled pseudocode. No intermediate state to track, no context window spent on addresses it only needed temporarily. And if the LLM decides the approach is wrong, it’s only wasted one tool call finding out.

Any “get a list, then process each item” workflow collapses from O(n) tool calls to one. The bigger gain is for workflows that don’t reduce to sequential calls: conditional logic, data transformation, or cross-referencing between results.

Automated renaming based on string references:

A stripped binary might have thousands of sub_* functions with no meaningful names, but many of them reference string literals that hint at their purpose. A human analyst would scan through decompiled output, spot a string like "failed to parse header", and rename the function accordingly. With execute, the LLM can do this systematically across the entire binary in a single tool call:

import re

funcs = await invoke("list_functions", {"filter": "sub_"})
renamed = []
for func in funcs["functions"]:
    decomp = await invoke("decompile_function", {"address": func["address"]})
    strings = re.findall(r'"([^"]{4,})"', decomp["pseudocode"])
    if strings:
        candidate = re.sub(r'[^a-zA-Z0-9_]', '_', strings[0])[:40]
        await invoke("rename_function", {
            "address": func["address"],
            "new_name": f"uses_{candidate}"
        })
        renamed.append({"old": func["name"], "new": f"uses_{candidate}"})
return {"renamed": len(renamed), "functions": renamed}

The names this generates are rough: a first pass rather than a final answer. But uses_failed_to_parse_header is vastly more useful than sub_140001A30 when you’re trying to understand a binary’s structure, and the LLM can refine them in a second pass once it understands the broader architecture.

Cross-database patch diffing:

Patch analysis requires comparing function lists between two versions of a library, identifying what was added or removed, and diffing the implementations that exist in both. Without execute, the LLM would pull function lists from each database in separate tool calls, hold both in context, compute set differences itself, and decompile changed functions one at a time. Dozens of round trips, large intermediate results sitting in context.

With execute, the entire triage happens server-side:

old_funcs = await invoke("list_functions", {"database": "libcrypto_1.1.1"})
new_funcs = await invoke("list_functions", {"database": "libcrypto_1.1.2"})

old_names = {f["name"] for f in old_funcs["functions"]}
new_names = {f["name"] for f in new_funcs["functions"]}

added = sorted(new_names - old_names)
removed = sorted(old_names - new_names)

# Spot-check shared functions for implementation changes
changed = []
for name in sorted(old_names & new_names)[:30]:
    old_dec = await invoke("decompile_function", {
        "address": name, "database": "libcrypto_1.1.1"
    })
    new_dec = await invoke("decompile_function", {
        "address": name, "database": "libcrypto_1.1.2"
    })
    if old_dec["pseudocode"] != new_dec["pseudocode"]:
        changed.append(name)

return {
    "added": added[:50],
    "removed": removed[:50],
    "changed": changed,
    "summary": {
        "added": len(added),
        "removed": len(removed),
        "shared_checked": min(30, len(old_names & new_names)),
        "shared_changed": len(changed)
    }
}

The database parameter override lets a single execute block work across multiple open databases. Each invoke call can target a different database by name. The LLM gets back a structured summary of what changed between versions, and can then drill into specific changed functions in follow-up calls. The set operations, sorting, and conditional comparison all happen server-side rather than burning context on intermediate data the LLM only needs to pass through.

The sandbox

The code runs in a RestrictedPython sandbox. The LLM can import re, struct, json, math, collections, itertools, functools, and a few other safe standard library modules. It cannot access the filesystem, open network connections, or spawn subprocesses. Attribute access to dunder names (__class__, __globals__, __code__) is blocked at the AST level, closing Python sandbox escape hatches. Print output is capped at ~1 MiB to prevent runaway loops from exhausting worker memory.

Database lifecycle tools (open_database, close_database, wait_for_analysis) are blocked inside the sandbox; an execute block shouldn’t be spawning or tearing down workers as a side effect. The meta-tools themselves (execute, batch, call) are also blocked to prevent recursion. Everything else (decompilation, disassembly, renaming, commenting, type manipulation, structure editing) is available through await invoke().

A failed invoke call raises a Python exception that the script can catch with try/except, or that terminates the block with an error message if uncaught.

If the LLM writes an execute block that contains a single invoke call with no processing logic around it, the server detects this and returns a hint suggesting the simpler call meta-tool instead. Small nudges like this help the LLM learn the right tool for the job over the course of a session.

batch: bulk operations without scripting overhead

Not every multi-call workflow needs control flow. Sometimes it’s the same operation twenty times: decompile a list of functions, rename a set of symbols, add comments at known addresses. For these, execute is overkill: sandbox overhead just to loop over a list. batch handles this directly: a list of operations, run sequentially with per-item error handling.

{
  "operations": [
    {"tool": "decompile_function", "params": {"address": "0x401000"}},
    {"tool": "decompile_function", "params": {"address": "0x401100"}},
    {"tool": "rename_function", "params": {"address": "0x401000", "new_name": "parse_header"}},
    {"tool": "rename_function", "params": {"address": "0x401100", "new_name": "validate_checksum"}},
    {"tool": "set_comment", "params": {"address": "0x401000", "comment": "Entry point for packet parsing"}},
    {"tool": "set_comment", "params": {"address": "0x401100", "comment": "CRC-32 validation"}}
  ]
}

Up to 50 operations per call, mixing different tools freely. This example decompiles two functions, renames them, and annotates them: six operations that would have been six separate tool calls in 2.1, collapsed into one.

In 2.1, batching was baked into individual tools: decompile_function accepted up to 50 addresses, get_xrefs_to accepted up to 50, each with its own batch parameter format. The LLM had to remember which tools supported batching and how each one worked. The unified batch meta-tool replaces all of that: a list of {tool, params} objects. Any tool can be batched.

stop_on_error controls whether the batch aborts on the first failure or continues collecting results. The default is to continue: if 30 functions are being renamed and one address is invalid, the other 29 still succeed. The response includes per-operation success/failure status, so the LLM can see exactly what failed and decide whether to retry or move on.

The split is straightforward: if there’s no data dependency between operations (the output of one doesn’t feed into another), the LLM uses batch. If the workflow chains outputs, filters intermediate results, or applies conditional logic, it writes an execute script.

call and get_schema: the discovery layer

2.1 introduced progressive tool discovery: ~20 core tools registered upfront, the rest discoverable via search_tools and callable through call_tool. 2.2 refines this into a cleaner surface:

• search_tools: regex search over tool names, descriptions, and tags. Returns compact signatures by default; pass detail="detailed" for descriptions or detail="full" for complete schemas.
• get_schema: fetch the full parameter schema for a specific tool by name, skipping the search when the LLM already knows what it wants.
• call: invoke any tool by name (renamed from call_tool), including hidden tools not in the client’s tool list.

~25 tools are now pinned (up from ~20), and the total count is down to ~125 after 2.1’s resource consolidation. The remaining ~100 specialized tools are discoverable through search_tools and callable through call, batch, or execute.

Together, the five meta-tools form a hierarchy:

The LLM picks the right level without prompting. A quick rename uses a pinned tool directly. A bulk annotation uses batch. A multi-step investigation uses execute. When it needs something specialized (applying a calling convention, editing register variables), it searches, checks the schema, and calls through call.

Daemon mode

The meta-tools only pay off if the server stays alive long enough to use them. In 2.1, ida-mcp ran as a stdio subprocess of the MCP client. When the client disconnected (closing an editor, cycling a session, restarting after a crash), the server process died and took all worker state with it. Every open database, every completed auto-analysis pass, every renamed function: gone. For quick, single-session analysis, this was acceptable. But reverse engineering work rarely fits in a single session. You open a binary, let auto-analysis run, rename a few hundred functions, apply types, and then come back the next day to continue. Or the session cycles for an unrelated reason and you lose everything.

The problem was worse in Claude Code, where subagents share a single MCP session. A subagent halfway through analyzing a firmware image (hundreds of functions renamed, types applied) loses everything when the session cycles. It reconnects, but has to reopen, re-analyze, and reconstruct its progress from whatever survived context compaction.

In 2.2, the server runs as a persistent HTTP daemon behind a lightweight stdio proxy:

LLM Client  <──stdio──>  Proxy  <──HTTP──>  Daemon
                                             Workers + Databases

The first time an MCP client connects, the proxy spawns a daemon process and detaches it. Subsequent connections (including reconnections after a session cycle, from a different editor, or from a completely new conversation) reuse the running daemon. Workers and their databases persist across disconnects: renamed symbols, added comments, applied types all survive.

The daemon also supports collaboration across clients. If a human analyst has been annotating a binary through one MCP session, a second session connecting to the same daemon sees all those annotations immediately. The daemon doesn’t care who made the changes; it just maintains the databases.

The daemon listens on 127.0.0.1 with a per-instance 256-bit bearer token. The state file is written with 0600 permissions so only the spawning user can read the token. To stop the daemon:

ida-mcp stop

This is the default transport now. Existing MCP client configurations (ida-mcp as the command) work without changes. The proxy handles daemon lifecycle transparently.

Raw binary and firmware support

ida-mcp could already open ELF, PE, and Mach-O files, where IDA auto-detects the architecture and load address from file headers. But firmware analysis (bootloaders, ROM dumps, flash extractions) starts with a blob of bytes and no metadata. Previously, you had to preprocess the binary in IDA’s GUI or write a loader script before ida-mcp could work with it. In 2.2, open_database accepts three new parameters that give the LLM what it needs to bootstrap analysis on raw binaries:

• processor: the IDA processor module with an optional variant (e.g., arm:ARMv7-M for Cortex-M firmware, metapc:80386p for 32-bit x86, mips:mipsl for little-endian MIPS)
• loader: explicit loader selection (e.g., "Binary file" for raw blobs)
• base_address: the load address in hex or decimal (e.g., "0x08000000" for a typical STM32 flash base)

For structured formats, these parameters are optional. IDA figures them out from the file headers. For raw binaries, the LLM needs to provide them. If the user says “analyze this Cortex-M firmware dump loaded at 0x08000000,” those three parameters map directly:

{
  "file_path": "/path/to/firmware.bin",
  "processor": "arm:ARMv7-M",
  "loader": "Binary file",
  "base_address": "0x08000000"
}

The server validates processor names and catches a subtle headless-mode pitfall: processor names like arm, metapc, and mips are ambiguous. In IDA’s GUI, selecting one of these pops up a dialog asking which variant you mean: ARM or AArch64? 32-bit or 64-bit x86? But headless idalib never shows that dialog. It silently picks a default, and the default is often wrong. A Cortex-M firmware blob opened with bare arm ends up disassembled as AArch64, producing nonsense.

The server rejects these bare names on raw binaries and returns the available variants with descriptions:

"arm" is ambiguous for raw binaries. It defaults to AArch64 in headless mode.
Use a specific variant:
  arm:ARMv7-M    Cortex-M (32-bit Thumb-2)
  arm:ARMv7-A    32-bit A-profile
  arm:AArch64    64-bit (explicit)

The LLM can also call list_targets to enumerate all available processors and loaders, so it can match an unknown binary to the right target without guessing.

Fat Mach-O support

macOS universal binaries pack multiple architecture slices into a single file. In 2.1, opening one would silently pick whichever slice IDA defaulted to, usually arm64, even when the target was x86_64. Nothing indicated the wrong slice had been selected until the disassembly didn’t make sense.

In 2.2, the server parses the fat header, identifies the available slices, and requires the caller to choose explicitly:

AmbiguousFatBinary: universal binary contains multiple architectures.
Available slices: arm64, arm64e, x86_64
Pass fat_arch="arm64" to select a slice.

Each slice gets its own .i64 sidecar (binary.arm64.i64, binary.x86_64.i64), so multiple architectures can be opened simultaneously in separate workers. Combined with execute’s cross-database support, the LLM can decompile the same function in both the arm64 and x86_64 slices and diff the pseudocode. This helps when finding platform-specific behavior, verifying that a vulnerability affects all architectures, or understanding how the compiler optimized differently for each target.

The fat header parser also handles an edge case that has bitten other tools: Java .class files share the same magic bytes (0xCAFEBABE) as Mach-O fat binaries. The parser validates slice counts and CPU types to distinguish the two, so a directory full of Java classes won’t trigger false fat-binary detection.

Tuning for your model and client

Not every model writes good Python, and not every MCP client needs server-side tool discovery. The meta-tools are designed to be independently useful, so you can enable the ones that match your setup and disable the ones that don’t.

Three environment variables control which meta-tools are available:

• IDA_MCP_DISABLE_EXECUTE: hides the execute meta-tool. Smaller models or those without strong code generation can produce unreliable Python in execute blocks: wrong parameter names, broken control flow, off-by-one iteration. For these models, discrete tool calls are more reliable: each call is independently validated, and errors are clear and localized. Disabling execute keeps batch for bulk operations and call for hidden tools.

• IDA_MCP_DISABLE_BATCH: hides the batch meta-tool. Useful if your workflow routes all multi-step work through execute anyway, since having both visible can lead the LLM to pick the wrong one.

• IDA_MCP_DISABLE_TOOL_SEARCH: disables server-side progressive disclosure entirely. All ~125 tools become directly visible in the client’s tool list, and search_tools and get_schema are removed. This is the right setting for clients like Claude Code that already implement their own tool deferral. Claude Code defers tool schemas and loads them on demand. If ida-mcp is also hiding tools behind search_tools, the LLM has to go through two layers of discovery to reach a specialized tool. Disabling the server-side layer removes the redundancy.

These are environment variables on the server process, so they apply to all sessions against that daemon. Set them in your MCP client configuration:

{
  "command": "ida-mcp",
  "env": {
    "IDA_MCP_DISABLE_TOOL_SEARCH": "1"
  }
}

As a starting point: if you’re using Claude (Opus or Sonnet) through Claude Code, disable tool search. If you’re using a smaller model or a client without native tool deferral, leave everything enabled and let server-side progressive disclosure handle it.

Other improvements

• Per-run log files: Each server run writes to its own timestamped log file, and open_database warnings (e.g., loader compatibility issues) are surfaced to the client instead of silently swallowed. When something goes wrong, you can find the relevant log without scrolling through a monolithic file.
• Heartbeat progress reporting: save_database and execute blocks send progress notifications every 5 seconds to prevent client timeouts on large databases. Saving a database with millions of functions and extensive annotations can take minutes; without heartbeats, the MCP client would assume the server had hung and disconnect.
• Database reopen fix: Reopening an existing .i64 no longer passes stale loader options that caused idalib to exit(1) on format mismatch. This was annoying because the failure mode was a silent exit with no error message: the worker just disappeared.

Upgrading

uv tool install --upgrade ida-mcp

Or with pip:

pip install --upgrade ida-mcp

The MCP interface is backward compatible. Existing client configurations work without changes. The daemon spawns automatically on first connection.

Links

• Repository: github.com/jtsylve/ida-mcp
• PyPI: pypi.org/project/ida-mcp
• Previous post: ida-mcp 2.1: Progressive Tool Discovery, Background Analysis, and Batch Operations

If you run into issues or have feature requests, please open an issue on GitHub.

IDA Pro and Hex-Rays are trademarks of Hex-Rays SA. ida-mcp is an independent project and is not affiliated with or endorsed by Hex-Rays.