HDDSCAP File Format

The .hddscap format is the native capture format for hdds_viewer, optimized for high-throughput recording and efficient replay.

Overview

Property

Value

Extension	`.hddscap`
Magic bytes	`HDDS` (0x48 0x44 0x44 0x53)
Version	1.0
Endianness	Little-endian
Compression	Optional (LZ4)

File Structure

┌──────────────────────────────────────┐ │ Header (256 bytes) │ ├──────────────────────────────────────┤ │ Type Registry (variable) │ ├──────────────────────────────────────┤ │ Segment 1 │ ├──────────────────────────────────────┤ │ Segment 2 │ ├──────────────────────────────────────┤ │ ... │ ├──────────────────────────────────────┤ │ Segment N │ ├──────────────────────────────────────┤ │ Footer (64 bytes) │

└──────────────────────────────────────┘

Header Format (256 bytes)

Offset

Size

Field

Description

0x00	4	magic	"HDDS" (0x48444453)
0x04	2	version_major	Format major version
0x06	2	version_minor	Format minor version
0x08	8	timestamp_start	Capture start (ns since epoch)
0x10	8	timestamp_end	Capture end (set in footer)
0x18	8	frame_count	Total frames (set in footer)
0x20	4	flags	Compression, encryption flags
0x24	4	type_registry_offset	Offset to type registry
0x28	4	type_registry_size	Size of type registry
0x2C	4	first_segment_offset	Offset to first segment
0x30	32	capture_name	UTF-8 capture name
0x50	32	hostname	Capturing host
0x70	16	hdds_version	HDDS version string
0x80	128	reserved	Future use (zero-filled)

Flags

Bit

Meaning

0	Compressed (LZ4)
1	Encrypted (AES-256-GCM)
2	Indexed (fast seek)
3-31	Reserved

Type Registry

Maps type IDs to type descriptors:

┌─────────────────────────────────────┐ │ Entry Count (u32) │ ├─────────────────────────────────────┤ │ Type Entry 1 │ │ ├─ type_id (u16) │ │ ├─ name_len (u16) │ │ ├─ name (UTF-8, name_len bytes) │ │ └─ schema (CDR2 TypeObject) │ ├─────────────────────────────────────┤ │ Type Entry 2 │ ├─────────────────────────────────────┤ │ ... │

└─────────────────────────────────────┘

Segment Format

Each segment contains up to 1000 frames:

┌─────────────────────────────────────┐ │ Segment Header (32 bytes) │ │ ├─ segment_id (u32) │ │ ├─ frame_count (u32) │ │ ├─ compressed_size (u32) │ │ ├─ uncompressed_size (u32) │ │ ├─ timestamp_start (u64) │ │ ├─ timestamp_end (u64) │ │ └─ crc32 (u32) │ ├─────────────────────────────────────┤ │ Frame 1 │ ├─────────────────────────────────────┤ │ Frame 2 │ ├─────────────────────────────────────┤ │ ... │

└─────────────────────────────────────┘

Frame Format

┌─────────────────────────────────────┐ │ Frame Header (24 bytes) │ │ ├─ timestamp_ns (u64) │ │ ├─ topic_id (u16) │ │ ├─ type_id (u16) │ │ ├─ qos_hash (u32) │ │ ├─ payload_size (u32) │ │ └─ flags (u32) │ ├─────────────────────────────────────┤ │ Payload (CDR2 encoded) │

└─────────────────────────────────────┘

Frame Flags

Bit

Meaning

0	Key sample
1	Dispose
2	Unregister
3	Inline QoS present
4-31	Reserved

Footer Format (64 bytes)

Offset

Size

Field

Description

0x00	8	timestamp_end	Capture end time
0x08	8	frame_count	Total frames captured
0x10	8	byte_count	Total payload bytes
0x18	4	segment_count	Number of segments
0x1C	4	topic_count	Unique topics
0x20	4	type_count	Unique types
0x24	4	crc32_header	CRC32 of header
0x28	4	crc32_footer	CRC32 of footer
0x2C	20	reserved	Future use

Performance

Metric

Value

Write throughput	625 MB/s
Read throughput	800+ MB/s
Compression ratio	3:1 typical
Max file size	16 TB
Max frames	2^64 - 1

Tools

Create Capture

# From live traffic hdds-viewer --domain 0 --record output.hddscap With compression

hdds-viewer --domain 0 --record output.hddscap --compress

Inspect Capture

# Show header info
hdds-viewer --info capture.hddscap

Output:
File: capture.hddscap
Version: 1.0
Duration: 5m 23s
Frames: 1,234,567
Size: 156 MB (compressed)
Topics: 12
Types: 8

Convert to PCAP

hdds-viewer --convert capture.hddscap --output capture.pcap

Merge Captures

hdds-viewer --merge capture1.hddscap capture2.hddscap -o merged.hddscap

Extract Time Range

hdds-viewer --extract capture.hddscap \ --start "2025-01-01T12:00:00" \ --end "2025-01-01T12:05:00" \

-o subset.hddscap

Comparison with PCAP

Feature

HDDSCAP

PCAP/PCAPNG

DDS-native	Yes	No (raw packets)
Type information	Embedded	External IDL
Compression	LZ4 (built-in)	External tool
Random access	Yes (indexed)	Sequential
CDR decoding	Pre-parsed	Runtime
File size	3x smaller	Larger
Wireshark compatible	Via export	Native

API (Rust)

Reading

use viewer_core::capture::{CaptureReader, Frame};

let reader = CaptureReader::open("capture.hddscap")?;
println!("Frames: {}", reader.header().frame_count);

for frame in reader.frames() {
    let frame: Frame = frame?;
    println!("{}: {} bytes", frame.topic_name, frame.payload.len());
}

Writing

use viewer_core::capture::{CaptureWriter, CaptureConfig};

let config = CaptureConfig::new("my_capture")
    .with_compression(true)
    .with_hostname("node-1");

let mut writer = CaptureWriter::create("output.hddscap", config)?;

writer.write_frame(&frame)?;
writer.finalize()?;  // Writes footer

Security

Encryption (Enterprise)

Captures can be encrypted with AES-256-GCM:

hdds-viewer --record output.hddscap --encrypt --key-file key.bin

Integrity

CRC32 checksums protect:

Header integrity
Segment integrity
Footer integrity

# Verify capture

hdds-viewer --verify capture.hddscap