Test Types¶

BIT provides three distinct types of tests, each designed for different stages of the system lifecycle and different monitoring needs.

Overview¶

PBIT (Power-On Built-In Test)¶

Purpose¶

PBIT tests verify that hardware components are present, functional, and properly configured when the system boots. Think of them as a "system health check" at power-on.

Characteristics¶

• Run once at system startup
• Quick execution (typically 1-5 seconds each)
• Boolean result - system is either healthy or not
• No frequency setting in configuration
• Validation-focused - checks against expected state

When to Use PBIT¶

• ✓ Verify hardware after system boot
• ✓ Detect missing or failed components early
• ✓ Validate security configuration
• ✓ Check system meets baseline requirements
• ✓ CI/CD validation gates

Common PBIT Tests¶

Hardware Validation:

• pbit_cpu_cores - Verify expected CPU count
• pbit_disk_health - Check disk SMART status
• pbit_ethernet - Verify network interfaces exist
• pbit_gpio - Check GPIO pins are accessible
• pbit_can - Verify CAN interfaces are available

Resource Checks:

• pbit_cpu_usage - Ensure CPU isn't overloaded at boot
• pbit_memory_usage - Verify sufficient memory available
• pbit_disk_usage - Check disk space not exhausted
• pbit_temperature - Verify sensors within range

Security & Configuration:

• pbit_selinux_apparmor_status - Verify security system active
• pbit_ssh_configuration - Check SSH daemon settings
• pbit_firewall_configuration - Verify firewall status
• pbit_permissions_verification - Check file permissions

Device Whitelists:

• pbit_usb_whitelist - Verify only approved USB devices
• pbit_pci_whitelist - Verify only approved PCI devices

PBIT Execution Flow¶

In Service Mode (bit_manager.service):

1. System boots
2. bit_manager.service starts
3. All enabled PBIT tests load
4. Each test runs once sequentially
5. Results logged to systemd journal
6. CBIT tests begin periodic execution

In One-Shot Mode:

1. User runs bit-manager -o
2. All enabled PBIT tests load
3. Each test runs once
4. Results print to stdout
5. Process exits with status code

PBIT tests can be enabled/disabled remotely using bit-monitor-cli or bit-monitor-gui. Use --enable-suite PBIT or --disable-suite PBIT to control all PBIT tests at once.

CBIT (Continuous Built-In Test)¶

Purpose¶

CBIT tests continuously monitor system health, watching for degradation, resource exhaustion, or configuration drift over time.

Characteristics¶

• Run periodically at configured intervals
• Ongoing monitoring until system shutdown
• Frequency-based execution (e.g., every 30 seconds)
• Trend detection possible with historical data
• Alert-focused - notify when issues develop

When to Use CBIT¶

• ✓ Monitor resource utilization (CPU, memory, disk)
• ✓ Detect performance degradation
• ✓ Watch for temperature increases
• ✓ Verify interfaces stay operational
• ✓ Detect unauthorized configuration changes
• ✓ Monitor for security policy violations

Common CBIT Tests¶

Resource Monitoring: - cbit_cpu_usage - Continuous CPU utilization monitoring - cbit_memory_usage - Track memory consumption over time - cbit_disk_usage - Watch disk space growth - cbit_temperature - Monitor thermal conditions

Hardware Monitoring: - cbit_disk_health - Continuous SMART monitoring - cbit_ethernet - Monitor interface status - cbit_can - Watch CAN interface health - cbit_gpio - Monitor GPIO pin states - cbit_serial_ports - Track serial port availability

System Monitoring: - cbit_dmesg_check - Watch kernel messages for errors - cbit_syslog_analysis - Monitor system logs - cbit_power_consumption - Track power usage

Security Monitoring: - cbit_selinux_apparmor_status - Ensure security stays enabled - cbit_ssh_configuration - Watch for SSH config changes - cbit_firewall_configuration - Monitor firewall status - cbit_permissions_verification - Detect permission changes

Device Monitoring: - cbit_usb_whitelist - Detect unauthorized USB devices - cbit_pci_whitelist - Detect unauthorized PCI devices

CBIT Execution Flow¶

In Service Mode (bit_manager.service):

1. Test loads with frequency setting from config
2. Test executes immediately (first run)
3. Sleeps for frequency seconds
4. Repeats execution and sleep until service stops
5. Each iteration logs status to systemd journal

Typical Frequencies:

• Fast (5-10s): Security checks (USB/PCI whitelist)
• Medium (30-60s): Resource monitoring (CPU, memory, disk, temperature)
• Slow (300-600s): Configuration drift, disk health

CBIT tests can be controlled remotely using bit-monitor-cli or bit-monitor-gui:

• --enable-suite CBIT / --disable-suite CBIT - Control all CBIT tests
• --run-enabled CBIT - Trigger immediate execution of all enabled CBIT tests
• Individual tests can be enabled/disabled without restarting bit-manager

FBIT (Factory Built-In Test)¶

Purpose¶

FBIT tests validate hardware functionality during manufacturing, testing actual data throughput and hardware capabilities beyond presence checks.

Characteristics¶

• Run on demand (not automatic)
• Longer execution time (seconds to minutes)
• Invasive testing - creates files, sends data
• Hardware-specific - requires actual interfaces
• Production validation focused

When to Use FBIT¶

• ✓ Manufacturing line testing
• ✓ Hardware acceptance testing
• ✓ Post-repair validation
• ✓ Performance benchmarking
• ✓ Hardware certification

Available FBIT Tests¶

Communication Tests: - fbit_can_data - CAN bus loopback test (requires physical loopback) - fbit_serial_data - Serial port loopback test (100KB transfer) - fbit_tcp_data - Network throughput test (requires iPerf server)

Storage Tests: - fbit_ssd - SSD performance test (creates 1MB file)

I/O Tests: - fbit_gpio_data - GPIO loopback test (requires wiring) - fbit_usb - USB device test (creates test file in /tmp)

System Tests: - fbit_system_data - Internal systems monitoring - fbit_video - V4L2 video device validation - fbit_pci - PCI capability and status checks

FBIT Execution¶

FBIT tests run only when explicitly triggered:

# Run specific FBIT test
bit-manager -t fbit_serial_data -o

Or trigger remotely:

# Via bit-monitor-cli
bit-monitor-cli --hostname=<device> --run-test fbit_serial_data

# Run all FBIT tests
bit-monitor-cli --hostname=<device> --run-suite FBIT

See bit-manager for more execution options.

Hardware Requirements¶

Many FBIT tests require physical setup:

Loopback Requirements: - CAN: Physical loopback connector or two CAN interfaces cross-wired - Serial: TX/RX pins wired together or null modem cable - GPIO: Output pins wired to input pins

Network Requirements: - TCP: iPerf server accessible on network

Storage Requirements: - SSD: Writable filesystem with sufficient space

Comparing Test Types¶

Relationship Between Types¶

Many tests come in PBIT/CBIT pairs with shared configuration:

Shared Whitelist Pattern: - PBIT generates baseline (detect current hardware) - CBIT monitors against baseline (detect changes) - Example: pbit_usb_whitelist / cbit_usb_whitelist

Shared Threshold Pattern: - Both use same threshold values - PBIT checks once at boot, CBIT monitors continuously - Example: pbit_cpu_usage / cbit_cpu_usage

Independent Tests: - FBIT tests are typically standalone - Different purpose and configuration than PBIT/CBIT - Example: fbit_serial_data (loopback test)

See Configuration Guide for TOML structure details.

Choosing the Right Test Type¶

Use PBIT When:¶

• ❓ "Is the hardware present and working at boot?"
• ❓ "Does the system meet baseline requirements?"
• ❓ "Are critical components operational?"

Use CBIT When:¶

• ❓ "Is the system staying healthy over time?"
• ❓ "Are resources being exhausted?"
• ❓ "Have there been unauthorized changes?"

Use FBIT When:¶

• ❓ "Does the hardware actually perform as expected?"
• ❓ "Can we transfer data successfully?"
• ❓ "Does the device meet performance specs?"

Test Selection Strategy¶

Embedded System Deployment¶

Minimal (resource-constrained):

# Enable only critical PBIT tests
pbit_cpu_cores
pbit_disk_health
pbit_memory_usage

# Essential CBIT monitoring
cbit_disk_usage (frequency = 300)
cbit_temperature (frequency = 120)

Standard (typical deployment):

# Most PBIT tests
pbit_cpu_usage, pbit_memory_usage, pbit_disk_*
pbit_ethernet, pbit_temperature
pbit_usb_whitelist, pbit_pci_whitelist

# Active CBIT monitoring
cbit_cpu_usage (30s), cbit_memory_usage (30s)
cbit_disk_usage (60s), cbit_temperature (60s)
cbit_usb_whitelist (5s), cbit_pci_whitelist (5s)

Comprehensive (security-critical):

# All PBIT tests
All 21 PBIT tests enabled

# All CBIT monitoring
All 21 CBIT tests with appropriate frequencies

Manufacturing Line¶

# Run all FBIT tests
fbit_can_data      # CAN loopback
fbit_serial_data   # Serial loopback
fbit_gpio_data     # GPIO loopback
fbit_ssd           # Storage performance
fbit_usb           # USB functionality
fbit_tcp_data      # Network throughput

Best Practices¶

PBIT Best Practices¶

1. Enable all critical hardware checks
2. Set realistic thresholds (not too tight)
3. Don't disable security checks
4. Test after configuration changes

CBIT Best Practices¶

1. Choose appropriate frequencies (don't poll too fast)
2. Monitor critical resources (CPU, memory, disk)
3. Enable security monitoring (whitelist checks)
4. Review logs regularly for trends

FBIT Best Practices¶

1. Run in controlled environment (not production)
2. Ensure proper hardware setup (loopbacks, etc.)
3. Document pass/fail criteria clearly
4. Include in manufacturing checklist

Next Steps¶

• bit-learn Guide - Generate test configurations
• bit-inspect Guide - Inspect test details
• Running Tests - Execute tests
• Test Reference - Complete test catalog