# Linux Hello

**Secure facial authentication for Linux using IR cameras and TPM2**

A Windows Hello-equivalent biometric authentication system for Linux, designed with security-first principles: TPM-backed credential storage, anti-spoofing detection, and proper PAM integration.

---

## Table of Contents

1. [Project Overview](#project-overview)
2. [Architecture](#architecture)
3. [Security Model](#security-model)
4. [Components](#components)
5. [Technical Specifications](#technical-specifications)
6. [Development Phases](#development-phases)
7. [Build Instructions](#build-instructions)
8. [Testing Strategy](#testing-strategy)
9. [Threat Model](#threat-model)
10. [Contributing](#contributing)

---

## Project Overview

### Problem Statement

Current Linux facial authentication solutions (primarily Howdy) have significant security limitations:

- Face templates stored unencrypted on disk
- No TPM integration for secure credential storage
- Basic or no anti-spoofing capabilities
- Vulnerable to photo and video replay attacks
- No secure enclave processing

### Goals

1. **Security parity with Windows Hello** — TPM2-backed storage, anti-spoofing ML models, secure template handling
2. **Privacy by design** — all biometric data stays local, encrypted at rest, never leaves the device
3. **Seamless UX** — sub-second authentication, works with login managers, sudo, lock screens
4. **Broad compatibility** — support major distros, common IR camera hardware, multiple desktop environments

### Non-Goals

- RGB-only camera support (IR is required for security)
- Cloud-based authentication
- Multi-device sync of biometric data
- Fingerprint or other biometric modalities (future scope)

---

## Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                        Desktop Environment                          │
│                  (GNOME, KDE, SDDM, GDM, etc.)                      │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────┐
│                           PAM Module                                │
│                      (pam_linux_hello.so)                           │
│         - Authentication entry point                                │
│         - Communicates with daemon via Unix socket                  │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      D-Bus Service Daemon                           │
│                     (linux-hello-daemon)                            │
│                                                                     │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌────────────┐  │
│  │   Camera    │  │    Face     │  │    Anti-    │  │    TPM2    │  │
│  │  Interface  │  │  Detection  │  │  Spoofing   │  │  Storage   │  │
│  └─────────────┘  └─────────────┘  └─────────────┘  └────────────┘  │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
              ┌──────────┐   ┌──────────┐    ┌──────────┐
              │ IR Camera│   │   TPM2   │    │  ONNX    │
              │ /dev/vid │   │  Device  │    │ Runtime  │
              └──────────┘   └──────────┘    └──────────┘
```

### Component Communication

| From | To | Protocol | Purpose |
|------|----|----------|---------|
| PAM Module | Daemon | Unix Socket (abstract) | Auth requests/responses |
| Desktop Env | Daemon | D-Bus (system bus) | Enrollment, settings, status |
| Daemon | Camera | V4L2 | Frame capture, IR emitter control |
| Daemon | TPM2 | tss-esapi / tpm2-tss | Secure key/template storage |

---

## Security Model

### Biometric Template Protection

1. **Enrollment**: Face embeddings are generated and immediately encrypted using a TPM2-bound key
2. **Storage**: Encrypted templates stored in `/var/lib/linux-hello/templates/<user>/`
3. **Decryption**: Only possible on this specific TPM, with user authentication
4. **Deletion**: Secure wipe on unenrollment, templates never backed up

### Anti-Spoofing Layers

| Layer | Technique | Defeats |
|-------|-----------|---------|
| 1 | IR-only capture | Printed photos, most screens |
| 2 | Depth estimation from structured light | 2D images, flat screens |
| 3 | Liveness CNN | Photos, static masks |
| 4 | Temporal micro-movement analysis | Static 3D prints |
| 5 | (Optional) Challenge-response | All static attacks |

### Authentication Flow

```
1. PAM requests authentication for <user>
2. Daemon activates IR emitter + camera
3. Capture N frames (configurable, default 5)
4. For each frame:
   a. Run anti-spoofing checks (must pass threshold)
   b. Extract face embedding
5. Compare embeddings against TPM-decrypted templates
6. Return success/failure to PAM
7. Zero-out all frame buffers and embeddings from memory
```

### Threat Model Summary

| Threat | Mitigation |
|--------|------------|
| Stolen laptop (off) | TPM-bound keys, no access without hardware |
| Stolen laptop (suspended) | Memory encryption (separate concern), short auth timeout |
| Photo attack | IR camera, depth detection |
| Video replay on screen | IR doesn't capture screens properly |
| 3D printed mask | Temporal liveness detection |
| Evil maid (template extraction) | TPM binding, encrypted at rest |
| Shoulder surfing | N/A (face auth inherently resistant) |
| Similar-looking attacker | Embedding distance threshold tuning |

---

## Components

### 1. PAM Module (`pam_linux_hello.so`)

**Language**: C (for maximum PAM compatibility)

**Responsibilities**:
- Integrate with PAM authentication stack
- Communicate with daemon via Unix socket
- Handle timeouts and fallback to password
- Provide configuration via `/etc/pam.d/` and `/etc/linux-hello/pam.conf`

**Key Functions**:
```c
PAM_EXTERN int pam_sm_authenticate(pam_handle_t *pamh, int flags, int argc, const char **argv);
PAM_EXTERN int pam_sm_setcred(pam_handle_t *pamh, int flags, int argc, const char **argv);
```

**Configuration Options**:
- `timeout=<seconds>` — max time to attempt face auth (default: 5)
- `fallback=password` — allow password if face fails
- `debug` — verbose logging to syslog

---

### 2. Daemon (`linux-hello-daemon`)

**Language**: Rust

**Crates**:
- `tss-esapi` — TPM2 integration
- `v4l` — Video4Linux camera control
- `zbus` — D-Bus service
- `ort` — ONNX Runtime bindings for ML inference
- `opencv` — Image processing (via rust bindings or FFI)
- `tokio` — Async runtime
- `secrecy` — Secure memory handling

**Responsibilities**:
- Manage camera lifecycle (open, configure, capture, close)
- Control IR emitter via UVC extension units
- Run face detection and embedding models
- Run anti-spoofing models
- Interface with TPM2 for key management and encryption
- Expose D-Bus API for enrollment and management
- Handle concurrent authentication requests

**D-Bus Interface** (`org.linux-hello.Manager`):

```xml
<interface name="org.linux-hello.Manager">
  <method name="Enroll">
    <arg name="user" type="s" direction="in"/>
    <arg name="label" type="s" direction="in"/>
    <arg name="success" type="b" direction="out"/>
    <arg name="message" type="s" direction="out"/>
  </method>
  
  <method name="Remove">
    <arg name="user" type="s" direction="in"/>
    <arg name="label" type="s" direction="in"/>
    <arg name="success" type="b" direction="out"/>
  </method>
  
  <method name="List">
    <arg name="user" type="s" direction="in"/>
    <arg name="labels" type="as" direction="out"/>
  </method>
  
  <method name="Test">
    <arg name="user" type="s" direction="in"/>
    <arg name="result" type="b" direction="out"/>
    <arg name="confidence" type="d" direction="out"/>
  </method>
  
  <property name="Status" type="s" access="read"/>
  <property name="CameraAvailable" type="b" access="read"/>
  
  <signal name="AuthenticationAttempt">
    <arg name="user" type="s"/>
    <arg name="success" type="b"/>
    <arg name="method" type="s"/>
  </signal>
</interface>
```

---

### 3. IR Camera Interface

**Driver Layer**: V4L2 (Video4Linux2)

**Key Challenges**:
- IR emitter control varies by hardware (UVC extension units)
- Need to identify IR camera vs RGB camera
- Handle camera already in use by other applications

**IR Emitter Control**:
Most IR cameras use USB Video Class (UVC) extension units. We'll need to:
1. Enumerate UVC extension units
2. Find the IR emitter control (vendor-specific)
3. Send enable/disable commands

Reference implementation: [linux-enable-ir-emitter](https://github.com/EmixamPP/linux-enable-ir-emitter)

**Camera Detection Heuristic**:
```
1. Enumerate /dev/video* devices
2. Query V4L2 capabilities
3. Look for IR-specific formats (Y8, GREY, etc.)
4. Check for multiple cameras (typically IR is second device)
5. Attempt IR emitter activation as confirmation
```

---

### 4. Face Detection & Embedding

**Model Options**:

| Model | Size | Speed | Accuracy | License |
|-------|------|-------|----------|---------|
| RetinaFace | ~100MB | ~50ms | High | MIT |
| MTCNN | ~2MB | ~100ms | Medium | MIT |
| BlazeFace | ~500KB | ~10ms | Medium | Apache 2.0 |

**Embedding Model**:

| Model | Embedding Size | Accuracy (LFW) | License |
|-------|----------------|----------------|---------|
| ArcFace | 512 | 99.8% | MIT |
| FaceNet | 128/512 | 99.6% | Apache 2.0 |
| MobileFaceNet | 128 | 99.4% | MIT |

**Recommendation**: 
- Detection: BlazeFace (speed) or RetinaFace (accuracy)
- Embedding: MobileFaceNet (good balance for on-device)

**Inference Runtime**: ONNX Runtime with CPU execution provider (GPU optional)

---

### 5. Anti-Spoofing Module

**Multi-Stage Pipeline**:

```
Frame → IR Validation → Depth Check → Liveness CNN → Temporal Analysis → Decision
           │                │              │               │
           ▼                ▼              ▼               ▼
        Reject if       Reject if      Score 0-1      Micro-movement
        not IR-like    depth < thresh   threshold       detection
```

**Stage 1: IR Validation**
- Verify frame characteristics match IR capture (histogram analysis)
- Reject frames that appear to be visible light

**Stage 2: Depth Estimation**
- Use structured light patterns from IR emitter
- Estimate depth map, reject if too flat
- Reference: [3DDFA_V2](https://github.com/cleardusk/3DDFA_V2)

**Stage 3: Liveness CNN**
- Binary classifier: real face vs spoof
- Training data: CelebA-Spoof, CASIA-FASD, Replay-Attack datasets
- Architecture: MobileNetV3-Small backbone

**Stage 4: Temporal Analysis**
- Capture sequence of frames (not just one)
- Detect micro-movements (eye blinks, slight head motion)
- Use optical flow or landmark tracking

**Confidence Scoring**:
```
final_score = w1 * depth_score + w2 * liveness_score + w3 * temporal_score

if final_score < threshold:
    reject("Spoof detected")
```

Default weights: `w1=0.3, w2=0.5, w3=0.2`
Default threshold: `0.7`

---

### 6. TPM2 Storage Module

**Key Hierarchy**:
```
TPM Storage Root Key (SRK)
    └── Linux Hello Primary Key (sealed to PCRs)
            └── User Template Encryption Key (per-user)
                    └── Encrypted face template
```

**Operations**:

| Operation | TPM2 Command | Purpose |
|-----------|--------------|---------|
| Create primary key | `TPM2_CreatePrimary` | One-time setup |
| Create user key | `TPM2_Create` | Per-user enrollment |
| Encrypt template | `TPM2_EncryptDecrypt` | Secure storage |
| Decrypt template | `TPM2_EncryptDecrypt` | Authentication |
| Seal to PCRs | `TPM2_PolicyPCR` | Bind to boot state (optional) |

**PCR Binding (Optional)**:
- PCR 7: Secure Boot state
- PCR 11: BitLocker-like (unified kernel image)
- Effect: Templates inaccessible if boot chain modified

**Key Storage**:
- Primary key handle persisted at `0x81000001`
- User keys stored as TPM2 context blobs in `/var/lib/linux-hello/keys/`

---

### 7. CLI Tool (`linux-hello`)

**Commands**:

```bash
# Enrollment
linux-hello enroll [--label <name>]     # Add face model
linux-hello enroll --glasses            # Add variant

# Management
linux-hello list                         # Show enrolled models
linux-hello remove <label>               # Remove specific model
linux-hello remove --all                 # Remove all models
linux-hello clear                        # Full reset

# Testing
linux-hello test                         # Test recognition
linux-hello test --verbose               # Show confidence scores
linux-hello test --debug                 # Save debug frames

# Status
linux-hello status                       # Show daemon status
linux-hello status --camera              # Camera diagnostics

# Configuration
linux-hello config                       # Show current config
linux-hello config --set <key>=<value>   # Modify settings
```

---

## Technical Specifications

### Hardware Requirements

| Component | Minimum | Recommended |
|-----------|---------|-------------|
| CPU | x86_64 or ARM64 | — |
| RAM | 512MB available | 1GB available |
| Camera | IR camera with emitter | Windows Hello certified |
| TPM | TPM 2.0 | fTPM or discrete |
| Storage | 100MB | 500MB (for models) |

### Supported IR Cameras

Initial target hardware:

| Manufacturer | Models | Notes |
|--------------|--------|-------|
| Lenovo | Yoga series IR cams | Well-documented |
| Dell | XPS series IR cams | UVC extension units |
| HP | EliteBook/Spectre IR | Tested with linux-enable-ir-emitter |
| Microsoft | Surface cameras | May need special handling |
| Logitech | Brio 4K | External USB, IR available |

### Performance Targets

| Metric | Target |
|--------|--------|
| Cold start (daemon not running) | < 2s |
| Warm authentication | < 500ms |
| Memory usage (daemon idle) | < 50MB |
| Memory usage (during auth) | < 200MB |
| CPU usage (during auth) | < 50% single core |
| False Acceptance Rate (FAR) | < 0.001% |
| False Rejection Rate (FRR) | < 5% |

---

## Development Phases

### Phase 1: Foundation (Weeks 1-4)

**Goals**: Basic infrastructure, camera integration, simple face detection

**Deliverables**:
- [ ] Project scaffolding (Rust workspace, C PAM module skeleton)
- [ ] V4L2 camera enumeration and capture
- [ ] IR emitter control (at least for Lenovo hardware)
- [ ] Basic face detection with ONNX model
- [ ] Frame capture CLI tool for testing

**Milestone**: Capture IR frames and detect faces in them

---

### Phase 2: Core Authentication (Weeks 5-8)

**Goals**: Face embedding, template storage, basic PAM integration

**Deliverables**:
- [ ] Face embedding model integration
- [ ] Template matching (cosine similarity)
- [ ] Local encrypted storage (no TPM yet)
- [ ] PAM module communicating with daemon
- [ ] Basic enrollment flow
- [ ] Basic authentication flow

**Milestone**: Login with face on test system (insecure, no anti-spoofing)

---

### Phase 3: Security Hardening (Weeks 9-14)

**Goals**: TPM integration, anti-spoofing, secure architecture

**Deliverables**:
- [ ] TPM2 key hierarchy setup
- [ ] Template encryption with TPM-bound keys
- [ ] Depth estimation anti-spoofing
- [ ] Liveness CNN model training and integration
- [ ] Temporal analysis implementation
- [ ] Secure memory handling (zeroization)
- [ ] Security audit of IPC

**Milestone**: Secure authentication resistant to photo/video attacks

---

### Phase 4: Polish & Integration (Weeks 15-18)

**Goals**: UX, desktop integration, broad compatibility

**Deliverables**:
- [ ] D-Bus service for desktop integration
- [ ] GNOME Settings integration (optional)
- [ ] KDE System Settings integration (optional)
- [ ] Multiple camera hardware support
- [ ] Comprehensive error handling and logging
- [ ] User documentation
- [ ] Packaging (deb, rpm, AUR)

**Milestone**: Release candidate

---

### Phase 5: Hardening & Release (Weeks 19-22)

**Goals**: Testing, security review, release

**Deliverables**:
- [ ] Penetration testing (spoofing attacks)
- [ ] Fuzzing (IPC, image parsing)
- [ ] Performance optimization
- [ ] External security audit (if budget allows)
- [ ] v1.0 release

---

## Build Instructions

### Prerequisites

```bash
# Ubuntu/Debian
sudo apt install build-essential cmake pkg-config \
    libpam0g-dev libv4l-dev libudev-dev \
    libssl-dev libtss2-dev \
    libonnxruntime-dev \
    libopencv-dev

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

### Building

```bash
git clone https://github.com/<org>/linux-hello.git
cd linux-hello

# Build daemon and CLI
cargo build --release

# Build PAM module
cd pam-module
make
sudo make install

# Install daemon
sudo install -m 755 target/release/linux-hello-daemon /usr/libexec/
sudo install -m 755 target/release/linux-hello /usr/bin/

# Install systemd service
sudo install -m 644 dist/linux-hello.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now linux-hello
```

### Configuration

```bash
# Create config directory
sudo mkdir -p /etc/linux-hello

# Copy default config
sudo cp dist/config.toml /etc/linux-hello/

# Edit as needed
sudo nano /etc/linux-hello/config.toml
```

**Default Configuration** (`/etc/linux-hello/config.toml`):

```toml
[general]
log_level = "info"
timeout_seconds = 5

[camera]
device = "auto"  # or /dev/video2
ir_emitter = "auto"
resolution = [640, 480]
fps = 30

[detection]
model = "blazeface"
min_face_size = 80
confidence_threshold = 0.9

[embedding]
model = "mobilefacenet"
distance_threshold = 0.6

[anti_spoofing]
enabled = true
depth_check = true
liveness_model = true
temporal_check = true
min_score = 0.7

[tpm]
enabled = true
pcr_binding = false
```

### PAM Configuration

Add to `/etc/pam.d/common-auth` (Debian/Ubuntu) or equivalent:

```
# Linux Hello face authentication
auth    sufficient    pam_linux_hello.so timeout=5 fallback=password
```

---

## Testing Strategy

### Unit Tests

```bash
cargo test                    # Rust unit tests
make -C pam-module test       # PAM module tests
```

### Integration Tests

```bash
# Requires camera hardware
./tests/integration/run.sh

# Tests:
# - Camera detection
# - IR emitter control
# - Face detection pipeline
# - Enrollment flow
# - Authentication flow
# - Anti-spoofing (with test images)
```

### Security Tests

```bash
# Spoofing test suite
./tests/security/spoof_test.sh

# Includes:
# - Printed photo attack
# - Screen display attack
# - Video replay attack
# - 3D printed mask (requires physical mask)
```

### Performance Benchmarks

```bash
cargo bench                   # Microbenchmarks
./tests/perf/e2e_latency.sh   # End-to-end timing
```

---

## Threat Model

### In Scope

| Attacker | Capabilities | Mitigation |
|----------|--------------|------------|
| Remote attacker | Network access only | Face auth is local-only |
| Local user (unprivileged) | Shell access | Unix permissions, polkit |
| Physical attacker (laptop off) | Full physical access | TPM binding |
| Physical attacker (laptop on) | Brief physical access | Timeout, lockout |
| Photo attacker | Has victim's photo | IR camera, depth check |
| Video attacker | Has victim's video | Liveness detection |
| Mask attacker | Has 3D print of victim | Temporal analysis |
| Twin/lookalike | Genetic similarity | Embedding threshold |

### Out of Scope

| Attacker | Why |
|----------|-----|
| Nation-state with TPM exploits | Too sophisticated, use password |
| Evil maid with unlimited time | Can't defend, use FDE + password |
| Attacker who can compel biometrics | Legal issue, not technical |
| Deepfake video on IR screen | Unlikely attack vector, revisit if needed |

### Security Invariants

1. Face templates MUST never exist unencrypted on disk
2. Templates MUST be bound to this device's TPM
3. Authentication MUST fail closed (any error → reject)
4. Anti-spoofing MUST run before embedding comparison
5. Frame data MUST be zeroized after use

---

## Contributing

### Code of Conduct

We follow the [Contributor Covenant](https://www.contributor-covenant.org/).

### Getting Started

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Write tests for your changes
4. Ensure all tests pass (`cargo test && make -C pam-module test`)
5. Run lints (`cargo clippy && cargo fmt --check`)
6. Commit your changes (`git commit -m 'Add amazing feature'`)
7. Push to the branch (`git push origin feature/amazing-feature`)
8. Open a Pull Request

### Code Style

- Rust: Follow `rustfmt` defaults
- C: Follow Linux kernel style
- Commits: Conventional Commits format

### Security Issues

Report security vulnerabilities privately to: security@<project-domain>.org

Do NOT open public issues for security vulnerabilities.

---

## License

This project is licensed under the **GNU General Public License v3.0** — see [LICENSE](LICENSE) for details.

TPM integration code may have additional licensing considerations due to TCG specifications.

---

## Acknowledgments

- [Howdy](https://github.com/boltgolt/howdy) — inspiration and proof of concept
- [linux-enable-ir-emitter](https://github.com/EmixamPP/linux-enable-ir-emitter) — IR emitter research
- [dlib](http://dlib.net/) — face recognition foundations
- [ONNX Runtime](https://onnxruntime.ai/) — cross-platform ML inference

---

## Contact

- Project Lead: [TBD]
- Mailing List: [TBD]
- Matrix/IRC: [TBD]

---

*Last updated: January 2026*