Files

2026-01-02 19:08:56 +01:00

23 KiB

Raw Blame History

Linux Hello

Secure facial authentication for Linux using IR cameras and TPM2

A Windows Hello-equivalent biometric authentication system for Linux, designed with security-first principles: TPM-backed credential storage, anti-spoofing detection, and proper PAM integration.

Project Overview
Architecture
Security Model
Components
Technical Specifications
Development Phases
Build Instructions
Testing Strategy
Threat Model
Contributing

Project Overview

Problem Statement

Current Linux facial authentication solutions (primarily Howdy) have significant security limitations:

Face templates stored unencrypted on disk
No TPM integration for secure credential storage
Basic or no anti-spoofing capabilities
Vulnerable to photo and video replay attacks
No secure enclave processing

Goals

Security parity with Windows Hello — TPM2-backed storage, anti-spoofing ML models, secure template handling
Privacy by design — all biometric data stays local, encrypted at rest, never leaves the device
Seamless UX — sub-second authentication, works with login managers, sudo, lock screens
Broad compatibility — support major distros, common IR camera hardware, multiple desktop environments

Non-Goals

RGB-only camera support (IR is required for security)
Cloud-based authentication
Multi-device sync of biometric data
Fingerprint or other biometric modalities (future scope)

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                        Desktop Environment                          │
│                  (GNOME, KDE, SDDM, GDM, etc.)                      │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────┐
│                           PAM Module                                │
│                      (pam_linux_hello.so)                           │
│         - Authentication entry point                                │
│         - Communicates with daemon via Unix socket                  │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      D-Bus Service Daemon                           │
│                     (linux-hello-daemon)                            │
│                                                                     │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌────────────┐  │
│  │   Camera    │  │    Face     │  │    Anti-    │  │    TPM2    │  │
│  │  Interface  │  │  Detection  │  │  Spoofing   │  │  Storage   │  │
│  └─────────────┘  └─────────────┘  └─────────────┘  └────────────┘  │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
              ┌──────────┐   ┌──────────┐    ┌──────────┐
              │ IR Camera│   │   TPM2   │    │  ONNX    │
              │ /dev/vid │   │  Device  │    │ Runtime  │
              └──────────┘   └──────────┘    └──────────┘

Component Communication

From	To	Protocol	Purpose
PAM Module	Daemon	Unix Socket (abstract)	Auth requests/responses
Desktop Env	Daemon	D-Bus (system bus)	Enrollment, settings, status
Daemon	Camera	V4L2	Frame capture, IR emitter control
Daemon	TPM2	tss-esapi / tpm2-tss	Secure key/template storage

Security Model

Biometric Template Protection

Enrollment: Face embeddings are generated and immediately encrypted using a TPM2-bound key
Storage: Encrypted templates stored in /var/lib/linux-hello/templates/<user>/
Decryption: Only possible on this specific TPM, with user authentication
Deletion: Secure wipe on unenrollment, templates never backed up

Anti-Spoofing Layers

Layer	Technique	Defeats
1	IR-only capture	Printed photos, most screens
2	Depth estimation from structured light	2D images, flat screens
3	Liveness CNN	Photos, static masks
4	Temporal micro-movement analysis	Static 3D prints
5	(Optional) Challenge-response	All static attacks

Authentication Flow

1. PAM requests authentication for <user>
2. Daemon activates IR emitter + camera
3. Capture N frames (configurable, default 5)
4. For each frame:
   a. Run anti-spoofing checks (must pass threshold)
   b. Extract face embedding
5. Compare embeddings against TPM-decrypted templates
6. Return success/failure to PAM
7. Zero-out all frame buffers and embeddings from memory

Threat Model Summary

Threat	Mitigation
Stolen laptop (off)	TPM-bound keys, no access without hardware
Stolen laptop (suspended)	Memory encryption (separate concern), short auth timeout
Photo attack	IR camera, depth detection
Video replay on screen	IR doesn't capture screens properly
3D printed mask	Temporal liveness detection
Evil maid (template extraction)	TPM binding, encrypted at rest
Shoulder surfing	N/A (face auth inherently resistant)
Similar-looking attacker	Embedding distance threshold tuning

Components

1. PAM Module (`pam_linux_hello.so`)

Language: C (for maximum PAM compatibility)

Responsibilities:

Integrate with PAM authentication stack
Communicate with daemon via Unix socket
Handle timeouts and fallback to password
Provide configuration via /etc/pam.d/ and /etc/linux-hello/pam.conf

Key Functions:

PAM_EXTERN int pam_sm_authenticate(pam_handle_t *pamh, int flags, int argc, const char **argv);
PAM_EXTERN int pam_sm_setcred(pam_handle_t *pamh, int flags, int argc, const char **argv);

Configuration Options:

timeout=<seconds> — max time to attempt face auth (default: 5)
fallback=password — allow password if face fails
debug — verbose logging to syslog

2. Daemon (`linux-hello-daemon`)

Language: Rust

Crates:

tss-esapi — TPM2 integration
v4l — Video4Linux camera control
zbus — D-Bus service
ort — ONNX Runtime bindings for ML inference
opencv — Image processing (via rust bindings or FFI)
tokio — Async runtime
secrecy — Secure memory handling

Responsibilities:

Manage camera lifecycle (open, configure, capture, close)
Control IR emitter via UVC extension units
Run face detection and embedding models
Run anti-spoofing models
Interface with TPM2 for key management and encryption
Expose D-Bus API for enrollment and management
Handle concurrent authentication requests

D-Bus Interface (org.linux-hello.Manager):

<interface name="org.linux-hello.Manager">
  <method name="Enroll">
    <arg name="user" type="s" direction="in"/>
    <arg name="label" type="s" direction="in"/>
    <arg name="success" type="b" direction="out"/>
    <arg name="message" type="s" direction="out"/>
  </method>
  
  <method name="Remove">
    <arg name="user" type="s" direction="in"/>
    <arg name="label" type="s" direction="in"/>
    <arg name="success" type="b" direction="out"/>
  </method>
  
  <method name="List">
    <arg name="user" type="s" direction="in"/>
    <arg name="labels" type="as" direction="out"/>
  </method>
  
  <method name="Test">
    <arg name="user" type="s" direction="in"/>
    <arg name="result" type="b" direction="out"/>
    <arg name="confidence" type="d" direction="out"/>
  </method>
  
  <property name="Status" type="s" access="read"/>
  <property name="CameraAvailable" type="b" access="read"/>
  
  <signal name="AuthenticationAttempt">
    <arg name="user" type="s"/>
    <arg name="success" type="b"/>
    <arg name="method" type="s"/>
  </signal>
</interface>

3. IR Camera Interface

Driver Layer: V4L2 (Video4Linux2)

Key Challenges:

IR emitter control varies by hardware (UVC extension units)
Need to identify IR camera vs RGB camera
Handle camera already in use by other applications

IR Emitter Control: Most IR cameras use USB Video Class (UVC) extension units. We'll need to:

Enumerate UVC extension units
Find the IR emitter control (vendor-specific)
Send enable/disable commands

Reference implementation: linux-enable-ir-emitter

Camera Detection Heuristic:

1. Enumerate /dev/video* devices
2. Query V4L2 capabilities
3. Look for IR-specific formats (Y8, GREY, etc.)
4. Check for multiple cameras (typically IR is second device)
5. Attempt IR emitter activation as confirmation

4. Face Detection & Embedding

Model Options:

Model	Size	Speed	Accuracy	License
RetinaFace	~100MB	~50ms	High	MIT
MTCNN	~2MB	~100ms	Medium	MIT
BlazeFace	~500KB	~10ms	Medium	Apache 2.0

Embedding Model:

Model	Embedding Size	Accuracy (LFW)	License
ArcFace	512	99.8%	MIT
FaceNet	128/512	99.6%	Apache 2.0
MobileFaceNet	128	99.4%	MIT

Recommendation:

Detection: BlazeFace (speed) or RetinaFace (accuracy)
Embedding: MobileFaceNet (good balance for on-device)

Inference Runtime: ONNX Runtime with CPU execution provider (GPU optional)

5. Anti-Spoofing Module

Multi-Stage Pipeline:

Frame → IR Validation → Depth Check → Liveness CNN → Temporal Analysis → Decision
           │                │              │               │
           ▼                ▼              ▼               ▼
        Reject if       Reject if      Score 0-1      Micro-movement
        not IR-like    depth < thresh   threshold       detection

Stage 1: IR Validation

Verify frame characteristics match IR capture (histogram analysis)
Reject frames that appear to be visible light

Stage 2: Depth Estimation

Use structured light patterns from IR emitter
Estimate depth map, reject if too flat
Reference: 3DDFA_V2

Stage 3: Liveness CNN

Binary classifier: real face vs spoof
Training data: CelebA-Spoof, CASIA-FASD, Replay-Attack datasets
Architecture: MobileNetV3-Small backbone

Stage 4: Temporal Analysis

Capture sequence of frames (not just one)
Detect micro-movements (eye blinks, slight head motion)
Use optical flow or landmark tracking

Confidence Scoring:

final_score = w1 * depth_score + w2 * liveness_score + w3 * temporal_score

if final_score < threshold:
    reject("Spoof detected")

Default weights: w1=0.3, w2=0.5, w3=0.2 Default threshold: 0.7

6. TPM2 Storage Module

Key Hierarchy:

TPM Storage Root Key (SRK)
    └── Linux Hello Primary Key (sealed to PCRs)
            └── User Template Encryption Key (per-user)
                    └── Encrypted face template

Operations:

Operation	TPM2 Command	Purpose
Create primary key	`TPM2_CreatePrimary`	One-time setup
Create user key	`TPM2_Create`	Per-user enrollment
Encrypt template	`TPM2_EncryptDecrypt`	Secure storage
Decrypt template	`TPM2_EncryptDecrypt`	Authentication
Seal to PCRs	`TPM2_PolicyPCR`	Bind to boot state (optional)

PCR Binding (Optional):

PCR 7: Secure Boot state
PCR 11: BitLocker-like (unified kernel image)
Effect: Templates inaccessible if boot chain modified

Key Storage:

Primary key handle persisted at 0x81000001
User keys stored as TPM2 context blobs in /var/lib/linux-hello/keys/

7. CLI Tool (`linux-hello`)

Commands:

# Enrollment
linux-hello enroll [--label <name>]     # Add face model
linux-hello enroll --glasses            # Add variant

# Management
linux-hello list                         # Show enrolled models
linux-hello remove <label>               # Remove specific model
linux-hello remove --all                 # Remove all models
linux-hello clear                        # Full reset

# Testing
linux-hello test                         # Test recognition
linux-hello test --verbose               # Show confidence scores
linux-hello test --debug                 # Save debug frames

# Status
linux-hello status                       # Show daemon status
linux-hello status --camera              # Camera diagnostics

# Configuration
linux-hello config                       # Show current config
linux-hello config --set <key>=<value>   # Modify settings

Technical Specifications

Hardware Requirements

Component	Minimum	Recommended
CPU	x86_64 or ARM64	—
RAM	512MB available	1GB available
Camera	IR camera with emitter	Windows Hello certified
TPM	TPM 2.0	fTPM or discrete
Storage	100MB	500MB (for models)

Supported IR Cameras

Initial target hardware:

Manufacturer	Models	Notes
Lenovo	Yoga series IR cams	Well-documented
Dell	XPS series IR cams	UVC extension units
HP	EliteBook/Spectre IR	Tested with linux-enable-ir-emitter
Microsoft	Surface cameras	May need special handling
Logitech	Brio 4K	External USB, IR available

Performance Targets

Metric	Target
Cold start (daemon not running)	< 2s
Warm authentication	< 500ms
Memory usage (daemon idle)	< 50MB
Memory usage (during auth)	< 200MB
CPU usage (during auth)	< 50% single core
False Acceptance Rate (FAR)	< 0.001%
False Rejection Rate (FRR)	< 5%

Development Phases

Phase 1: Foundation (Weeks 1-4)

Goals: Basic infrastructure, camera integration, simple face detection

Deliverables:

Project scaffolding (Rust workspace, C PAM module skeleton)
V4L2 camera enumeration and capture
IR emitter control (at least for Lenovo hardware)
Basic face detection with ONNX model
Frame capture CLI tool for testing

Milestone: Capture IR frames and detect faces in them

Phase 2: Core Authentication (Weeks 5-8)

Goals: Face embedding, template storage, basic PAM integration

Deliverables:

Face embedding model integration
Template matching (cosine similarity)
Local encrypted storage (no TPM yet)
PAM module communicating with daemon
Basic enrollment flow
Basic authentication flow

Milestone: Login with face on test system (insecure, no anti-spoofing)

Phase 3: Security Hardening (Weeks 9-14)

Goals: TPM integration, anti-spoofing, secure architecture

Deliverables:

TPM2 key hierarchy setup
Template encryption with TPM-bound keys
Depth estimation anti-spoofing
Liveness CNN model training and integration
Temporal analysis implementation
Secure memory handling (zeroization)
Security audit of IPC

Milestone: Secure authentication resistant to photo/video attacks

Phase 4: Polish & Integration (Weeks 15-18)

Goals: UX, desktop integration, broad compatibility

Deliverables:

D-Bus service for desktop integration
GNOME Settings integration (optional)
KDE System Settings integration (optional)
Multiple camera hardware support
Comprehensive error handling and logging
User documentation
Packaging (deb, rpm, AUR)

Milestone: Release candidate

Phase 5: Hardening & Release (Weeks 19-22)

Goals: Testing, security review, release

Deliverables:

Penetration testing (spoofing attacks)
Fuzzing (IPC, image parsing)
Performance optimization
External security audit (if budget allows)
v1.0 release

Build Instructions

Prerequisites

# Ubuntu/Debian
sudo apt install build-essential cmake pkg-config \
    libpam0g-dev libv4l-dev libudev-dev \
    libssl-dev libtss2-dev \
    libonnxruntime-dev \
    libopencv-dev

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Building

git clone https://github.com/<org>/linux-hello.git
cd linux-hello

# Build daemon and CLI
cargo build --release

# Build PAM module
cd pam-module
make
sudo make install

# Install daemon
sudo install -m 755 target/release/linux-hello-daemon /usr/libexec/
sudo install -m 755 target/release/linux-hello /usr/bin/

# Install systemd service
sudo install -m 644 dist/linux-hello.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now linux-hello

Configuration

# Create config directory
sudo mkdir -p /etc/linux-hello

# Copy default config
sudo cp dist/config.toml /etc/linux-hello/

# Edit as needed
sudo nano /etc/linux-hello/config.toml

Default Configuration (/etc/linux-hello/config.toml):

[general]
log_level = "info"
timeout_seconds = 5

[camera]
device = "auto"  # or /dev/video2
ir_emitter = "auto"
resolution = [640, 480]
fps = 30

[detection]
model = "blazeface"
min_face_size = 80
confidence_threshold = 0.9

[embedding]
model = "mobilefacenet"
distance_threshold = 0.6

[anti_spoofing]
enabled = true
depth_check = true
liveness_model = true
temporal_check = true
min_score = 0.7

[tpm]
enabled = true
pcr_binding = false

PAM Configuration

Add to /etc/pam.d/common-auth (Debian/Ubuntu) or equivalent:

# Linux Hello face authentication
auth    sufficient    pam_linux_hello.so timeout=5 fallback=password

Testing Strategy

Unit Tests

cargo test                    # Rust unit tests
make -C pam-module test       # PAM module tests

Integration Tests

# Requires camera hardware
./tests/integration/run.sh

# Tests:
# - Camera detection
# - IR emitter control
# - Face detection pipeline
# - Enrollment flow
# - Authentication flow
# - Anti-spoofing (with test images)

Security Tests

# Spoofing test suite
./tests/security/spoof_test.sh

# Includes:
# - Printed photo attack
# - Screen display attack
# - Video replay attack
# - 3D printed mask (requires physical mask)

Performance Benchmarks

cargo bench                   # Microbenchmarks
./tests/perf/e2e_latency.sh   # End-to-end timing

Threat Model

In Scope

Attacker	Capabilities	Mitigation
Remote attacker	Network access only	Face auth is local-only
Local user (unprivileged)	Shell access	Unix permissions, polkit
Physical attacker (laptop off)	Full physical access	TPM binding
Physical attacker (laptop on)	Brief physical access	Timeout, lockout
Photo attacker	Has victim's photo	IR camera, depth check
Video attacker	Has victim's video	Liveness detection
Mask attacker	Has 3D print of victim	Temporal analysis
Twin/lookalike	Genetic similarity	Embedding threshold

Out of Scope

Attacker	Why
Nation-state with TPM exploits	Too sophisticated, use password
Evil maid with unlimited time	Can't defend, use FDE + password
Attacker who can compel biometrics	Legal issue, not technical
Deepfake video on IR screen	Unlikely attack vector, revisit if needed

Security Invariants

Face templates MUST never exist unencrypted on disk
Templates MUST be bound to this device's TPM
Authentication MUST fail closed (any error → reject)
Anti-spoofing MUST run before embedding comparison
Frame data MUST be zeroized after use

Contributing

Code of Conduct

We follow the Contributor Covenant.

Getting Started

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Write tests for your changes
Ensure all tests pass (cargo test && make -C pam-module test)
Run lints (cargo clippy && cargo fmt --check)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Code Style

Rust: Follow rustfmt defaults
C: Follow Linux kernel style
Commits: Conventional Commits format

Security Issues

Report security vulnerabilities privately to: security@.org

Do NOT open public issues for security vulnerabilities.

License

This project is licensed under the GNU General Public License v3.0 — see LICENSE for details.

TPM integration code may have additional licensing considerations due to TCG specifications.

Acknowledgments

Howdy — inspiration and proof of concept
linux-enable-ir-emitter — IR emitter research
dlib — face recognition foundations
ONNX Runtime — cross-platform ML inference

Contact

Project Lead: [TBD]
Mailing List: [TBD]
Matrix/IRC: [TBD]

Last updated: January 2026

23 KiB Raw Blame History

Linux Hello

Table of Contents

Project Overview

Problem Statement

Goals

Non-Goals

Architecture

Component Communication

Security Model

Biometric Template Protection

Anti-Spoofing Layers

Authentication Flow

Threat Model Summary

Components

1. PAM Module (pam_linux_hello.so)

2. Daemon (linux-hello-daemon)

3. IR Camera Interface

4. Face Detection & Embedding

5. Anti-Spoofing Module

6. TPM2 Storage Module

7. CLI Tool (linux-hello)

Technical Specifications

Hardware Requirements

Supported IR Cameras

Performance Targets

Development Phases

Phase 1: Foundation (Weeks 1-4)

Phase 2: Core Authentication (Weeks 5-8)

Phase 3: Security Hardening (Weeks 9-14)

Phase 4: Polish & Integration (Weeks 15-18)

Phase 5: Hardening & Release (Weeks 19-22)

Build Instructions

Prerequisites

Building

Configuration

PAM Configuration

Testing Strategy

Unit Tests

Integration Tests

Security Tests

Performance Benchmarks

Threat Model

In Scope

Out of Scope

Security Invariants

Contributing

Code of Conduct

Getting Started

Code Style

Security Issues

License

Acknowledgments

Contact

23 KiB

Raw Blame History

1. PAM Module (`pam_linux_hello.so`)

2. Daemon (`linux-hello-daemon`)

7. CLI Tool (`linux-hello`)