- Add new Snapchat story extractor with spotlight and user story support
- Expand browser cookie extraction to support Zen Browser and multi-platform profiles
- Significantly enhance XenForo extractor with gallery, media, and attachment support
- Add APPDATA-based profile discovery for Windows browsers
- Update main.rs with new extractor wiring and improved CLI handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add set_oauth() method to Extractor trait (base.rs)
- Implement set_oauth() in PixivExtractor (pixiv.rs)
- Implement set_oauth() in DeviantArtExtractor (deviantart.rs)
- Wire OAuth config to extractors in main.rs (was placeholder)
This fixes the gap found during Phase 6 verification where OAuth
tokens from config were loaded but never passed to extractors.
- Created summary for plan 06-04
- Updated STATE.md with new plan completion and metrics
- Implemented --simulate dry-run mode, --destination wiring, and OAuth config support
- Added OauthConfig struct with access_token, refresh_token, client_id, client_secret
- Added oauth field to ExtractorConfig (HashMap<String, OauthConfig>)
- Config file can now specify OAuth tokens per extractor (e.g., pixiv, deviantart)
- Merges OAuth config when loading multiple config files
- Added OAuth config lookup in main.rs for Pixiv and DeviantArt
- Use CLI --destination > config.downloader.directory > default (.)
- Create destination directory if it doesn't exist before downloading
- Supports both CLI argument and config file directory setting
- Added simulate mode check before download loop
- When --simulate is set, prints URLs that would be downloaded but skips actual download
- Added destination directory creation if it doesn't exist
- Uses --destination argument for download location
- Added cookie loading from --cookies and --cookies-from-browser CLI args
- Added set_cookies() method to Extractor trait (default no-op)
- Implemented set_cookies() for TwitterExtractor and InstagramExtractor
- Extractors now receive cookies during initialization for authenticated requests
- All 145 library tests pass
- Added load_urls_from_file() function to parse URLs from input files
- Supports comments (lines starting with #) and empty lines
- Combines CLI URLs with input file URLs for processing
- Input files processed in order, duplicates allowed
- Summary documents browser cookie extraction implementation
- Firefox and Chrome profile detection and cookie extraction working
- Updated STATE.md with progress and metrics
- Simplified Chrome extraction to avoid duplicate code
- Fixed borrow of moved value in Firefox extraction match arms
- Removed unused temp_path shadowing in Chrome extraction
- All 145 library tests pass
- Implements find_firefox_profile() to locate Firefox profile directories
- Implements extract_firefox_cookies() to read cookies from Firefox SQLite database
- Implements find_chrome_profile() to locate Chrome profile directories
- Implements extract_chrome_cookies() to read cookies from Chrome SQLite database
- Handles encrypted Chrome cookies gracefully with warnings
- Implements extract_browser_cookies() as unified API for both browsers
- Uses tempfile to avoid database locking issues
- Adds comprehensive error handling and logging
- Added rusqlite dependency with bundled feature
- Created src/archive/mod.rs with DownloadArchive trait and SqliteArchive
- Added --download-archive CLI option for archive database path
- Added --download-archive-skip-duplicates flag with default path
- Integrated archive checking in DownloadManager before download
- Records successful downloads to archive after completion
- All 129 tests pass
- Created src/postprocess/metadata.rs with MetadataPostProcessor struct
- Implements PostProcessor trait with process() and finalize() methods
- Writes .metadata.json files next to downloaded files
- Includes ExtendedMetadata for additional structured fields
- Added tests for metadata path generation and JSON writing
- All 106 tests pass
- Created src/postprocess/zip.rs with ZipPostProcessor struct
- Implements PostProcessor trait with process() and finalize() methods
- Supports compression (deflate) and storage modes
- Added StreamingZipWriter for large archive handling
- Added tests for compression methods
- All 106 tests pass
- Added src/postprocess/mod.rs with PostProcessor trait
- DownloadMetadata struct for tracking file download information
- PostProcessorConfig enum for configuring post-processors
- PostProcessorBuilder for composing multiple post-processors
- ZipConfig and MetadataConfig for specific configurations
- Added chrono dependency for timestamp support
- Added zip = { version = "8.0", features = ["deflate"] } for ZIP archive creation
- Added walkdir = "2.5" for directory traversal
- Dependencies resolve without conflicts
- Added --filter-size-min option for minimum file size
- Added --filter-size-max option for maximum file size
- Added --filter-type option for allowed file types
- Added parse_size() utility to convert sizes like '1kb', '1mb', '1gb' to bytes
- Added tests for parse_size() function
- All 106 tests pass
- Added FileFilter struct with min_size, max_size, and allowed_types
- Added filter() method to check if download should proceed based on size/type
- Added Filtered error variant to DownloadError
- Updated DownloadManager::download() to apply filters before downloading
- Filter checks Content-Type header and Content-Length against criteria
- All 105 tests pass
