Archivst Desktop Manual

Interface preview

Labels below match the callouts you see in the app.

BackupSet JSON Configuration

Archivst Desktop stores your backup configuration as JSON in your user profile. Understanding this structure helps with troubleshooting and advanced scenarios.

File Location

• Default path: %APPDATA%\ArchivstDesktop\defaultset.json
• Debug builds: %APPDATA%\ArchivstDesktop\debug.defaultset.json
• Custom location: Configure via AppSettings.DefaultBackupSetDirectory in appsettings.json
• Backup copy: Automatic backup created as defaultset.json.backup (weekly)
• Cloud sync: BackupSet is synchronized to cloud storage for recovery

JSON Structure

The BackupSet JSON contains these key sections:

{
  "ScheduleEnabled": true,
  "TotalFiles": 15234,
  "TotalBytes": 52428800,
  "TotalFilesOnline": 12000,
  "TotalBytesOnline": 40960000,
  "LastBackup": "2024-01-15T14:30:00Z",
  "Folders": [
    {
      "Name": "Documents",
      "Path": "C:\\Users\\YourName\\Documents",
      "IsCloud": true,
      "IsEncrypted": true,
      "AutoSelectFiles": true,
      "AutoSelectDaysAge": 3,
      "AutoCompressImages": true,
      "AutoCompressFormat": 1,
      "AutoCompressQuality": 85,
      "AutoCompressMaxRes": 2048,
      "DuplicateFileHandling": "Ignore",
      "FilesConditions": {
        "Wildcard": {
          "Include": [],
          "Exclude": ["*.tmp", "*.log"]
        }
      }
    }
  ],
  "Hydration": {
    "Documents/photo.jpg": "Hydrated",
    "Documents/archive.zip": "Archived"
  }
}

Key Properties

• ScheduleEnabled: Master switch for scheduled backups
• TotalFiles/TotalBytes: Aggregate statistics across all folders
• TotalFilesOnline/TotalBytesOnline: Count of hydrated (downloadable) items
• LastBackup: UTC timestamp of last successful backup run
• Folders: Array of folder configurations (see Folder Properties below)
• Hydration: Dictionary mapping file paths to restoration states (Archived/Hydrating/Hydrated)

Folder Properties

• Basic: Name, Path, Created timestamp
• Destinations: IsCloud, IsLocal (with LocalDestination), IsFtp (with FtpDomain, FtpPath, FtpUser)
• Security: IsEncrypted (files encrypted before storage)
• Filters: FoldersConditions, FilesConditions (Exact/Wildcard/Regex Include/Exclude lists)
• Wildcards: UseWildcard, Wildcards[], WildcardsAlways[], UseWildcardVideo, UseAlwaysWildcard
• Auto-selection: AutoSelectFiles, AutoSelectDaysAge (default: 3 days)
• Image processing: See Image Compression section below
• Deduplication: DuplicateFileHandling (Copy or Ignore mode)
• Schedule: Per-folder schedule override (optional)

Manual Editing

Warning: Always close Archivst Desktop before editing the BackupSet JSON. Invalid JSON will prevent the app from loading your configuration.

• Close Archivst Desktop completely (exit from system tray)
• Make a backup copy of defaultset.json before editing
• Use a JSON validator to verify syntax after changes
• Restart the application; check Logs panel for parse errors

Recovery

• Local recovery: If corrupt, app automatically tries defaultset.json.backup
• Cloud recovery: If both local copies fail, downloads from cloud storage
• Fresh start: Delete both JSON files; app creates empty BackupSet on next launch

File Deduplication

Archivst Desktop intelligently handles duplicate files to save storage space and transfer bandwidth. The deduplication system operates at two levels: detection and handling.

How Deduplication Works

• Content hashing: Each file is hashed using MD5 during backup. The hash (ContentHash) is stored in the file's metadata.
• Hash index: A local text file (hash_index.txt) maps content hashes to file paths for fast duplicate detection.
• Detection: Before uploading, the system checks if a file with identical content already exists in your backup.
• Handling modes: Configure per-folder via DuplicateFileHandling property.

Deduplication Modes

• Copy (default): Always backs up duplicates. Use when you want every copy preserved (e.g., identical files in different project folders).
• Ignore: Skips files with identical content already backed up. Saves storage but means only one copy exists in cloud.

Hash Index Location

The hash index is stored per-folder at:

%APPDATA%\ArchivstDesktop\fileDetail\[FOLDER_ID]\hash_index.txt

Format: Each line contains HASH|RELATIVE_PATH

File Metadata Storage

Each backed-up file has metadata stored in:

%APPDATA%\ArchivstDesktop\fileDetail\[FOLDER_ID]\[RELATIVE_PATH].json

This JSON file tracks:

• ContentHash: MD5 hash of file content (hex string)
• IsDuplicate: Boolean flag if file is a duplicate
• DuplicateOfPath: Path to original file (if duplicate)
• Versions[]: Array of version metadata (see Versioning section)
• OriginalSize/DestinationSize: File sizes before/after compression
• LastModified: Source file modification timestamp
• Outcome: Last backup result (Success, Skipped, FailureFile, etc.)

Moving vs Copying Files

Archivst detects when files are moved within a backup folder:

• Move detection: If a file disappears from one path but appears at another with identical hash, it's considered moved.
• Safety threshold: If more than 75% of files appear moved, backup is aborted to prevent data loss (configurable via BackupSettings.AbortIfMovedPercent).
• Pre-scan mode: Enable BackupSettings.PreScanVerbose for detailed move detection logs.

Performance Considerations

• Hash index is rebuilt at the start of each backup from existing file metadata
• Large backups (100k+ files) may take 10-30 seconds to build the index
• Hash calculation adds ~5-10% overhead per file during initial backup
• Subsequent backups skip unchanged files, so overhead is minimal

Manual Management

To reset deduplication state for a folder:

• Navigate to %APPDATA%\ArchivstDesktop\fileDetail\[FOLDER_ID]\
• Delete hash_index.txt and all file JSON metadata
• Next backup will treat all files as new

Settings File & DateTime Metadata

Archivst Desktop uses appsettings.json to persist application-wide settings and preferences. This file is separate from the BackupSet JSON and controls app behavior.

Settings File Location

• User settings: %APPDATA%\ArchivstDesktop\appsettings.json
• CLI fallback: In CLI mode, can use executable directory's appsettings.json
• Priority: Roaming user settings override local application settings

Settings Structure

{
  "backupsettings": {
    "CopyThreads": 8,
    "CopyCalmThreads": 2,
    "RestoreThreads": 2,
    "VideoRegex": "^.*\\.(webm|mkv|flv|mp4|...)$",
    "RawRegex": "^.*\\.(3fr|arw|cr2|cr3|dng|nef|...)$",
    "FtpTimeout": 0,
    "AbortIfMovedPercent": 75,
    "PreScanVerbose": false
  },
  "cloudsettings": {
    "CopyThreads": 2,
    "LimitUploadKbps": 0
  },
  "appsettings": {
    "Env": "live",
    "WebDomain": "https://www.archivst.com",
    "FunctionDomain": "https://apis.archivst.com/",
    "AutoUpdate": true,
    "DarkMode": true,
    "StartWithWindows": false,
    "ScheduleDisabledPollMinutes": 60,
    "DefaultBackupSetDirectory": null
  }
}

DateTime Metadata Handling

Archivst preserves file datetime metadata throughout the backup and restore lifecycle:

Captured Timestamps

• LastModified: Source file's last write time (preserved from FileInfo.LastWriteTime)
• Created: File creation timestamp (preserved during restore)
• DateStored: UTC timestamp when file was backed up to destination

Version Metadata (FileDetailsVersionModel)

Each backup creates a version entry with:

• DateStored: UTC time when this version was uploaded
• LastModified: Source file modification time for this version
• VersionId: Azure Blob version identifier (for cloud backups)
• Location/AccountHost: Storage endpoint where version is stored
• Container: Container or folder path
• BlobName: Blob name or file path in destination
• AccessTier: Azure storage tier (Hot, Cool, Archive)

Point-in-Time Restore

• Versions array allows restore from any previous backup date
• UI presents version history; select a datetime to restore older file version
• Cloud versioning: Azure Blob versioning tracks all changes automatically
• Local/FTP: Versions tracked via filename suffixes and metadata

Timestamp Accuracy

• Storage format: ISO 8601 UTC format (yyyy-MM-ddTHH:mm:ssZ)
• Comparison: Uses ToUniversalString() extension for consistent matching
• File matching: Files are considered unchanged if OriginalSize and LastModified match exactly
• Timezone handling: All timestamps converted to UTC for storage; displayed in local time in UI

Settings Editor (In-App)

The Settings panel in Archivst Desktop provides a datagrid editor:

• Properties marked with [Hide] attribute are not shown in UI (e.g., Token, Env)
• Changes saved immediately to appsettings.json
• Advanced users can edit JSON directly (close app first)

Key Settings Explained

• CopyThreads: Concurrent file transfers during backup (8 default; reduce if network is unstable)
• CopyCalmThreads: Throttled thread count during calm periods (2 default)
• LimitUploadKbps: Bandwidth throttle in KB/s (0 = unlimited)
• AbortIfMovedPercent: Safety: abort backup if >75% of files appear moved (prevents accidental mass deletion)
• PreScanVerbose: Enable detailed logging for file move detection diagnostics
• VideoRegex/RawRegex: Regular expressions to identify video and RAW photo formats
• ScheduleDisabledPollMinutes: Polling interval when scheduling is off (60 min default)

Image Compression & Sizing Options

Archivst Desktop can automatically compress and resize images during backup to reduce storage costs and transfer time. This is especially useful for photo libraries.

Enabling Image Compression

Image compression is configured per-folder. To enable:

• Select a folder in the Backup panel
• Click folder options (gear icon)
• Enable "Compress Images" checkbox
• Configure compression settings (see below)

Compression Settings

These properties control how images are processed:

AutoCompressImages (boolean)

• Default: false (disabled)
• Effect: Master switch; if false, all other compression settings are ignored
• Usage: Enable for folders containing photos, screenshots, or other image-heavy content

AutoCompressFormat (integer)

• Values: 1 = JPEG/PNG (typical), other formats may be added in future
• Default: 1
• Behavior: Determines output format for compressed images
• Smart conversion: Preserves transparency (PNG) or converts to JPEG for photos

AutoCompressQuality (integer, 0-100)

• Range: 0 (lowest quality, smallest size) to 100 (highest quality, largest size)
• Recommended: 80-90 for photos, 85-95 for documents/screenshots
• Default: 0 (app may default to 85 if enabled)
• Trade-off: Quality 85 typically reduces size by 50-70% with minimal visible loss

AutoCompressIgnoreSize (decimal, MB)

• Purpose: Skip compression for small images already optimized
• Default: 0 (compress all images)
• Example: Set to 0.5 to skip images under 500 KB
• Use case: Avoid re-compressing thumbnails, icons, or web-optimized images

AutoCompressMaxRes (integer, pixels)

• Purpose: Resize images down to maximum resolution (longest edge)
• Default: 0 (no resizing, preserve original dimensions)
• Examples:
  • 2048 = suitable for 2K displays, good balance
  • 4096 = 4K resolution, high quality
  • 1920 = Full HD, smaller files
• Behavior: Only downscales; never upscales smaller images
• Aspect ratio: Always preserved during resize

FallbackCompressionFailure (boolean)

• Default: false
• True: If compression fails (e.g., corrupted image, unsupported format), upload original file instead
• False: Skip file and log error if compression fails
• Recommended: Enable to ensure no files are missed due to conversion issues

OnlyRaw (boolean)

• Default: false
• Purpose: Photography workflow option to process only RAW image formats
• RAW formats: CR2, CR3, NEF, ARW, DNG, RAF, etc. (defined by BackupSettings.RawRegex)
• Use case: Photographers who want to compress RAW previews but preserve original RAW files

Advanced: Image Verification

Optional quality checks to ensure compressed images are acceptable:

AutoCompressVerify (boolean)

• Default: false (disabled for performance)
• Effect: Enables similarity checking after compression
• Performance: Adds ~30-50% overhead; only enable if quality is critical

AutoCompressMinPsnr (double)

• Default: 25 dB (decibels)
• Metric: Peak Signal-to-Noise Ratio; higher = better quality
• Threshold: Compressed image must meet this PSNR or compression is rejected
• Typical values: 30+ = excellent, 25-30 = good, <25 = visible artifacts

AutoCompressMinSimilarity (double, 0.0-1.0)

• Default: 0.90 (90% similarity required)
• Metric: Normalized structural similarity index
• Threshold: Compressed image must be at least this similar to original
• Recommended: 0.85-0.95 for most use cases

File Metadata After Compression

When an image is compressed, metadata records both versions:

• OriginalSize: Size of source image file
• DestinationSize: Size after compression
• DestinationName: Filename with new extension if format changed
• ConversionFailedUploadedOriginal: Flag indicating fallback to original occurred
• AlteredFileWithRelativePath: Path to compressed version

Supported Image Formats

• Input: JPEG, PNG, BMP, TIFF, WebP, and common RAW formats
• Output: JPEG (photos), PNG (transparency/graphics), WebP (future)
• Excluded: GIF, SVG, ICO (copied without compression)

Example Configurations

Casual Photos (Balanced)

AutoCompressImages = true
AutoCompressQuality = 85
AutoCompressMaxRes = 2048
AutoCompressIgnoreSize = 0.5
FallbackCompressionFailure = true

Result: 2K images at 85% quality, skip files under 500 KB, fallback if needed. Typical 60-70% size reduction.

Professional Photography (High Quality)

AutoCompressImages = true
AutoCompressQuality = 95
AutoCompressMaxRes = 4096
AutoCompressVerify = true
AutoCompressMinPsnr = 30
AutoCompressMinSimilarity = 0.95
OnlyRaw = false

Result: 4K images at 95% quality with verification. Minimal quality loss, ~30-40% size reduction.

Archive/Web Images (Maximum Compression)

AutoCompressImages = true
AutoCompressQuality = 75
AutoCompressMaxRes = 1920
AutoCompressIgnoreSize = 0

Result: Full HD images at 75% quality. Aggressive compression, ~75-85% size reduction.

Troubleshooting Compression

• Images not compressing: Check AutoCompressImages is enabled and file is not below AutoCompressIgnoreSize threshold
• Quality too low: Increase AutoCompressQuality or AutoCompressMaxRes
• Files too large: Decrease AutoCompressQuality or AutoCompressMaxRes
• Conversion errors: Enable FallbackCompressionFailure to upload originals; check Logs panel for details
• Slow backups: Disable AutoCompressVerify; reduce AutoCompressMaxRes

Performance Impact

• Compression time: ~50-200ms per image (varies by size and settings)
• Verification time: +30-50% if AutoCompressVerify enabled
• CPU usage: High during compression; uses multi-threading where possible
• Memory: Loads full image into memory; large RAW files (50+ MB) may slow systems with <8GB RAM
• Disk I/O: Temporary files created in %TEMP% during conversion (cleaned up after)

Best Practices

• Test settings on a small subset of files before applying to entire library
• Use AutoCompressIgnoreSize to avoid reprocessing already-optimized images
• For mixed content folders, disable compression and use separate folders for photos vs documents
• Monitor storage savings in folder statistics after first backup
• Keep originals locally until you verify restored images meet your quality standards