Compare commits

..

9 Commits

Author SHA1 Message Date
Connor Yoh
65e3141760 Fix error with expansion 2025-09-10 20:30:44 +01:00
Connor Yoh
4e5789e8f4 Clean up 2025-09-10 20:24:54 +01:00
Connor Yoh
3b3a2df392 Simplifying adding raw files 2025-09-10 20:19:56 +01:00
Connor Yoh
055d3acc82 Removed not needed processed files 2025-09-10 19:44:27 +01:00
Connor Yoh
aaa0eb4e0f Fixes page editor loading 2025-09-10 19:37:23 +01:00
Connor Yoh
dc9acdd7cc Show history 2025-09-10 19:05:30 +01:00
Connor Yoh
3cea686acd Semi working 2025-09-10 18:31:47 +01:00
Connor Yoh
f88c3e25d1 4 types 2025-09-10 10:03:35 +01:00
Connor Yoh
f8bdeabe35 Translations 2025-09-09 14:45:44 +01:00
33 changed files with 1494 additions and 2247 deletions

View File

@ -2,275 +2,318 @@
## Overview ## Overview
Stirling PDF implements a comprehensive file history tracking system that embeds metadata directly into PDF documents using the PDF keywords field. This system tracks tool operations, version progression, and file lineage through the processing pipeline. Stirling PDF implements a client-side file history system using IndexedDB storage. File metadata, including version history and tool chains, are stored as `StirlingFileStub` objects that travel alongside the actual file data. This enables comprehensive version tracking, tool history, and file lineage management without modifying PDF content.
## PDF Metadata Format ## Storage Architecture
### Storage Mechanism ### IndexedDB-Based Storage
File history is stored in the PDF **Keywords** field as a JSON string with the prefix `stirling-history:`. File history is stored in the browser's IndexedDB using the `fileStorage` service, providing:
- **Persistent storage**: Survives browser sessions and page reloads
- **Large capacity**: Supports files up to 100GB+ with full metadata
- **Fast queries**: Optimized for file browsing and history lookups
- **Type safety**: Structured TypeScript interfaces
### Metadata Structure ### Core Data Structures
```typescript ```typescript
interface PDFHistoryMetadata { interface StirlingFileStub extends BaseFileMetadata {
stirlingHistory: { id: FileId; // Unique file identifier (UUID)
originalFileId: string; // UUID of the root file in the version chain quickKey: string; // Deduplication key: name|size|lastModified
parentFileId?: string; // UUID of the immediate parent file thumbnailUrl?: string; // Generated thumbnail blob URL
versionNumber: number; // Version number (1, 2, 3, etc.) processedFile?: ProcessedFileMetadata; // PDF page data and processing results
toolChain: ToolOperation[]; // Array of applied tool operations
formatVersion: '1.0'; // Metadata format version // File Metadata
}; name: string;
size: number;
type: string;
lastModified: number;
createdAt: number;
// Version Control
isLeaf: boolean; // True if this is the latest version
versionNumber?: number; // Version number (1, 2, 3, etc.)
originalFileId?: string; // UUID of the root file in version chain
parentFileId?: string; // UUID of immediate parent file
// Tool History
toolHistory?: ToolOperation[]; // Complete sequence of applied tools
} }
interface ToolOperation { interface ToolOperation {
toolName: string; // Tool identifier (e.g., 'compress', 'sanitize') toolName: string; // Tool identifier (e.g., 'compress', 'sanitize')
timestamp: number; // When the tool was applied timestamp: number; // When the tool was applied
parameters?: Record<string, any>; // Tool-specific parameters (optional) }
interface StoredStirlingFileRecord extends StirlingFileStub {
data: ArrayBuffer; // Actual file content
fileId: FileId; // Duplicate for indexing
} }
``` ```
### Standard PDF Metadata Fields Used ## Version Management System
The system uses industry-standard PDF document information fields:
- **Creator**: Set to "Stirling-PDF" (identifies the application)
- **Producer**: Set to "Stirling-PDF" (identifies the PDF library/processor)
- **Title, Author, Subject, CreationDate**: Automatically preserved by pdf-lib during processing
- **Keywords**: Enhanced with Stirling history data while preserving user keywords
**Date Handling Strategy**:
- **PDF CreationDate**: Preserved automatically (document creation date)
- **File.lastModified**: Source of truth for "when file was last changed" (original upload time or tool processing time)
- **No duplication**: Single timestamp approach using File.lastModified for all UI displays
### Example PDF Document Information
```
PDF Document Info:
Title: "User Document Title" (preserved from original)
Author: "Document Author" (preserved from original)
Creator: "Stirling-PDF"
Producer: "Stirling-PDF"
CreationDate: "2025-01-01T10:30:00Z" (preserved from original)
Keywords: ["user-keyword", "stirling-history:{\"stirlingHistory\":{\"originalFileId\":\"abc123\",\"versionNumber\":2,\"toolChain\":[{\"toolName\":\"compress\",\"timestamp\":1756825614618},{\"toolName\":\"sanitize\",\"timestamp\":1756825631545}],\"formatVersion\":\"1.0\"}}"]
File System:
lastModified: 1756825631545 (tool processing time - source of truth for "when file was last changed")
```
## Version Numbering System
### Version Progression ### Version Progression
- **v0**: Original uploaded file (no Stirling PDF processing) - **v1**: Original uploaded file (first version)
- **v1**: First tool applied to original file - **v2**: First tool applied to original
- **v2**: Second tool applied (inherits from v1) - **v3**: Second tool applied (inherits from v2)
- **v3**: Third tool applied (inherits from v2) - **v4**: Third tool applied (inherits from v3)
- **etc.** - **etc.**
### Version Relationships ### Leaf Node System
Only the latest version of each file family is marked as `isLeaf: true`:
- **Leaf files**: Show in default file list, available for tool processing
- **History files**: Hidden by default, accessible via history expansion
### File Relationships
``` ```
document.pdf (v0) document.pdf (v1, isLeaf: false)
↓ compress ↓ compress
document.pdf (v1: compress) document.pdf (v2, isLeaf: false)
↓ sanitize ↓ sanitize
document.pdf (v2: compress → sanitize) document.pdf (v3, isLeaf: true) ← Current active version
↓ ocr
document.pdf (v3: compress → sanitize → ocr)
``` ```
## File Lineage Tracking
### Original File ID
The `originalFileId` remains constant throughout the entire version chain, enabling grouping of all versions of the same logical document.
### Parent-Child Relationships
Each processed file references its immediate parent via `parentFileId`, creating a complete audit trail.
### Tool Chain
The `toolChain` array maintains the complete sequence of tool operations applied to reach the current version.
## Implementation Architecture ## Implementation Architecture
### Frontend Components ### 1. FileStorage Service (`fileStorage.ts`)
#### 1. PDF Metadata Service (`pdfMetadataService.ts`) **Core Methods:**
- **PDF-lib Integration**: Uses pdf-lib for metadata injection/extraction
- **Caching**: ContentCache with 10-minute TTL for performance
- **Encryption Support**: Handles encrypted PDFs with `ignoreEncryption: true`
**Key Methods:**
```typescript ```typescript
// Inject metadata into PDF // Store file with complete metadata
injectHistoryMetadata(pdfBytes: ArrayBuffer, originalFileId: string, parentFileId?: string, toolChain: ToolOperation[], versionNumber: number): Promise<ArrayBuffer> async storeStirlingFile(stirlingFile: StirlingFile, stub: StirlingFileStub): Promise<void>
// Extract metadata from PDF // Load file with metadata
extractHistoryMetadata(pdfBytes: ArrayBuffer): Promise<PDFHistoryMetadata | null> async getStirlingFile(id: FileId): Promise<StirlingFile | null>
async getStirlingFileStub(id: FileId): Promise<StirlingFileStub | null>
// Create new version with incremented number // Query operations
createNewVersion(pdfBytes: ArrayBuffer, parentFileId: string, toolOperation: ToolOperation): Promise<ArrayBuffer> async getLeafStirlingFileStubs(): Promise<StirlingFileStub[]>
async getAllStirlingFileStubs(): Promise<StirlingFileStub[]>
// Version management
async markFileAsProcessed(fileId: FileId): Promise<boolean> // Set isLeaf = false
async markFileAsLeaf(fileId: FileId): Promise<boolean> // Set isLeaf = true
``` ```
#### 2. File History Utilities (`fileHistoryUtils.ts`) ### 2. File Context Integration
- **FileContext Integration**: Links PDF metadata with React state management
- **Version Management**: Handles version grouping and latest version filtering
- **Tool Integration**: Prepares files for tool processing with history injection
**Key Functions:** **FileContext** manages runtime state with `StirlingFileStub[]` in memory:
```typescript ```typescript
// Extract history from File and update FileRecord interface FileContextState {
extractFileHistory(file: File, record: FileRecord): Promise<FileRecord> files: {
ids: FileId[];
// Inject history before tool processing byId: Record<FileId, StirlingFileStub>;
injectHistoryForTool(file: File, sourceFileRecord: FileRecord, toolName: string, parameters?): Promise<File> };
}
// Group files by original ID for version management
groupFilesByOriginal(fileRecords: FileRecord[]): Map<string, FileRecord[]>
// Get only latest version of each file group
getLatestVersions(fileRecords: FileRecord[]): FileRecord[]
``` ```
#### 3. Tool Operation Integration (`useToolOperation.ts`) **Key Operations:**
- **Automatic Injection**: All tool operations automatically inject history metadata - `addFiles()`: Stores new files with initial metadata
- **Version Progression**: Reads current version from PDF and increments appropriately - `addStirlingFileStubs()`: Loads existing files from storage with preserved metadata
- **Universal Support**: Works with single-file, multi-file, and custom tool patterns - `consumeFiles()`: Processes files through tools, creating new versions
### Data Flow ### 3. Tool Operation Integration
``` **Tool Processing Flow:**
1. User uploads PDF → No history (v0) 1. **Input**: User selects files (marked as `isLeaf: true`)
2. Tool processing begins → prepareFilesWithHistory() injects current state 2. **Processing**: Backend processes files and returns results
3. Backend processes PDF → Returns processed file with embedded history 3. **History Creation**: New `StirlingFileStub` created with:
4. FileContext adds result → extractFileHistory() reads embedded metadata - Incremented version number
5. UI displays file → Shows version badges and tool chain - Updated tool history
- Parent file reference
4. **Storage**: Both parent (marked `isLeaf: false`) and child (marked `isLeaf: true`) stored
5. **UI Update**: FileContext updated with new file state
**Child Stub Creation:**
```typescript
export function createChildStub(
parentStub: StirlingFileStub,
operation: { toolName: string; timestamp: number },
resultingFile: File,
thumbnail?: string
): StirlingFileStub {
return {
id: createFileId(),
name: resultingFile.name,
size: resultingFile.size,
type: resultingFile.type,
lastModified: resultingFile.lastModified,
quickKey: createQuickKey(resultingFile),
createdAt: Date.now(),
isLeaf: true,
// Version Control
versionNumber: (parentStub.versionNumber || 1) + 1,
originalFileId: parentStub.originalFileId || parentStub.id,
parentFileId: parentStub.id,
// Tool History
toolHistory: [...(parentStub.toolHistory || []), operation],
thumbnailUrl: thumbnail
};
}
``` ```
## UI Integration ## UI Integration
### File Manager ### File Manager History Display
- **Version Toggle**: Switch between "Latest Only" and "All Versions" views
- **Version Badges**: v0, v1, v2 indicators on file items
- **History Dropdown**: Version timeline with restore functionality
- **Tool Chain Display**: Complete processing history in file details panel
### Active Files Workbench **FileManager** (`FileManager.tsx`) provides:
- **Version Metadata**: Version number in file metadata line (e.g., "PDF file - 3 Pages - v2") - **Default View**: Shows only leaf files (`isLeaf: true`)
- **Tool Chain Overlay**: Bottom overlay showing tool sequence (e.g., "compress → sanitize") - **History Expansion**: Click to show all versions of a file family
- **Real-time Updates**: Immediate display after tool processing - **History Groups**: Nested display using `FileHistoryGroup.tsx`
## Storage and Persistence **FileListItem** (`FileListItem.tsx`) displays:
- **Version Badges**: v1, v2, v3 indicators
- **Tool Chain**: Complete processing history in tooltips
- **History Actions**: "Show/Hide History" toggle, "Restore" for history files
### PDF Metadata ### FileManagerContext Integration
- **Embedded in PDF**: History travels with the document across downloads/uploads
- **Keywords Field**: Uses standard PDF metadata field for maximum compatibility
- **Multiple Keywords**: System handles multiple history entries and extracts latest version
### IndexedDB Storage **File Selection Flow:**
- **Client-side Persistence**: FileMetadata includes extracted history information
- **Lazy Loading**: History extracted when files are accessed from storage
- **Batch Processing**: Large collections processed in batches of 5 to prevent memory issues
### Memory Management
- **ContentCache**: 10-minute TTL, 50-file capacity for metadata extraction results
- **Cleanup**: Automatic cache eviction and expired entry removal
- **Large File Support**: No artificial size limits (supports 100GB+ PDFs)
## Tool Configuration
### Filename Preservation
Most tools preserve the original filename to maintain file identity:
**No Prefix (Filename Preserved):**
- compress, repair, sanitize, addPassword, removePassword, changePermissions, removeCertificateSign, unlockPdfForms, ocr, addWatermark
**With Prefix (Different Content):**
- split (`split_` - creates multiple files)
- convert (`converted_` - changes file format)
### Configuration Pattern
```typescript ```typescript
export const toolOperationConfig = { // Recent files (from storage)
toolType: ToolType.singleFile, onRecentFileSelect: (stirlingFileStubs: StirlingFileStub[]) => void
operationType: 'toolName', // Calls: actions.addStirlingFileStubs(stirlingFileStubs, options)
endpoint: '/api/v1/category/tool-endpoint',
filePrefix: '', // Empty for filename preservation // New uploads
buildFormData: buildToolFormData, onFileUpload: (files: File[]) => void
defaultParameters // Calls: actions.addFiles(files, options)
```
**History Management:**
```typescript
// Toggle history visibility
const { expandedFileIds, onToggleExpansion } = useFileManagerContext();
// Restore history file to current
const handleAddToRecents = (file: StirlingFileStub) => {
fileStorage.markFileAsLeaf(file.id); // Make this version current
}; };
``` ```
### Metadata Preservation Strategy ## Data Flow
The system uses a **minimal touch approach** for PDF metadata:
```typescript ### New File Upload
// Only modify necessary fields, let pdf-lib preserve everything else ```
pdfDoc.setCreator('Stirling-PDF'); 1. User uploads files → addFiles()
pdfDoc.setProducer('Stirling-PDF'); 2. Generate thumbnails and page count
pdfDoc.setKeywords([...existingKeywords, historyKeyword]); 3. Create StirlingFileStub with isLeaf: true, versionNumber: 1
4. Store both StirlingFile + StirlingFileStub in IndexedDB
// File.lastModified = Date.now() for processed files (source of truth) 5. Dispatch to FileContext state
// PDF internal dates (CreationDate, etc.) preserved automatically by pdf-lib
``` ```
**Benefits:** ### Tool Processing
- **Automatic Preservation**: pdf-lib preserves Title, Author, Subject, CreationDate without explicit re-setting ```
- **No Duplication**: File.lastModified is single source of truth for "when file changed" 1. User selects tool + files → useToolOperation()
- **Simpler Code**: Minimal metadata operations reduce complexity and bugs 2. API processes files → returns processed File objects
- **Better Performance**: Fewer PDF reads/writes during processing 3. createChildStub() for each result:
- Parent marked isLeaf: false
- Child created with isLeaf: true, incremented version
4. Store all files with updated metadata
5. Update FileContext with new state
```
## Error Handling and Resilience ### File Loading (Recent Files)
```
1. User selects from FileManager → onRecentFileSelect()
2. addStirlingFileStubs() with preserved metadata
3. Load actual StirlingFile data from storage
4. Files appear in workbench with complete history intact
```
## Performance Optimizations
### Metadata Regeneration
When loading files from storage, missing `processedFile` data is regenerated:
```typescript
// In addStirlingFileStubs()
const needsProcessing = !record.processedFile ||
!record.processedFile.pages ||
record.processedFile.pages.length === 0;
if (needsProcessing) {
const result = await generateThumbnailWithMetadata(stirlingFile);
record.processedFile = createProcessedFile(result.pageCount, result.thumbnail);
}
```
### Memory Management
- **Blob URL Tracking**: Automatic cleanup of thumbnail URLs
- **Lazy Loading**: Files loaded from storage only when needed
- **LRU Caching**: File objects cached in memory with size limits
## File Deduplication
### QuickKey System
Files are deduplicated using `quickKey` format:
```typescript
const quickKey = `${file.name}|${file.size}|${file.lastModified}`;
```
This prevents duplicate uploads while allowing different versions of the same logical file.
## Error Handling
### Graceful Degradation ### Graceful Degradation
- **Extraction Failures**: Files display normally without history if metadata extraction fails - **Storage Failures**: Files continue to work without persistence
- **Encrypted PDFs**: System handles encrypted documents with `ignoreEncryption` option - **Metadata Issues**: Missing metadata regenerated on demand
- **Corrupted Metadata**: Invalid history metadata is silently ignored with fallback to basic file info - **Version Conflicts**: Automatic version number resolution
### Performance Considerations ### Recovery Scenarios
- **Caching**: Metadata extraction results are cached to avoid re-parsing - **Corrupted Storage**: Automatic cleanup and re-initialization
- **Batch Processing**: Large file collections processed in controlled batches - **Missing Files**: Stubs cleaned up automatically
- **Async Extraction**: History extraction doesn't block file operations - **Version Mismatches**: Automatic version chain reconstruction
## Developer Guidelines ## Developer Guidelines
### Adding History to New Tools ### Adding File History to New Components
1. **Set `filePrefix: ''`** in tool configuration to preserve filenames
2. **Use existing patterns**: Tool operations automatically inherit history injection 1. **Use FileContext Actions**:
3. **Custom processors**: Must handle history injection manually if using custom response handlers ```typescript
const { actions } = useFileActions();
await actions.addFiles(files); // For new uploads
await actions.addStirlingFileStubs(stubs); // For existing files
```
2. **Preserve Metadata When Processing**:
```typescript
const childStub = createChildStub(parentStub, {
toolName: 'compress',
timestamp: Date.now()
}, processedFile, thumbnail);
```
3. **Handle Storage Operations**:
```typescript
await fileStorage.storeStirlingFile(stirlingFile, stirlingFileStub);
const stub = await fileStorage.getStirlingFileStub(fileId);
```
### Testing File History ### Testing File History
1. **Upload a PDF**: Should show no version (v0), original File.lastModified preserved
2. **Apply any tool**: Should show v1 with tool name, File.lastModified updated to processing time
3. **Apply another tool**: Should show v2 with tool chain sequence
4. **Check file manager**: Version toggle, history dropdown, standard PDF metadata should all work
5. **Check workbench**: Tool chain overlay should appear on thumbnails
### Backend Tool Monitoring 1. **Upload files**: Should show v1, marked as leaf
The system automatically logs metadata preservation: 2. **Apply tool**: Should create v2, mark v1 as non-leaf
- **Success**: `✅ METADATA PRESERVED: Tool 'ocr' correctly preserved all PDF metadata` 3. **Check FileManager**: History should show both versions
- **Issues**: `⚠️ METADATA LOSS: Tool 'compress' did not preserve PDF metadata: CreationDate modified, Author stripped` 4. **Restore old version**: Should mark old version as leaf
5. **Check storage**: Both versions should persist in IndexedDB
This helps identify which backend tools need to be updated to preserve standard PDF metadata fields.
### Debugging
Enable development mode logging to see:
- History injection: `📄 Injected PDF history metadata`
- History extraction: `📄 History extraction completed`
- Version progression: Version number increments and tool chain updates
- Metadata issues: Warnings for tools that strip PDF metadata
## Future Enhancements ## Future Enhancements
### Possible Extensions ### Potential Improvements
- **Branching**: Support for parallel processing branches from same source - **Branch History**: Support for parallel processing branches
- **Diff Tracking**: Track specific changes made by each tool - **History Export**: Export complete version history as JSON
- **User Attribution**: Add user information to tool operations - **Conflict Resolution**: Handle concurrent modifications
- **Timestamp Precision**: Enhanced timestamp tracking for audit trails - **Cloud Sync**: Sync history across devices
- **Export Options**: Export complete processing history as JSON/XML - **Compression**: Compress historical file data
### Compatibility ### API Extensions
- **PDF Standard Compliance**: Uses standard PDF Keywords field for broad compatibility - **Batch Operations**: Process multiple version chains simultaneously
- **Backwards Compatibility**: PDFs without history metadata work normally - **Search Integration**: Search within tool history and file metadata
- **Future Versions**: Format version field enables future metadata schema evolution - **Analytics**: Track usage patterns and tool effectiveness
--- ---
**Last Updated**: January 2025 **Last Updated**: January 2025
**Format Version**: 1.0 **Implementation**: Stirling PDF Frontend v2
**Implementation**: Stirling PDF Frontend v2 **Storage Version**: IndexedDB with fileStorage service

View File

@ -2146,6 +2146,13 @@
"storageLow": "Storage is running low. Consider removing old files.", "storageLow": "Storage is running low. Consider removing old files.",
"supportMessage": "Powered by browser database storage for unlimited capacity", "supportMessage": "Powered by browser database storage for unlimited capacity",
"noFileSelected": "No files selected", "noFileSelected": "No files selected",
"showHistory": "Show History",
"hideHistory": "Hide History",
"fileHistory": "FileHistory",
"loadingHistory": "Loading History...",
"lastModified": "Last Modified",
"toolChain": "Tools Applied",
"restore": "Restore",
"searchFiles": "Search files...", "searchFiles": "Search files...",
"recent": "Recent", "recent": "Recent",
"localFiles": "Local Files", "localFiles": "Local Files",

View File

@ -1,7 +1,7 @@
import React, { useState, useCallback, useEffect } from 'react'; import React, { useState, useCallback, useEffect } from 'react';
import { Modal } from '@mantine/core'; import { Modal } from '@mantine/core';
import { Dropzone } from '@mantine/dropzone'; import { Dropzone } from '@mantine/dropzone';
import { FileMetadata } from '../types/file'; import { StirlingFileStub } from '../types/fileContext';
import { useFileManager } from '../hooks/useFileManager'; import { useFileManager } from '../hooks/useFileManager';
import { useFilesModalContext } from '../contexts/FilesModalContext'; import { useFilesModalContext } from '../contexts/FilesModalContext';
import { Tool } from '../types/tool'; import { Tool } from '../types/tool';
@ -15,12 +15,12 @@ interface FileManagerProps {
} }
const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => { const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => {
const { isFilesModalOpen, closeFilesModal, onFilesSelect, onStoredFilesSelect } = useFilesModalContext(); const { isFilesModalOpen, closeFilesModal, onFileUpload, onRecentFileSelect } = useFilesModalContext();
const [recentFiles, setRecentFiles] = useState<FileMetadata[]>([]); const [recentFiles, setRecentFiles] = useState<StirlingFileStub[]>([]);
const [isDragging, setIsDragging] = useState(false); const [isDragging, setIsDragging] = useState(false);
const [isMobile, setIsMobile] = useState(false); const [isMobile, setIsMobile] = useState(false);
const { loadRecentFiles, handleRemoveFile, convertToFile } = useFileManager(); const { loadRecentFiles, handleRemoveFile } = useFileManager();
// File management handlers // File management handlers
const isFileSupported = useCallback((fileName: string) => { const isFileSupported = useCallback((fileName: string) => {
@ -34,33 +34,26 @@ const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => {
setRecentFiles(files); setRecentFiles(files);
}, [loadRecentFiles]); }, [loadRecentFiles]);
const handleFilesSelected = useCallback(async (files: FileMetadata[]) => { const handleRecentFilesSelected = useCallback(async (files: StirlingFileStub[]) => {
try { try {
// Use stored files flow that preserves original IDs // Use StirlingFileStubs directly - preserves all metadata!
const filesWithMetadata = await Promise.all( onRecentFileSelect(files);
files.map(async (metadata) => ({
file: await convertToFile(metadata),
originalId: metadata.id,
metadata
}))
);
onStoredFilesSelect(filesWithMetadata);
} catch (error) { } catch (error) {
console.error('Failed to process selected files:', error); console.error('Failed to process selected files:', error);
} }
}, [convertToFile, onStoredFilesSelect]); }, [onRecentFileSelect]);
const handleNewFileUpload = useCallback(async (files: File[]) => { const handleNewFileUpload = useCallback(async (files: File[]) => {
if (files.length > 0) { if (files.length > 0) {
try { try {
// Files will get IDs assigned through onFilesSelect -> FileContext addFiles // Files will get IDs assigned through onFilesSelect -> FileContext addFiles
onFilesSelect(files); onFileUpload(files);
await refreshRecentFiles(); await refreshRecentFiles();
} catch (error) { } catch (error) {
console.error('Failed to process dropped files:', error); console.error('Failed to process dropped files:', error);
} }
} }
}, [onFilesSelect, refreshRecentFiles]); }, [onFileUpload, refreshRecentFiles]);
const handleRemoveFileByIndex = useCallback(async (index: number) => { const handleRemoveFileByIndex = useCallback(async (index: number) => {
await handleRemoveFile(index, recentFiles, setRecentFiles); await handleRemoveFile(index, recentFiles, setRecentFiles);
@ -85,7 +78,7 @@ const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => {
// Cleanup any blob URLs when component unmounts // Cleanup any blob URLs when component unmounts
useEffect(() => { useEffect(() => {
return () => { return () => {
// FileMetadata doesn't have blob URLs, so no cleanup needed // StoredFileMetadata doesn't have blob URLs, so no cleanup needed
// Blob URLs are managed by FileContext and tool operations // Blob URLs are managed by FileContext and tool operations
console.log('FileManager unmounting - FileContext handles blob URL cleanup'); console.log('FileManager unmounting - FileContext handles blob URL cleanup');
}; };
@ -146,7 +139,7 @@ const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => {
> >
<FileManagerProvider <FileManagerProvider
recentFiles={recentFiles} recentFiles={recentFiles}
onFilesSelected={handleFilesSelected} onRecentFilesSelected={handleRecentFilesSelected}
onNewFilesSelect={handleNewFileUpload} onNewFilesSelect={handleNewFileUpload}
onClose={closeFilesModal} onClose={closeFilesModal}
isFileSupported={isFileSupported} isFileSupported={isFileSupported}

View File

@ -69,7 +69,7 @@ const FileEditorThumbnail = ({
const fileRecord = selectors.getStirlingFileStub(file.id); const fileRecord = selectors.getStirlingFileStub(file.id);
const toolHistory = fileRecord?.toolHistory || []; const toolHistory = fileRecord?.toolHistory || [];
const hasToolHistory = toolHistory.length > 0; const hasToolHistory = toolHistory.length > 0;
const versionNumber = fileRecord?.versionNumber || 0; const versionNumber = fileRecord?.versionNumber || 1;
const downloadSelectedFile = useCallback(() => { const downloadSelectedFile = useCallback(() => {

View File

@ -5,12 +5,12 @@ import ChevronLeftIcon from '@mui/icons-material/ChevronLeft';
import ChevronRightIcon from '@mui/icons-material/ChevronRight'; import ChevronRightIcon from '@mui/icons-material/ChevronRight';
import { useTranslation } from 'react-i18next'; import { useTranslation } from 'react-i18next';
import { getFileSize } from '../../utils/fileUtils'; import { getFileSize } from '../../utils/fileUtils';
import { FileMetadata } from '../../types/file'; import { StirlingFileStub } from '../../types/fileContext';
interface CompactFileDetailsProps { interface CompactFileDetailsProps {
currentFile: FileMetadata | null; currentFile: StirlingFileStub | null;
thumbnail: string | null; thumbnail: string | null;
selectedFiles: FileMetadata[]; selectedFiles: StirlingFileStub[];
currentFileIndex: number; currentFileIndex: number;
numberOfFiles: number; numberOfFiles: number;
isAnimating: boolean; isAnimating: boolean;
@ -72,7 +72,7 @@ const CompactFileDetails: React.FC<CompactFileDetailsProps> = ({
<Text size="xs" c="dimmed"> <Text size="xs" c="dimmed">
{currentFile ? getFileSize(currentFile) : ''} {currentFile ? getFileSize(currentFile) : ''}
{selectedFiles.length > 1 && `${selectedFiles.length} files`} {selectedFiles.length > 1 && `${selectedFiles.length} files`}
{currentFile && ` • v${currentFile.versionNumber || 0}`} {currentFile && ` • v${currentFile.versionNumber || 1}`}
</Text> </Text>
{hasMultipleFiles && ( {hasMultipleFiles && (
<Text size="xs" c="blue"> <Text size="xs" c="blue">
@ -80,9 +80,9 @@ const CompactFileDetails: React.FC<CompactFileDetailsProps> = ({
</Text> </Text>
)} )}
{/* Compact tool chain for mobile */} {/* Compact tool chain for mobile */}
{currentFile?.historyInfo?.toolChain && currentFile.historyInfo.toolChain.length > 0 && ( {currentFile?.toolHistory && currentFile.toolHistory.length > 0 && (
<Text size="xs" c="dimmed"> <Text size="xs" c="dimmed">
{currentFile.historyInfo.toolChain.map(tool => tool.toolName).join(' → ')} {currentFile.toolHistory.map((tool: any) => tool.toolName).join(' → ')}
</Text> </Text>
)} )}
</Box> </Box>

View File

@ -0,0 +1,68 @@
import React from 'react';
import { Box, Text, Collapse, Group } from '@mantine/core';
import { useTranslation } from 'react-i18next';
import { StirlingFileStub } from '../../types/fileContext';
import FileListItem from './FileListItem';
interface FileHistoryGroupProps {
leafFile: StirlingFileStub;
historyFiles: StirlingFileStub[];
isExpanded: boolean;
onDownloadSingle: (file: StirlingFileStub) => void;
onFileDoubleClick: (file: StirlingFileStub) => void;
onFileRemove: (index: number) => void;
isFileSupported: (fileName: string) => boolean;
}
const FileHistoryGroup: React.FC<FileHistoryGroupProps> = ({
leafFile,
historyFiles,
isExpanded,
onDownloadSingle,
onFileDoubleClick,
onFileRemove,
isFileSupported,
}) => {
const { t } = useTranslation();
// Sort history files by version number (oldest first, excluding the current leaf file)
const sortedHistory = historyFiles
.filter(file => file.id !== leafFile.id) // Exclude the leaf file itself
.sort((a, b) => (b.versionNumber || 1) - (a.versionNumber || 1));
if (!isExpanded || sortedHistory.length === 0) {
return null;
}
return (
<Collapse in={isExpanded}>
<Box ml="md" mt="xs" mb="sm">
<Group align="center" mb="sm">
<Text size="xs" fw={600} c="dimmed" tt="uppercase">
{t('fileManager.fileHistory', 'File History')} ({sortedHistory.length})
</Text>
</Group>
<Box ml="md">
{sortedHistory.map((historyFile, index) => (
<FileListItem
key={`history-${historyFile.id}-${historyFile.versionNumber || 1}`}
file={historyFile}
isSelected={false} // History files are not selectable
isSupported={isFileSupported(historyFile.name)}
onSelect={() => {}} // No selection for history files
onRemove={() => onFileRemove(index)} // Pass through remove handler
onDownload={() => onDownloadSingle(historyFile)}
onDoubleClick={() => onFileDoubleClick(historyFile)}
isHistoryFile={true} // This enables "Add to Recents" in menu
isLatestVersion={false} // History files are never latest
// onAddToRecents is accessed from context by FileListItem
/>
))}
</Box>
</Box>
</Collapse>
);
};
export default FileHistoryGroup;

View File

@ -2,11 +2,11 @@ import React from 'react';
import { Stack, Card, Box, Text, Badge, Group, Divider, ScrollArea } from '@mantine/core'; import { Stack, Card, Box, Text, Badge, Group, Divider, ScrollArea } from '@mantine/core';
import { useTranslation } from 'react-i18next'; import { useTranslation } from 'react-i18next';
import { detectFileExtension, getFileSize } from '../../utils/fileUtils'; import { detectFileExtension, getFileSize } from '../../utils/fileUtils';
import { FileMetadata } from '../../types/file'; import { StirlingFileStub } from '../../types/fileContext';
import ToolChain from '../shared/ToolChain'; import ToolChain from '../shared/ToolChain';
interface FileInfoCardProps { interface FileInfoCardProps {
currentFile: FileMetadata | null; currentFile: StirlingFileStub | null;
modalHeight: string; modalHeight: string;
} }
@ -114,19 +114,19 @@ const FileInfoCard: React.FC<FileInfoCardProps> = ({
<Text size="sm" c="dimmed">{t('fileManager.fileVersion', 'Version')}</Text> <Text size="sm" c="dimmed">{t('fileManager.fileVersion', 'Version')}</Text>
{currentFile && {currentFile &&
<Badge size="sm" variant="light" color={currentFile?.versionNumber ? 'blue' : 'gray'}> <Badge size="sm" variant="light" color={currentFile?.versionNumber ? 'blue' : 'gray'}>
v{currentFile ? (currentFile.versionNumber || 0) : ''} v{currentFile ? (currentFile.versionNumber || 1) : ''}
</Badge>} </Badge>}
</Group> </Group>
{/* Tool Chain Display */} {/* Tool Chain Display */}
{currentFile?.historyInfo?.toolChain && currentFile.historyInfo.toolChain.length > 0 && ( {currentFile?.toolHistory && currentFile.toolHistory.length > 0 && (
<> <>
<Divider /> <Divider />
<Box py="xs"> <Box py="xs">
<Text size="xs" c="dimmed" mb="xs">{t('fileManager.toolChain', 'Tools Applied')}</Text> <Text size="xs" c="dimmed" mb="xs">{t('fileManager.toolChain', 'Tools Applied')}</Text>
<ToolChain <ToolChain
toolChain={currentFile.historyInfo.toolChain} toolChain={currentFile.toolHistory}
displayStyle="badges" displayStyle="badges"
size="xs" size="xs"
maxWidth={'180px'} maxWidth={'180px'}

View File

@ -4,6 +4,7 @@ import CloudIcon from '@mui/icons-material/Cloud';
import HistoryIcon from '@mui/icons-material/History'; import HistoryIcon from '@mui/icons-material/History';
import { useTranslation } from 'react-i18next'; import { useTranslation } from 'react-i18next';
import FileListItem from './FileListItem'; import FileListItem from './FileListItem';
import FileHistoryGroup from './FileHistoryGroup';
import { useFileManagerContext } from '../../contexts/FileManagerContext'; import { useFileManagerContext } from '../../contexts/FileManagerContext';
interface FileListAreaProps { interface FileListAreaProps {
@ -20,8 +21,8 @@ const FileListArea: React.FC<FileListAreaProps> = ({
recentFiles, recentFiles,
filteredFiles, filteredFiles,
selectedFilesSet, selectedFilesSet,
fileGroups,
expandedFileIds, expandedFileIds,
loadedHistoryFiles,
onFileSelect, onFileSelect,
onFileRemove, onFileRemove,
onFileDoubleClick, onFileDoubleClick,
@ -53,24 +54,34 @@ const FileListArea: React.FC<FileListAreaProps> = ({
</Center> </Center>
) : ( ) : (
filteredFiles.map((file, index) => { filteredFiles.map((file, index) => {
// Determine if this is a history file based on whether it's in the recent files or loaded as history // All files in filteredFiles are now leaf files only
const isLeafFile = recentFiles.some(rf => rf.id === file.id); const historyFiles = loadedHistoryFiles.get(file.id) || [];
const isHistoryFile = !isLeafFile; // If not in recent files, it's a loaded history file const isExpanded = expandedFileIds.has(file.id);
const isLatestVersion = isLeafFile; // Only leaf files (from recent files) are latest versions
return ( return (
<FileListItem <React.Fragment key={file.id}>
key={file.id} <FileListItem
file={file} file={file}
isSelected={selectedFilesSet.has(file.id)} isSelected={selectedFilesSet.has(file.id)}
isSupported={isFileSupported(file.name)} isSupported={isFileSupported(file.name)}
onSelect={(shiftKey) => onFileSelect(file, index, shiftKey)} onSelect={(shiftKey) => onFileSelect(file, index, shiftKey)}
onRemove={() => onFileRemove(index)} onRemove={() => onFileRemove(index)}
onDownload={() => onDownloadSingle(file)} onDownload={() => onDownloadSingle(file)}
onDoubleClick={() => onFileDoubleClick(file)} onDoubleClick={() => onFileDoubleClick(file)}
isHistoryFile={isHistoryFile} isHistoryFile={false} // All files here are leaf files
isLatestVersion={isLatestVersion} isLatestVersion={true} // All files here are the latest versions
/> />
<FileHistoryGroup
leafFile={file}
historyFiles={historyFiles}
isExpanded={isExpanded}
onDownloadSingle={onDownloadSingle}
onFileDoubleClick={onFileDoubleClick}
onFileRemove={onFileRemove}
isFileSupported={isFileSupported}
/>
</React.Fragment>
); );
}) })
)} )}

View File

@ -1,18 +1,19 @@
import React, { useState } from 'react'; import React, { useState } from 'react';
import { Group, Box, Text, ActionIcon, Checkbox, Divider, Menu, Badge, Button, Loader } from '@mantine/core'; import { Group, Box, Text, ActionIcon, Checkbox, Divider, Menu, Badge, Loader } from '@mantine/core';
import MoreVertIcon from '@mui/icons-material/MoreVert'; import MoreVertIcon from '@mui/icons-material/MoreVert';
import DeleteIcon from '@mui/icons-material/Delete'; import DeleteIcon from '@mui/icons-material/Delete';
import DownloadIcon from '@mui/icons-material/Download'; import DownloadIcon from '@mui/icons-material/Download';
import AddIcon from '@mui/icons-material/Add'; import AddIcon from '@mui/icons-material/Add';
import HistoryIcon from '@mui/icons-material/History'; import HistoryIcon from '@mui/icons-material/History';
import RestoreIcon from '@mui/icons-material/Restore';
import { useTranslation } from 'react-i18next'; import { useTranslation } from 'react-i18next';
import { getFileSize, getFileDate } from '../../utils/fileUtils'; import { getFileSize, getFileDate } from '../../utils/fileUtils';
import { FileMetadata } from '../../types/file'; import { StirlingFileStub } from '../../types/fileContext';
import { useFileManagerContext } from '../../contexts/FileManagerContext'; import { useFileManagerContext } from '../../contexts/FileManagerContext';
import ToolChain from '../shared/ToolChain'; import ToolChain from '../shared/ToolChain';
interface FileListItemProps { interface FileListItemProps {
file: FileMetadata; file: StirlingFileStub;
isSelected: boolean; isSelected: boolean;
isSupported: boolean; isSupported: boolean;
onSelect: (shiftKey?: boolean) => void; onSelect: (shiftKey?: boolean) => void;
@ -38,29 +39,26 @@ const FileListItem: React.FC<FileListItemProps> = ({
const [isHovered, setIsHovered] = useState(false); const [isHovered, setIsHovered] = useState(false);
const [isMenuOpen, setIsMenuOpen] = useState(false); const [isMenuOpen, setIsMenuOpen] = useState(false);
const { t } = useTranslation(); const { t } = useTranslation();
const { fileGroups, expandedFileIds, onToggleExpansion, onAddToRecents, isLoadingHistory, getHistoryError } = useFileManagerContext(); const {expandedFileIds, onToggleExpansion, onAddToRecents } = useFileManagerContext();
// Keep item in hovered state if menu is open // Keep item in hovered state if menu is open
const shouldShowHovered = isHovered || isMenuOpen; const shouldShowHovered = isHovered || isMenuOpen;
// Get version information for this file // Get version information for this file
const leafFileId = isLatestVersion ? file.id : (file.originalFileId || file.id); const leafFileId = isLatestVersion ? file.id : (file.originalFileId || file.id);
const lineagePath = fileGroups.get(leafFileId) || []; const hasVersionHistory = (file.versionNumber || 1) > 1; // Show history for any processed file (v2+)
const hasVersionHistory = (file.versionNumber || 0) > 0; // Show history for any processed file (v1+) const currentVersion = file.versionNumber || 1; // Display original files as v1
const currentVersion = file.versionNumber || 0; // Display original files as v0
const isExpanded = expandedFileIds.has(leafFileId); const isExpanded = expandedFileIds.has(leafFileId);
// Get loading state for this file's history
const isLoadingFileHistory = isLoadingHistory(file.id);
const historyError = getHistoryError(file.id);
return ( return (
<> <>
<Box <Box
p="sm" p="sm"
style={{ style={{
cursor: 'pointer', cursor: isHistoryFile ? 'default' : 'pointer',
backgroundColor: isSelected ? 'var(--mantine-color-gray-1)' : (shouldShowHovered ? 'var(--mantine-color-gray-1)' : 'var(--bg-file-list)'), backgroundColor: isSelected
? 'var(--mantine-color-gray-1)'
: (shouldShowHovered ? 'var(--mantine-color-gray-1)' : 'var(--bg-file-list)'),
opacity: isSupported ? 1 : 0.5, opacity: isSupported ? 1 : 0.5,
transition: 'background-color 0.15s ease', transition: 'background-color 0.15s ease',
userSelect: 'none', userSelect: 'none',
@ -70,49 +68,47 @@ const FileListItem: React.FC<FileListItemProps> = ({
paddingLeft: isHistoryFile ? '2rem' : '0.75rem', // Indent history files paddingLeft: isHistoryFile ? '2rem' : '0.75rem', // Indent history files
borderLeft: isHistoryFile ? '3px solid var(--mantine-color-blue-4)' : 'none' // Visual indicator for history borderLeft: isHistoryFile ? '3px solid var(--mantine-color-blue-4)' : 'none' // Visual indicator for history
}} }}
onClick={(e) => onSelect(e.shiftKey)} onClick={isHistoryFile ? undefined : (e) => onSelect(e.shiftKey)}
onDoubleClick={onDoubleClick} onDoubleClick={onDoubleClick}
onMouseEnter={() => setIsHovered(true)} onMouseEnter={() => setIsHovered(true)}
onMouseLeave={() => setIsHovered(false)} onMouseLeave={() => setIsHovered(false)}
> >
<Group gap="sm"> <Group gap="sm">
<Box> {!isHistoryFile && (
{/* Checkbox for all files */} <Box>
<Checkbox {/* Checkbox for regular files only */}
checked={isSelected} <Checkbox
onChange={() => {}} // Handled by parent onClick checked={isSelected}
size="sm" onChange={() => {}} // Handled by parent onClick
pl="sm" size="sm"
pr="xs" pl="sm"
styles={{ pr="xs"
input: { styles={{
cursor: 'pointer' input: {
} cursor: 'pointer'
}} }
/> }}
</Box> />
</Box>
)}
<Box style={{ flex: 1, minWidth: 0 }}> <Box style={{ flex: 1, minWidth: 0 }}>
<Group gap="xs" align="center"> <Group gap="xs" align="center">
<Text size="sm" fw={500} truncate style={{ flex: 1 }}>{file.name}</Text> <Text size="sm" fw={500} truncate style={{ flex: 1 }}>{file.name}</Text>
{isLoadingFileHistory && <Loader size={14} />} <Badge size="xs" variant="light" color={"blue"}>
<Badge size="xs" variant="light" color={currentVersion > 0 ? "blue" : "gray"}> v{currentVersion}
v{currentVersion} </Badge>
</Badge>
</Group> </Group>
<Group gap="xs" align="center"> <Group gap="xs" align="center">
<Text size="xs" c="dimmed"> <Text size="xs" c="dimmed">
{getFileSize(file)} {getFileDate(file)} {getFileSize(file)} {getFileDate(file)}
{hasVersionHistory && (
<Text span c="dimmed"> has history</Text>
)}
</Text> </Text>
{/* Tool chain for processed files */} {/* Tool chain for processed files */}
{file.historyInfo?.toolChain && file.historyInfo.toolChain.length > 0 && ( {file.toolHistory && file.toolHistory.length > 0 && (
<ToolChain <ToolChain
toolChain={file.historyInfo.toolChain} toolChain={file.toolHistory}
maxWidth={'150px'} maxWidth={'150px'}
displayStyle="text" displayStyle="text"
size="xs" size="xs"
@ -163,44 +159,35 @@ const FileListItem: React.FC<FileListItemProps> = ({
<> <>
<Menu.Item <Menu.Item
leftSection={ leftSection={
isLoadingFileHistory ?
<Loader size={16} /> :
<HistoryIcon style={{ fontSize: 16 }} /> <HistoryIcon style={{ fontSize: 16 }} />
} }
onClick={(e) => { onClick={(e) => {
e.stopPropagation(); e.stopPropagation();
onToggleExpansion(leafFileId); onToggleExpansion(leafFileId);
}} }}
disabled={isLoadingFileHistory}
> >
{isLoadingFileHistory ? {
t('fileManager.loadingHistory', 'Loading History...') :
(isExpanded ? (isExpanded ?
t('fileManager.hideHistory', 'Hide History') : t('fileManager.hideHistory', 'Hide History') :
t('fileManager.showHistory', 'Show History') t('fileManager.showHistory', 'Show History')
) )
} }
</Menu.Item> </Menu.Item>
{historyError && (
<Menu.Item disabled c="red" style={{ fontSize: '12px' }}>
{t('fileManager.historyError', 'Error loading history')}
</Menu.Item>
)}
<Menu.Divider /> <Menu.Divider />
</> </>
)} )}
{/* Add to Recents option for history files */} {/* Restore option for history files */}
{isHistoryFile && ( {isHistoryFile && (
<> <>
<Menu.Item <Menu.Item
leftSection={<AddIcon style={{ fontSize: 16 }} />} leftSection={<RestoreIcon style={{ fontSize: 16 }} />}
onClick={(e) => { onClick={(e) => {
e.stopPropagation(); e.stopPropagation();
onAddToRecents(file); onAddToRecents(file);
}} }}
> >
{t('fileManager.addToRecents', 'Add to Recents')} {t('fileManager.restore', 'Restore')}
</Menu.Item> </Menu.Item>
<Menu.Divider /> <Menu.Divider />
</> </>

View File

@ -42,7 +42,7 @@ export default function Workbench() {
// Get tool registry to look up selected tool // Get tool registry to look up selected tool
const { toolRegistry } = useToolManagement(); const { toolRegistry } = useToolManagement();
const selectedTool = selectedToolId ? toolRegistry[selectedToolId] : null; const selectedTool = selectedToolId ? toolRegistry[selectedToolId] : null;
const { addToActiveFiles } = useFileHandler(); const { addFiles } = useFileHandler();
const handlePreviewClose = () => { const handlePreviewClose = () => {
setPreviewFile(null); setPreviewFile(null);
@ -81,7 +81,7 @@ export default function Workbench() {
setCurrentView("pageEditor"); setCurrentView("pageEditor");
}, },
onMergeFiles: (filesToMerge) => { onMergeFiles: (filesToMerge) => {
filesToMerge.forEach(addToActiveFiles); addFiles(filesToMerge);
setCurrentView("viewer"); setCurrentView("viewer");
} }
})} })}

View File

@ -1,7 +1,7 @@
import React from 'react'; import React from 'react';
import { Box, Center } from '@mantine/core'; import { Box, Center } from '@mantine/core';
import InsertDriveFileIcon from '@mui/icons-material/InsertDriveFile'; import InsertDriveFileIcon from '@mui/icons-material/InsertDriveFile';
import { FileMetadata } from '../../types/file'; import { StirlingFileStub } from '../../types/fileContext';
import DocumentThumbnail from './filePreview/DocumentThumbnail'; import DocumentThumbnail from './filePreview/DocumentThumbnail';
import DocumentStack from './filePreview/DocumentStack'; import DocumentStack from './filePreview/DocumentStack';
import HoverOverlay from './filePreview/HoverOverlay'; import HoverOverlay from './filePreview/HoverOverlay';
@ -9,7 +9,7 @@ import NavigationArrows from './filePreview/NavigationArrows';
export interface FilePreviewProps { export interface FilePreviewProps {
// Core file data // Core file data
file: File | FileMetadata | null; file: File | StirlingFileStub | null;
thumbnail?: string | null; thumbnail?: string | null;
// Optional features // Optional features
@ -22,7 +22,7 @@ export interface FilePreviewProps {
isAnimating?: boolean; isAnimating?: boolean;
// Event handlers // Event handlers
onFileClick?: (file: File | FileMetadata | null) => void; onFileClick?: (file: File | StirlingFileStub | null) => void;
onPrevious?: () => void; onPrevious?: () => void;
onNext?: () => void; onNext?: () => void;
} }

View File

@ -7,7 +7,7 @@ import { useFileHandler } from '../../hooks/useFileHandler';
import { useFilesModalContext } from '../../contexts/FilesModalContext'; import { useFilesModalContext } from '../../contexts/FilesModalContext';
const LandingPage = () => { const LandingPage = () => {
const { addMultipleFiles } = useFileHandler(); const { addFiles } = useFileHandler();
const fileInputRef = React.useRef<HTMLInputElement>(null); const fileInputRef = React.useRef<HTMLInputElement>(null);
const { colorScheme } = useMantineColorScheme(); const { colorScheme } = useMantineColorScheme();
const { t } = useTranslation(); const { t } = useTranslation();
@ -15,7 +15,7 @@ const LandingPage = () => {
const [isUploadHover, setIsUploadHover] = React.useState(false); const [isUploadHover, setIsUploadHover] = React.useState(false);
const handleFileDrop = async (files: File[]) => { const handleFileDrop = async (files: File[]) => {
await addMultipleFiles(files); await addFiles(files);
}; };
const handleOpenFilesModal = () => { const handleOpenFilesModal = () => {
@ -29,7 +29,7 @@ const LandingPage = () => {
const handleFileSelect = async (event: React.ChangeEvent<HTMLInputElement>) => { const handleFileSelect = async (event: React.ChangeEvent<HTMLInputElement>) => {
const files = Array.from(event.target.files || []); const files = Array.from(event.target.files || []);
if (files.length > 0) { if (files.length > 0) {
await addMultipleFiles(files); await addFiles(files);
} }
// Reset the input so the same file can be selected again // Reset the input so the same file can be selected again
event.target.value = ''; event.target.value = '';

View File

@ -1,10 +1,10 @@
import React from 'react'; import React from 'react';
import { Box, Center, Image } from '@mantine/core'; import { Box, Center, Image } from '@mantine/core';
import PictureAsPdfIcon from '@mui/icons-material/PictureAsPdf'; import PictureAsPdfIcon from '@mui/icons-material/PictureAsPdf';
import { FileMetadata } from '../../../types/file'; import { StirlingFileStub } from '../../../types/fileContext';
export interface DocumentThumbnailProps { export interface DocumentThumbnailProps {
file: File | FileMetadata | null; file: File | StirlingFileStub | null;
thumbnail?: string | null; thumbnail?: string | null;
style?: React.CSSProperties; style?: React.CSSProperties;
onClick?: () => void; onClick?: () => void;

View File

@ -17,7 +17,7 @@ const FileStatusIndicator = ({
selectedFiles = [], selectedFiles = [],
}: FileStatusIndicatorProps) => { }: FileStatusIndicatorProps) => {
const { t } = useTranslation(); const { t } = useTranslation();
const { openFilesModal, onFilesSelect } = useFilesModalContext(); const { openFilesModal, onFileUpload } = useFilesModalContext();
const { files: stirlingFileStubs } = useAllFiles(); const { files: stirlingFileStubs } = useAllFiles();
const { loadRecentFiles } = useFileManager(); const { loadRecentFiles } = useFileManager();
const [hasRecentFiles, setHasRecentFiles] = useState<boolean | null>(null); const [hasRecentFiles, setHasRecentFiles] = useState<boolean | null>(null);
@ -44,7 +44,7 @@ const FileStatusIndicator = ({
input.onchange = (event) => { input.onchange = (event) => {
const files = Array.from((event.target as HTMLInputElement).files || []); const files = Array.from((event.target as HTMLInputElement).files || []);
if (files.length > 0) { if (files.length > 0) {
onFilesSelect(files); onFileUpload(files);
} }
}; };
input.click(); input.click();

View File

@ -372,11 +372,12 @@ const Viewer = ({
else if (effectiveFile.url?.startsWith('indexeddb:')) { else if (effectiveFile.url?.startsWith('indexeddb:')) {
const fileId = effectiveFile.url.replace('indexeddb:', '') as FileId /* FIX ME: Not sure this is right - at least probably not the right place for this logic */; const fileId = effectiveFile.url.replace('indexeddb:', '') as FileId /* FIX ME: Not sure this is right - at least probably not the right place for this logic */;
// Get data directly from IndexedDB // Get file directly from IndexedDB
const arrayBuffer = await fileStorage.getFileData(fileId); const file = await fileStorage.getStirlingFile(fileId);
if (!arrayBuffer) { if (!file) {
throw new Error('File not found in IndexedDB - may have been purged by browser'); throw new Error('File not found in IndexedDB - may have been purged by browser');
} }
const arrayBuffer = await file.arrayBuffer();
// Store reference for cleanup // Store reference for cleanup
currentArrayBufferRef.current = arrayBuffer; currentArrayBufferRef.current = arrayBuffer;

View File

@ -22,13 +22,12 @@ import {
FileId, FileId,
StirlingFileStub, StirlingFileStub,
StirlingFile, StirlingFile,
createStirlingFile
} from '../types/fileContext'; } from '../types/fileContext';
// Import modular components // Import modular components
import { fileContextReducer, initialFileContextState } from './file/FileReducer'; import { fileContextReducer, initialFileContextState } from './file/FileReducer';
import { createFileSelectors } from './file/fileSelectors'; import { createFileSelectors } from './file/fileSelectors';
import { AddedFile, addFiles, consumeFiles, undoConsumeFiles, createFileActions } from './file/fileActions'; import { addFiles, addStirlingFileStubs, consumeFiles, undoConsumeFiles, createFileActions } from './file/fileActions';
import { FileLifecycleManager } from './file/lifecycle'; import { FileLifecycleManager } from './file/lifecycle';
import { FileStateContext, FileActionsContext } from './file/contexts'; import { FileStateContext, FileActionsContext } from './file/contexts';
import { IndexedDBProvider, useIndexedDB } from './IndexedDBContext'; import { IndexedDBProvider, useIndexedDB } from './IndexedDBContext';
@ -73,58 +72,44 @@ function FileContextInner({
dispatch({ type: 'SET_UNSAVED_CHANGES', payload: { hasChanges } }); dispatch({ type: 'SET_UNSAVED_CHANGES', payload: { hasChanges } });
}, []); }, []);
const selectFiles = (addedFilesWithIds: AddedFile[]) => { const selectFiles = (stirlingFiles: StirlingFile[]) => {
const currentSelection = stateRef.current.ui.selectedFileIds; const currentSelection = stateRef.current.ui.selectedFileIds;
const newFileIds = addedFilesWithIds.map(({ id }) => id); const newFileIds = stirlingFiles.map(stirlingFile => stirlingFile.fileId);
dispatch({ type: 'SET_SELECTED_FILES', payload: { fileIds: [...currentSelection, ...newFileIds] } }); dispatch({ type: 'SET_SELECTED_FILES', payload: { fileIds: [...currentSelection, ...newFileIds] } });
} }
// File operations using unified addFiles helper with persistence // File operations using unified addFiles helper with persistence
const addRawFiles = useCallback(async (files: File[], options?: { insertAfterPageId?: string; selectFiles?: boolean }): Promise<StirlingFile[]> => { const addRawFiles = useCallback(async (files: File[], options?: { insertAfterPageId?: string; selectFiles?: boolean }): Promise<StirlingFile[]> => {
const addedFilesWithIds = await addFiles('raw', { files, ...options }, stateRef, filesRef, dispatch, lifecycleManager); const stirlingFiles = await addFiles({ files, ...options }, stateRef, filesRef, dispatch, lifecycleManager, enablePersistence);
// Auto-select the newly added files if requested // Auto-select the newly added files if requested
if (options?.selectFiles && addedFilesWithIds.length > 0) { if (options?.selectFiles && stirlingFiles.length > 0) {
selectFiles(addedFilesWithIds); selectFiles(stirlingFiles);
} }
// Persist to IndexedDB if enabled return stirlingFiles;
if (indexedDB && enablePersistence && addedFilesWithIds.length > 0) { }, [enablePersistence]);
await Promise.all(addedFilesWithIds.map(async ({ file, id, thumbnail }) => {
try {
await indexedDB.saveFile(file, id, thumbnail);
} catch (error) {
console.error('Failed to persist file to IndexedDB:', file.name, error);
}
}));
}
return addedFilesWithIds.map(({ file, id }) => createStirlingFile(file, id)); const addStirlingFileStubsAction = useCallback(async (stirlingFileStubs: StirlingFileStub[], options?: { insertAfterPageId?: string; selectFiles?: boolean }): Promise<StirlingFile[]> => {
}, [indexedDB, enablePersistence]); // StirlingFileStubs preserve all metadata - perfect for FileManager use case!
const result = await addStirlingFileStubs(stirlingFileStubs, options, stateRef, filesRef, dispatch, lifecycleManager);
const addProcessedFiles = useCallback(async (filesWithThumbnails: Array<{ file: File; thumbnail?: string; pageCount?: number }>): Promise<StirlingFile[]> => {
const result = await addFiles('processed', { filesWithThumbnails }, stateRef, filesRef, dispatch, lifecycleManager);
return result.map(({ file, id }) => createStirlingFile(file, id));
}, []);
const addStoredFiles = useCallback(async (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: any }>, options?: { selectFiles?: boolean }): Promise<StirlingFile[]> => {
const result = await addFiles('stored', { filesWithMetadata }, stateRef, filesRef, dispatch, lifecycleManager);
// Auto-select the newly added files if requested // Auto-select the newly added files if requested
if (options?.selectFiles && result.length > 0) { if (options?.selectFiles && result.length > 0) {
selectFiles(result); selectFiles(result);
} }
return result.map(({ file, id }) => createStirlingFile(file, id)); return result;
}, []); }, []);
// Action creators // Action creators
const baseActions = useMemo(() => createFileActions(dispatch), []); const baseActions = useMemo(() => createFileActions(dispatch), []);
// Helper functions for pinned files // Helper functions for pinned files
const consumeFilesWrapper = useCallback(async (inputFileIds: FileId[], outputFiles: File[]): Promise<FileId[]> => { const consumeFilesWrapper = useCallback(async (inputFileIds: FileId[], outputStirlingFiles: StirlingFile[], outputStirlingFileStubs: StirlingFileStub[]): Promise<FileId[]> => {
return consumeFiles(inputFileIds, outputFiles, filesRef, dispatch, indexedDB); return consumeFiles(inputFileIds, outputStirlingFiles, outputStirlingFileStubs, filesRef, dispatch);
}, [indexedDB]); }, []);
const undoConsumeFilesWrapper = useCallback(async (inputFiles: File[], inputStirlingFileStubs: StirlingFileStub[], outputFileIds: FileId[]): Promise<void> => { const undoConsumeFilesWrapper = useCallback(async (inputFiles: File[], inputStirlingFileStubs: StirlingFileStub[], outputFileIds: FileId[]): Promise<void> => {
return undoConsumeFiles(inputFiles, inputStirlingFileStubs, outputFileIds, filesRef, dispatch, indexedDB); return undoConsumeFiles(inputFiles, inputStirlingFileStubs, outputFileIds, filesRef, dispatch, indexedDB);
@ -143,8 +128,7 @@ function FileContextInner({
const actions = useMemo<FileContextActions>(() => ({ const actions = useMemo<FileContextActions>(() => ({
...baseActions, ...baseActions,
addFiles: addRawFiles, addFiles: addRawFiles,
addProcessedFiles, addStirlingFileStubs: addStirlingFileStubsAction,
addStoredFiles,
removeFiles: async (fileIds: FileId[], deleteFromStorage?: boolean) => { removeFiles: async (fileIds: FileId[], deleteFromStorage?: boolean) => {
// Remove from memory and cleanup resources // Remove from memory and cleanup resources
lifecycleManager.removeFiles(fileIds, stateRef); lifecycleManager.removeFiles(fileIds, stateRef);
@ -199,8 +183,7 @@ function FileContextInner({
}), [ }), [
baseActions, baseActions,
addRawFiles, addRawFiles,
addProcessedFiles, addStirlingFileStubsAction,
addStoredFiles,
lifecycleManager, lifecycleManager,
setHasUnsavedChanges, setHasUnsavedChanges,
consumeFilesWrapper, consumeFilesWrapper,

View File

@ -1,10 +1,9 @@
import React, { createContext, useContext, useState, useRef, useCallback, useEffect, useMemo } from 'react'; import React, { createContext, useContext, useState, useRef, useCallback, useEffect, useMemo } from 'react';
import { FileMetadata } from '../types/file';
import { fileStorage } from '../services/fileStorage'; import { fileStorage } from '../services/fileStorage';
import { StirlingFileStub } from '../types/fileContext';
import { downloadFiles } from '../utils/downloadUtils'; import { downloadFiles } from '../utils/downloadUtils';
import { FileId } from '../types/file'; import { FileId } from '../types/file';
import { getLatestVersions, groupFilesByOriginal, getVersionHistory, createFileMetadataWithHistory } from '../utils/fileHistoryUtils'; import { groupFilesByOriginal } from '../utils/fileHistoryUtils';
import { useMultiFileHistory } from '../hooks/useFileHistory';
// Type for the context value - now contains everything directly // Type for the context value - now contains everything directly
interface FileManagerContextValue { interface FileManagerContextValue {
@ -12,36 +11,33 @@ interface FileManagerContextValue {
activeSource: 'recent' | 'local' | 'drive'; activeSource: 'recent' | 'local' | 'drive';
selectedFileIds: FileId[]; selectedFileIds: FileId[];
searchTerm: string; searchTerm: string;
selectedFiles: FileMetadata[]; selectedFiles: StirlingFileStub[];
filteredFiles: FileMetadata[]; filteredFiles: StirlingFileStub[];
fileInputRef: React.RefObject<HTMLInputElement | null>; fileInputRef: React.RefObject<HTMLInputElement | null>;
selectedFilesSet: Set<string>; selectedFilesSet: Set<string>;
expandedFileIds: Set<string>; expandedFileIds: Set<string>;
fileGroups: Map<string, FileMetadata[]>; fileGroups: Map<string, StirlingFileStub[]>;
loadedHistoryFiles: Map<FileId, StirlingFileStub[]>;
// History loading state
isLoadingHistory: (fileId: FileId) => boolean;
getHistoryError: (fileId: FileId) => string | null;
// Handlers // Handlers
onSourceChange: (source: 'recent' | 'local' | 'drive') => void; onSourceChange: (source: 'recent' | 'local' | 'drive') => void;
onLocalFileClick: () => void; onLocalFileClick: () => void;
onFileSelect: (file: FileMetadata, index: number, shiftKey?: boolean) => void; onFileSelect: (file: StirlingFileStub, index: number, shiftKey?: boolean) => void;
onFileRemove: (index: number) => void; onFileRemove: (index: number) => void;
onFileDoubleClick: (file: FileMetadata) => void; onFileDoubleClick: (file: StirlingFileStub) => void;
onOpenFiles: () => void; onOpenFiles: () => void;
onSearchChange: (value: string) => void; onSearchChange: (value: string) => void;
onFileInputChange: (event: React.ChangeEvent<HTMLInputElement>) => void; onFileInputChange: (event: React.ChangeEvent<HTMLInputElement>) => void;
onSelectAll: () => void; onSelectAll: () => void;
onDeleteSelected: () => void; onDeleteSelected: () => void;
onDownloadSelected: () => void; onDownloadSelected: () => void;
onDownloadSingle: (file: FileMetadata) => void; onDownloadSingle: (file: StirlingFileStub) => void;
onToggleExpansion: (fileId: string) => void; onToggleExpansion: (fileId: FileId) => void;
onAddToRecents: (file: FileMetadata) => void; onAddToRecents: (file: StirlingFileStub) => void;
onNewFilesSelect: (files: File[]) => void; onNewFilesSelect: (files: File[]) => void;
// External props // External props
recentFiles: FileMetadata[]; recentFiles: StirlingFileStub[];
isFileSupported: (fileName: string) => boolean; isFileSupported: (fileName: string) => boolean;
modalHeight: string; modalHeight: string;
} }
@ -52,8 +48,8 @@ const FileManagerContext = createContext<FileManagerContextValue | null>(null);
// Provider component props // Provider component props
interface FileManagerProviderProps { interface FileManagerProviderProps {
children: React.ReactNode; children: React.ReactNode;
recentFiles: FileMetadata[]; recentFiles: StirlingFileStub[];
onFilesSelected: (files: FileMetadata[]) => void; // For selecting stored files onRecentFilesSelected: (files: StirlingFileStub[]) => void; // For selecting stored files
onNewFilesSelect: (files: File[]) => void; // For uploading new local files onNewFilesSelect: (files: File[]) => void; // For uploading new local files
onClose: () => void; onClose: () => void;
isFileSupported: (fileName: string) => boolean; isFileSupported: (fileName: string) => boolean;
@ -66,7 +62,7 @@ interface FileManagerProviderProps {
export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
children, children,
recentFiles, recentFiles,
onFilesSelected, onRecentFilesSelected,
onNewFilesSelect, onNewFilesSelect,
onClose, onClose,
isFileSupported, isFileSupported,
@ -80,19 +76,12 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
const [searchTerm, setSearchTerm] = useState(''); const [searchTerm, setSearchTerm] = useState('');
const [lastClickedIndex, setLastClickedIndex] = useState<number | null>(null); const [lastClickedIndex, setLastClickedIndex] = useState<number | null>(null);
const [expandedFileIds, setExpandedFileIds] = useState<Set<string>>(new Set()); const [expandedFileIds, setExpandedFileIds] = useState<Set<string>>(new Set());
const [loadedHistoryFiles, setLoadedHistoryFiles] = useState<Map<FileId, FileMetadata[]>>(new Map()); // Cache for loaded history const [loadedHistoryFiles, setLoadedHistoryFiles] = useState<Map<FileId, StirlingFileStub[]>>(new Map()); // Cache for loaded history
const fileInputRef = useRef<HTMLInputElement>(null); const fileInputRef = useRef<HTMLInputElement>(null);
// Track blob URLs for cleanup // Track blob URLs for cleanup
const createdBlobUrls = useRef<Set<string>>(new Set()); const createdBlobUrls = useRef<Set<string>>(new Set());
// History loading hook
const {
loadFileHistory,
getHistory,
isLoadingHistory,
getError: getHistoryError
} = useMultiFileHistory();
// Computed values (with null safety) // Computed values (with null safety)
const selectedFilesSet = new Set(selectedFileIds); const selectedFilesSet = new Set(selectedFileIds);
@ -100,39 +89,24 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
// Group files by original file ID for version management // Group files by original file ID for version management
const fileGroups = useMemo(() => { const fileGroups = useMemo(() => {
if (!recentFiles || recentFiles.length === 0) return new Map(); if (!recentFiles || recentFiles.length === 0) return new Map();
// Convert FileMetadata to FileRecord-like objects for grouping utility // Convert StirlingFileStub to FileRecord-like objects for grouping utility
const recordsForGrouping = recentFiles.map(file => ({ const recordsForGrouping = recentFiles.map(file => ({
...file, ...file,
originalFileId: file.originalFileId, originalFileId: file.originalFileId,
versionNumber: file.versionNumber || 0 versionNumber: file.versionNumber || 1
})); }));
return groupFilesByOriginal(recordsForGrouping); return groupFilesByOriginal(recordsForGrouping);
}, [recentFiles]); }, [recentFiles]);
// Get files to display with expansion logic // Get files to display with expansion logic
const displayFiles = useMemo(() => { const displayFiles = useMemo(() => {
if (!recentFiles || recentFiles.length === 0) return []; if (!recentFiles || recentFiles.length === 0) return [];
const expandedFiles = []; // Only return leaf files - history files will be handled by separate components
return recentFiles;
// Since we now only load leaf files, iterate through recent files directly }, [recentFiles]);
for (const leafFile of recentFiles) {
// Add the leaf file (main file shown in list)
expandedFiles.push(leafFile);
// If expanded, add the loaded history files
if (expandedFileIds.has(leafFile.id)) {
const historyFiles = loadedHistoryFiles.get(leafFile.id) || [];
// Sort history files by version number (oldest first)
const sortedHistory = historyFiles.sort((a, b) => (a.versionNumber || 0) - (b.versionNumber || 0));
expandedFiles.push(...sortedHistory);
}
}
return expandedFiles;
}, [recentFiles, expandedFileIds, loadedHistoryFiles]);
const selectedFiles = selectedFileIds.length === 0 ? [] : const selectedFiles = selectedFileIds.length === 0 ? [] :
displayFiles.filter(file => selectedFilesSet.has(file.id)); displayFiles.filter(file => selectedFilesSet.has(file.id));
@ -155,7 +129,7 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
fileInputRef.current?.click(); fileInputRef.current?.click();
}, []); }, []);
const handleFileSelect = useCallback((file: FileMetadata, currentIndex: number, shiftKey?: boolean) => { const handleFileSelect = useCallback((file: StirlingFileStub, currentIndex: number, shiftKey?: boolean) => {
const fileId = file.id; const fileId = file.id;
if (!fileId) return; if (!fileId) return;
@ -196,61 +170,136 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
} }
}, [filteredFiles, lastClickedIndex]); }, [filteredFiles, lastClickedIndex]);
// Helper function to safely determine which files can be deleted
const getSafeFilesToDelete = useCallback((
leafFileIds: string[],
allStoredStubs: StirlingFileStub[]
): string[] => {
const fileMap = new Map(allStoredStubs.map(f => [f.id as string, f]));
const filesToDelete = new Set<string>();
const filesToPreserve = new Set<string>();
// First, identify all files in the lineages of the leaf files being deleted
for (const leafFileId of leafFileIds) {
const currentFile = fileMap.get(leafFileId);
if (!currentFile) continue;
// Always include the leaf file itself for deletion
filesToDelete.add(leafFileId);
// If this is a processed file with history, trace back through its lineage
if (currentFile.versionNumber && currentFile.versionNumber > 1) {
const originalFileId = currentFile.originalFileId || currentFile.id;
// Find all files in this history chain
const chainFiles = allStoredStubs.filter((file: StirlingFileStub) =>
(file.originalFileId || file.id) === originalFileId
);
// Add all files in this lineage as candidates for deletion
chainFiles.forEach(file => filesToDelete.add(file.id));
}
}
// Now identify files that must be preserved because they're referenced by OTHER lineages
for (const file of allStoredStubs) {
const fileOriginalId = file.originalFileId || file.id;
// If this file is a leaf node (not being deleted) and its lineage overlaps with files we want to delete
if (file.isLeaf !== false && !leafFileIds.includes(file.id)) {
// Find all files in this preserved lineage
const preservedChainFiles = allStoredStubs.filter((chainFile: StirlingFileStub) =>
(chainFile.originalFileId || chainFile.id) === fileOriginalId
);
// Mark all files in this preserved lineage as must-preserve
preservedChainFiles.forEach(chainFile => filesToPreserve.add(chainFile.id));
}
}
// Final list: files to delete minus files that must be preserved
const safeToDelete = Array.from(filesToDelete).filter(fileId => !filesToPreserve.has(fileId));
console.log('Deletion analysis:', {
candidatesForDeletion: Array.from(filesToDelete),
mustPreserve: Array.from(filesToPreserve),
safeToDelete
});
return safeToDelete;
}, []);
const handleFileRemove = useCallback(async (index: number) => { const handleFileRemove = useCallback(async (index: number) => {
const fileToRemove = filteredFiles[index]; const fileToRemove = filteredFiles[index];
if (fileToRemove) { if (fileToRemove) {
const deletedFileId = fileToRemove.id; const deletedFileId = fileToRemove.id;
// Get all stored files to analyze lineages
const allStoredStubs = await fileStorage.getAllStirlingFileStubs();
// Get safe files to delete (respecting shared lineages)
const filesToDelete = getSafeFilesToDelete([deletedFileId as string], allStoredStubs);
console.log(`Safely deleting files for ${fileToRemove.name}:`, filesToDelete);
// Clear from selection immediately // Clear from selection immediately
setSelectedFileIds(prev => prev.filter(id => id !== deletedFileId)); setSelectedFileIds(prev => prev.filter(id => !filesToDelete.includes(id)));
// Clear from expanded state to prevent ghost entries // Clear from expanded state to prevent ghost entries
setExpandedFileIds(prev => { setExpandedFileIds(prev => {
const newExpanded = new Set(prev); const newExpanded = new Set(prev);
newExpanded.delete(deletedFileId); filesToDelete.forEach(id => newExpanded.delete(id));
return newExpanded; return newExpanded;
}); });
// Clear from history cache - need to remove this file from any cached history // Clear from history cache - remove all files in the chain
setLoadedHistoryFiles(prev => { setLoadedHistoryFiles(prev => {
const newCache = new Map(prev); const newCache = new Map(prev);
// If the deleted file was a main file with cached history, remove its cache // Remove cache entries for all deleted files
newCache.delete(deletedFileId); filesToDelete.forEach(id => newCache.delete(id as FileId));
// Also remove the deleted file from any other file's history cache // Also remove deleted files from any other file's history cache
for (const [mainFileId, historyFiles] of newCache.entries()) { for (const [mainFileId, historyFiles] of newCache.entries()) {
const filteredHistory = historyFiles.filter(histFile => histFile.id !== deletedFileId); const filteredHistory = historyFiles.filter(histFile => !filesToDelete.includes(histFile.id as string));
if (filteredHistory.length !== historyFiles.length) { if (filteredHistory.length !== historyFiles.length) {
// The deleted file was in this history, update the cache
newCache.set(mainFileId, filteredHistory); newCache.set(mainFileId, filteredHistory);
} }
} }
return newCache; return newCache;
}); });
// Call the parent's deletion logic // Delete safe files from IndexedDB
await onFileRemove(index); try {
for (const fileId of filesToDelete) {
await fileStorage.deleteStirlingFile(fileId as FileId);
}
} catch (error) {
console.error('Failed to delete files from chain:', error);
}
// Call the parent's deletion logic for the main file only
onFileRemove(index);
// Refresh to ensure consistent state // Refresh to ensure consistent state
await refreshRecentFiles(); await refreshRecentFiles();
} }
}, [filteredFiles, onFileRemove, refreshRecentFiles]); }, [filteredFiles, onFileRemove, refreshRecentFiles, getSafeFilesToDelete]);
const handleFileDoubleClick = useCallback((file: FileMetadata) => { const handleFileDoubleClick = useCallback((file: StirlingFileStub) => {
if (isFileSupported(file.name)) { if (isFileSupported(file.name)) {
onFilesSelected([file]); onRecentFilesSelected([file]);
onClose(); onClose();
} }
}, [isFileSupported, onFilesSelected, onClose]); }, [isFileSupported, onRecentFilesSelected, onClose]);
const handleOpenFiles = useCallback(() => { const handleOpenFiles = useCallback(() => {
if (selectedFiles.length > 0) { if (selectedFiles.length > 0) {
onFilesSelected(selectedFiles); onRecentFilesSelected(selectedFiles);
onClose(); onClose();
} }
}, [selectedFiles, onFilesSelected, onClose]); }, [selectedFiles, onRecentFilesSelected, onClose]);
const handleSearchChange = useCallback((value: string) => { const handleSearchChange = useCallback((value: string) => {
setSearchTerm(value); setSearchTerm(value);
@ -288,59 +337,45 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
if (selectedFileIds.length === 0) return; if (selectedFileIds.length === 0) return;
try { try {
// Use the same logic as individual file deletion for consistency // Get all stored files to analyze lineages
// Delete each selected file individually using the same cache update logic const allStoredStubs = await fileStorage.getAllStirlingFileStubs();
const allFilesToDelete = filteredFiles.filter(file =>
selectedFileIds.includes(file.id) // Get safe files to delete (respecting shared lineages)
); const filesToDelete = getSafeFilesToDelete(selectedFileIds, allStoredStubs);
// Deduplicate by file ID since shared files can appear multiple times in the display console.log(`Bulk safely deleting files and their history chains:`, filesToDelete);
const uniqueFilesToDelete = allFilesToDelete.reduce((unique: typeof allFilesToDelete, file) => {
if (!unique.some(f => f.id === file.id)) {
unique.push(file);
}
return unique;
}, []);
const filesToDelete = uniqueFilesToDelete;
const deletedFileIds = new Set(filesToDelete.map(f => f.id));
// Update history cache synchronously // Update history cache synchronously
setLoadedHistoryFiles(prev => { setLoadedHistoryFiles(prev => {
const newCache = new Map(prev); const newCache = new Map(prev);
for (const fileToDelete of filesToDelete) { // Remove cache entries for all deleted files
// If the deleted file was a main file with cached history, remove its cache filesToDelete.forEach(id => newCache.delete(id as FileId));
newCache.delete(fileToDelete.id);
// Also remove deleted files from any other file's history cache
// Also remove the deleted file from any other file's history cache for (const [mainFileId, historyFiles] of newCache.entries()) {
for (const [mainFileId, historyFiles] of newCache.entries()) { const filteredHistory = historyFiles.filter(histFile => !filesToDelete.includes(histFile.id as string));
const filteredHistory = historyFiles.filter(histFile => histFile.id !== fileToDelete.id); if (filteredHistory.length !== historyFiles.length) {
if (filteredHistory.length !== historyFiles.length) { newCache.set(mainFileId, filteredHistory);
// The deleted file was in this history, update the cache
newCache.set(mainFileId, filteredHistory);
}
} }
} }
return newCache; return newCache;
}); });
// Also clear any expanded state for deleted files to prevent ghost entries // Also clear any expanded state for deleted files to prevent ghost entries
setExpandedFileIds(prev => { setExpandedFileIds(prev => {
const newExpanded = new Set(prev); const newExpanded = new Set(prev);
for (const deletedId of deletedFileIds) { filesToDelete.forEach(id => newExpanded.delete(id));
newExpanded.delete(deletedId);
}
return newExpanded; return newExpanded;
}); });
// Clear selection immediately to prevent ghost selections // Clear selection immediately to prevent ghost selections
setSelectedFileIds(prev => prev.filter(id => !deletedFileIds.has(id))); setSelectedFileIds(prev => prev.filter(id => !filesToDelete.includes(id)));
// Delete files from IndexedDB // Delete safe files from IndexedDB
for (const file of filesToDelete) { for (const fileId of filesToDelete) {
await fileStorage.deleteFile(file.id); await fileStorage.deleteStirlingFile(fileId as FileId);
} }
// Refresh the file list to get updated data // Refresh the file list to get updated data
@ -348,7 +383,7 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
} catch (error) { } catch (error) {
console.error('Failed to delete selected files:', error); console.error('Failed to delete selected files:', error);
} }
}, [selectedFileIds, filteredFiles, refreshRecentFiles]); }, [selectedFileIds, filteredFiles, refreshRecentFiles, getSafeFilesToDelete]);
const handleDownloadSelected = useCallback(async () => { const handleDownloadSelected = useCallback(async () => {
@ -369,7 +404,7 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
} }
}, [selectedFileIds, filteredFiles]); }, [selectedFileIds, filteredFiles]);
const handleDownloadSingle = useCallback(async (file: FileMetadata) => { const handleDownloadSingle = useCallback(async (file: StirlingFileStub) => {
try { try {
await downloadFiles([file]); await downloadFiles([file]);
} catch (error) { } catch (error) {
@ -377,9 +412,9 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
} }
}, []); }, []);
const handleToggleExpansion = useCallback(async (fileId: string) => { const handleToggleExpansion = useCallback(async (fileId: FileId) => {
const isCurrentlyExpanded = expandedFileIds.has(fileId); const isCurrentlyExpanded = expandedFileIds.has(fileId);
// Update expansion state // Update expansion state
setExpandedFileIds(prev => { setExpandedFileIds(prev => {
const newSet = new Set(prev); const newSet = new Set(prev);
@ -394,107 +429,52 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
// Load complete history chain if expanding // Load complete history chain if expanding
if (!isCurrentlyExpanded) { if (!isCurrentlyExpanded) {
const currentFileMetadata = recentFiles.find(f => f.id === fileId); const currentFileMetadata = recentFiles.find(f => f.id === fileId);
if (currentFileMetadata && (currentFileMetadata.versionNumber || 0) > 0) { if (currentFileMetadata && (currentFileMetadata.versionNumber || 1) > 1) {
try { try {
// Load the current file to get its full history // Get all stored file metadata for chain traversal
const storedFile = await fileStorage.getFile(fileId as FileId); const allStoredStubs = await fileStorage.getAllStirlingFileStubs();
if (storedFile) { const fileMap = new Map(allStoredStubs.map(f => [f.id, f]));
const file = new File([storedFile.data], storedFile.name, {
type: storedFile.type, // Get the current file's IndexedDB data
lastModified: storedFile.lastModified const currentStoredStub = fileMap.get(fileId as FileId);
}); if (!currentStoredStub) {
console.warn(`No stored file found for ${fileId}`);
// Get the complete history metadata (this will give us original/parent IDs) return;
const historyData = await loadFileHistory(file, fileId as FileId); }
if (historyData?.originalFileId) { // Build complete history chain using IndexedDB metadata
// Load complete history chain by traversing parent relationships const historyFiles: StirlingFileStub[] = [];
const historyFiles: FileMetadata[] = [];
// Find the original file
// Get all stored files for chain traversal
const allStoredMetadata = await fileStorage.getAllFileMetadata(); // Collect only files in this specific branch (ancestors of current file)
const fileMap = new Map(allStoredMetadata.map(f => [f.id, f])); const chainFiles: StirlingFileStub[] = [];
const allFiles = Array.from(fileMap.values());
// Build complete chain by following parent relationships backwards
const visitedIds = new Set([fileId]); // Don't include the current file // Build a map for fast parent lookups
const toProcess = [historyData]; // Start with current file's history data const fileIdMap = new Map<FileId, StirlingFileStub>();
allFiles.forEach(f => fileIdMap.set(f.id, f));
while (toProcess.length > 0) {
const currentHistoryData = toProcess.shift()!; // Trace back from current file through parent chain
let currentFile = fileIdMap.get(fileId);
// Add original file if we haven't seen it while (currentFile?.parentFileId) {
if (currentHistoryData.originalFileId && !visitedIds.has(currentHistoryData.originalFileId)) { const parentFile = fileIdMap.get(currentFile.parentFileId);
visitedIds.add(currentHistoryData.originalFileId); if (parentFile) {
const originalMeta = fileMap.get(currentHistoryData.originalFileId as FileId); chainFiles.push(parentFile);
if (originalMeta) { currentFile = parentFile;
try { } else {
const origStoredFile = await fileStorage.getFile(originalMeta.id); break; // Parent not found, stop tracing
if (origStoredFile) {
const origFile = new File([origStoredFile.data], origStoredFile.name, {
type: origStoredFile.type,
lastModified: origStoredFile.lastModified
});
const origMetadata = await createFileMetadataWithHistory(origFile, originalMeta.id, originalMeta.thumbnail);
historyFiles.push(origMetadata);
}
} catch (error) {
console.warn(`Failed to load original file ${originalMeta.id}:`, error);
}
}
}
// Add parent file if we haven't seen it
if (currentHistoryData.parentFileId && !visitedIds.has(currentHistoryData.parentFileId)) {
visitedIds.add(currentHistoryData.parentFileId);
const parentMeta = fileMap.get(currentHistoryData.parentFileId);
if (parentMeta) {
try {
const parentStoredFile = await fileStorage.getFile(parentMeta.id);
if (parentStoredFile) {
const parentFile = new File([parentStoredFile.data], parentStoredFile.name, {
type: parentStoredFile.type,
lastModified: parentStoredFile.lastModified
});
const parentMetadata = await createFileMetadataWithHistory(parentFile, parentMeta.id, parentMeta.thumbnail);
historyFiles.push(parentMetadata);
// Load parent's history to continue the chain
const parentHistoryData = await loadFileHistory(parentFile, parentMeta.id);
if (parentHistoryData) {
toProcess.push(parentHistoryData);
}
}
} catch (error) {
console.warn(`Failed to load parent file ${parentMeta.id}:`, error);
}
}
}
}
// Also find any files that have the current file as their original (siblings/alternatives)
for (const [metaId, meta] of fileMap) {
if (!visitedIds.has(metaId) && (meta as any).originalFileId === historyData.originalFileId) {
visitedIds.add(metaId);
try {
const siblingStoredFile = await fileStorage.getFile(meta.id);
if (siblingStoredFile) {
const siblingFile = new File([siblingStoredFile.data], siblingStoredFile.name, {
type: siblingStoredFile.type,
lastModified: siblingStoredFile.lastModified
});
const siblingMetadata = await createFileMetadataWithHistory(siblingFile, meta.id, meta.thumbnail);
historyFiles.push(siblingMetadata);
}
} catch (error) {
console.warn(`Failed to load sibling file ${meta.id}:`, error);
}
}
}
// Cache the loaded history files
setLoadedHistoryFiles(prev => new Map(prev.set(fileId as FileId, historyFiles)));
} }
} }
// Sort by version number (oldest first for history display)
chainFiles.sort((a, b) => (a.versionNumber || 1) - (b.versionNumber || 1));
// StirlingFileStubs already have all the data we need - no conversion required!
historyFiles.push(...chainFiles);
// Cache the loaded history files
setLoadedHistoryFiles(prev => new Map(prev.set(fileId as FileId, historyFiles)));
} catch (error) { } catch (error) {
console.warn(`Failed to load history chain for file ${fileId}:`, error); console.warn(`Failed to load history chain for file ${fileId}:`, error);
} }
@ -507,30 +487,19 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
return newMap; return newMap;
}); });
} }
}, [expandedFileIds, recentFiles, loadFileHistory]); }, [expandedFileIds, recentFiles]);
const handleAddToRecents = useCallback(async (file: FileMetadata) => { const handleAddToRecents = useCallback(async (file: StirlingFileStub) => {
try { try {
console.log('Promoting to recents:', file.name, 'version:', file.versionNumber); // Mark the file as a leaf node so it appears in recent files
await fileStorage.markFileAsLeaf(file.id);
// Load the file from storage and create a copy with new ID and timestamp
const storedFile = await fileStorage.getFile(file.id); // Refresh the recent files list to show updated state
if (storedFile) { await refreshRecentFiles();
// Create new file with current timestamp to appear at top
const promotedFile = new File([storedFile.data], file.name, {
type: file.type,
lastModified: Date.now() // Current timestamp makes it appear at top
});
// Add as new file through the normal flow (creates new ID)
onNewFilesSelect([promotedFile]);
console.log('Successfully promoted to recents:', file.name, 'v' + file.versionNumber);
}
} catch (error) { } catch (error) {
console.error('Failed to promote to recents:', error); console.error('Failed to add to recents:', error);
} }
}, [onNewFilesSelect]); }, [refreshRecentFiles]);
// Cleanup blob URLs when component unmounts // Cleanup blob URLs when component unmounts
useEffect(() => { useEffect(() => {
@ -564,10 +533,7 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
selectedFilesSet, selectedFilesSet,
expandedFileIds, expandedFileIds,
fileGroups, fileGroups,
loadedHistoryFiles,
// History loading state
isLoadingHistory,
getHistoryError,
// Handlers // Handlers
onSourceChange: handleSourceChange, onSourceChange: handleSourceChange,
@ -599,8 +565,7 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
fileInputRef, fileInputRef,
expandedFileIds, expandedFileIds,
fileGroups, fileGroups,
isLoadingHistory, loadedHistoryFiles,
getHistoryError,
handleSourceChange, handleSourceChange,
handleLocalFileClick, handleLocalFileClick,
handleFileSelect, handleFileSelect,

View File

@ -1,15 +1,15 @@
import React, { createContext, useContext, useState, useCallback, useMemo } from 'react'; import React, { createContext, useContext, useState, useCallback, useMemo } from 'react';
import { useFileHandler } from '../hooks/useFileHandler'; import { useFileHandler } from '../hooks/useFileHandler';
import { FileMetadata } from '../types/file'; import { useFileActions } from './FileContext';
import { FileId } from '../types/file'; import { StirlingFileStub } from '../types/fileContext';
import { fileStorage } from '../services/fileStorage';
interface FilesModalContextType { interface FilesModalContextType {
isFilesModalOpen: boolean; isFilesModalOpen: boolean;
openFilesModal: (options?: { insertAfterPage?: number; customHandler?: (files: File[], insertAfterPage?: number) => void }) => void; openFilesModal: (options?: { insertAfterPage?: number; customHandler?: (files: File[], insertAfterPage?: number) => void }) => void;
closeFilesModal: () => void; closeFilesModal: () => void;
onFileSelect: (file: File) => void; onFileUpload: (files: File[]) => void;
onFilesSelect: (files: File[]) => void; onRecentFileSelect: (stirlingFileStubs: StirlingFileStub[]) => void;
onStoredFilesSelect: (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>) => void;
onModalClose?: () => void; onModalClose?: () => void;
setOnModalClose: (callback: () => void) => void; setOnModalClose: (callback: () => void) => void;
} }
@ -17,7 +17,8 @@ interface FilesModalContextType {
const FilesModalContext = createContext<FilesModalContextType | null>(null); const FilesModalContext = createContext<FilesModalContextType | null>(null);
export const FilesModalProvider: React.FC<{ children: React.ReactNode }> = ({ children }) => { export const FilesModalProvider: React.FC<{ children: React.ReactNode }> = ({ children }) => {
const { addToActiveFiles, addMultipleFiles, addStoredFiles } = useFileHandler(); const { addFiles } = useFileHandler();
const { actions } = useFileActions();
const [isFilesModalOpen, setIsFilesModalOpen] = useState(false); const [isFilesModalOpen, setIsFilesModalOpen] = useState(false);
const [onModalClose, setOnModalClose] = useState<(() => void) | undefined>(); const [onModalClose, setOnModalClose] = useState<(() => void) | undefined>();
const [insertAfterPage, setInsertAfterPage] = useState<number | undefined>(); const [insertAfterPage, setInsertAfterPage] = useState<number | undefined>();
@ -36,39 +37,45 @@ export const FilesModalProvider: React.FC<{ children: React.ReactNode }> = ({ ch
onModalClose?.(); onModalClose?.();
}, [onModalClose]); }, [onModalClose]);
const handleFileSelect = useCallback((file: File) => { const handleFileUpload = useCallback((files: File[]) => {
if (customHandler) {
// Use custom handler for special cases (like page insertion)
customHandler([file], insertAfterPage);
} else {
// Use normal file handling
addToActiveFiles(file);
}
closeFilesModal();
}, [addToActiveFiles, closeFilesModal, insertAfterPage, customHandler]);
const handleFilesSelect = useCallback((files: File[]) => {
if (customHandler) { if (customHandler) {
// Use custom handler for special cases (like page insertion) // Use custom handler for special cases (like page insertion)
customHandler(files, insertAfterPage); customHandler(files, insertAfterPage);
} else { } else {
// Use normal file handling // Use normal file handling
addMultipleFiles(files); addFiles(files);
} }
closeFilesModal(); closeFilesModal();
}, [addMultipleFiles, closeFilesModal, insertAfterPage, customHandler]); }, [addFiles, closeFilesModal, insertAfterPage, customHandler]);
const handleStoredFilesSelect = useCallback((filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>) => { const handleRecentFileSelect = useCallback(async (stirlingFileStubs: StirlingFileStub[]) => {
if (customHandler) { if (customHandler) {
// Use custom handler for special cases (like page insertion) // Load the actual files from storage for custom handler
const files = filesWithMetadata.map(item => item.file); try {
customHandler(files, insertAfterPage); const loadedFiles: File[] = [];
for (const stub of stirlingFileStubs) {
const stirlingFile = await fileStorage.getStirlingFile(stub.id);
if (stirlingFile) {
loadedFiles.push(stirlingFile);
}
}
if (loadedFiles.length > 0) {
customHandler(loadedFiles, insertAfterPage);
}
} catch (error) {
console.error('Failed to load files for custom handler:', error);
}
} else { } else {
// Use normal file handling // Normal case - use addStirlingFileStubs to preserve metadata
addStoredFiles(filesWithMetadata); if (actions.addStirlingFileStubs) {
actions.addStirlingFileStubs(stirlingFileStubs, { selectFiles: true });
} else {
console.error('addStirlingFileStubs action not available');
}
} }
closeFilesModal(); closeFilesModal();
}, [addStoredFiles, closeFilesModal, insertAfterPage, customHandler]); }, [actions.addStirlingFileStubs, closeFilesModal, customHandler, insertAfterPage]);
const setModalCloseCallback = useCallback((callback: () => void) => { const setModalCloseCallback = useCallback((callback: () => void) => {
setOnModalClose(() => callback); setOnModalClose(() => callback);
@ -78,18 +85,16 @@ export const FilesModalProvider: React.FC<{ children: React.ReactNode }> = ({ ch
isFilesModalOpen, isFilesModalOpen,
openFilesModal, openFilesModal,
closeFilesModal, closeFilesModal,
onFileSelect: handleFileSelect, onFileUpload: handleFileUpload,
onFilesSelect: handleFilesSelect, onRecentFileSelect: handleRecentFileSelect,
onStoredFilesSelect: handleStoredFilesSelect,
onModalClose, onModalClose,
setOnModalClose: setModalCloseCallback, setOnModalClose: setModalCloseCallback,
}), [ }), [
isFilesModalOpen, isFilesModalOpen,
openFilesModal, openFilesModal,
closeFilesModal, closeFilesModal,
handleFileSelect, handleFileUpload,
handleFilesSelect, handleRecentFileSelect,
handleStoredFilesSelect,
onModalClose, onModalClose,
setModalCloseCallback, setModalCloseCallback,
]); ]);

View File

@ -4,24 +4,23 @@
*/ */
import React, { createContext, useContext, useCallback, useRef } from 'react'; import React, { createContext, useContext, useCallback, useRef } from 'react';
const DEBUG = process.env.NODE_ENV === 'development';
import { fileStorage } from '../services/fileStorage'; import { fileStorage } from '../services/fileStorage';
import { FileId } from '../types/file'; import { FileId } from '../types/file';
import { FileMetadata } from '../types/file'; import { StirlingFileStub, createStirlingFile } from '../types/fileContext';
import { generateThumbnailForFile } from '../utils/thumbnailUtils'; import { generateThumbnailForFile } from '../utils/thumbnailUtils';
import { createFileMetadataWithHistory } from '../utils/fileHistoryUtils';
const DEBUG = process.env.NODE_ENV === 'development';
interface IndexedDBContextValue { interface IndexedDBContextValue {
// Core CRUD operations // Core CRUD operations
saveFile: (file: File, fileId: FileId, existingThumbnail?: string) => Promise<FileMetadata>; saveFile: (file: File, fileId: FileId, existingThumbnail?: string) => Promise<StirlingFileStub>;
loadFile: (fileId: FileId) => Promise<File | null>; loadFile: (fileId: FileId) => Promise<File | null>;
loadMetadata: (fileId: FileId) => Promise<FileMetadata | null>; loadMetadata: (fileId: FileId) => Promise<StirlingFileStub | null>;
deleteFile: (fileId: FileId) => Promise<void>; deleteFile: (fileId: FileId) => Promise<void>;
// Batch operations // Batch operations
loadAllMetadata: () => Promise<FileMetadata[]>; loadAllMetadata: () => Promise<StirlingFileStub[]>;
loadLeafMetadata: () => Promise<FileMetadata[]>; // Only leaf files for recent files list loadLeafMetadata: () => Promise<StirlingFileStub[]>; // Only leaf files for recent files list
deleteMultiple: (fileIds: FileId[]) => Promise<void>; deleteMultiple: (fileIds: FileId[]) => Promise<void>;
clearAll: () => Promise<void>; clearAll: () => Promise<void>;
@ -59,22 +58,42 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
if (DEBUG) console.log(`🗂️ Evicted ${toRemove.length} LRU cache entries`); if (DEBUG) console.log(`🗂️ Evicted ${toRemove.length} LRU cache entries`);
}, []); }, []);
const saveFile = useCallback(async (file: File, fileId: FileId, existingThumbnail?: string): Promise<FileMetadata> => { const saveFile = useCallback(async (file: File, fileId: FileId, existingThumbnail?: string): Promise<StirlingFileStub> => {
// Use existing thumbnail or generate new one if none provided // Use existing thumbnail or generate new one if none provided
const thumbnail = existingThumbnail || await generateThumbnailForFile(file); const thumbnail = existingThumbnail || await generateThumbnailForFile(file);
// Store in IndexedDB // Store in IndexedDB (no history data - that's handled by direct fileStorage calls now)
await fileStorage.storeFile(file, fileId, thumbnail); const stirlingFile = createStirlingFile(file, fileId);
// Create minimal stub for storage
const stub: StirlingFileStub = {
id: fileId,
name: file.name,
size: file.size,
type: file.type,
lastModified: file.lastModified,
quickKey: `${file.name}|${file.size}|${file.lastModified}`,
thumbnailUrl: thumbnail,
isLeaf: true,
createdAt: Date.now(),
versionNumber: 1,
originalFileId: fileId,
toolHistory: []
};
await fileStorage.storeStirlingFile(stirlingFile, stub);
const storedFile = await fileStorage.getStirlingFileStub(fileId);
// Cache the file object for immediate reuse // Cache the file object for immediate reuse
fileCache.current.set(fileId, { file, lastAccessed: Date.now() }); fileCache.current.set(fileId, { file, lastAccessed: Date.now() });
evictLRUEntries(); evictLRUEntries();
// Extract history metadata for PDFs and return enhanced metadata // Return StirlingFileStub from the stored file (no conversion needed)
const metadata = await createFileMetadataWithHistory(file, fileId, thumbnail); if (!storedFile) {
throw new Error(`Failed to retrieve stored file after saving: ${file.name}`);
}
return storedFile;
return metadata;
}, []); }, []);
const loadFile = useCallback(async (fileId: FileId): Promise<File | null> => { const loadFile = useCallback(async (fileId: FileId): Promise<File | null> => {
@ -87,14 +106,11 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
} }
// Load from IndexedDB // Load from IndexedDB
const storedFile = await fileStorage.getFile(fileId); const storedFile = await fileStorage.getStirlingFile(fileId);
if (!storedFile) return null; if (!storedFile) return null;
// Reconstruct File object // StirlingFile is already a File object, no reconstruction needed
const file = new File([storedFile.data], storedFile.name, { const file = storedFile;
type: storedFile.type,
lastModified: storedFile.lastModified
});
// Cache for future use with LRU eviction // Cache for future use with LRU eviction
fileCache.current.set(fileId, { file, lastAccessed: Date.now() }); fileCache.current.set(fileId, { file, lastAccessed: Date.now() });
@ -103,34 +119,9 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
return file; return file;
}, [evictLRUEntries]); }, [evictLRUEntries]);
const loadMetadata = useCallback(async (fileId: FileId): Promise<FileMetadata | null> => { const loadMetadata = useCallback(async (fileId: FileId): Promise<StirlingFileStub | null> => {
// Try to get from cache first (no IndexedDB hit) // Load stub directly from storage service
const cached = fileCache.current.get(fileId); return await fileStorage.getStirlingFileStub(fileId);
if (cached) {
const file = cached.file;
return {
id: fileId,
name: file.name,
type: file.type,
size: file.size,
lastModified: file.lastModified
};
}
// Load metadata from IndexedDB (efficient - no data field)
const metadata = await fileStorage.getAllFileMetadata();
const fileMetadata = metadata.find(m => m.id === fileId);
if (!fileMetadata) return null;
return {
id: fileMetadata.id,
name: fileMetadata.name,
type: fileMetadata.type,
size: fileMetadata.size,
lastModified: fileMetadata.lastModified,
thumbnail: fileMetadata.thumbnail
};
}, []); }, []);
const deleteFile = useCallback(async (fileId: FileId): Promise<void> => { const deleteFile = useCallback(async (fileId: FileId): Promise<void> => {
@ -138,121 +129,22 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
fileCache.current.delete(fileId); fileCache.current.delete(fileId);
// Remove from IndexedDB // Remove from IndexedDB
await fileStorage.deleteFile(fileId); await fileStorage.deleteStirlingFile(fileId);
}, []); }, []);
const loadLeafMetadata = useCallback(async (): Promise<FileMetadata[]> => { const loadLeafMetadata = useCallback(async (): Promise<StirlingFileStub[]> => {
const metadata = await fileStorage.getLeafFileMetadata(); // Only get leaf files const metadata = await fileStorage.getLeafStirlingFileStubs(); // Only get leaf files
// Separate PDF and non-PDF files for different processing // All files are already StirlingFileStub objects, no processing needed
const pdfFiles = metadata.filter(m => m.type.includes('pdf')); return metadata;
const nonPdfFiles = metadata.filter(m => !m.type.includes('pdf'));
// Process non-PDF files immediately (no history extraction needed)
const nonPdfMetadata: FileMetadata[] = nonPdfFiles.map(m => ({
id: m.id,
name: m.name,
type: m.type,
size: m.size,
lastModified: m.lastModified,
thumbnail: m.thumbnail,
isLeaf: m.isLeaf
}));
// Process PDF files with controlled concurrency to avoid memory issues
const BATCH_SIZE = 5; // Process 5 PDFs at a time to avoid overwhelming memory
const pdfMetadata: FileMetadata[] = [];
for (let i = 0; i < pdfFiles.length; i += BATCH_SIZE) {
const batch = pdfFiles.slice(i, i + BATCH_SIZE);
const batchResults = await Promise.all(batch.map(async (m) => {
try {
// For PDF files, load and extract basic history for display only
const storedFile = await fileStorage.getFile(m.id);
if (storedFile?.data) {
const file = new File([storedFile.data], m.name, {
type: m.type,
lastModified: m.lastModified
});
return await createFileMetadataWithHistory(file, m.id, m.thumbnail);
}
} catch (error) {
if (DEBUG) console.warn('🗂️ Failed to extract basic metadata from leaf file:', m.name, error);
}
// Fallback to basic metadata without history
return {
id: m.id,
name: m.name,
type: m.type,
size: m.size,
lastModified: m.lastModified,
thumbnail: m.thumbnail,
isLeaf: m.isLeaf
};
}));
pdfMetadata.push(...batchResults);
}
return [...nonPdfMetadata, ...pdfMetadata];
}, []); }, []);
const loadAllMetadata = useCallback(async (): Promise<FileMetadata[]> => { const loadAllMetadata = useCallback(async (): Promise<StirlingFileStub[]> => {
const metadata = await fileStorage.getAllFileMetadata(); const metadata = await fileStorage.getAllStirlingFileStubs();
// Separate PDF and non-PDF files for different processing // All files are already StirlingFileStub objects, no processing needed
const pdfFiles = metadata.filter(m => m.type.includes('pdf')); return metadata;
const nonPdfFiles = metadata.filter(m => !m.type.includes('pdf'));
// Process non-PDF files immediately (no history extraction needed)
const nonPdfMetadata: FileMetadata[] = nonPdfFiles.map(m => ({
id: m.id,
name: m.name,
type: m.type,
size: m.size,
lastModified: m.lastModified,
thumbnail: m.thumbnail
}));
// Process PDF files with controlled concurrency to avoid memory issues
const BATCH_SIZE = 5; // Process 5 PDFs at a time to avoid overwhelming memory
const pdfMetadata: FileMetadata[] = [];
for (let i = 0; i < pdfFiles.length; i += BATCH_SIZE) {
const batch = pdfFiles.slice(i, i + BATCH_SIZE);
const batchResults = await Promise.all(batch.map(async (m) => {
try {
// For PDF files, load and extract history with timeout
const storedFile = await fileStorage.getFile(m.id);
if (storedFile?.data) {
const file = new File([storedFile.data], m.name, {
type: m.type,
lastModified: m.lastModified
});
return await createFileMetadataWithHistory(file, m.id, m.thumbnail);
}
} catch (error) {
if (DEBUG) console.warn('🗂️ Failed to extract history from stored file:', m.name, error);
}
// Fallback to basic metadata if history extraction fails
return {
id: m.id,
name: m.name,
type: m.type,
size: m.size,
lastModified: m.lastModified,
thumbnail: m.thumbnail
};
}));
pdfMetadata.push(...batchResults);
}
return [...nonPdfMetadata, ...pdfMetadata];
}, []); }, []);
const deleteMultiple = useCallback(async (fileIds: FileId[]): Promise<void> => { const deleteMultiple = useCallback(async (fileIds: FileId[]): Promise<void> => {
@ -260,7 +152,7 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
fileIds.forEach(id => fileCache.current.delete(id)); fileIds.forEach(id => fileCache.current.delete(id));
// Remove from IndexedDB in parallel // Remove from IndexedDB in parallel
await Promise.all(fileIds.map(id => fileStorage.deleteFile(id))); await Promise.all(fileIds.map(id => fileStorage.deleteStirlingFile(id)));
}, []); }, []);
const clearAll = useCallback(async (): Promise<void> => { const clearAll = useCallback(async (): Promise<void> => {

View File

@ -8,14 +8,15 @@ import {
FileContextState, FileContextState,
toStirlingFileStub, toStirlingFileStub,
createFileId, createFileId,
createQuickKey createQuickKey,
createStirlingFile,
} from '../../types/fileContext'; } from '../../types/fileContext';
import { FileId, FileMetadata } from '../../types/file'; import { FileId } from '../../types/file';
import { generateThumbnailWithMetadata } from '../../utils/thumbnailUtils'; import { generateThumbnailWithMetadata } from '../../utils/thumbnailUtils';
import { FileLifecycleManager } from './lifecycle'; import { FileLifecycleManager } from './lifecycle';
import { buildQuickKeySet } from './fileSelectors'; import { buildQuickKeySet } from './fileSelectors';
import { extractBasicFileMetadata } from '../../utils/fileHistoryUtils'; import { StirlingFile } from '../../types/fileContext';
import { fileStorage } from '../../services/fileStorage';
const DEBUG = process.env.NODE_ENV === 'development'; const DEBUG = process.env.NODE_ENV === 'development';
/** /**
@ -70,432 +71,268 @@ export function createProcessedFile(pageCount: number, thumbnail?: string) {
} }
/** /**
* File addition types * Create a child StirlingFileStub from a parent stub with proper history management.
* Used when a tool processes an existing file to create a new version with incremented history.
*
* @param parentStub - The parent StirlingFileStub to create a child from
* @param operation - Tool operation information (toolName, timestamp)
* @returns New child StirlingFileStub with proper version history
*/ */
type AddFileKind = 'raw' | 'processed' | 'stored'; export function createChildStub(
parentStub: StirlingFileStub,
operation: { toolName: string; timestamp: number },
resultingFile: File,
thumbnail?: string
): StirlingFileStub {
const newFileId = createFileId();
// Build new tool history by appending to parent's history
const parentToolHistory = parentStub.toolHistory || [];
const newToolHistory = [...parentToolHistory, operation];
// Calculate new version number
const newVersionNumber = (parentStub.versionNumber || 1) + 1;
// Determine original file ID (root of the version chain)
const originalFileId = parentStub.originalFileId || parentStub.id;
// Update the child stub's name to match the processed file
return {
// Copy all parent metadata
...parentStub,
// Update identity and version info
id: newFileId,
versionNumber: newVersionNumber,
parentFileId: parentStub.id,
originalFileId: originalFileId,
toolHistory: newToolHistory,
createdAt: Date.now(),
isLeaf: true, // New child is the current leaf node
name: resultingFile.name,
size: resultingFile.size,
type: resultingFile.type,
lastModified: resultingFile.lastModified,
thumbnailUrl: thumbnail
// Preserve thumbnails and processing metadata from parent
// These will be updated if the child has new thumbnails, but fallback to parent
};
}
interface AddFileOptions { interface AddFileOptions {
// For 'raw' files
files?: File[]; files?: File[];
// For 'processed' files // For 'processed' files
filesWithThumbnails?: Array<{ file: File; thumbnail?: string; pageCount?: number }>; filesWithThumbnails?: Array<{ file: File; thumbnail?: string; pageCount?: number }>;
// For 'stored' files
filesWithMetadata?: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>;
// Insertion position // Insertion position
insertAfterPageId?: string; insertAfterPageId?: string;
}
export interface AddedFile { // Auto-selection after adding
file: File; selectFiles?: boolean;
id: FileId;
thumbnail?: string;
} }
/** /**
* Unified file addition helper - replaces addFiles/addProcessedFiles/addStoredFiles * Unified file addition helper - replaces addFiles
*/ */
export async function addFiles( export async function addFiles(
kind: AddFileKind,
options: AddFileOptions, options: AddFileOptions,
stateRef: React.MutableRefObject<FileContextState>, stateRef: React.MutableRefObject<FileContextState>,
filesRef: React.MutableRefObject<Map<FileId, File>>, filesRef: React.MutableRefObject<Map<FileId, File>>,
dispatch: React.Dispatch<FileContextAction>, dispatch: React.Dispatch<FileContextAction>,
lifecycleManager: FileLifecycleManager lifecycleManager: FileLifecycleManager,
): Promise<AddedFile[]> { enablePersistence: boolean = false
): Promise<StirlingFile[]> {
// Acquire mutex to prevent race conditions // Acquire mutex to prevent race conditions
await addFilesMutex.lock(); await addFilesMutex.lock();
try { try {
const stirlingFileStubs: StirlingFileStub[] = []; const stirlingFileStubs: StirlingFileStub[] = [];
const addedFiles: AddedFile[] = []; const stirlingFiles: StirlingFile[] = [];
// Build quickKey lookup from existing files for deduplication // Build quickKey lookup from existing files for deduplication
const existingQuickKeys = buildQuickKeySet(stateRef.current.files.byId); const existingQuickKeys = buildQuickKeySet(stateRef.current.files.byId);
if (DEBUG) console.log(`📄 addFiles(${kind}): Existing quickKeys for deduplication:`, Array.from(existingQuickKeys));
switch (kind) { const { files = [] } = options;
case 'raw': { if (DEBUG) console.log(`📄 addFiles(raw): Adding ${files.length} files with immediate thumbnail generation`);
const { files = [] } = options;
if (DEBUG) console.log(`📄 addFiles(raw): Adding ${files.length} files with immediate thumbnail generation`);
for (const file of files) { for (const file of files) {
const quickKey = createQuickKey(file); const quickKey = createQuickKey(file);
// Soft deduplication: Check if file already exists by metadata // Soft deduplication: Check if file already exists by metadata
if (existingQuickKeys.has(quickKey)) { if (existingQuickKeys.has(quickKey)) {
if (DEBUG) console.log(`📄 Skipping duplicate file: ${file.name} (quickKey: ${quickKey})`); if (DEBUG) console.log(`📄 Skipping duplicate file: ${file.name} (quickKey: ${quickKey})`);
continue; continue;
}
if (DEBUG) console.log(`📄 Adding new file: ${file.name} (quickKey: ${quickKey})`);
const fileId = createFileId();
filesRef.current.set(fileId, file);
// Generate thumbnail and page count immediately
let thumbnail: string | undefined;
let pageCount: number = 1;
// Route based on file type - PDFs through full metadata pipeline, non-PDFs through simple path
if (file.type.startsWith('application/pdf')) {
try {
if (DEBUG) console.log(`📄 Generating PDF metadata for ${file.name}`);
const result = await generateThumbnailWithMetadata(file);
thumbnail = result.thumbnail;
pageCount = result.pageCount;
if (DEBUG) console.log(`📄 Generated PDF metadata for ${file.name}: ${pageCount} pages, thumbnail: SUCCESS`);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate PDF metadata for ${file.name}:`, error);
}
} else {
// Non-PDF files: simple thumbnail generation, no page count
try {
if (DEBUG) console.log(`📄 Generating simple thumbnail for non-PDF file ${file.name}`);
const { generateThumbnailForFile } = await import('../../utils/thumbnailUtils');
thumbnail = await generateThumbnailForFile(file);
pageCount = 0; // Non-PDFs have no page count
if (DEBUG) console.log(`📄 Generated simple thumbnail for ${file.name}: no page count, thumbnail: SUCCESS`);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate simple thumbnail for ${file.name}:`, error);
}
}
// Create record with immediate thumbnail and page metadata
const record = toStirlingFileStub(file, fileId);
if (thumbnail) {
record.thumbnailUrl = thumbnail;
// Track blob URLs for cleanup (images return blob URLs that need revocation)
if (thumbnail.startsWith('blob:')) {
lifecycleManager.trackBlobUrl(thumbnail);
}
}
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Create initial processedFile metadata with page count
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, thumbnail);
if (DEBUG) console.log(`📄 addFiles(raw): Created initial processedFile metadata for ${file.name} with ${pageCount} pages`);
}
// Extract basic metadata (version number and tool chain) for display
extractBasicFileMetadata(file, record).then(updatedRecord => {
if (updatedRecord !== record && (updatedRecord.versionNumber || updatedRecord.toolHistory)) {
// Basic metadata found, dispatch update to trigger re-render
dispatch({
type: 'UPDATE_FILE_RECORD',
payload: {
id: fileId,
updates: {
versionNumber: updatedRecord.versionNumber,
toolHistory: updatedRecord.toolHistory
}
}
});
}
}).catch(error => {
if (DEBUG) console.warn(`📄 Failed to extract basic metadata for ${file.name}:`, error);
});
existingQuickKeys.add(quickKey);
stirlingFileStubs.push(record);
addedFiles.push({ file, id: fileId, thumbnail });
}
break;
} }
if (DEBUG) console.log(`📄 Adding new file: ${file.name} (quickKey: ${quickKey})`);
case 'processed': { const fileId = createFileId();
const { filesWithThumbnails = [] } = options; filesRef.current.set(fileId, file);
if (DEBUG) console.log(`📄 addFiles(processed): Adding ${filesWithThumbnails.length} processed files with pre-existing thumbnails`);
for (const { file, thumbnail, pageCount = 1 } of filesWithThumbnails) { // Generate thumbnail and page count immediately
const quickKey = createQuickKey(file); let thumbnail: string | undefined;
let pageCount: number = 1;
if (existingQuickKeys.has(quickKey)) { // Route based on file type - PDFs through full metadata pipeline, non-PDFs through simple path
if (DEBUG) console.log(`📄 Skipping duplicate processed file: ${file.name}`); if (file.type.startsWith('application/pdf')) {
continue; try {
} if (DEBUG) console.log(`📄 Generating PDF metadata for ${file.name}`);
const result = await generateThumbnailWithMetadata(file);
const fileId = createFileId(); thumbnail = result.thumbnail;
filesRef.current.set(fileId, file); pageCount = result.pageCount;
if (DEBUG) console.log(`📄 Generated PDF metadata for ${file.name}: ${pageCount} pages, thumbnail: SUCCESS`);
const record = toStirlingFileStub(file, fileId); } catch (error) {
if (thumbnail) { if (DEBUG) console.warn(`📄 Failed to generate PDF metadata for ${file.name}:`, error);
record.thumbnailUrl = thumbnail;
// Track blob URLs for cleanup (images return blob URLs that need revocation)
if (thumbnail.startsWith('blob:')) {
lifecycleManager.trackBlobUrl(thumbnail);
}
}
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Create processedFile with provided metadata
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, thumbnail);
if (DEBUG) console.log(`📄 addFiles(processed): Created initial processedFile metadata for ${file.name} with ${pageCount} pages`);
}
// Extract basic metadata (version number and tool chain) for display
extractBasicFileMetadata(file, record).then(updatedRecord => {
if (updatedRecord !== record && (updatedRecord.versionNumber || updatedRecord.toolHistory)) {
// Basic metadata found, dispatch update to trigger re-render
dispatch({
type: 'UPDATE_FILE_RECORD',
payload: {
id: fileId,
updates: {
versionNumber: updatedRecord.versionNumber,
toolHistory: updatedRecord.toolHistory
}
}
});
}
}).catch(error => {
if (DEBUG) console.warn(`📄 Failed to extract basic metadata for ${file.name}:`, error);
});
existingQuickKeys.add(quickKey);
stirlingFileStubs.push(record);
addedFiles.push({ file, id: fileId, thumbnail });
} }
break; } else {
} // Non-PDF files: simple thumbnail generation, no page count
try {
case 'stored': { if (DEBUG) console.log(`📄 Generating simple thumbnail for non-PDF file ${file.name}`);
const { filesWithMetadata = [] } = options; const { generateThumbnailForFile } = await import('../../utils/thumbnailUtils');
if (DEBUG) console.log(`📄 addFiles(stored): Restoring ${filesWithMetadata.length} files from storage with existing metadata`); thumbnail = await generateThumbnailForFile(file);
pageCount = 0; // Non-PDFs have no page count
for (const { file, originalId, metadata } of filesWithMetadata) { if (DEBUG) console.log(`📄 Generated simple thumbnail for ${file.name}: no page count, thumbnail: SUCCESS`);
const quickKey = createQuickKey(file); } catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate simple thumbnail for ${file.name}:`, error);
if (existingQuickKeys.has(quickKey)) {
if (DEBUG) console.log(`📄 Skipping duplicate stored file: ${file.name} (quickKey: ${quickKey})`);
continue;
}
if (DEBUG) console.log(`📄 Adding stored file: ${file.name} (quickKey: ${quickKey})`);
// Try to preserve original ID, but generate new if it conflicts
let fileId = originalId;
if (filesRef.current.has(originalId)) {
fileId = createFileId();
if (DEBUG) console.log(`📄 ID conflict for stored file ${file.name}, using new ID: ${fileId}`);
}
filesRef.current.set(fileId, file);
const record = toStirlingFileStub(file, fileId);
// Generate processedFile metadata for stored files
let pageCount: number = 1;
// Only process PDFs through PDF worker manager, non-PDFs have no page count
if (file.type.startsWith('application/pdf')) {
try {
if (DEBUG) console.log(`📄 addFiles(stored): Generating PDF metadata for stored file ${file.name}`);
// Get page count from PDF
const arrayBuffer = await file.arrayBuffer();
const { pdfWorkerManager } = await import('../../services/pdfWorkerManager');
const pdf = await pdfWorkerManager.createDocument(arrayBuffer);
pageCount = pdf.numPages;
pdfWorkerManager.destroyDocument(pdf);
if (DEBUG) console.log(`📄 addFiles(stored): Found ${pageCount} pages in PDF ${file.name}`);
} catch (error) {
if (DEBUG) console.warn(`📄 addFiles(stored): Failed to generate PDF metadata for ${file.name}:`, error);
}
} else {
pageCount = 0; // Non-PDFs have no page count
if (DEBUG) console.log(`📄 addFiles(stored): Non-PDF file ${file.name}, no page count`);
}
// Restore metadata from storage
if (metadata.thumbnail) {
record.thumbnailUrl = metadata.thumbnail;
// Track blob URLs for cleanup (images return blob URLs that need revocation)
if (metadata.thumbnail.startsWith('blob:')) {
lifecycleManager.trackBlobUrl(metadata.thumbnail);
}
}
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Create processedFile metadata with correct page count
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, metadata.thumbnail);
if (DEBUG) console.log(`📄 addFiles(stored): Created processedFile metadata for ${file.name} with ${pageCount} pages`);
}
// Extract basic metadata (version number and tool chain) for display
extractBasicFileMetadata(file, record).then(updatedRecord => {
if (updatedRecord !== record && (updatedRecord.versionNumber || updatedRecord.toolHistory)) {
// Basic metadata found, dispatch update to trigger re-render
dispatch({
type: 'UPDATE_FILE_RECORD',
payload: {
id: fileId,
updates: {
versionNumber: updatedRecord.versionNumber,
toolHistory: updatedRecord.toolHistory
}
}
});
}
}).catch(error => {
if (DEBUG) console.warn(`📄 Failed to extract basic metadata for ${file.name}:`, error);
});
existingQuickKeys.add(quickKey);
stirlingFileStubs.push(record);
addedFiles.push({ file, id: fileId, thumbnail: metadata.thumbnail });
} }
break;
} }
// Create record with immediate thumbnail and page metadata
const record = toStirlingFileStub(file, fileId, thumbnail);
if (thumbnail) {
// Track blob URLs for cleanup (images return blob URLs that need revocation)
if (thumbnail.startsWith('blob:')) {
lifecycleManager.trackBlobUrl(thumbnail);
}
}
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Create initial processedFile metadata with page count
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, thumbnail);
if (DEBUG) console.log(`📄 addFiles(raw): Created initial processedFile metadata for ${file.name} with ${pageCount} pages`);
}
existingQuickKeys.add(quickKey);
stirlingFileStubs.push(record);
// Create StirlingFile directly
const stirlingFile = createStirlingFile(file, fileId);
stirlingFiles.push(stirlingFile);
}
// Persist to storage if enabled using fileStorage service
if (enablePersistence && stirlingFiles.length > 0) {
await Promise.all(stirlingFiles.map(async (stirlingFile, index) => {
try {
// Get corresponding stub with all metadata
const fileStub = stirlingFileStubs[index];
// Store using the cleaner signature - pass StirlingFile + StirlingFileStub directly
await fileStorage.storeStirlingFile(stirlingFile, fileStub);
if (DEBUG) console.log(`📄 addFiles: Stored file ${stirlingFile.name} with metadata:`, fileStub);
} catch (error) {
console.error('Failed to persist file to storage:', stirlingFile.name, error);
}
}));
} }
// Dispatch ADD_FILES action if we have new files // Dispatch ADD_FILES action if we have new files
if (stirlingFileStubs.length > 0) { if (stirlingFileStubs.length > 0) {
dispatch({ type: 'ADD_FILES', payload: { stirlingFileStubs } }); dispatch({ type: 'ADD_FILES', payload: { stirlingFileStubs } });
if (DEBUG) console.log(`📄 addFiles(${kind}): Successfully added ${stirlingFileStubs.length} files`);
} }
return addedFiles; return stirlingFiles;
} finally { } finally {
// Always release mutex even if error occurs // Always release mutex even if error occurs
addFilesMutex.unlock(); addFilesMutex.unlock();
} }
} }
/**
* Helper function to process files into records with thumbnails and metadata
*/
async function processFilesIntoRecords(
files: File[],
filesRef: React.MutableRefObject<Map<FileId, File>>
): Promise<Array<{ record: StirlingFileStub; file: File; fileId: FileId; thumbnail?: string }>> {
return Promise.all(
files.map(async (file) => {
const fileId = createFileId();
filesRef.current.set(fileId, file);
// Generate thumbnail and page count
let thumbnail: string | undefined;
let pageCount: number = 1;
try {
if (DEBUG) console.log(`📄 Generating thumbnail for file ${file.name}`);
const result = await generateThumbnailWithMetadata(file);
thumbnail = result.thumbnail;
pageCount = result.pageCount;
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate thumbnail for file ${file.name}:`, error);
}
const record = toStirlingFileStub(file, fileId);
if (thumbnail) {
record.thumbnailUrl = thumbnail;
}
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, thumbnail);
}
// Extract basic metadata synchronously during consumeFiles for immediate display
if (file.type.includes('pdf')) {
try {
const updatedRecord = await extractBasicFileMetadata(file, record);
if (updatedRecord !== record && (updatedRecord.versionNumber || updatedRecord.toolHistory)) {
// Update the record directly with basic metadata
Object.assign(record, {
versionNumber: updatedRecord.versionNumber,
toolHistory: updatedRecord.toolHistory
});
}
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to extract basic metadata for ${file.name}:`, error);
}
}
return { record, file, fileId, thumbnail };
})
);
}
/**
* Helper function to persist files to IndexedDB
*/
async function persistFilesToIndexedDB(
stirlingFileStubs: Array<{ file: File; fileId: FileId; thumbnail?: string }>,
indexedDB: { saveFile: (file: File, fileId: FileId, existingThumbnail?: string) => Promise<any> }
): Promise<void> {
await Promise.all(stirlingFileStubs.map(async ({ file, fileId, thumbnail }) => {
try {
await indexedDB.saveFile(file, fileId, thumbnail);
} catch (error) {
console.error('Failed to persist file to IndexedDB:', file.name, error);
}
}));
}
/** /**
* Consume files helper - replace unpinned input files with output files * Consume files helper - replace unpinned input files with output files
* Now accepts pre-created StirlingFiles and StirlingFileStubs to preserve all metadata
*/ */
export async function consumeFiles( export async function consumeFiles(
inputFileIds: FileId[], inputFileIds: FileId[],
outputFiles: File[], outputStirlingFiles: StirlingFile[],
outputStirlingFileStubs: StirlingFileStub[],
filesRef: React.MutableRefObject<Map<FileId, File>>, filesRef: React.MutableRefObject<Map<FileId, File>>,
dispatch: React.Dispatch<FileContextAction>, dispatch: React.Dispatch<FileContextAction>
indexedDB?: { saveFile: (file: File, fileId: FileId, existingThumbnail?: string) => Promise<any>; markFileAsProcessed: (fileId: FileId) => Promise<boolean> } | null
): Promise<FileId[]> { ): Promise<FileId[]> {
if (DEBUG) console.log(`📄 consumeFiles: Processing ${inputFileIds.length} input files, ${outputFiles.length} output files`); if (DEBUG) console.log(`📄 consumeFiles: Processing ${inputFileIds.length} input files, ${outputStirlingFiles.length} output files with pre-created stubs`);
// Process output files with thumbnails and metadata // Validate that we have matching files and stubs
const outputStirlingFileStubs = await processFilesIntoRecords(outputFiles, filesRef); if (outputStirlingFiles.length !== outputStirlingFileStubs.length) {
throw new Error(`Mismatch between output files (${outputStirlingFiles.length}) and stubs (${outputStirlingFileStubs.length})`);
// Mark input files as processed in IndexedDB (no longer leaf nodes) and save output files
if (indexedDB) {
// Mark input files as processed (isLeaf = false)
await Promise.all(
inputFileIds.map(async (fileId) => {
try {
await indexedDB.markFileAsProcessed(fileId);
if (DEBUG) console.log(`📄 Marked file ${fileId} as processed (no longer leaf)`);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to mark file ${fileId} as processed:`, error);
}
})
);
// Save output files to IndexedDB
await persistFilesToIndexedDB(outputStirlingFileStubs, indexedDB);
} }
// Dispatch the consume action // Store StirlingFiles in filesRef using their existing IDs (no ID generation needed)
for (let i = 0; i < outputStirlingFiles.length; i++) {
const stirlingFile = outputStirlingFiles[i];
const stub = outputStirlingFileStubs[i];
if (stirlingFile.fileId !== stub.id) {
console.warn(`📄 consumeFiles: ID mismatch between StirlingFile (${stirlingFile.fileId}) and stub (${stub.id})`);
}
filesRef.current.set(stirlingFile.fileId, stirlingFile);
if (DEBUG) console.log(`📄 consumeFiles: Stored StirlingFile ${stirlingFile.name} with ID ${stirlingFile.fileId}`);
}
// Mark input files as processed in storage (no longer leaf nodes)
await Promise.all(
inputFileIds.map(async (fileId) => {
try {
await fileStorage.markFileAsProcessed(fileId);
if (DEBUG) console.log(`📄 Marked file ${fileId} as processed (no longer leaf)`);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to mark file ${fileId} as processed:`, error);
}
})
);
// Save output files directly to fileStorage with complete metadata
for (let i = 0; i < outputStirlingFiles.length; i++) {
const stirlingFile = outputStirlingFiles[i];
const stub = outputStirlingFileStubs[i];
try {
// Use fileStorage directly with complete metadata from stub
await fileStorage.storeStirlingFile(stirlingFile, stub);
if (DEBUG) console.log(`📄 Saved StirlingFile ${stirlingFile.name} directly to storage with complete metadata:`, {
fileId: stirlingFile.fileId,
versionNumber: stub.versionNumber,
originalFileId: stub.originalFileId,
parentFileId: stub.parentFileId,
toolChainLength: stub.toolHistory?.length || 0
});
} catch (error) {
console.error('Failed to persist output file to fileStorage:', stirlingFile.name, error);
}
}
// Dispatch the consume action with pre-created stubs (no processing needed)
dispatch({ dispatch({
type: 'CONSUME_FILES', type: 'CONSUME_FILES',
payload: { payload: {
inputFileIds, inputFileIds,
outputStirlingFileStubs: outputStirlingFileStubs.map(({ record }) => record) outputStirlingFileStubs: outputStirlingFileStubs
} }
}); });
if (DEBUG) console.log(`📄 consumeFiles: Successfully consumed files - removed ${inputFileIds.length} inputs, added ${outputStirlingFileStubs.length} outputs`); if (DEBUG) console.log(`📄 consumeFiles: Successfully consumed files - removed ${inputFileIds.length} inputs, added ${outputStirlingFileStubs.length} outputs`);
// Return the output file IDs for undo tracking // Return the output file IDs for undo tracking
return outputStirlingFileStubs.map(({ fileId }) => fileId); return outputStirlingFileStubs.map(stub => stub.id);
} }
/** /**
@ -606,6 +443,96 @@ export async function undoConsumeFiles(
/** /**
* Action factory functions * Action factory functions
*/ */
/**
* Add files using existing StirlingFileStubs from storage - preserves all metadata
* Use this when loading files that already exist in storage (FileManager, etc.)
* StirlingFileStubs come with proper thumbnails, history, processing state
*/
export async function addStirlingFileStubs(
stirlingFileStubs: StirlingFileStub[],
options: { insertAfterPageId?: string; selectFiles?: boolean } = {},
stateRef: React.MutableRefObject<FileContextState>,
filesRef: React.MutableRefObject<Map<FileId, File>>,
dispatch: React.Dispatch<FileContextAction>,
_lifecycleManager: FileLifecycleManager
): Promise<StirlingFile[]> {
await addFilesMutex.lock();
try {
if (DEBUG) console.log(`📄 addStirlingFileStubs: Adding ${stirlingFileStubs.length} StirlingFileStubs preserving metadata`);
const existingQuickKeys = buildQuickKeySet(stateRef.current.files.byId);
const validStubs: StirlingFileStub[] = [];
const loadedFiles: StirlingFile[] = [];
for (const stub of stirlingFileStubs) {
// Check for duplicates using quickKey
if (existingQuickKeys.has(stub.quickKey || '')) {
if (DEBUG) console.log(`📄 Skipping duplicate StirlingFileStub: ${stub.name}`);
continue;
}
// Load the actual StirlingFile from storage
const stirlingFile = await fileStorage.getStirlingFile(stub.id);
if (!stirlingFile) {
console.warn(`📄 Failed to load StirlingFile for stub: ${stub.name} (${stub.id})`);
continue;
}
// Store the loaded file in filesRef
filesRef.current.set(stub.id, stirlingFile);
// Use the original stub (preserves thumbnails, history, metadata!)
const record = { ...stub };
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Check if processedFile data needs regeneration for proper Page Editor support
if (stirlingFile.type.startsWith('application/pdf')) {
const needsProcessing = !record.processedFile ||
!record.processedFile.pages ||
record.processedFile.pages.length === 0 ||
record.processedFile.totalPages !== record.processedFile.pages.length;
if (needsProcessing) {
if (DEBUG) console.log(`📄 addStirlingFileStubs: Regenerating processedFile for ${record.name}`);
try {
// Generate basic processedFile structure with page count
const result = await generateThumbnailWithMetadata(stirlingFile);
record.processedFile = createProcessedFile(result.pageCount, result.thumbnail);
record.thumbnailUrl = result.thumbnail; // Update thumbnail if needed
if (DEBUG) console.log(`📄 addStirlingFileStubs: Regenerated processedFile for ${record.name} with ${result.pageCount} pages`);
} catch (error) {
if (DEBUG) console.warn(`📄 addStirlingFileStubs: Failed to regenerate processedFile for ${record.name}:`, error);
// Ensure we have at least basic structure
if (!record.processedFile) {
record.processedFile = createProcessedFile(1); // Fallback to 1 page
}
}
}
}
existingQuickKeys.add(stub.quickKey || '');
validStubs.push(record);
loadedFiles.push(stirlingFile);
}
// Dispatch ADD_FILES action if we have new files
if (validStubs.length > 0) {
dispatch({ type: 'ADD_FILES', payload: { stirlingFileStubs: validStubs } });
if (DEBUG) console.log(`📄 addStirlingFileStubs: Successfully added ${validStubs.length} files with preserved metadata`);
}
return loadedFiles;
} finally {
addFilesMutex.unlock();
}
}
export const createFileActions = (dispatch: React.Dispatch<FileContextAction>) => ({ export const createFileActions = (dispatch: React.Dispatch<FileContextAction>) => ({
setSelectedFiles: (fileIds: FileId[]) => dispatch({ type: 'SET_SELECTED_FILES', payload: { fileIds } }), setSelectedFiles: (fileIds: FileId[]) => dispatch({ type: 'SET_SELECTED_FILES', payload: { fileIds } }),
setSelectedPages: (pageNumbers: number[]) => dispatch({ type: 'SET_SELECTED_PAGES', payload: { pageNumbers } }), setSelectedPages: (pageNumbers: number[]) => dispatch({ type: 'SET_SELECTED_PAGES', payload: { pageNumbers } }),

View File

@ -6,9 +6,9 @@ import { useToolState, type ProcessingProgress } from './useToolState';
import { useToolApiCalls, type ApiCallsConfig } from './useToolApiCalls'; import { useToolApiCalls, type ApiCallsConfig } from './useToolApiCalls';
import { useToolResources } from './useToolResources'; import { useToolResources } from './useToolResources';
import { extractErrorMessage } from '../../../utils/toolErrorHandler'; import { extractErrorMessage } from '../../../utils/toolErrorHandler';
import { StirlingFile, extractFiles, FileId, StirlingFileStub } from '../../../types/fileContext'; import { StirlingFile, extractFiles, FileId, StirlingFileStub, createStirlingFile, toStirlingFileStub } from '../../../types/fileContext';
import { ResponseHandler } from '../../../utils/toolResponseProcessor'; import { ResponseHandler } from '../../../utils/toolResponseProcessor';
import { prepareStirlingFilesWithHistory, verifyToolMetadataPreservation } from '../../../utils/fileHistoryUtils'; import { createChildStub } from '../../../contexts/file/fileActions';
// Re-export for backwards compatibility // Re-export for backwards compatibility
export type { ProcessingProgress, ResponseHandler }; export type { ProcessingProgress, ResponseHandler };
@ -129,7 +129,7 @@ export const useToolOperation = <TParams>(
config: ToolOperationConfig<TParams> config: ToolOperationConfig<TParams>
): ToolOperationHook<TParams> => { ): ToolOperationHook<TParams> => {
const { t } = useTranslation(); const { t } = useTranslation();
const { addFiles, consumeFiles, undoConsumeFiles, selectors, findFileId } = useFileContext(); const { addFiles, consumeFiles, undoConsumeFiles, selectors } = useFileContext();
// Composed hooks // Composed hooks
const { state, actions } = useToolState(); const { state, actions } = useToolState();
@ -166,24 +166,13 @@ export const useToolOperation = <TParams>(
cleanupBlobUrls(); cleanupBlobUrls();
// Prepare files with history metadata injection (for PDFs) // Prepare files with history metadata injection (for PDFs)
actions.setStatus('Preparing files...'); actions.setStatus('Processing files...');
const getFileStubById = (fileId: FileId) => {
return selectors.getStirlingFileStub(fileId);
};
const filesWithHistory = await prepareStirlingFilesWithHistory(
validFiles,
getFileStubById,
config.operationType,
params as Record<string, any>
);
try { try {
let processedFiles: File[]; let processedFiles: File[];
// Convert StirlingFiles with history to regular Files for API processing // Use original files directly (no PDF metadata injection - history stored in IndexedDB)
// The history is already injected into the File data, we just need to extract the File objects const filesForAPI = extractFiles(validFiles);
const filesForAPI = extractFiles(filesWithHistory);
switch (config.toolType) { switch (config.toolType) {
case ToolType.singleFile: { case ToolType.singleFile: {
@ -242,8 +231,6 @@ export const useToolOperation = <TParams>(
if (processedFiles.length > 0) { if (processedFiles.length > 0) {
actions.setFiles(processedFiles); actions.setFiles(processedFiles);
// Verify metadata preservation for backend quality tracking
await verifyToolMetadataPreservation(validFiles, processedFiles, config.operationType);
// Generate thumbnails and download URL concurrently // Generate thumbnails and download URL concurrently
actions.setGeneratingThumbnails(true); actions.setGeneratingThumbnails(true);
@ -272,7 +259,25 @@ export const useToolOperation = <TParams>(
} }
} }
const outputFileIds = await consumeFiles(inputFileIds, processedFiles); // Create new tool operation
const newToolOperation = {
toolName: config.operationType,
timestamp: Date.now()
};
console.log("tool complete inputs ")
const outputStirlingFileStubs = processedFiles.length != inputStirlingFileStubs.length
? processedFiles.map((file, index) => toStirlingFileStub(file, undefined, thumbnails[index]))
: processedFiles.map((resultingFile, index) =>
createChildStub(inputStirlingFileStubs[index], newToolOperation, resultingFile, thumbnails[index])
);
// Create StirlingFile objects from processed files and child stubs
const outputStirlingFiles = processedFiles.map((file, index) => {
const childStub = outputStirlingFileStubs[index];
return createStirlingFile(file, childStub.id);
});
const outputFileIds = await consumeFiles(inputFileIds, outputStirlingFiles, outputStirlingFileStubs);
// Store operation data for undo (only store what we need to avoid memory bloat) // Store operation data for undo (only store what we need to avoid memory bloat)
lastOperationRef.current = { lastOperationRef.current = {

View File

@ -1,39 +1,17 @@
import { useCallback } from 'react'; import { useCallback } from 'react';
import { useFileState, useFileActions } from '../contexts/FileContext'; import { useFileActions } from '../contexts/FileContext';
import { FileMetadata } from '../types/file';
import { FileId } from '../types/file';
export const useFileHandler = () => { export const useFileHandler = () => {
const { state } = useFileState(); // Still needed for addStoredFiles
const { actions } = useFileActions(); const { actions } = useFileActions();
const addToActiveFiles = useCallback(async (file: File) => { const addFiles = useCallback(async (files: File[], options: { insertAfterPageId?: string; selectFiles?: boolean } = {}) => {
// Merge default options with passed options - passed options take precedence
const mergedOptions = { selectFiles: true, ...options };
// Let FileContext handle deduplication with quickKey logic // Let FileContext handle deduplication with quickKey logic
await actions.addFiles([file], { selectFiles: true }); await actions.addFiles(files, mergedOptions);
}, [actions.addFiles]); }, [actions.addFiles]);
const addMultipleFiles = useCallback(async (files: File[]) => {
// Let FileContext handle deduplication with quickKey logic
await actions.addFiles(files, { selectFiles: true });
}, [actions.addFiles]);
// Add stored files preserving their original IDs to prevent session duplicates
const addStoredFiles = useCallback(async (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>) => {
// Filter out files that already exist with the same ID (exact match)
const newFiles = filesWithMetadata.filter(({ originalId }) => {
return state.files.byId[originalId] === undefined;
});
if (newFiles.length > 0) {
await actions.addStoredFiles(newFiles, { selectFiles: true });
}
console.log(`📁 Added ${newFiles.length} stored files (${filesWithMetadata.length - newFiles.length} skipped as duplicates)`);
}, [state.files.byId, actions.addStoredFiles]);
return { return {
addToActiveFiles, addFiles,
addMultipleFiles,
addStoredFiles,
}; };
}; };

View File

@ -6,7 +6,7 @@
import { useState, useCallback } from 'react'; import { useState, useCallback } from 'react';
import { FileId } from '../types/file'; import { FileId } from '../types/file';
import { StirlingFileStub } from '../types/fileContext'; import { StirlingFileStub } from '../types/fileContext';
import { loadFileHistoryOnDemand } from '../utils/fileHistoryUtils'; // loadFileHistoryOnDemand removed - history now comes from IndexedDB directly
interface FileHistoryState { interface FileHistoryState {
originalFileId?: string; originalFileId?: string;
@ -33,16 +33,17 @@ export function useFileHistory(): UseFileHistoryResult {
const [error, setError] = useState<string | null>(null); const [error, setError] = useState<string | null>(null);
const loadHistory = useCallback(async ( const loadHistory = useCallback(async (
file: File, _file: File,
fileId: FileId, _fileId: FileId,
updateFileStub?: (id: FileId, updates: Partial<StirlingFileStub>) => void _updateFileStub?: (id: FileId, updates: Partial<StirlingFileStub>) => void
) => { ) => {
setIsLoading(true); setIsLoading(true);
setError(null); setError(null);
try { try {
const history = await loadFileHistoryOnDemand(file, fileId, updateFileStub); // History is now loaded from IndexedDB, not PDF metadata
setHistoryData(history); // This function is deprecated
throw new Error('loadFileHistoryOnDemand is deprecated - use IndexedDB history directly');
} catch (err) { } catch (err) {
const errorMessage = err instanceof Error ? err.message : 'Failed to load file history'; const errorMessage = err instanceof Error ? err.message : 'Failed to load file history';
setError(errorMessage); setError(errorMessage);
@ -76,9 +77,9 @@ export function useMultiFileHistory() {
const [errors, setErrors] = useState<Map<FileId, string>>(new Map()); const [errors, setErrors] = useState<Map<FileId, string>>(new Map());
const loadFileHistory = useCallback(async ( const loadFileHistory = useCallback(async (
file: File, _file: File,
fileId: FileId, fileId: FileId,
updateFileStub?: (id: FileId, updates: Partial<StirlingFileStub>) => void _updateFileStub?: (id: FileId, updates: Partial<StirlingFileStub>) => void
) => { ) => {
// Don't reload if already loaded or currently loading // Don't reload if already loaded or currently loading
if (historyCache.has(fileId) || loadingFiles.has(fileId)) { if (historyCache.has(fileId) || loadingFiles.has(fileId)) {
@ -93,13 +94,9 @@ export function useMultiFileHistory() {
}); });
try { try {
const history = await loadFileHistoryOnDemand(file, fileId, updateFileStub); // History is now loaded from IndexedDB, not PDF metadata
// This function is deprecated
if (history) { throw new Error('loadFileHistoryOnDemand is deprecated - use IndexedDB history directly');
setHistoryCache(prev => new Map(prev).set(fileId, history));
}
return history;
} catch (err) { } catch (err) {
const errorMessage = err instanceof Error ? err.message : 'Failed to load file history'; const errorMessage = err instanceof Error ? err.message : 'Failed to load file history';
setErrors(prev => new Map(prev).set(fileId, errorMessage)); setErrors(prev => new Map(prev).set(fileId, errorMessage));

View File

@ -1,28 +1,29 @@
import { useState, useCallback } from 'react'; import { useState, useCallback } from 'react';
import { useIndexedDB } from '../contexts/IndexedDBContext'; import { useIndexedDB } from '../contexts/IndexedDBContext';
import { FileMetadata } from '../types/file'; import { fileStorage } from '../services/fileStorage';
import { StirlingFileStub, StirlingFile } from '../types/fileContext';
import { FileId } from '../types/fileContext'; import { FileId } from '../types/fileContext';
export const useFileManager = () => { export const useFileManager = () => {
const [loading, setLoading] = useState(false); const [loading, setLoading] = useState(false);
const indexedDB = useIndexedDB(); const indexedDB = useIndexedDB();
const convertToFile = useCallback(async (fileMetadata: FileMetadata): Promise<File> => { const convertToFile = useCallback(async (fileStub: StirlingFileStub): Promise<File> => {
if (!indexedDB) { if (!indexedDB) {
throw new Error('IndexedDB context not available'); throw new Error('IndexedDB context not available');
} }
// Regular file loading // Regular file loading
if (fileMetadata.id) { if (fileStub.id) {
const file = await indexedDB.loadFile(fileMetadata.id); const file = await indexedDB.loadFile(fileStub.id);
if (file) { if (file) {
return file; return file;
} }
} }
throw new Error(`File not found in storage: ${fileMetadata.name} (ID: ${fileMetadata.id})`); throw new Error(`File not found in storage: ${fileStub.name} (ID: ${fileStub.id})`);
}, [indexedDB]); }, [indexedDB]);
const loadRecentFiles = useCallback(async (): Promise<FileMetadata[]> => { const loadRecentFiles = useCallback(async (): Promise<StirlingFileStub[]> => {
setLoading(true); setLoading(true);
try { try {
if (!indexedDB) { if (!indexedDB) {
@ -30,11 +31,10 @@ export const useFileManager = () => {
} }
// Load only leaf files metadata (processed files that haven't been used as input for other tools) // Load only leaf files metadata (processed files that haven't been used as input for other tools)
const storedFileMetadata = await indexedDB.loadLeafMetadata(); const stirlingFileStubs = await fileStorage.getLeafStirlingFileStubs();
// For now, only regular files - drafts will be handled separately in the future // For now, only regular files - drafts will be handled separately in the future
const allFiles = storedFileMetadata; const sortedFiles = stirlingFileStubs.sort((a, b) => (b.lastModified || 0) - (a.lastModified || 0));
const sortedFiles = allFiles.sort((a, b) => (b.lastModified || 0) - (a.lastModified || 0));
return sortedFiles; return sortedFiles;
} catch (error) { } catch (error) {
@ -45,7 +45,7 @@ export const useFileManager = () => {
} }
}, [indexedDB]); }, [indexedDB]);
const handleRemoveFile = useCallback(async (index: number, files: FileMetadata[], setFiles: (files: FileMetadata[]) => void) => { const handleRemoveFile = useCallback(async (index: number, files: StirlingFileStub[], setFiles: (files: StirlingFileStub[]) => void) => {
const file = files[index]; const file = files[index];
if (!file.id) { if (!file.id) {
throw new Error('File ID is required for removal'); throw new Error('File ID is required for removal');
@ -70,10 +70,10 @@ export const useFileManager = () => {
// Store file with provided UUID from FileContext (thumbnail generated internally) // Store file with provided UUID from FileContext (thumbnail generated internally)
const metadata = await indexedDB.saveFile(file, fileId); const metadata = await indexedDB.saveFile(file, fileId);
// Convert file to ArrayBuffer for StoredFile interface compatibility // Convert file to ArrayBuffer for storage compatibility
const arrayBuffer = await file.arrayBuffer(); const arrayBuffer = await file.arrayBuffer();
// Return StoredFile format for compatibility with old API // This method is deprecated - use FileStorage directly instead
return { return {
id: fileId, id: fileId,
name: file.name, name: file.name,
@ -81,7 +81,7 @@ export const useFileManager = () => {
size: file.size, size: file.size,
lastModified: file.lastModified, lastModified: file.lastModified,
data: arrayBuffer, data: arrayBuffer,
thumbnail: metadata.thumbnail thumbnail: metadata.thumbnailUrl
}; };
} catch (error) { } catch (error) {
console.error('Failed to store file:', error); console.error('Failed to store file:', error);
@ -105,23 +105,24 @@ export const useFileManager = () => {
setSelectedFiles([]); setSelectedFiles([]);
}; };
const selectMultipleFiles = async (files: FileMetadata[], onStoredFilesSelect: (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>) => void) => { const selectMultipleFiles = async (files: StirlingFileStub[], onStirlingFilesSelect: (stirlingFiles: StirlingFile[]) => void) => {
if (selectedFiles.length === 0) return; if (selectedFiles.length === 0) return;
try { try {
// Filter by UUID and convert to File objects // Filter by UUID and load full StirlingFile objects directly
const selectedFileObjects = files.filter(f => selectedFiles.includes(f.id)); const selectedFileObjects = files.filter(f => selectedFiles.includes(f.id));
// Use stored files flow that preserves IDs const stirlingFiles = await Promise.all(
const filesWithMetadata = await Promise.all( selectedFileObjects.map(async (stub) => {
selectedFileObjects.map(async (metadata) => ({ const stirlingFile = await fileStorage.getStirlingFile(stub.id);
file: await convertToFile(metadata), if (!stirlingFile) {
originalId: metadata.id, throw new Error(`File not found in storage: ${stub.name}`);
metadata }
})) return stirlingFile;
})
); );
onStoredFilesSelect(filesWithMetadata);
onStirlingFilesSelect(stirlingFiles);
clearSelection(); clearSelection();
} catch (error) { } catch (error) {
console.error('Failed to load selected files:', error); console.error('Failed to load selected files:', error);

View File

@ -1,5 +1,5 @@
import { useState, useEffect } from "react"; import { useState, useEffect } from "react";
import { FileMetadata } from "../types/file"; import { StirlingFileStub } from "../types/fileContext";
import { useIndexedDB } from "../contexts/IndexedDBContext"; import { useIndexedDB } from "../contexts/IndexedDBContext";
import { generateThumbnailForFile } from "../utils/thumbnailUtils"; import { generateThumbnailForFile } from "../utils/thumbnailUtils";
import { FileId } from "../types/fileContext"; import { FileId } from "../types/fileContext";
@ -9,7 +9,7 @@ import { FileId } from "../types/fileContext";
* Hook for IndexedDB-aware thumbnail loading * Hook for IndexedDB-aware thumbnail loading
* Handles thumbnail generation for files not in IndexedDB * Handles thumbnail generation for files not in IndexedDB
*/ */
export function useIndexedDBThumbnail(file: FileMetadata | undefined | null): { export function useIndexedDBThumbnail(file: StirlingFileStub | undefined | null): {
thumbnail: string | null; thumbnail: string | null;
isGenerating: boolean isGenerating: boolean
} { } {
@ -27,8 +27,8 @@ export function useIndexedDBThumbnail(file: FileMetadata | undefined | null): {
} }
// First priority: use stored thumbnail // First priority: use stored thumbnail
if (file.thumbnail) { if (file.thumbnailUrl) {
setThumb(file.thumbnail); setThumb(file.thumbnailUrl);
return; return;
} }
@ -77,7 +77,7 @@ export function useIndexedDBThumbnail(file: FileMetadata | undefined | null): {
loadThumbnail(); loadThumbnail();
return () => { cancelled = true; }; return () => { cancelled = true; };
}, [file, file?.thumbnail, file?.id, indexedDB, generating]); }, [file, file?.thumbnailUrl, file?.id, indexedDB, generating]);
return { thumbnail: thumb, isGenerating: generating }; return { thumbnail: thumb, isGenerating: generating };
} }

View File

@ -1,22 +1,23 @@
/** /**
* IndexedDB File Storage Service * Stirling File Storage Service
* Provides high-capacity file storage for PDF processing * Single-table architecture with typed query methods
* Now uses centralized IndexedDB manager * Forces correct usage patterns through service API design
*/ */
import { FileId } from '../types/file'; import { FileId, BaseFileMetadata } from '../types/file';
import { StirlingFile, StirlingFileStub, createStirlingFile } from '../types/fileContext';
import { indexedDBManager, DATABASE_CONFIGS } from './indexedDBManager'; import { indexedDBManager, DATABASE_CONFIGS } from './indexedDBManager';
export interface StoredFile { /**
id: FileId; * Storage record - single source of truth
name: string; * Contains all data needed for both StirlingFile and StirlingFileStub
type: string; */
size: number; export interface StoredStirlingFileRecord extends BaseFileMetadata {
lastModified: number;
data: ArrayBuffer; data: ArrayBuffer;
fileId: FileId; // Matches runtime StirlingFile.fileId exactly
quickKey: string; // Matches runtime StirlingFile.quickKey exactly
thumbnail?: string; thumbnail?: string;
url?: string; // For compatibility with existing components url?: string; // For compatibility with existing components
isLeaf?: boolean; // True if this file is a leaf node (hasn't been processed yet)
} }
export interface StorageStats { export interface StorageStats {
@ -38,47 +39,49 @@ class FileStorageService {
} }
/** /**
* Store a file in IndexedDB with external UUID * Store a StirlingFile with its metadata from StirlingFileStub
*/ */
async storeFile(file: File, fileId: FileId, thumbnail?: string, isLeaf: boolean = true): Promise<StoredFile> { async storeStirlingFile(stirlingFile: StirlingFile, stub: StirlingFileStub): Promise<void> {
const db = await this.getDatabase(); const db = await this.getDatabase();
const arrayBuffer = await stirlingFile.arrayBuffer();
const arrayBuffer = await file.arrayBuffer(); const record: StoredStirlingFileRecord = {
id: stirlingFile.fileId,
const storedFile: StoredFile = { fileId: stirlingFile.fileId, // Explicit field for clarity
id: fileId, // Use provided UUID quickKey: stirlingFile.quickKey,
name: file.name, name: stirlingFile.name,
type: file.type, type: stirlingFile.type,
size: file.size, size: stirlingFile.size,
lastModified: file.lastModified, lastModified: stirlingFile.lastModified,
data: arrayBuffer, data: arrayBuffer,
thumbnail, thumbnail: stub.thumbnailUrl,
isLeaf isLeaf: stub.isLeaf ?? true,
// History data from stub
versionNumber: stub.versionNumber ?? 1,
originalFileId: stub.originalFileId ?? stirlingFile.fileId,
parentFileId: stub.parentFileId ?? undefined,
toolHistory: stub.toolHistory ?? []
}; };
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
try { try {
// Verify store exists before creating transaction
if (!db.objectStoreNames.contains(this.storeName)) {
throw new Error(`Object store '${this.storeName}' not found. Available stores: ${Array.from(db.objectStoreNames).join(', ')}`);
}
const transaction = db.transaction([this.storeName], 'readwrite'); const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName); const store = transaction.objectStore(this.storeName);
// Debug logging const request = store.add(record);
console.log('📄 LEAF FLAG DEBUG - Storing file:', {
id: storedFile.id,
name: storedFile.name,
isLeaf: storedFile.isLeaf,
dataSize: storedFile.data.byteLength
});
const request = store.add(storedFile);
request.onerror = () => { request.onerror = () => {
console.error('IndexedDB add error:', request.error); console.error('IndexedDB add error:', request.error);
console.error('Failed object:', storedFile);
reject(request.error); reject(request.error);
}; };
request.onsuccess = () => { request.onsuccess = () => {
console.log('File stored successfully with ID:', storedFile.id); resolve();
resolve(storedFile);
}; };
} catch (error) { } catch (error) {
console.error('Transaction error:', error); console.error('Transaction error:', error);
@ -88,9 +91,9 @@ class FileStorageService {
} }
/** /**
* Retrieve a file from IndexedDB * Get StirlingFile with full data - for loading into workbench
*/ */
async getFile(id: FileId): Promise<StoredFile | null> { async getStirlingFile(id: FileId): Promise<StirlingFile | null> {
const db = await this.getDatabase(); const db = await this.getDatabase();
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
@ -98,77 +101,167 @@ class FileStorageService {
const store = transaction.objectStore(this.storeName); const store = transaction.objectStore(this.storeName);
const request = store.get(id); const request = store.get(id);
request.onerror = () => reject(request.error);
request.onsuccess = () => resolve(request.result || null);
});
}
/**
* Get all stored files (WARNING: loads all data into memory)
*/
async getAllFiles(): Promise<StoredFile[]> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.getAll();
request.onerror = () => reject(request.error); request.onerror = () => reject(request.error);
request.onsuccess = () => { request.onsuccess = () => {
// Filter out null/corrupted entries const record = request.result as StoredStirlingFileRecord | undefined;
const files = request.result.filter(file => if (!record) {
file && resolve(null);
file.data && return;
file.name && }
typeof file.size === 'number'
); // Create File from stored data
resolve(files); const blob = new Blob([record.data], { type: record.type });
const file = new File([blob], record.name, {
type: record.type,
lastModified: record.lastModified
});
// Convert to StirlingFile with preserved IDs
const stirlingFile = createStirlingFile(file, record.fileId);
resolve(stirlingFile);
}; };
}); });
} }
/** /**
* Get metadata of all stored files (without loading data into memory) * Get multiple StirlingFiles - for batch loading
*/ */
async getAllFileMetadata(): Promise<Omit<StoredFile, 'data'>[]> { async getStirlingFiles(ids: FileId[]): Promise<StirlingFile[]> {
const results = await Promise.all(ids.map(id => this.getStirlingFile(id)));
return results.filter((file): file is StirlingFile => file !== null);
}
/**
* Get StirlingFileStub (metadata only) - for UI browsing
*/
async getStirlingFileStub(id: FileId): Promise<StirlingFileStub | null> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.get(id);
request.onerror = () => reject(request.error);
request.onsuccess = () => {
const record = request.result as StoredStirlingFileRecord | undefined;
if (!record) {
resolve(null);
return;
}
// Create StirlingFileStub from metadata (no file data)
const stub: StirlingFileStub = {
id: record.id,
name: record.name,
type: record.type,
size: record.size,
lastModified: record.lastModified,
quickKey: record.quickKey,
thumbnailUrl: record.thumbnail,
isLeaf: record.isLeaf,
versionNumber: record.versionNumber,
originalFileId: record.originalFileId,
parentFileId: record.parentFileId,
toolHistory: record.toolHistory,
createdAt: Date.now() // Current session
};
resolve(stub);
};
});
}
/**
* Get all StirlingFileStubs (metadata only) - for FileManager browsing
*/
async getAllStirlingFileStubs(): Promise<StirlingFileStub[]> {
const db = await this.getDatabase(); const db = await this.getDatabase();
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly'); const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName); const store = transaction.objectStore(this.storeName);
const request = store.openCursor(); const request = store.openCursor();
const files: Omit<StoredFile, 'data'>[] = []; const stubs: StirlingFileStub[] = [];
request.onerror = () => reject(request.error); request.onerror = () => reject(request.error);
request.onsuccess = (event) => { request.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result; const cursor = (event.target as IDBRequest).result;
if (cursor) { if (cursor) {
const storedFile = cursor.value; const record = cursor.value as StoredStirlingFileRecord;
// Only extract metadata, skip the data field if (record && record.name && typeof record.size === 'number') {
if (storedFile && storedFile.name && typeof storedFile.size === 'number') { // Extract metadata only - no file data
files.push({ stubs.push({
id: storedFile.id, id: record.id,
name: storedFile.name, name: record.name,
type: storedFile.type, type: record.type,
size: storedFile.size, size: record.size,
lastModified: storedFile.lastModified, lastModified: record.lastModified,
thumbnail: storedFile.thumbnail quickKey: record.quickKey,
thumbnailUrl: record.thumbnail,
isLeaf: record.isLeaf,
versionNumber: record.versionNumber || 1,
originalFileId: record.originalFileId || record.id,
parentFileId: record.parentFileId,
toolHistory: record.toolHistory || [],
createdAt: Date.now()
}); });
} }
cursor.continue(); cursor.continue();
} else { } else {
// Metadata loaded efficiently without file data resolve(stubs);
resolve(files);
} }
}; };
}); });
} }
/** /**
* Delete a file from IndexedDB * Get leaf StirlingFileStubs only - for unprocessed files
*/ */
async deleteFile(id: FileId): Promise<void> { async getLeafStirlingFileStubs(): Promise<StirlingFileStub[]> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.openCursor();
const leafStubs: StirlingFileStub[] = [];
request.onerror = () => reject(request.error);
request.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result;
if (cursor) {
const record = cursor.value as StoredStirlingFileRecord;
// Only include leaf files (default to true if undefined)
if (record && record.name && typeof record.size === 'number' && record.isLeaf !== false) {
leafStubs.push({
id: record.id,
name: record.name,
type: record.type,
size: record.size,
lastModified: record.lastModified,
quickKey: record.quickKey,
thumbnailUrl: record.thumbnail,
isLeaf: record.isLeaf,
versionNumber: record.versionNumber || 1,
originalFileId: record.originalFileId || record.id,
parentFileId: record.parentFileId,
toolHistory: record.toolHistory || [],
createdAt: Date.now()
});
}
cursor.continue();
} else {
resolve(leafStubs);
}
};
});
}
/**
* Delete StirlingFile - single operation, no sync issues
*/
async deleteStirlingFile(id: FileId): Promise<void> {
const db = await this.getDatabase(); const db = await this.getDatabase();
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
@ -182,415 +275,7 @@ class FileStorageService {
} }
/** /**
* Update the lastModified timestamp of a file (for most recently used sorting) * Update thumbnail for existing file
*/
async touchFile(id: FileId): Promise<boolean> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const getRequest = store.get(id);
getRequest.onsuccess = () => {
const file = getRequest.result;
if (file) {
// Update lastModified to current timestamp
file.lastModified = Date.now();
const updateRequest = store.put(file);
updateRequest.onsuccess = () => resolve(true);
updateRequest.onerror = () => reject(updateRequest.error);
} else {
resolve(false); // File not found
}
};
getRequest.onerror = () => reject(getRequest.error);
});
}
/**
* Mark a file as no longer being a leaf (it has been processed)
*/
async markFileAsProcessed(id: FileId): Promise<boolean> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const getRequest = store.get(id);
getRequest.onsuccess = () => {
const file = getRequest.result;
if (file) {
console.log('📄 LEAF FLAG DEBUG - Marking as processed:', {
id: file.id,
name: file.name,
wasLeaf: file.isLeaf,
nowLeaf: false
});
file.isLeaf = false;
const updateRequest = store.put(file);
updateRequest.onsuccess = () => resolve(true);
updateRequest.onerror = () => reject(updateRequest.error);
} else {
console.warn('📄 LEAF FLAG DEBUG - File not found for processing:', id);
resolve(false); // File not found
}
};
getRequest.onerror = () => reject(getRequest.error);
});
}
/**
* Get only leaf files (files that haven't been processed yet)
*/
async getLeafFiles(): Promise<StoredFile[]> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.openCursor();
const leafFiles: StoredFile[] = [];
request.onerror = () => reject(request.error);
request.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result;
if (cursor) {
const storedFile = cursor.value;
if (storedFile && storedFile.isLeaf !== false) { // Default to true if undefined
leafFiles.push(storedFile);
}
cursor.continue();
} else {
resolve(leafFiles);
}
};
});
}
/**
* Get metadata of only leaf files (without loading data into memory)
*/
async getLeafFileMetadata(): Promise<Omit<StoredFile, 'data'>[]> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.openCursor();
const files: Omit<StoredFile, 'data'>[] = [];
request.onerror = () => reject(request.error);
request.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result;
if (cursor) {
const storedFile = cursor.value;
// Only include leaf files (default to true if undefined for backward compatibility)
if (storedFile && storedFile.name && typeof storedFile.size === 'number' && storedFile.isLeaf !== false) {
files.push({
id: storedFile.id,
name: storedFile.name,
type: storedFile.type,
size: storedFile.size,
lastModified: storedFile.lastModified,
thumbnail: storedFile.thumbnail,
isLeaf: storedFile.isLeaf
});
}
cursor.continue();
} else {
console.log('📄 LEAF FLAG DEBUG - Found leaf files:', files.map(f => ({ id: f.id, name: f.name, isLeaf: f.isLeaf })));
resolve(files);
}
};
});
}
/**
* Clear all stored files
*/
async clearAll(): Promise<void> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const request = store.clear();
request.onerror = () => reject(request.error);
request.onsuccess = () => resolve();
});
}
/**
* Get storage statistics (only our IndexedDB usage)
*/
async getStorageStats(): Promise<StorageStats> {
let used = 0;
let available = 0;
let quota: number | undefined;
let fileCount = 0;
try {
// Get browser quota for context
if ('storage' in navigator && 'estimate' in navigator.storage) {
const estimate = await navigator.storage.estimate();
quota = estimate.quota;
available = estimate.quota || 0;
}
// Calculate our actual IndexedDB usage from file metadata
const files = await this.getAllFileMetadata();
used = files.reduce((total, file) => total + (file?.size || 0), 0);
fileCount = files.length;
// Adjust available space
if (quota) {
available = quota - used;
}
} catch (error) {
console.warn('Could not get storage stats:', error);
// If we can't read metadata, database might be purged
used = 0;
fileCount = 0;
}
return {
used,
available,
fileCount,
quota
};
}
/**
* Get file count quickly without loading metadata
*/
async getFileCount(): Promise<number> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.count();
request.onerror = () => reject(request.error);
request.onsuccess = () => resolve(request.result);
});
}
/**
* Check all IndexedDB databases to see if files are in another version
*/
async debugAllDatabases(): Promise<void> {
console.log('=== Checking All IndexedDB Databases ===');
if ('databases' in indexedDB) {
try {
const databases = await indexedDB.databases();
console.log('Found databases:', databases);
for (const dbInfo of databases) {
if (dbInfo.name?.includes('stirling') || dbInfo.name?.includes('pdf')) {
console.log(`Checking database: ${dbInfo.name} (version: ${dbInfo.version})`);
try {
const db = await new Promise<IDBDatabase>((resolve, reject) => {
const request = indexedDB.open(dbInfo.name!, dbInfo.version);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
console.log(`Database ${dbInfo.name} object stores:`, Array.from(db.objectStoreNames));
db.close();
} catch (error) {
console.error(`Failed to open database ${dbInfo.name}:`, error);
}
}
}
} catch (error) {
console.error('Failed to list databases:', error);
}
} else {
console.log('indexedDB.databases() not supported');
}
// Also check our specific database with different versions
for (let version = 1; version <= 3; version++) {
try {
console.log(`Trying to open ${this.dbConfig.name} version ${version}...`);
const db = await new Promise<IDBDatabase>((resolve, reject) => {
const request = indexedDB.open(this.dbConfig.name, version);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
request.onupgradeneeded = () => {
// Don't actually upgrade, just check
request.transaction?.abort();
};
});
console.log(`Version ${version} object stores:`, Array.from(db.objectStoreNames));
if (db.objectStoreNames.contains('files')) {
const transaction = db.transaction(['files'], 'readonly');
const store = transaction.objectStore('files');
const countRequest = store.count();
countRequest.onsuccess = () => {
console.log(`Version ${version} files store has ${countRequest.result} entries`);
};
}
db.close();
} catch (error) {
if (error instanceof Error) {
console.log(`Version ${version} not accessible:`, error.message);
}
}
}
}
/**
* Debug method to check what's actually in the database
*/
async debugDatabaseContents(): Promise<void> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
// First try getAll to see if there's anything
const getAllRequest = store.getAll();
getAllRequest.onsuccess = () => {
console.log('=== Raw getAll() result ===');
console.log('Raw entries found:', getAllRequest.result.length);
getAllRequest.result.forEach((item, index) => {
console.log(`Raw entry ${index}:`, {
keys: Object.keys(item || {}),
id: item?.id,
name: item?.name,
size: item?.size,
type: item?.type,
hasData: !!item?.data,
dataSize: item?.data?.byteLength,
fullObject: item
});
});
};
// Then try cursor
const cursorRequest = store.openCursor();
console.log('=== IndexedDB Cursor Debug ===');
let count = 0;
cursorRequest.onerror = () => {
console.error('Cursor error:', cursorRequest.error);
reject(cursorRequest.error);
};
cursorRequest.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result;
if (cursor) {
count++;
const value = cursor.value;
console.log(`Cursor File ${count}:`, {
id: value?.id,
name: value?.name,
size: value?.size,
type: value?.type,
hasData: !!value?.data,
dataSize: value?.data?.byteLength,
hasThumbnail: !!value?.thumbnail,
allKeys: Object.keys(value || {})
});
cursor.continue();
} else {
console.log(`=== End Cursor Debug - Found ${count} files ===`);
resolve();
}
};
});
}
/**
* Convert StoredFile back to pure File object without mutations
* Returns a clean File object - use FileContext.addStoredFiles() for proper metadata handling
*/
createFileFromStored(storedFile: StoredFile): File {
if (!storedFile || !storedFile.data) {
throw new Error('Invalid stored file: missing data');
}
if (!storedFile.name || typeof storedFile.size !== 'number') {
throw new Error('Invalid stored file: missing metadata');
}
const blob = new Blob([storedFile.data], { type: storedFile.type });
const file = new File([blob], storedFile.name, {
type: storedFile.type,
lastModified: storedFile.lastModified
});
// Use FileContext.addStoredFiles() to properly associate with metadata
return file;
}
/**
* Convert StoredFile to the format expected by FileContext.addStoredFiles()
* This is the recommended way to load stored files into FileContext
*/
createFileWithMetadata(storedFile: StoredFile): { file: File; originalId: FileId; metadata: { thumbnail?: string } } {
const file = this.createFileFromStored(storedFile);
return {
file,
originalId: storedFile.id,
metadata: {
thumbnail: storedFile.thumbnail
}
};
}
/**
* Create blob URL for stored file
*/
createBlobUrl(storedFile: StoredFile): string {
const blob = new Blob([storedFile.data], { type: storedFile.type });
return URL.createObjectURL(blob);
}
/**
* Get file data as ArrayBuffer for streaming/chunked processing
*/
async getFileData(id: FileId): Promise<ArrayBuffer | null> {
try {
const storedFile = await this.getFile(id);
return storedFile ? storedFile.data : null;
} catch (error) {
console.warn(`Failed to get file data for ${id}:`, error);
return null;
}
}
/**
* Create a temporary blob URL that gets revoked automatically
*/
async createTemporaryBlobUrl(id: FileId): Promise<string | null> {
const data = await this.getFileData(id);
if (!data) return null;
const blob = new Blob([data], { type: 'application/pdf' });
const url = URL.createObjectURL(blob);
// Auto-revoke after a short delay to free memory
setTimeout(() => {
URL.revokeObjectURL(url);
}, 10000); // 10 seconds
return url;
}
/**
* Update thumbnail for an existing file
*/ */
async updateThumbnail(id: FileId, thumbnail: string): Promise<boolean> { async updateThumbnail(id: FileId, thumbnail: string): Promise<boolean> {
const db = await this.getDatabase(); const db = await this.getDatabase();
@ -602,13 +287,12 @@ class FileStorageService {
const getRequest = store.get(id); const getRequest = store.get(id);
getRequest.onsuccess = () => { getRequest.onsuccess = () => {
const storedFile = getRequest.result; const record = getRequest.result as StoredStirlingFileRecord;
if (storedFile) { if (record) {
storedFile.thumbnail = thumbnail; record.thumbnail = thumbnail;
const updateRequest = store.put(storedFile); const updateRequest = store.put(record);
updateRequest.onsuccess = () => { updateRequest.onsuccess = () => {
console.log('Thumbnail updated for file:', id);
resolve(true); resolve(true);
}; };
updateRequest.onerror = () => { updateRequest.onerror = () => {
@ -632,31 +316,161 @@ class FileStorageService {
} }
/** /**
* Check if storage quota is running low * Clear all stored files
*/ */
async isStorageLow(): Promise<boolean> { async clearAll(): Promise<void> {
const stats = await this.getStorageStats(); const db = await this.getDatabase();
if (!stats.quota) return false;
const usagePercent = stats.used / stats.quota; return new Promise((resolve, reject) => {
return usagePercent > 0.8; // Consider low if over 80% used const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const request = store.clear();
request.onerror = () => reject(request.error);
request.onsuccess = () => resolve();
});
} }
/** /**
* Clean up old files if storage is low * Get storage statistics
*/ */
async cleanupOldFiles(maxFiles: number = 50): Promise<void> { async getStorageStats(): Promise<StorageStats> {
const files = await this.getAllFileMetadata(); let used = 0;
let available = 0;
let quota: number | undefined;
let fileCount = 0;
if (files.length <= maxFiles) return; try {
// Get browser quota for context
if ('storage' in navigator && 'estimate' in navigator.storage) {
const estimate = await navigator.storage.estimate();
quota = estimate.quota;
available = estimate.quota || 0;
}
// Sort by last modified (oldest first) // Calculate our actual IndexedDB usage from file metadata
files.sort((a, b) => a.lastModified - b.lastModified); const stubs = await this.getAllStirlingFileStubs();
used = stubs.reduce((total, stub) => total + (stub?.size || 0), 0);
fileCount = stubs.length;
// Delete oldest files // Adjust available space
const filesToDelete = files.slice(0, files.length - maxFiles); if (quota) {
for (const file of filesToDelete) { available = quota - used;
await this.deleteFile(file.id); }
} catch (error) {
console.warn('Could not get storage stats:', error);
used = 0;
fileCount = 0;
}
return {
used,
available,
fileCount,
quota
};
}
/**
* Create blob URL for stored file data
*/
async createBlobUrl(id: FileId): Promise<string | null> {
try {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.get(id);
request.onerror = () => reject(request.error);
request.onsuccess = () => {
const record = request.result as StoredStirlingFileRecord | undefined;
if (record) {
const blob = new Blob([record.data], { type: record.type });
const url = URL.createObjectURL(blob);
resolve(url);
} else {
resolve(null);
}
};
});
} catch (error) {
console.warn(`Failed to create blob URL for ${id}:`, error);
return null;
}
}
/**
* Mark a file as processed (no longer a leaf file)
* Used when a file becomes input to a tool operation
*/
async markFileAsProcessed(fileId: FileId): Promise<boolean> {
try {
const db = await this.getDatabase();
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const record = await new Promise<StoredStirlingFileRecord | undefined>((resolve, reject) => {
const request = store.get(fileId);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
if (!record) {
return false; // File not found
}
// Update the isLeaf flag to false
record.isLeaf = false;
await new Promise<void>((resolve, reject) => {
const request = store.put(record);
request.onsuccess = () => resolve();
request.onerror = () => reject(request.error);
});
return true;
} catch (error) {
console.error('Failed to mark file as processed:', error);
return false;
}
}
/**
* Mark a file as leaf (opposite of markFileAsProcessed)
* Used when promoting a file back to "recent" status
*/
async markFileAsLeaf(fileId: FileId): Promise<boolean> {
try {
const db = await this.getDatabase();
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const record = await new Promise<StoredStirlingFileRecord | undefined>((resolve, reject) => {
const request = store.get(fileId);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
if (!record) {
return false; // File not found
}
// Update the isLeaf flag to true
record.isLeaf = true;
await new Promise<void>((resolve, reject) => {
const request = store.put(record);
request.onsuccess = () => resolve();
request.onerror = () => reject(request.error);
});
return true;
} catch (error) {
console.error('Failed to mark file as leaf:', error);
return false;
} }
} }
} }

View File

@ -201,13 +201,16 @@ class IndexedDBManager {
export const DATABASE_CONFIGS = { export const DATABASE_CONFIGS = {
FILES: { FILES: {
name: 'stirling-pdf-files', name: 'stirling-pdf-files',
version: 2, version: 3,
stores: [{ stores: [{
name: 'files', name: 'files',
keyPath: 'id', keyPath: 'id',
indexes: [ indexes: [
{ name: 'name', keyPath: 'name', unique: false }, { name: 'name', keyPath: 'name', unique: false },
{ name: 'lastModified', keyPath: 'lastModified', unique: false } { name: 'lastModified', keyPath: 'lastModified', unique: false },
{ name: 'originalFileId', keyPath: 'originalFileId', unique: false },
{ name: 'parentFileId', keyPath: 'parentFileId', unique: false },
{ name: 'versionNumber', keyPath: 'versionNumber', unique: false }
] ]
}] }]
} as DatabaseConfig, } as DatabaseConfig,
@ -219,7 +222,8 @@ export const DATABASE_CONFIGS = {
name: 'drafts', name: 'drafts',
keyPath: 'id' keyPath: 'id'
}] }]
} as DatabaseConfig } as DatabaseConfig,
} as const; } as const;
export const indexedDBManager = IndexedDBManager.getInstance(); export const indexedDBManager = IndexedDBManager.getInstance();

View File

@ -7,18 +7,17 @@
*/ */
import { PDFDocument } from 'pdf-lib'; import { PDFDocument } from 'pdf-lib';
import { FileId } from '../types/file';
import { ContentCache, type CacheConfig } from '../utils/ContentCache'; import { ContentCache, type CacheConfig } from '../utils/ContentCache';
const DEBUG = process.env.NODE_ENV === 'development'; const DEBUG = process.env.NODE_ENV === 'development';
/** /**
* Tool operation metadata for history tracking * Tool operation metadata for history tracking
* Note: Parameters removed for security - sensitive data like passwords should not be stored
*/ */
export interface ToolOperation { export interface ToolOperation {
toolName: string; toolName: string;
timestamp: number; timestamp: number;
parameters?: Record<string, any>;
} }
/** /**
@ -182,7 +181,7 @@ export class PDFMetadataService {
latestVersionNumber = parsed.stirlingHistory.versionNumber; latestVersionNumber = parsed.stirlingHistory.versionNumber;
historyJson = json; historyJson = json;
} }
} catch (error) { } catch {
// Silent fallback for corrupted history // Silent fallback for corrupted history
} }
} }

View File

@ -8,11 +8,11 @@ export type FileId = string & { readonly [tag]: 'FileId' };
/** /**
* Tool operation metadata for history tracking * Tool operation metadata for history tracking
* Note: Parameters removed for security - sensitive data like passwords should not be stored in history
*/ */
export interface ToolOperation { export interface ToolOperation {
toolName: string; toolName: string;
timestamp: number; timestamp: number;
parameters?: Record<string, any>;
} }
/** /**
@ -21,31 +21,32 @@ export interface ToolOperation {
*/ */
export interface FileHistoryInfo { export interface FileHistoryInfo {
originalFileId: string; originalFileId: string;
parentFileId?: string; parentFileId?: FileId;
versionNumber: number; versionNumber: number;
toolChain: ToolOperation[]; toolChain: ToolOperation[];
} }
/** /**
* File metadata for efficient operations without loading full file data * Base file metadata shared between storage and runtime layers
* Used by IndexedDBContext and FileContext for lazy file loading * Contains all common file properties and history tracking
*/ */
export interface FileMetadata { export interface BaseFileMetadata {
id: FileId; id: FileId;
name: string; name: string;
type: string; type: string;
size: number; size: number;
lastModified: number; lastModified: number;
thumbnail?: string; createdAt?: number; // When file was added to system
isLeaf?: boolean; // True if this file is a leaf node (hasn't been processed yet)
// File history tracking
// File history tracking (extracted from PDF metadata) isLeaf?: boolean; // True if this file hasn't been processed yet
historyInfo?: FileHistoryInfo;
// Quick access version information
originalFileId?: string; // Root file ID for grouping versions originalFileId?: string; // Root file ID for grouping versions
versionNumber?: number; // Version number in chain versionNumber?: number; // Version number in chain
parentFileId?: FileId; // Immediate parent file ID parentFileId?: FileId; // Immediate parent file ID
toolHistory?: Array<{
toolName: string;
timestamp: number;
}>; // Tool chain for history tracking
// Standard PDF document metadata // Standard PDF document metadata
pdfMetadata?: { pdfMetadata?: {
@ -59,6 +60,10 @@ export interface FileMetadata {
}; };
} }
// FileMetadata has been replaced with StoredFileMetadata from '../services/fileStorage'
// This ensures clear type relationships and eliminates duplication
export interface StorageConfig { export interface StorageConfig {
useIndexedDB: boolean; useIndexedDB: boolean;
maxFileSize: number; // Maximum size per file in bytes maxFileSize: number; // Maximum size per file in bytes

View File

@ -3,7 +3,7 @@
*/ */
import { PageOperation } from './pageEditor'; import { PageOperation } from './pageEditor';
import { FileId, FileMetadata } from './file'; import { FileId, BaseFileMetadata } from './file';
// Re-export FileId for convenience // Re-export FileId for convenience
export type { FileId }; export type { FileId };
@ -51,30 +51,20 @@ export interface ProcessedFileMetadata {
* separately in refs for memory efficiency. Supports multi-tool workflows * separately in refs for memory efficiency. Supports multi-tool workflows
* where files persist across tool operations. * where files persist across tool operations.
*/ */
export interface StirlingFileStub { /**
id: FileId; // UUID primary key for collision-free operations * StirlingFileStub - Runtime UI metadata for files in the active workbench session
name: string; // Display name for UI *
size: number; // File size for progress indicators * Contains UI display data and processing state. Actual File objects stored
type: string; // MIME type for format validation * separately in refs for memory efficiency. Supports multi-tool workflows
lastModified: number; // Original timestamp for deduplication * where files persist across tool operations.
*/
export interface StirlingFileStub extends BaseFileMetadata {
quickKey?: string; // Fast deduplication key: name|size|lastModified quickKey?: string; // Fast deduplication key: name|size|lastModified
thumbnailUrl?: string; // Generated thumbnail blob URL for visual display thumbnailUrl?: string; // Generated thumbnail blob URL for visual display
blobUrl?: string; // File access blob URL for downloads/processing blobUrl?: string; // File access blob URL for downloads/processing
createdAt?: number; // When added to workbench for sorting
processedFile?: ProcessedFileMetadata; // PDF page data and processing results processedFile?: ProcessedFileMetadata; // PDF page data and processing results
insertAfterPageId?: string; // Page ID after which this file should be inserted insertAfterPageId?: string; // Page ID after which this file should be inserted
isPinned?: boolean; // Protected from tool consumption (replace/remove) isPinned?: boolean; // Protected from tool consumption (replace/remove)
isLeaf?: boolean; // True if this file is a leaf node (hasn't been processed yet)
// File history tracking (from PDF metadata)
originalFileId?: string; // Root file ID for grouping versions
versionNumber?: number; // Version number in chain
parentFileId?: FileId; // Immediate parent file ID
toolHistory?: Array<{
toolName: string;
timestamp: number;
parameters?: Record<string, any>;
}>;
// Note: File object stored in provider ref, not in state // Note: File object stored in provider ref, not in state
} }
@ -117,6 +107,11 @@ export function isStirlingFile(file: File): file is StirlingFile {
// Create a StirlingFile from a regular File object // Create a StirlingFile from a regular File object
export function createStirlingFile(file: File, id?: FileId): StirlingFile { export function createStirlingFile(file: File, id?: FileId): StirlingFile {
// Check if file is already a StirlingFile to avoid property redefinition
if (isStirlingFile(file)) {
return file; // Already has fileId and quickKey properties
}
const fileId = id || createFileId(); const fileId = id || createFileId();
const quickKey = createQuickKey(file); const quickKey = createQuickKey(file);
@ -163,7 +158,9 @@ export function isFileObject(obj: any): obj is File | StirlingFile {
export function toStirlingFileStub( export function toStirlingFileStub(
file: File, file: File,
id?: FileId id?: FileId,
thumbnail?: string
): StirlingFileStub { ): StirlingFileStub {
const fileId = id || createFileId(); const fileId = id || createFileId();
return { return {
@ -174,7 +171,8 @@ export function toStirlingFileStub(
lastModified: file.lastModified, lastModified: file.lastModified,
quickKey: createQuickKey(file), quickKey: createQuickKey(file),
createdAt: Date.now(), createdAt: Date.now(),
isLeaf: true // New files are leaf nodes by default isLeaf: true, // New files are leaf nodes by default
thumbnailUrl: thumbnail
}; };
} }
@ -220,7 +218,6 @@ export interface FileOperation {
metadata?: { metadata?: {
originalFileName?: string; originalFileName?: string;
outputFileNames?: string[]; outputFileNames?: string[];
parameters?: Record<string, any>;
fileSize?: number; fileSize?: number;
pageCount?: number; pageCount?: number;
error?: string; error?: string;
@ -297,8 +294,7 @@ export type FileContextAction =
export interface FileContextActions { export interface FileContextActions {
// File management - lightweight actions only // File management - lightweight actions only
addFiles: (files: File[], options?: { insertAfterPageId?: string; selectFiles?: boolean }) => Promise<StirlingFile[]>; addFiles: (files: File[], options?: { insertAfterPageId?: string; selectFiles?: boolean }) => Promise<StirlingFile[]>;
addProcessedFiles: (filesWithThumbnails: Array<{ file: File; thumbnail?: string; pageCount?: number }>) => Promise<StirlingFile[]>; addStirlingFileStubs: (stirlingFileStubs: StirlingFileStub[], options?: { insertAfterPageId?: string; selectFiles?: boolean }) => Promise<StirlingFile[]>;
addStoredFiles: (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>, options?: { selectFiles?: boolean }) => Promise<StirlingFile[]>;
removeFiles: (fileIds: FileId[], deleteFromStorage?: boolean) => Promise<void>; removeFiles: (fileIds: FileId[], deleteFromStorage?: boolean) => Promise<void>;
updateStirlingFileStub: (id: FileId, updates: Partial<StirlingFileStub>) => void; updateStirlingFileStub: (id: FileId, updates: Partial<StirlingFileStub>) => void;
reorderFiles: (orderedFileIds: FileId[]) => void; reorderFiles: (orderedFileIds: FileId[]) => void;
@ -310,7 +306,7 @@ export interface FileContextActions {
unpinFile: (file: StirlingFile) => void; unpinFile: (file: StirlingFile) => void;
// File consumption (replace unpinned files with outputs) // File consumption (replace unpinned files with outputs)
consumeFiles: (inputFileIds: FileId[], outputFiles: File[]) => Promise<FileId[]>; consumeFiles: (inputFileIds: FileId[], outputStirlingFiles: StirlingFile[], outputStirlingFileStubs: StirlingFileStub[]) => Promise<FileId[]>;
undoConsumeFiles: (inputFiles: File[], inputStirlingFileStubs: StirlingFileStub[], outputFileIds: FileId[]) => Promise<void>; undoConsumeFiles: (inputFiles: File[], inputStirlingFileStubs: StirlingFileStub[], outputFileIds: FileId[]) => Promise<void>;
// Selection management // Selection management
setSelectedFiles: (fileIds: FileId[]) => void; setSelectedFiles: (fileIds: FileId[]) => void;

View File

@ -1,4 +1,4 @@
import { FileMetadata } from '../types/file'; import { StirlingFileStub } from '../types/fileContext';
import { fileStorage } from '../services/fileStorage'; import { fileStorage } from '../services/fileStorage';
import { zipFileService } from '../services/zipFileService'; import { zipFileService } from '../services/zipFileService';
@ -9,14 +9,14 @@ import { zipFileService } from '../services/zipFileService';
*/ */
export function downloadBlob(blob: Blob, filename: string): void { export function downloadBlob(blob: Blob, filename: string): void {
const url = URL.createObjectURL(blob); const url = URL.createObjectURL(blob);
const link = document.createElement('a'); const link = document.createElement('a');
link.href = url; link.href = url;
link.download = filename; link.download = filename;
document.body.appendChild(link); document.body.appendChild(link);
link.click(); link.click();
document.body.removeChild(link); document.body.removeChild(link);
// Clean up the blob URL // Clean up the blob URL
URL.revokeObjectURL(url); URL.revokeObjectURL(url);
} }
@ -26,23 +26,23 @@ export function downloadBlob(blob: Blob, filename: string): void {
* @param file - The file object with storage information * @param file - The file object with storage information
* @throws Error if file cannot be retrieved from storage * @throws Error if file cannot be retrieved from storage
*/ */
export async function downloadFileFromStorage(file: FileMetadata): Promise<void> { export async function downloadFileFromStorage(file: StirlingFileStub): Promise<void> {
const lookupKey = file.id; const lookupKey = file.id;
const storedFile = await fileStorage.getFile(lookupKey); const stirlingFile = await fileStorage.getStirlingFile(lookupKey);
if (!storedFile) { if (!stirlingFile) {
throw new Error(`File "${file.name}" not found in storage`); throw new Error(`File "${file.name}" not found in storage`);
} }
const blob = new Blob([storedFile.data], { type: storedFile.type }); // StirlingFile is already a File object, just download it
downloadBlob(blob, storedFile.name); downloadBlob(stirlingFile, stirlingFile.name);
} }
/** /**
* Downloads multiple files as individual downloads * Downloads multiple files as individual downloads
* @param files - Array of files to download * @param files - Array of files to download
*/ */
export async function downloadMultipleFiles(files: FileMetadata[]): Promise<void> { export async function downloadMultipleFiles(files: StirlingFileStub[]): Promise<void> {
for (const file of files) { for (const file of files) {
await downloadFileFromStorage(file); await downloadFileFromStorage(file);
} }
@ -53,36 +53,33 @@ export async function downloadMultipleFiles(files: FileMetadata[]): Promise<void
* @param files - Array of files to include in ZIP * @param files - Array of files to include in ZIP
* @param zipFilename - Optional custom ZIP filename (defaults to timestamped name) * @param zipFilename - Optional custom ZIP filename (defaults to timestamped name)
*/ */
export async function downloadFilesAsZip(files: FileMetadata[], zipFilename?: string): Promise<void> { export async function downloadFilesAsZip(files: StirlingFileStub[], zipFilename?: string): Promise<void> {
if (files.length === 0) { if (files.length === 0) {
throw new Error('No files provided for ZIP download'); throw new Error('No files provided for ZIP download');
} }
// Convert stored files to File objects // Convert stored files to File objects
const fileObjects: File[] = []; const filesToZip: File[] = [];
for (const fileWithUrl of files) { for (const fileWithUrl of files) {
const lookupKey = fileWithUrl.id; const lookupKey = fileWithUrl.id;
const storedFile = await fileStorage.getFile(lookupKey); const stirlingFile = await fileStorage.getStirlingFile(lookupKey);
if (storedFile) { if (stirlingFile) {
const file = new File([storedFile.data], storedFile.name, { // StirlingFile is already a File object!
type: storedFile.type, filesToZip.push(stirlingFile);
lastModified: storedFile.lastModified
});
fileObjects.push(file);
} }
} }
if (fileObjects.length === 0) { if (filesToZip.length === 0) {
throw new Error('No valid files found in storage for ZIP download'); throw new Error('No valid files found in storage for ZIP download');
} }
// Generate default filename if not provided // Generate default filename if not provided
const finalZipFilename = zipFilename || const finalZipFilename = zipFilename ||
`files-${new Date().toISOString().slice(0, 19).replace(/[:-]/g, '')}.zip`; `files-${new Date().toISOString().slice(0, 19).replace(/[:-]/g, '')}.zip`;
// Create and download ZIP // Create and download ZIP
const { zipFile } = await zipFileService.createZipFromFiles(fileObjects, finalZipFilename); const { zipFile } = await zipFileService.createZipFromFiles(filesToZip, finalZipFilename);
downloadBlob(zipFile, finalZipFilename); downloadBlob(zipFile, finalZipFilename);
} }
@ -94,7 +91,7 @@ export async function downloadFilesAsZip(files: FileMetadata[], zipFilename?: st
* @param options - Download options * @param options - Download options
*/ */
export async function downloadFiles( export async function downloadFiles(
files: FileMetadata[], files: StirlingFileStub[],
options: { options: {
forceZip?: boolean; forceZip?: boolean;
zipFilename?: string; zipFilename?: string;
@ -133,8 +130,8 @@ export function downloadFileObject(file: File, filename?: string): void {
* @param mimeType - MIME type (defaults to text/plain) * @param mimeType - MIME type (defaults to text/plain)
*/ */
export function downloadTextAsFile( export function downloadTextAsFile(
content: string, content: string,
filename: string, filename: string,
mimeType: string = 'text/plain' mimeType: string = 'text/plain'
): void { ): void {
const blob = new Blob([content], { type: mimeType }); const blob = new Blob([content], { type: mimeType });
@ -149,4 +146,4 @@ export function downloadTextAsFile(
export function downloadJsonAsFile(data: any, filename: string): void { export function downloadJsonAsFile(data: any, filename: string): void {
const content = JSON.stringify(data, null, 2); const content = JSON.stringify(data, null, 2);
downloadTextAsFile(content, filename, 'application/json'); downloadTextAsFile(content, filename, 'application/json');
} }

View File

@ -1,206 +1,10 @@
/** /**
* File History Utilities * File History Utilities
* *
* Helper functions for integrating PDF metadata service with FileContext operations. * Helper functions for IndexedDB-based file history management.
* Handles extraction of history from files and preparation for metadata injection. * Handles file history operations and lineage tracking.
*/ */
import { pdfMetadataService, type ToolOperation } from '../services/pdfMetadataService';
import { StirlingFileStub } from '../types/fileContext'; import { StirlingFileStub } from '../types/fileContext';
import { FileId, FileMetadata } from '../types/file';
import { createFileId } from '../types/fileContext';
const DEBUG = process.env.NODE_ENV === 'development';
/**
* Extract history information from a PDF file and update StirlingFileStub
*/
export async function extractFileHistory(
file: File,
record: StirlingFileStub
): Promise<StirlingFileStub> {
// Only process PDF files
if (!file.type.includes('pdf')) {
return record;
}
try {
const arrayBuffer = await file.arrayBuffer();
const historyMetadata = await pdfMetadataService.extractHistoryMetadata(arrayBuffer);
if (historyMetadata) {
const history = historyMetadata.stirlingHistory;
// Update record with history information
return {
...record,
originalFileId: history.originalFileId,
versionNumber: history.versionNumber,
parentFileId: history.parentFileId as FileId | undefined,
toolHistory: history.toolChain
};
}
} catch (error) {
if (DEBUG) console.warn('📄 Failed to extract file history:', file.name, error);
}
return record;
}
/**
* Inject history metadata into a PDF file for tool operations
*/
export async function injectHistoryForTool(
file: File,
sourceStirlingFileStub: StirlingFileStub,
toolName: string,
parameters?: Record<string, any>
): Promise<File> {
// Only process PDF files
if (!file.type.includes('pdf')) {
return file;
}
try {
const arrayBuffer = await file.arrayBuffer();
// Create tool operation record
const toolOperation: ToolOperation = {
toolName,
timestamp: Date.now(),
parameters
};
let modifiedBytes: ArrayBuffer;
// Extract version info directly from the PDF metadata to ensure accuracy
const existingHistoryMetadata = await pdfMetadataService.extractHistoryMetadata(arrayBuffer);
let newVersionNumber: number;
let originalFileId: string;
let parentFileId: string;
let parentToolChain: ToolOperation[];
if (existingHistoryMetadata) {
// File already has embedded history - increment version
const history = existingHistoryMetadata.stirlingHistory;
newVersionNumber = history.versionNumber + 1;
originalFileId = history.originalFileId;
parentFileId = sourceStirlingFileStub.id; // This file becomes the parent
parentToolChain = history.toolChain || [];
} else if (sourceStirlingFileStub.originalFileId && sourceStirlingFileStub.versionNumber) {
// File record has history but PDF doesn't (shouldn't happen, but fallback)
newVersionNumber = sourceStirlingFileStub.versionNumber + 1;
originalFileId = sourceStirlingFileStub.originalFileId;
parentFileId = sourceStirlingFileStub.id;
parentToolChain = sourceStirlingFileStub.toolHistory || [];
} else {
// File has no history - this becomes version 1
newVersionNumber = 1;
originalFileId = sourceStirlingFileStub.id; // Use source file ID as original
parentFileId = sourceStirlingFileStub.id; // Parent is the source file
parentToolChain = []; // No previous tools
}
// Create new tool chain with the new operation
const newToolChain = [...parentToolChain, toolOperation];
modifiedBytes = await pdfMetadataService.injectHistoryMetadata(
arrayBuffer,
originalFileId,
parentFileId,
newToolChain,
newVersionNumber
);
// Create new file with updated metadata
return new File([modifiedBytes], file.name, { type: file.type });
} catch (error) {
if (DEBUG) console.warn('📄 Failed to inject history for tool operation:', error);
return file; // Return original file if injection fails
}
}
/**
* Prepare StirlingFiles with history-injected PDFs for tool operations
* Preserves fileId and all StirlingFile metadata while injecting history
*/
export async function prepareStirlingFilesWithHistory(
stirlingFiles: import('../types/fileContext').StirlingFile[],
getStirlingFileStub: (fileId: import('../types/file').FileId) => StirlingFileStub | undefined,
toolName: string,
parameters?: Record<string, any>
): Promise<import('../types/fileContext').StirlingFile[]> {
const processedFiles: import('../types/fileContext').StirlingFile[] = [];
for (const stirlingFile of stirlingFiles) {
const fileStub = getStirlingFileStub(stirlingFile.fileId);
if (!fileStub) {
// If no stub found, keep original file
processedFiles.push(stirlingFile);
continue;
}
// Inject history into the file data
const fileWithHistory = await injectHistoryForTool(stirlingFile, fileStub, toolName, parameters);
// Create new StirlingFile with the updated file data but preserve fileId and quickKey
const updatedStirlingFile = new File([fileWithHistory], fileWithHistory.name, {
type: fileWithHistory.type,
lastModified: fileWithHistory.lastModified
}) as import('../types/fileContext').StirlingFile;
// Preserve the original fileId and quickKey
Object.defineProperty(updatedStirlingFile, 'fileId', {
value: stirlingFile.fileId,
writable: false,
enumerable: true,
configurable: false
});
Object.defineProperty(updatedStirlingFile, 'quickKey', {
value: stirlingFile.quickKey,
writable: false,
enumerable: true,
configurable: false
});
processedFiles.push(updatedStirlingFile);
}
return processedFiles;
}
/**
* Verify that processed files preserved metadata from originals
* Logs warnings for tools that strip standard PDF metadata
*/
export async function verifyToolMetadataPreservation(
originalFiles: File[],
processedFiles: File[],
toolName: string
): Promise<void> {
if (originalFiles.length === 0 || processedFiles.length === 0) return;
try {
// For single-file tools, compare the original with the processed file
if (originalFiles.length === 1 && processedFiles.length === 1) {
const originalBytes = await originalFiles[0].arrayBuffer();
const processedBytes = await processedFiles[0].arrayBuffer();
await pdfMetadataService.verifyMetadataPreservation(
originalBytes,
processedBytes,
toolName
);
}
// For multi-file tools, we could add more complex verification later
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to verify metadata preservation for ${toolName}:`, error);
}
}
/** /**
* Group files by processing branches - each branch ends in a leaf file * Group files by processing branches - each branch ends in a leaf file
@ -264,49 +68,6 @@ export function groupFilesByOriginal(StirlingFileStubs: StirlingFileStub[]): Map
return groups; return groups;
} }
/**
* Get the latest version of each file group (optimized version using leaf flags)
*/
export function getLatestVersions(fileStubs: StirlingFileStub[]): StirlingFileStub[] {
// If we have leaf flags, use them for much faster filtering
const hasLeafFlags = fileStubs.some(fileStub => fileStub.isLeaf !== undefined);
if (hasLeafFlags) {
// Fast path: just return files marked as leaf nodes
return fileStubs.filter(fileStub => fileStub.isLeaf !== false); // Default to true if undefined
} else {
// Fallback to expensive calculation for backward compatibility
const groups = groupFilesByOriginal(fileStubs);
const latestVersions: StirlingFileStub[] = [];
for (const [_, fileStubs] of groups) {
if (fileStubs.length > 0) {
// First item is the latest version (sorted desc by version number)
latestVersions.push(fileStubs[0]);
}
}
return latestVersions;
}
}
/**
* Get version history for a file
*/
export function getVersionHistory(
targetFileStub: StirlingFileStub,
allFileStubs: StirlingFileStub[]
): StirlingFileStub[] {
const originalId = targetFileStub.originalFileId || targetFileStub.id;
return allFileStubs
.filter(fileStub => {
const fileStubOriginalId = fileStub.originalFileId || fileStub.id;
return fileStubOriginalId === originalId;
})
.sort((a, b) => (b.versionNumber || 0) - (a.versionNumber || 0));
}
/** /**
* Check if a file has version history * Check if a file has version history
*/ */
@ -314,195 +75,4 @@ export function hasVersionHistory(fileStub: StirlingFileStub): boolean {
return !!(fileStub.originalFileId && fileStub.versionNumber && fileStub.versionNumber > 0); return !!(fileStub.originalFileId && fileStub.versionNumber && fileStub.versionNumber > 0);
} }
/**
* Generate a descriptive name for a file version
*/
export function generateVersionName(fileStub: StirlingFileStub): string {
const baseName = fileStub.name.replace(/\.pdf$/i, '');
if (!hasVersionHistory(fileStub)) {
return fileStub.name;
}
const versionInfo = fileStub.versionNumber ? ` (v${fileStub.versionNumber})` : '';
const toolInfo = fileStub.toolHistory && fileStub.toolHistory.length > 0
? ` - ${fileStub.toolHistory[fileStub.toolHistory.length - 1].toolName}`
: '';
return `${baseName}${versionInfo}${toolInfo}.pdf`;
}
/**
* Get recent files efficiently using leaf flags from IndexedDB
* This is much faster than loading all files and calculating leaf nodes
*/
export async function getRecentLeafFiles(): Promise<import('../services/fileStorage').StoredFile[]> {
try {
const { fileStorage } = await import('../services/fileStorage');
return await fileStorage.getLeafFiles();
} catch (error) {
console.warn('Failed to get recent leaf files from IndexedDB:', error);
return [];
}
}
/**
* Get recent file metadata efficiently using leaf flags from IndexedDB
* This is much faster than loading all files and calculating leaf nodes
*/
export async function getRecentLeafFileMetadata(): Promise<Omit<import('../services/fileStorage').StoredFile, 'data'>[]> {
try {
const { fileStorage } = await import('../services/fileStorage');
return await fileStorage.getLeafFileMetadata();
} catch (error) {
console.warn('Failed to get recent leaf file metadata from IndexedDB:', error);
return [];
}
}
/**
* Extract basic file metadata (version number and tool chain) without full history calculation
* This is lightweight and used for displaying essential info on file thumbnails
*/
export async function extractBasicFileMetadata(
file: File,
fileStub: StirlingFileStub
): Promise<StirlingFileStub> {
// Only process PDF files
if (!file.type.includes('pdf')) {
return fileStub;
}
try {
const arrayBuffer = await file.arrayBuffer();
const historyMetadata = await pdfMetadataService.extractHistoryMetadata(arrayBuffer);
if (historyMetadata) {
const history = historyMetadata.stirlingHistory;
// Update fileStub with essential metadata only (no parent/original relationships)
return {
...fileStub,
versionNumber: history.versionNumber,
toolHistory: history.toolChain
};
}
} catch (error) {
if (DEBUG) console.warn('📄 Failed to extract basic metadata:', file.name, error);
}
return fileStub;
}
/**
* Load file history on-demand for a specific file
* This replaces the automatic history extraction during file loading
*/
export async function loadFileHistoryOnDemand(
file: File,
fileId: FileId,
updateFileStub?: (id: FileId, updates: Partial<StirlingFileStub>) => void
): Promise<{
originalFileId?: string;
versionNumber?: number;
parentFileId?: FileId;
toolHistory?: Array<{
toolName: string;
timestamp: number;
parameters?: Record<string, any>;
}>;
} | null> {
// Only process PDF files
if (!file.type.includes('pdf')) {
return null;
}
try {
const baseFileStub: StirlingFileStub = {
id: fileId,
name: file.name,
size: file.size,
type: file.type,
lastModified: file.lastModified
};
const updatedFileStub = await extractFileHistory(file, baseFileStub);
if (updatedFileStub !== baseFileStub && (updatedFileStub.originalFileId || updatedFileStub.versionNumber)) {
const historyData = {
originalFileId: updatedFileStub.originalFileId,
versionNumber: updatedFileStub.versionNumber,
parentFileId: updatedFileStub.parentFileId,
toolHistory: updatedFileStub.toolHistory
};
// Update the file stub if update function is provided
if (updateFileStub) {
updateFileStub(fileId, historyData);
}
return historyData;
}
return null;
} catch (error) {
console.warn(`Failed to load history for ${file.name}:`, error);
return null;
}
}
/**
* Create metadata for storing files with history information
*/
export async function createFileMetadataWithHistory(
file: File,
fileId: FileId,
thumbnail?: string
): Promise<FileMetadata> {
const baseMetadata: FileMetadata = {
id: fileId,
name: file.name,
type: file.type,
size: file.size,
lastModified: file.lastModified,
thumbnail,
isLeaf: true // New files are leaf nodes by default
};
// Extract metadata for PDF files
if (file.type.includes('pdf')) {
try {
const arrayBuffer = await file.arrayBuffer();
const [historyMetadata, standardMetadata] = await Promise.all([
pdfMetadataService.extractHistoryMetadata(arrayBuffer),
pdfMetadataService.extractStandardMetadata(arrayBuffer)
]);
const result = { ...baseMetadata };
// Add standard PDF metadata if available
if (standardMetadata) {
result.pdfMetadata = standardMetadata;
}
// Add history metadata if available (basic version for display)
if (historyMetadata) {
const history = historyMetadata.stirlingHistory;
// Only add basic metadata needed for display, not full history relationships
result.versionNumber = history.versionNumber;
result.historyInfo = {
originalFileId: history.originalFileId,
parentFileId: history.parentFileId,
versionNumber: history.versionNumber,
toolChain: history.toolChain
};
}
return result;
} catch (error) {
if (DEBUG) console.warn('📄 Failed to extract metadata:', file.name, error);
}
}
return baseMetadata;
}

View File

@ -6,7 +6,7 @@ import { FileOperation } from '../types/fileContext';
*/ */
export const createOperation = <TParams = void>( export const createOperation = <TParams = void>(
operationType: string, operationType: string,
params: TParams, _params: TParams,
selectedFiles: File[] selectedFiles: File[]
): { operation: FileOperation; operationId: string; fileId: FileId } => { ): { operation: FileOperation; operationId: string; fileId: FileId } => {
const operationId = `${operationType}-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`; const operationId = `${operationType}-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
@ -20,7 +20,6 @@ export const createOperation = <TParams = void>(
status: 'pending', status: 'pending',
metadata: { metadata: {
originalFileName: selectedFiles[0]?.name, originalFileName: selectedFiles[0]?.name,
parameters: params,
fileSize: selectedFiles.reduce((sum, f) => sum + f.size, 0) fileSize: selectedFiles.reduce((sum, f) => sum + f.size, 0)
} }
} as any /* FIX ME*/; } as any /* FIX ME*/;