saulteafarmer/Stirling-PDF
1
0
Fork 0
mirror of https://github.com/Stirling-Tools/Stirling-PDF.git synced 2025-09-18 01:19:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

Feature/v2/filehistory (#4370)

Browse Source 
File History

---------

Co-authored-by: Connor Yoh <connor@stirlingpdf.com>
This commit is contained in:
ConnorYoh 2025-09-16 15:08:11 +01:00 committed by GitHub
parent 8e8b417f5e
commit 190178a471
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
61 changed files with 2279 additions and 1245 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

319
devGuide/FILE_HISTORY_SPECIFICATION.md Normal file
View File

@ -0,0 +1,319 @@
# Stirling PDF File History Specification
## Overview
Stirling PDF implements a client-side file history system using IndexedDB storage. File metadata, including version history and tool chains, are stored as `StirlingFileStub` objects that travel alongside the actual file data. This enables comprehensive version tracking, tool history, and file lineage management without modifying PDF content.
## Storage Architecture
### IndexedDB-Based Storage
File history is stored in the browser's IndexedDB using the `fileStorage` service, providing:
- **Persistent storage**: Survives browser sessions and page reloads
- **Large capacity**: Supports files up to 100GB+ with full metadata
- **Fast queries**: Optimized for file browsing and history lookups
- **Type safety**: Structured TypeScript interfaces
### Core Data Structures
```typescript
interface StirlingFileStub extends BaseFileMetadata {
id: FileId; // Unique file identifier (UUID)
quickKey: string; // Deduplication key: name|size|lastModified
thumbnailUrl?: string; // Generated thumbnail blob URL
processedFile?: ProcessedFileMetadata; // PDF page data and processing results
// File Metadata
name: string;
size: number;
type: string;
lastModified: number;
createdAt: number;
// Version Control
isLeaf: boolean; // True if this is the latest version
versionNumber?: number; // Version number (1, 2, 3, etc.)
originalFileId?: string; // UUID of the root file in version chain
parentFileId?: string; // UUID of immediate parent file
// Tool History
toolHistory?: ToolOperation[]; // Complete sequence of applied tools
}
interface ToolOperation {
toolName: string; // Tool identifier (e.g., 'compress', 'sanitize')
timestamp: number; // When the tool was applied
}
interface StoredStirlingFileRecord extends StirlingFileStub {
data: ArrayBuffer; // Actual file content
fileId: FileId; // Duplicate for indexing
}
```
## Version Management System
### Version Progression
- **v1**: Original uploaded file (first version)
- **v2**: First tool applied to original
- **v3**: Second tool applied (inherits from v2)
- **v4**: Third tool applied (inherits from v3)
- **etc.**
### Leaf Node System
Only the latest version of each file family is marked as `isLeaf: true`:
- **Leaf files**: Show in default file list, available for tool processing
- **History files**: Hidden by default, accessible via history expansion
### File Relationships
```
document.pdf (v1, isLeaf: false)
↓ compress
document.pdf (v2, isLeaf: false)
↓ sanitize
document.pdf (v3, isLeaf: true) ← Current active version
```
## Implementation Architecture
### 1. FileStorage Service (`fileStorage.ts`)
**Core Methods:**
```typescript
// Store file with complete metadata
async storeStirlingFile(stirlingFile: StirlingFile, stub: StirlingFileStub): Promise<void>
// Load file with metadata
async getStirlingFile(id: FileId): Promise<StirlingFile | null>
async getStirlingFileStub(id: FileId): Promise<StirlingFileStub | null>
// Query operations
async getLeafStirlingFileStubs(): Promise<StirlingFileStub[]>
async getAllStirlingFileStubs(): Promise<StirlingFileStub[]>
// Version management
async markFileAsProcessed(fileId: FileId): Promise<boolean> // Set isLeaf = false
async markFileAsLeaf(fileId: FileId): Promise<boolean> // Set isLeaf = true
```
### 2. File Context Integration
**FileContext** manages runtime state with `StirlingFileStub[]` in memory:
```typescript
interface FileContextState {
files: {
ids: FileId[];
byId: Record<FileId, StirlingFileStub>;
};
}
```
**Key Operations:**
- `addFiles()`: Stores new files with initial metadata
- `addStirlingFileStubs()`: Loads existing files from storage with preserved metadata
- `consumeFiles()`: Processes files through tools, creating new versions
### 3. Tool Operation Integration
**Tool Processing Flow:**
1. **Input**: User selects files (marked as `isLeaf: true`)
2. **Processing**: Backend processes files and returns results
3. **History Creation**: New `StirlingFileStub` created with:
- Incremented version number
- Updated tool history
- Parent file reference
4. **Storage**: Both parent (marked `isLeaf: false`) and child (marked `isLeaf: true`) stored
5. **UI Update**: FileContext updated with new file state
**Child Stub Creation:**
```typescript
export function createChildStub(
parentStub: StirlingFileStub,
operation: { toolName: string; timestamp: number },
resultingFile: File,
thumbnail?: string
): StirlingFileStub {
return {
id: createFileId(),
name: resultingFile.name,
size: resultingFile.size,
type: resultingFile.type,
lastModified: resultingFile.lastModified,
quickKey: createQuickKey(resultingFile),
createdAt: Date.now(),
isLeaf: true,
// Version Control
versionNumber: (parentStub.versionNumber || 1) + 1,
originalFileId: parentStub.originalFileId || parentStub.id,
parentFileId: parentStub.id,
// Tool History
toolHistory: [...(parentStub.toolHistory || []), operation],
thumbnailUrl: thumbnail
};
}
```
## UI Integration
### File Manager History Display
**FileManager** (`FileManager.tsx`) provides:
- **Default View**: Shows only leaf files (`isLeaf: true`)
- **History Expansion**: Click to show all versions of a file family
- **History Groups**: Nested display using `FileHistoryGroup.tsx`
**FileListItem** (`FileListItem.tsx`) displays:
- **Version Badges**: v1, v2, v3 indicators
- **Tool Chain**: Complete processing history in tooltips
- **History Actions**: "Show/Hide History" toggle, "Restore" for history files
### FileManagerContext Integration
**File Selection Flow:**
```typescript
// Recent files (from storage)
onRecentFileSelect: (stirlingFileStubs: StirlingFileStub[]) => void
// Calls: actions.addStirlingFileStubs(stirlingFileStubs, options)
// New uploads
onFileUpload: (files: File[]) => void
// Calls: actions.addFiles(files, options)
```
**History Management:**
```typescript
// Toggle history visibility
const { expandedFileIds, onToggleExpansion } = useFileManagerContext();
// Restore history file to current
const handleAddToRecents = (file: StirlingFileStub) => {
fileStorage.markFileAsLeaf(file.id); // Make this version current
};
```
## Data Flow
### New File Upload
```
1. User uploads files → addFiles()
2. Generate thumbnails and page count
3. Create StirlingFileStub with isLeaf: true, versionNumber: 1
4. Store both StirlingFile + StirlingFileStub in IndexedDB
5. Dispatch to FileContext state
```
### Tool Processing
```
1. User selects tool + files → useToolOperation()
2. API processes files → returns processed File objects
3. createChildStub() for each result:
- Parent marked isLeaf: false
- Child created with isLeaf: true, incremented version
4. Store all files with updated metadata
5. Update FileContext with new state
```
### File Loading (Recent Files)
```
1. User selects from FileManager → onRecentFileSelect()
2. addStirlingFileStubs() with preserved metadata
3. Load actual StirlingFile data from storage
4. Files appear in workbench with complete history intact
```
## Performance Optimizations
### Metadata Regeneration
When loading files from storage, missing `processedFile` data is regenerated:
```typescript
// In addStirlingFileStubs()
const needsProcessing = !record.processedFile ||
!record.processedFile.pages ||
record.processedFile.pages.length === 0;
if (needsProcessing) {
const result = await generateThumbnailWithMetadata(stirlingFile);
record.processedFile = createProcessedFile(result.pageCount, result.thumbnail);
}
```
### Memory Management
- **Blob URL Tracking**: Automatic cleanup of thumbnail URLs
- **Lazy Loading**: Files loaded from storage only when needed
- **LRU Caching**: File objects cached in memory with size limits
## File Deduplication
### QuickKey System
Files are deduplicated using `quickKey` format:
```typescript
const quickKey = `${file.name}|${file.size}|${file.lastModified}`;
```
This prevents duplicate uploads while allowing different versions of the same logical file.
## Error Handling
### Graceful Degradation
- **Storage Failures**: Files continue to work without persistence
- **Metadata Issues**: Missing metadata regenerated on demand
- **Version Conflicts**: Automatic version number resolution
### Recovery Scenarios
- **Corrupted Storage**: Automatic cleanup and re-initialization
- **Missing Files**: Stubs cleaned up automatically
- **Version Mismatches**: Automatic version chain reconstruction
## Developer Guidelines
### Adding File History to New Components
1. **Use FileContext Actions**:
```typescript
const { actions } = useFileActions();
await actions.addFiles(files); // For new uploads
await actions.addStirlingFileStubs(stubs); // For existing files
```
2. **Preserve Metadata When Processing**:
```typescript
const childStub = createChildStub(parentStub, {
toolName: 'compress',
timestamp: Date.now()
}, processedFile, thumbnail);
```
3. **Handle Storage Operations**:
```typescript
await fileStorage.storeStirlingFile(stirlingFile, stirlingFileStub);
const stub = await fileStorage.getStirlingFileStub(fileId);
```
### Testing File History
1. **Upload files**: Should show v1, marked as leaf
2. **Apply tool**: Should create v2, mark v1 as non-leaf
3. **Check FileManager**: History should show both versions
4. **Restore old version**: Should mark old version as leaf
5. **Check storage**: Both versions should persist in IndexedDB
## Future Enhancements
### Potential Improvements
- **Branch History**: Support for parallel processing branches
- **History Export**: Export complete version history as JSON
- **Conflict Resolution**: Handle concurrent modifications
- **Cloud Sync**: Sync history across devices
- **Compression**: Compress historical file data
### API Extensions
- **Batch Operations**: Process multiple version chains simultaneously
- **Search Integration**: Search within tool history and file metadata
- **Analytics**: Track usage patterns and tool effectiveness
---
**Last Updated**: January 2025
**Implementation**: Stirling PDF Frontend v2
**Storage Version**: IndexedDB with fileStorage service

7
frontend/public/locales/en-GB/translation.json
View File

@ -2406,6 +2406,13 @@
"storageLow": "Storage is running low. Consider removing old files.",
"supportMessage": "Powered by browser database storage for unlimited capacity",
"noFileSelected": "No files selected",
"showHistory": "Show History",
"hideHistory": "Hide History",
"fileHistory": "File History",
"loadingHistory": "Loading History...",
"lastModified": "Last Modified",
"toolChain": "Tools Applied",
"restore": "Restore",
"searchFiles": "Search files...",
"recent": "Recent",
"localFiles": "Local Files",

31
frontend/src/components/FileManager.tsx
View File

@ -1,7 +1,7 @@
import React, { useState, useCallback, useEffect } from 'react';
import { Modal } from '@mantine/core';
import { Dropzone } from '@mantine/dropzone';
import { FileMetadata } from '../types/file';
import { StirlingFileStub } from '../types/fileContext';
import { useFileManager } from '../hooks/useFileManager';
import { useFilesModalContext } from '../contexts/FilesModalContext';
import { Tool } from '../types/tool';
@ -15,12 +15,12 @@ interface FileManagerProps {
}
const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => {
const { isFilesModalOpen, closeFilesModal, onFilesSelect, onStoredFilesSelect } = useFilesModalContext();
const [recentFiles, setRecentFiles] = useState<FileMetadata[]>([]);
const { isFilesModalOpen, closeFilesModal, onFileUpload, onRecentFileSelect } = useFilesModalContext();
const [recentFiles, setRecentFiles] = useState<StirlingFileStub[]>([]);
const [isDragging, setIsDragging] = useState(false);
const [isMobile, setIsMobile] = useState(false);
const { loadRecentFiles, handleRemoveFile, convertToFile } = useFileManager();
const { loadRecentFiles, handleRemoveFile } = useFileManager();
// File management handlers
const isFileSupported = useCallback((fileName: string) => {
@ -34,33 +34,26 @@ const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => {
setRecentFiles(files);
}, [loadRecentFiles]);
const handleFilesSelected = useCallback(async (files: FileMetadata[]) => {
const handleRecentFilesSelected = useCallback(async (files: StirlingFileStub[]) => {
try {
// Use stored files flow that preserves original IDs
const filesWithMetadata = await Promise.all(
files.map(async (metadata) => ({
file: await convertToFile(metadata),
originalId: metadata.id,
metadata
}))
);
onStoredFilesSelect(filesWithMetadata);
// Use StirlingFileStubs directly - preserves all metadata!
onRecentFileSelect(files);
} catch (error) {
console.error('Failed to process selected files:', error);
}
}, [convertToFile, onStoredFilesSelect]);
}, [onRecentFileSelect]);
const handleNewFileUpload = useCallback(async (files: File[]) => {
if (files.length > 0) {
try {
// Files will get IDs assigned through onFilesSelect -> FileContext addFiles
onFilesSelect(files);
onFileUpload(files);
await refreshRecentFiles();
} catch (error) {
console.error('Failed to process dropped files:', error);
}
}
}, [onFilesSelect, refreshRecentFiles]);
}, [onFileUpload, refreshRecentFiles]);
const handleRemoveFileByIndex = useCallback(async (index: number) => {
await handleRemoveFile(index, recentFiles, setRecentFiles);
@ -85,7 +78,7 @@ const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => {
// Cleanup any blob URLs when component unmounts
useEffect(() => {
return () => {
// FileMetadata doesn't have blob URLs, so no cleanup needed
// StoredFileMetadata doesn't have blob URLs, so no cleanup needed
// Blob URLs are managed by FileContext and tool operations
console.log('FileManager unmounting - FileContext handles blob URL cleanup');
};
@ -146,7 +139,7 @@ const FileManager: React.FC<FileManagerProps> = ({ selectedTool }) => {
>
<FileManagerProvider
recentFiles={recentFiles}
onFilesSelected={handleFilesSelected}
onRecentFilesSelected={handleRecentFilesSelected}
onNewFilesSelect={handleNewFileUpload}
onClose={closeFilesModal}
isFileSupported={isFileSupported}

23
frontend/src/components/fileEditor/FileEditor.tsx
View File

@ -78,22 +78,6 @@ const FileEditor = ({
// Use activeStirlingFileStubs directly - no conversion needed
const localSelectedIds = contextSelectedIds;
// Helper to convert StirlingFileStub to FileThumbnail format
const recordToFileItem = useCallback((record: any) => {
const file = selectors.getFile(record.id);
if (!file) return null;
return {
id: record.id,
name: file.name,
pageCount: record.processedFile?.totalPages || 1,
thumbnail: record.thumbnailUrl || '',
size: file.size,
file: file
};
}, [selectors]);
// Process uploaded files using context
const handleFileUpload = useCallback(async (uploadedFiles: File[]) => {
setError(null);
@ -404,13 +388,10 @@ const FileEditor = ({
}}
>
{activeStirlingFileStubs.map((record, index) => {
const fileItem = recordToFileItem(record);
if (!fileItem) return null;
return (
<FileEditorThumbnail
key={record.id}
file={fileItem}
file={record}
index={index}
totalFiles={activeStirlingFileStubs.length}
selectedFiles={localSelectedIds}
@ -421,7 +402,7 @@ const FileEditor = ({
onSetStatus={setStatus}
onReorderFiles={handleReorderFiles}
toolMode={toolMode}
isSupported={isFileSupported(fileItem.name)}
isSupported={isFileSupported(record.name)}
/>
);
})}

56
frontend/src/components/fileEditor/FileEditorThumbnail.tsx
View File

@ -8,22 +8,17 @@ import PushPinIcon from '@mui/icons-material/PushPin';
import PushPinOutlinedIcon from '@mui/icons-material/PushPinOutlined';
import DragIndicatorIcon from '@mui/icons-material/DragIndicator';
import { draggable, dropTargetForElements } from '@atlaskit/pragmatic-drag-and-drop/element/adapter';
import { StirlingFileStub } from '../../types/fileContext';
import styles from './FileEditor.module.css';
import { useFileContext } from '../../contexts/FileContext';
import { FileId } from '../../types/file';
import ToolChain from '../shared/ToolChain';
interface FileItem {
id: FileId;
name: string;
pageCount: number;
thumbnail: string | null;
size: number;
modifiedAt?: number | string | Date;
}
interface FileEditorThumbnailProps {
file: FileItem;
file: StirlingFileStub;
index: number;
totalFiles: number;
selectedFiles: FileId[];
@ -64,6 +59,8 @@ const FileEditorThumbnail = ({
}, [activeFiles, file.id]);
const isPinned = actualFile ? isFilePinned(actualFile) : false;
const pageCount = file.processedFile?.totalPages || 0;
const downloadSelectedFile = useCallback(() => {
// Prefer parent-provided handler if available
if (typeof onDownloadFile === 'function') {
@ -109,22 +106,21 @@ const FileEditorThumbnail = ({
const pageLabel = useMemo(
() =>
file.pageCount > 0
? `${file.pageCount} ${file.pageCount === 1 ? 'Page' : 'Pages'}`
pageCount > 0
? `${pageCount} ${pageCount === 1 ? 'Page' : 'Pages'}`
: '',
[file.pageCount]
[pageCount]
);
const dateLabel = useMemo(() => {
const d =
file.modifiedAt != null ? new Date(file.modifiedAt) : new Date(); // fallback
const d = new Date(file.lastModified);
if (Number.isNaN(d.getTime())) return '';
return new Intl.DateTimeFormat(undefined, {
month: 'short',
day: '2-digit',
year: 'numeric',
}).format(d);
}, [file.modifiedAt]);
}, [file.lastModified]);
// ---- Drag & drop wiring ----
const fileElementRef = useCallback((element: HTMLDivElement | null) => {
@ -350,7 +346,8 @@ const FileEditorThumbnail = ({
lineClamp={3}
title={`${extUpper || 'FILE'} • ${prettySize}`}
>
{/* e.g., Jan 29, 2025 - PDF file - 3 Pages */}
{/* e.g., v2 - Jan 29, 2025 - PDF file - 3 Pages */}
{`v${file.versionNumber} - `}
{dateLabel}
{extUpper ? ` - ${extUpper} file` : ''}
{pageLabel ? ` - ${pageLabel}` : ''}
@ -360,9 +357,9 @@ const FileEditorThumbnail = ({
{/* Preview area */}
<div className={`${styles.previewBox} mx-6 mb-4 relative flex-1`}>
<div className={styles.previewPaper}>
{file.thumbnail && (
{file.thumbnailUrl && (
<img
src={file.thumbnail}
src={file.thumbnailUrl}
alt={file.name}
draggable={false}
loading="lazy"
@ -399,6 +396,29 @@ const FileEditorThumbnail = ({
<span ref={handleRef} className={styles.dragHandle} aria-hidden>
<DragIndicatorIcon fontSize="small" />
</span>
{/* Tool chain display at bottom */}
{file.toolHistory && (
<div style={{
position: 'absolute',
bottom: '4px',
left: '4px',
right: '4px',
padding: '4px 6px',
textAlign: 'center',
fontWeight: 600,
overflow: 'hidden',
whiteSpace: 'nowrap'
}}>
<ToolChain
toolChain={file.toolHistory}
displayStyle="text"
size="xs"
maxWidth={'100%'}
color='var(--mantine-color-gray-7)'
/>
</div>
)}
</div>
</div>
);

13
frontend/src/components/fileManager/CompactFileDetails.tsx
View File

@ -5,12 +5,12 @@ import ChevronLeftIcon from '@mui/icons-material/ChevronLeft';
import ChevronRightIcon from '@mui/icons-material/ChevronRight';
import { useTranslation } from 'react-i18next';
import { getFileSize } from '../../utils/fileUtils';
import { FileMetadata } from '../../types/file';
import { StirlingFileStub } from '../../types/fileContext';
interface CompactFileDetailsProps {
currentFile: FileMetadata | null;
currentFile: StirlingFileStub | null;
thumbnail: string | null;
selectedFiles: FileMetadata[];
selectedFiles: StirlingFileStub[];
currentFileIndex: number;
numberOfFiles: number;
isAnimating: boolean;
@ -72,12 +72,19 @@ const CompactFileDetails: React.FC<CompactFileDetailsProps> = ({
<Text size="xs" c="dimmed">
{currentFile ? getFileSize(currentFile) : ''}
{selectedFiles.length > 1 && ` • ${selectedFiles.length} files`}
{currentFile && ` • v${currentFile.versionNumber || 1}`}
</Text>
{hasMultipleFiles && (
<Text size="xs" c="blue">
{currentFileIndex + 1} of {selectedFiles.length}
</Text>
)}
{/* Compact tool chain for mobile */}
{currentFile?.toolHistory && currentFile.toolHistory.length > 0 && (
<Text size="xs" c="dimmed">
{currentFile.toolHistory.map((tool: any) => tool.toolName).join(' → ')}
</Text>
)}
</Box>
{/* Navigation arrows for multiple files */}

68
frontend/src/components/fileManager/FileHistoryGroup.tsx Normal file
View File

@ -0,0 +1,68 @@
import React from 'react';
import { Box, Text, Collapse, Group } from '@mantine/core';
import { useTranslation } from 'react-i18next';
import { StirlingFileStub } from '../../types/fileContext';
import FileListItem from './FileListItem';
interface FileHistoryGroupProps {
leafFile: StirlingFileStub;
historyFiles: StirlingFileStub[];
isExpanded: boolean;
onDownloadSingle: (file: StirlingFileStub) => void;
onFileDoubleClick: (file: StirlingFileStub) => void;
onHistoryFileRemove: (file: StirlingFileStub) => void;
isFileSupported: (fileName: string) => boolean;
}
const FileHistoryGroup: React.FC<FileHistoryGroupProps> = ({
leafFile,
historyFiles,
isExpanded,
onDownloadSingle,
onFileDoubleClick,
onHistoryFileRemove,
isFileSupported,
}) => {
const { t } = useTranslation();
// Sort history files by version number (oldest first, excluding the current leaf file)
const sortedHistory = historyFiles
.filter(file => file.id !== leafFile.id) // Exclude the leaf file itself
.sort((a, b) => (b.versionNumber || 1) - (a.versionNumber || 1));
if (!isExpanded || sortedHistory.length === 0) {
return null;
}
return (
<Collapse in={isExpanded}>
<Box ml="md" mt="xs" mb="sm">
<Group align="center" mb="sm">
<Text size="xs" fw={600} c="dimmed">
{t('fileManager.fileHistory', 'File History')} ({sortedHistory.length})
</Text>
</Group>
<Box ml="md">
{sortedHistory.map((historyFile, _index) => (
<FileListItem
key={`history-${historyFile.id}-${historyFile.versionNumber || 1}`}
file={historyFile}
isSelected={false} // History files are not selectable
isSupported={isFileSupported(historyFile.name)}
onSelect={() => {}} // No selection for history files
onRemove={() => onHistoryFileRemove(historyFile)} // Remove specific history file
onDownload={() => onDownloadSingle(historyFile)}
onDoubleClick={() => onFileDoubleClick(historyFile)}
isHistoryFile={true} // This enables "Add to Recents" in menu
isLatestVersion={false} // History files are never latest
// onAddToRecents is accessed from context by FileListItem
/>
))}
</Box>
</Box>
</Collapse>
);
};
export default FileHistoryGroup;

34
frontend/src/components/fileManager/FileInfoCard.tsx
View File

@ -2,10 +2,11 @@ import React from 'react';
import { Stack, Card, Box, Text, Badge, Group, Divider, ScrollArea } from '@mantine/core';
import { useTranslation } from 'react-i18next';
import { detectFileExtension, getFileSize } from '../../utils/fileUtils';
import { FileMetadata } from '../../types/file';
import { StirlingFileStub } from '../../types/fileContext';
import ToolChain from '../shared/ToolChain';
interface FileInfoCardProps {
currentFile: FileMetadata | null;
currentFile: StirlingFileStub | null;
modalHeight: string;
}
@ -53,11 +54,36 @@ const FileInfoCard: React.FC<FileInfoCardProps> = ({
<Divider />
<Group justify="space-between" py="xs">
<Text size="sm" c="dimmed">{t('fileManager.fileVersion', 'Version')}</Text>
<Text size="sm" c="dimmed">{t('fileManager.lastModified', 'Last Modified')}</Text>
<Text size="sm" fw={500}>
{currentFile ? '1.0' : ''}
{currentFile ? new Date(currentFile.lastModified).toLocaleDateString() : ''}
</Text>
</Group>
<Divider />
<Group justify="space-between" py="xs">
<Text size="sm" c="dimmed">{t('fileManager.fileVersion', 'Version')}</Text>
{currentFile &&
<Badge size="sm" variant="light" color={currentFile?.versionNumber ? 'blue' : 'gray'}>
v{currentFile ? (currentFile.versionNumber || 1) : ''}
</Badge>}
</Group>
{/* Tool Chain Display */}
{currentFile?.toolHistory && currentFile.toolHistory.length > 0 && (
<>
<Divider />
<Box py="xs">
<Text size="xs" c="dimmed" mb="xs">{t('fileManager.toolChain', 'Tools Applied')}</Text>
<ToolChain
toolChain={currentFile.toolHistory}
displayStyle="badges"
size="xs"
/>
</Box>
</>
)}
</Stack>
</ScrollArea>
</Card>

47
frontend/src/components/fileManager/FileListArea.tsx
View File

@ -4,6 +4,7 @@ import CloudIcon from '@mui/icons-material/Cloud';
import HistoryIcon from '@mui/icons-material/History';
import { useTranslation } from 'react-i18next';
import FileListItem from './FileListItem';
import FileHistoryGroup from './FileHistoryGroup';
import { useFileManagerContext } from '../../contexts/FileManagerContext';
interface FileListAreaProps {
@ -20,8 +21,11 @@ const FileListArea: React.FC<FileListAreaProps> = ({
recentFiles,
filteredFiles,
selectedFilesSet,
expandedFileIds,
loadedHistoryFiles,
onFileSelect,
onFileRemove,
onHistoryFileRemove,
onFileDoubleClick,
onDownloadSingle,
isFileSupported,
@ -50,18 +54,37 @@ const FileListArea: React.FC<FileListAreaProps> = ({
</Stack>
</Center>
) : (
filteredFiles.map((file, index) => (
<FileListItem
key={file.id}
file={file}
isSelected={selectedFilesSet.has(file.id)}
isSupported={isFileSupported(file.name)}
onSelect={(shiftKey) => onFileSelect(file, index, shiftKey)}
onRemove={() => onFileRemove(index)}
onDownload={() => onDownloadSingle(file)}
onDoubleClick={() => onFileDoubleClick(file)}
/>
))
filteredFiles.map((file, index) => {
// All files in filteredFiles are now leaf files only
const historyFiles = loadedHistoryFiles.get(file.id) || [];
const isExpanded = expandedFileIds.has(file.id);
return (
<React.Fragment key={file.id}>
<FileListItem
file={file}
isSelected={selectedFilesSet.has(file.id)}
isSupported={isFileSupported(file.name)}
onSelect={(shiftKey) => onFileSelect(file, index, shiftKey)}
onRemove={() => onFileRemove(index)}
onDownload={() => onDownloadSingle(file)}
onDoubleClick={() => onFileDoubleClick(file)}
isHistoryFile={false} // All files here are leaf files
isLatestVersion={true} // All files here are the latest versions
/>
<FileHistoryGroup
leafFile={file}
historyFiles={historyFiles}
isExpanded={isExpanded}
onDownloadSingle={onDownloadSingle}
onFileDoubleClick={onFileDoubleClick}
onHistoryFileRemove={onHistoryFileRemove}
isFileSupported={isFileSupported}
/>
</React.Fragment>
);
})
)}
</Stack>
</ScrollArea>

127
frontend/src/components/fileManager/FileListItem.tsx
View File

@ -3,12 +3,16 @@ import { Group, Box, Text, ActionIcon, Checkbox, Divider, Menu, Badge } from '@m
import MoreVertIcon from '@mui/icons-material/MoreVert';
import DeleteIcon from '@mui/icons-material/Delete';
import DownloadIcon from '@mui/icons-material/Download';
import HistoryIcon from '@mui/icons-material/History';
import RestoreIcon from '@mui/icons-material/Restore';
import { useTranslation } from 'react-i18next';
import { getFileSize, getFileDate } from '../../utils/fileUtils';
import { FileMetadata } from '../../types/file';
import { FileId, StirlingFileStub } from '../../types/fileContext';
import { useFileManagerContext } from '../../contexts/FileManagerContext';
import ToolChain from '../shared/ToolChain';
interface FileListItemProps {
file: FileMetadata;
file: StirlingFileStub;
isSelected: boolean;
isSupported: boolean;
onSelect: (shiftKey?: boolean) => void;
@ -16,6 +20,8 @@ interface FileListItemProps {
onDownload?: () => void;
onDoubleClick?: () => void;
isLast?: boolean;
isHistoryFile?: boolean; // Whether this is a history file (indented)
isLatestVersion?: boolean; // Whether this is the latest version (shows chevron)
}
const FileListItem: React.FC<FileListItemProps> = ({
@ -25,60 +31,89 @@ const FileListItem: React.FC<FileListItemProps> = ({
onSelect,
onRemove,
onDownload,
onDoubleClick
onDoubleClick,
isHistoryFile = false,
isLatestVersion = false
}) => {
const [isHovered, setIsHovered] = useState(false);
const [isMenuOpen, setIsMenuOpen] = useState(false);
const { t } = useTranslation();
const {expandedFileIds, onToggleExpansion, onAddToRecents } = useFileManagerContext();
// Keep item in hovered state if menu is open
const shouldShowHovered = isHovered || isMenuOpen;
// Get version information for this file
const leafFileId = (isLatestVersion ? file.id : (file.originalFileId || file.id)) as FileId;
const hasVersionHistory = (file.versionNumber || 1) > 1; // Show history for any processed file (v2+)
const currentVersion = file.versionNumber || 1; // Display original files as v1
const isExpanded = expandedFileIds.has(leafFileId);
return (
<>
<Box
p="sm"
style={{
cursor: 'pointer',
backgroundColor: isSelected ? 'var(--mantine-color-gray-1)' : (shouldShowHovered ? 'var(--mantine-color-gray-1)' : 'var(--bg-file-list)'),
cursor: isHistoryFile ? 'default' : 'pointer',
backgroundColor: isSelected
? 'var(--mantine-color-gray-1)'
: (shouldShowHovered ? 'var(--mantine-color-gray-1)' : 'var(--bg-file-list)'),
opacity: isSupported ? 1 : 0.5,
transition: 'background-color 0.15s ease',
userSelect: 'none',
WebkitUserSelect: 'none',
MozUserSelect: 'none',
msUserSelect: 'none'
msUserSelect: 'none',
paddingLeft: isHistoryFile ? '2rem' : '0.75rem', // Indent history files
borderLeft: isHistoryFile ? '3px solid var(--mantine-color-blue-4)' : 'none' // Visual indicator for history
}}
onClick={(e) => onSelect(e.shiftKey)}
onClick={isHistoryFile ? undefined : (e) => onSelect(e.shiftKey)}
onDoubleClick={onDoubleClick}
onMouseEnter={() => setIsHovered(true)}
onMouseLeave={() => setIsHovered(false)}
>
<Group gap="sm">
<Box>
<Checkbox
checked={isSelected}
onChange={() => {}} // Handled by parent onClick
size="sm"
pl="sm"
pr="xs"
styles={{
input: {
cursor: 'pointer'
}
}}
/>
</Box>
{!isHistoryFile && (
<Box>
{/* Checkbox for regular files only */}
<Checkbox
checked={isSelected}
onChange={() => {}} // Handled by parent onClick
size="sm"
pl="sm"
pr="xs"
styles={{
input: {
cursor: 'pointer'
}
}}
/>
</Box>
)}
<Box style={{ flex: 1, minWidth: 0 }}>
<Group gap="xs" align="center">
<Text size="sm" fw={500} truncate style={{ flex: 1 }}>{file.name}</Text>
{file.isDraft && (
<Badge size="xs" variant="light" color="orange">
DRAFT
</Badge>
<Badge size="xs" variant="light" color={"blue"}>
v{currentVersion}
</Badge>
</Group>
<Group gap="xs" align="center">
<Text size="xs" c="dimmed">
{getFileSize(file)} • {getFileDate(file)}
</Text>
{/* Tool chain for processed files */}
{file.toolHistory && file.toolHistory.length > 0 && (
<ToolChain
toolChain={file.toolHistory}
maxWidth={'150px'}
displayStyle="text"
size="xs"
/>
)}
</Group>
<Text size="xs" c="dimmed">{getFileSize(file)} • {getFileDate(file)}</Text>
</Box>
{/* Three dots menu - fades in/out on hover */}
@ -117,6 +152,46 @@ const FileListItem: React.FC<FileListItemProps> = ({
{t('fileManager.download', 'Download')}
</Menu.Item>
)}
{/* Show/Hide History option for latest version files */}
{isLatestVersion && hasVersionHistory && (
<>
<Menu.Item
leftSection={
<HistoryIcon style={{ fontSize: 16 }} />
}
onClick={(e) => {
e.stopPropagation();
onToggleExpansion(leafFileId);
}}
>
{
(isExpanded ?
t('fileManager.hideHistory', 'Hide History') :
t('fileManager.showHistory', 'Show History')
)
}
</Menu.Item>
<Menu.Divider />
</>
)}
{/* Restore option for history files */}
{isHistoryFile && (
<>
<Menu.Item
leftSection={<RestoreIcon style={{ fontSize: 16 }} />}
onClick={(e) => {
e.stopPropagation();
onAddToRecents(file);
}}
>
{t('fileManager.restore', 'Restore')}
</Menu.Item>
<Menu.Divider />
</>
)}
<Menu.Item
leftSection={<DeleteIcon style={{ fontSize: 16 }} />}
onClick={(e) => {

4
frontend/src/components/layout/Workbench.tsx
View File

@ -42,7 +42,7 @@ export default function Workbench() {
// Get tool registry to look up selected tool
const { toolRegistry } = useToolManagement();
const selectedTool = selectedToolId ? toolRegistry[selectedToolId] : null;
const { addToActiveFiles } = useFileHandler();
const { addFiles } = useFileHandler();
const handlePreviewClose = () => {
setPreviewFile(null);
@ -81,7 +81,7 @@ export default function Workbench() {
setCurrentView("pageEditor");
},
onMergeFiles: (filesToMerge) => {
filesToMerge.forEach(addToActiveFiles);
addFiles(filesToMerge);
setCurrentView("viewer");
}
})}

11
frontend/src/components/shared/FileCard.tsx
View File

@ -12,7 +12,7 @@ import { useIndexedDBThumbnail } from "../../hooks/useIndexedDBThumbnail";
interface FileCardProps {
file: File;
record?: StirlingFileStub;
fileStub?: StirlingFileStub;
onRemove: () => void;
onDoubleClick?: () => void;
onView?: () => void;
@ -22,12 +22,11 @@ interface FileCardProps {
isSupported?: boolean; // Whether the file format is supported by the current tool
}
const FileCard = ({ file, record, onRemove, onDoubleClick, onView, onEdit, isSelected, onSelect, isSupported = true }: FileCardProps) => {
const FileCard = ({ file, fileStub, onRemove, onDoubleClick, onView, onEdit, isSelected, onSelect, isSupported = true }: FileCardProps) => {
const { t } = useTranslation();
// Use record thumbnail if available, otherwise fall back to IndexedDB lookup
const fileMetadata = record ? { id: record.id, name: record.name, type: record.type, size: record.size, lastModified: record.lastModified } : null;
const { thumbnail: indexedDBThumb, isGenerating } = useIndexedDBThumbnail(fileMetadata);
const thumb = record?.thumbnailUrl || indexedDBThumb;
const { thumbnail: indexedDBThumb, isGenerating } = useIndexedDBThumbnail(fileStub);
const thumb = fileStub?.thumbnailUrl || indexedDBThumb;
const [isHovered, setIsHovered] = useState(false);
return (
@ -177,7 +176,7 @@ const FileCard = ({ file, record, onRemove, onDoubleClick, onView, onEdit, isSel
<Badge color="blue" variant="light" size="sm">
{getFileDate(file)}
</Badge>
{record?.id && (
{fileStub?.id && (
<Badge
color="green"
variant="light"

2
frontend/src/components/shared/FileGrid.tsx
View File

@ -139,7 +139,7 @@ const FileGrid = ({
<FileCard
key={fileId + idx}
file={item.file}
record={item.record}
fileStub={item.record}
onRemove={onRemove ? () => onRemove(originalIdx) : () => {}}
onDoubleClick={onDoubleClick && supported ? () => onDoubleClick(item) : undefined}
onView={onView && supported ? () => onView(item) : undefined}

6
frontend/src/components/shared/FilePreview.tsx
View File

@ -1,7 +1,7 @@
import React from 'react';
import { Box, Center } from '@mantine/core';
import InsertDriveFileIcon from '@mui/icons-material/InsertDriveFile';
import { FileMetadata } from '../../types/file';
import { StirlingFileStub } from '../../types/fileContext';
import DocumentThumbnail from './filePreview/DocumentThumbnail';
import DocumentStack from './filePreview/DocumentStack';
import HoverOverlay from './filePreview/HoverOverlay';
@ -9,7 +9,7 @@ import NavigationArrows from './filePreview/NavigationArrows';
export interface FilePreviewProps {
// Core file data
file: File | FileMetadata | null;
file: File | StirlingFileStub | null;
thumbnail?: string | null;
// Optional features
@ -22,7 +22,7 @@ export interface FilePreviewProps {
isAnimating?: boolean;
// Event handlers
onFileClick?: (file: File | FileMetadata | null) => void;
onFileClick?: (file: File | StirlingFileStub | null) => void;
onPrevious?: () => void;
onNext?: () => void;
}

6
frontend/src/components/shared/LandingPage.tsx
View File

@ -7,7 +7,7 @@ import { useFileHandler } from '../../hooks/useFileHandler';
import { useFilesModalContext } from '../../contexts/FilesModalContext';
const LandingPage = () => {
const { addMultipleFiles } = useFileHandler();
const { addFiles } = useFileHandler();
const fileInputRef = React.useRef<HTMLInputElement>(null);
const { colorScheme } = useMantineColorScheme();
const { t } = useTranslation();
@ -15,7 +15,7 @@ const LandingPage = () => {
const [isUploadHover, setIsUploadHover] = React.useState(false);
const handleFileDrop = async (files: File[]) => {
await addMultipleFiles(files);
await addFiles(files);
};
const handleOpenFilesModal = () => {
@ -29,7 +29,7 @@ const LandingPage = () => {
const handleFileSelect = async (event: React.ChangeEvent<HTMLInputElement>) => {
const files = Array.from(event.target.files || []);
if (files.length > 0) {
await addMultipleFiles(files);
await addFiles(files);
}
// Reset the input so the same file can be selected again
event.target.value = '';

153
frontend/src/components/shared/ToolChain.tsx Normal file
View File

@ -0,0 +1,153 @@
/**
* Reusable ToolChain component with smart truncation and tooltip expansion
* Used across FileListItem, FileDetails, and FileThumbnail for consistent display
*/
import React from 'react';
import { Text, Tooltip, Badge, Group } from '@mantine/core';
import { ToolOperation } from '../../types/file';
interface ToolChainProps {
toolChain: ToolOperation[];
maxWidth?: string;
displayStyle?: 'text' | 'badges' | 'compact';
size?: 'xs' | 'sm' | 'md';
color?: string;
}
const ToolChain: React.FC<ToolChainProps> = ({
toolChain,
maxWidth = '100%',
displayStyle = 'text',
size = 'xs',
color = 'var(--mantine-color-blue-7)'
}) => {
if (!toolChain || toolChain.length === 0) return null;
const toolNames = toolChain.map(tool => tool.toolName);
// Create full tool chain for tooltip
const fullChainDisplay = displayStyle === 'badges' ? (
<Group gap="xs" wrap="wrap">
{toolChain.map((tool, index) => (
<React.Fragment key={`${tool.toolName}-${index}`}>
<Badge size="sm" variant="light" color="blue">
{tool.toolName}
</Badge>
{index < toolChain.length - 1 && (
<Text size="sm" c="dimmed">→</Text>
)}
</React.Fragment>
))}
</Group>
) : (
<Text size="sm">{toolNames.join(' → ')}</Text>
);
// Create truncated display based on available space
const getTruncatedDisplay = () => {
if (toolNames.length <= 2) {
// Show all tools if 2 or fewer
return { text: toolNames.join(' → '), isTruncated: false };
} else {
// Show first tool ... last tool for longer chains
return {
text: `${toolNames[0]} → +${toolNames.length-2} → ${toolNames[toolNames.length - 1]}`,
isTruncated: true
};
}
};
const { text: truncatedText, isTruncated } = getTruncatedDisplay();
// Compact style for very small spaces
if (displayStyle === 'compact') {
const compactText = toolNames.length === 1 ? toolNames[0] : `${toolNames.length} tools`;
const isCompactTruncated = toolNames.length > 1;
const compactElement = (
<Text
size={size}
style={{
color,
fontWeight: 500,
whiteSpace: 'nowrap',
overflow: 'hidden',
textOverflow: 'ellipsis',
maxWidth: `${maxWidth}`,
cursor: isCompactTruncated ? 'help' : 'default'
}}
>
{compactText}
</Text>
);
return isCompactTruncated ? (
<Tooltip label={fullChainDisplay} multiline withinPortal>
{compactElement}
</Tooltip>
) : compactElement;
}
// Badge style for file details
if (displayStyle === 'badges') {
const isBadgesTruncated = toolChain.length > 3;
const badgesElement = (
<div style={{ maxWidth: `${maxWidth}`, overflow: 'hidden' }}>
<Group gap="2px" wrap="nowrap">
{toolChain.slice(0, 3).map((tool, index) => (
<React.Fragment key={`${tool.toolName}-${index}`}>
<Badge size={size} variant="light" color="blue">
{tool.toolName}
</Badge>
{index < Math.min(toolChain.length - 1, 2) && (
<Text size="xs" c="dimmed">→</Text>
)}
</React.Fragment>
))}
{toolChain.length > 3 && (
<>
<Text size="xs" c="dimmed">...</Text>
<Badge size={size} variant="light" color="blue">
{toolChain[toolChain.length - 1].toolName}
</Badge>
</>
)}
</Group>
</div>
);
return isBadgesTruncated ? (
<Tooltip label={`${toolNames.join(' → ')}`} withinPortal>
{badgesElement}
</Tooltip>
) : badgesElement;
}
// Text style (default) for file list items
const textElement = (
<Text
size={size}
style={{
color,
fontWeight: 500,
whiteSpace: 'nowrap',
overflow: 'hidden',
textOverflow: 'ellipsis',
maxWidth: `${maxWidth}`,
cursor: isTruncated ? 'help' : 'default'
}}
>
{truncatedText}
</Text>
);
return isTruncated ? (
<Tooltip label={fullChainDisplay} withinPortal>
{textElement}
</Tooltip>
) : textElement;
};
export default ToolChain;

4
frontend/src/components/shared/filePreview/DocumentThumbnail.tsx
View File

@ -1,10 +1,10 @@
import React from 'react';
import { Box, Center, Image } from '@mantine/core';
import PictureAsPdfIcon from '@mui/icons-material/PictureAsPdf';
import { FileMetadata } from '../../../types/file';
import { StirlingFileStub } from '../../../types/fileContext';
export interface DocumentThumbnailProps {
file: File | FileMetadata | null;
file: File | StirlingFileStub | null;
thumbnail?: string | null;
style?: React.CSSProperties;
onClick?: () => void;

4
frontend/src/components/tools/shared/FileStatusIndicator.tsx
View File

@ -18,7 +18,7 @@ const FileStatusIndicator = ({
minFiles = 1,
}: FileStatusIndicatorProps) => {
const { t } = useTranslation();
const { openFilesModal, onFilesSelect } = useFilesModalContext();
const { openFilesModal, onFileUpload } = useFilesModalContext();
const { files: stirlingFileStubs } = useAllFiles();
const { loadRecentFiles } = useFileManager();
const [hasRecentFiles, setHasRecentFiles] = useState<boolean | null>(null);
@ -45,7 +45,7 @@ const FileStatusIndicator = ({
input.onchange = (event) => {
const files = Array.from((event.target as HTMLInputElement).files || []);
if (files.length > 0) {
onFilesSelect(files);
onFileUpload(files);
}
};
input.click();

7
frontend/src/components/viewer/Viewer.tsx
View File

@ -372,11 +372,12 @@ const Viewer = ({
else if (effectiveFile.url?.startsWith('indexeddb:')) {
const fileId = effectiveFile.url.replace('indexeddb:', '') as FileId /* FIX ME: Not sure this is right - at least probably not the right place for this logic */;
// Get data directly from IndexedDB
const arrayBuffer = await fileStorage.getFileData(fileId);
if (!arrayBuffer) {
// Get file directly from IndexedDB
const file = await fileStorage.getStirlingFile(fileId);
if (!file) {
throw new Error('File not found in IndexedDB - may have been purged by browser');
}
const arrayBuffer = await file.arrayBuffer();
// Store reference for cleanup
currentArrayBufferRef.current = arrayBuffer;

53
frontend/src/contexts/FileContext.tsx
View File

@ -22,13 +22,12 @@ import {
FileId,
StirlingFileStub,
StirlingFile,
createStirlingFile
} from '../types/fileContext';
// Import modular components
import { fileContextReducer, initialFileContextState } from './file/FileReducer';
import { createFileSelectors } from './file/fileSelectors';
import { AddedFile, addFiles, consumeFiles, undoConsumeFiles, createFileActions } from './file/fileActions';
import { addFiles, addStirlingFileStubs, consumeFiles, undoConsumeFiles, createFileActions } from './file/fileActions';
import { FileLifecycleManager } from './file/lifecycle';
import { FileStateContext, FileActionsContext } from './file/contexts';
import { IndexedDBProvider, useIndexedDB } from './IndexedDBContext';
@ -73,58 +72,44 @@ function FileContextInner({
dispatch({ type: 'SET_UNSAVED_CHANGES', payload: { hasChanges } });
}, []);
const selectFiles = (addedFilesWithIds: AddedFile[]) => {
const selectFiles = (stirlingFiles: StirlingFile[]) => {
const currentSelection = stateRef.current.ui.selectedFileIds;
const newFileIds = addedFilesWithIds.map(({ id }) => id);
const newFileIds = stirlingFiles.map(stirlingFile => stirlingFile.fileId);
dispatch({ type: 'SET_SELECTED_FILES', payload: { fileIds: [...currentSelection, ...newFileIds] } });
}
// File operations using unified addFiles helper with persistence
const addRawFiles = useCallback(async (files: File[], options?: { insertAfterPageId?: string; selectFiles?: boolean }): Promise<StirlingFile[]> => {
const addedFilesWithIds = await addFiles('raw', { files, ...options }, stateRef, filesRef, dispatch, lifecycleManager);
const stirlingFiles = await addFiles({ files, ...options }, stateRef, filesRef, dispatch, lifecycleManager, enablePersistence);
// Auto-select the newly added files if requested
if (options?.selectFiles && addedFilesWithIds.length > 0) {
selectFiles(addedFilesWithIds);
if (options?.selectFiles && stirlingFiles.length > 0) {
selectFiles(stirlingFiles);
}
// Persist to IndexedDB if enabled
if (indexedDB && enablePersistence && addedFilesWithIds.length > 0) {
await Promise.all(addedFilesWithIds.map(async ({ file, id, thumbnail }) => {
try {
await indexedDB.saveFile(file, id, thumbnail);
} catch (error) {
console.error('Failed to persist file to IndexedDB:', file.name, error);
}
}));
}
return stirlingFiles;
}, [enablePersistence]);
return addedFilesWithIds.map(({ file, id }) => createStirlingFile(file, id));
}, [indexedDB, enablePersistence]);
const addProcessedFiles = useCallback(async (filesWithThumbnails: Array<{ file: File; thumbnail?: string; pageCount?: number }>): Promise<StirlingFile[]> => {
const result = await addFiles('processed', { filesWithThumbnails }, stateRef, filesRef, dispatch, lifecycleManager);
return result.map(({ file, id }) => createStirlingFile(file, id));
}, []);
const addStoredFiles = useCallback(async (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: any }>, options?: { selectFiles?: boolean }): Promise<StirlingFile[]> => {
const result = await addFiles('stored', { filesWithMetadata }, stateRef, filesRef, dispatch, lifecycleManager);
const addStirlingFileStubsAction = useCallback(async (stirlingFileStubs: StirlingFileStub[], options?: { insertAfterPageId?: string; selectFiles?: boolean }): Promise<StirlingFile[]> => {
// StirlingFileStubs preserve all metadata - perfect for FileManager use case!
const result = await addStirlingFileStubs(stirlingFileStubs, options, stateRef, filesRef, dispatch, lifecycleManager);
// Auto-select the newly added files if requested
if (options?.selectFiles && result.length > 0) {
selectFiles(result);
}
return result.map(({ file, id }) => createStirlingFile(file, id));
return result;
}, []);
// Action creators
const baseActions = useMemo(() => createFileActions(dispatch), []);
// Helper functions for pinned files
const consumeFilesWrapper = useCallback(async (inputFileIds: FileId[], outputFiles: File[]): Promise<FileId[]> => {
return consumeFiles(inputFileIds, outputFiles, filesRef, dispatch, indexedDB);
}, [indexedDB]);
const consumeFilesWrapper = useCallback(async (inputFileIds: FileId[], outputStirlingFiles: StirlingFile[], outputStirlingFileStubs: StirlingFileStub[]): Promise<FileId[]> => {
return consumeFiles(inputFileIds, outputStirlingFiles, outputStirlingFileStubs, filesRef, dispatch);
}, []);
const undoConsumeFilesWrapper = useCallback(async (inputFiles: File[], inputStirlingFileStubs: StirlingFileStub[], outputFileIds: FileId[]): Promise<void> => {
return undoConsumeFiles(inputFiles, inputStirlingFileStubs, outputFileIds, filesRef, dispatch, indexedDB);
@ -143,8 +128,7 @@ function FileContextInner({
const actions = useMemo<FileContextActions>(() => ({
...baseActions,
addFiles: addRawFiles,
addProcessedFiles,
addStoredFiles,
addStirlingFileStubs: addStirlingFileStubsAction,
removeFiles: async (fileIds: FileId[], deleteFromStorage?: boolean) => {
// Remove from memory and cleanup resources
lifecycleManager.removeFiles(fileIds, stateRef);
@ -199,8 +183,7 @@ function FileContextInner({
}), [
baseActions,
addRawFiles,
addProcessedFiles,
addStoredFiles,
addStirlingFileStubsAction,
lifecycleManager,
setHasUnsavedChanges,
consumeFilesWrapper,

394
frontend/src/contexts/FileManagerContext.tsx
View File

@ -1,8 +1,9 @@
import React, { createContext, useContext, useState, useRef, useCallback, useEffect, useMemo } from 'react';
import { FileMetadata } from '../types/file';
import { fileStorage } from '../services/fileStorage';
import { StirlingFileStub } from '../types/fileContext';
import { downloadFiles } from '../utils/downloadUtils';
import { FileId } from '../types/file';
import { groupFilesByOriginal } from '../utils/fileHistoryUtils';
// Type for the context value - now contains everything directly
interface FileManagerContextValue {
@ -10,27 +11,34 @@ interface FileManagerContextValue {
activeSource: 'recent' | 'local' | 'drive';
selectedFileIds: FileId[];
searchTerm: string;
selectedFiles: FileMetadata[];
filteredFiles: FileMetadata[];
selectedFiles: StirlingFileStub[];
filteredFiles: StirlingFileStub[];
fileInputRef: React.RefObject<HTMLInputElement | null>;
selectedFilesSet: Set<string>;
selectedFilesSet: Set<FileId>;
expandedFileIds: Set<FileId>;
fileGroups: Map<FileId, StirlingFileStub[]>;
loadedHistoryFiles: Map<FileId, StirlingFileStub[]>;
// Handlers
onSourceChange: (source: 'recent' | 'local' | 'drive') => void;
onLocalFileClick: () => void;
onFileSelect: (file: FileMetadata, index: number, shiftKey?: boolean) => void;
onFileSelect: (file: StirlingFileStub, index: number, shiftKey?: boolean) => void;
onFileRemove: (index: number) => void;
onFileDoubleClick: (file: FileMetadata) => void;
onHistoryFileRemove: (file: StirlingFileStub) => void;
onFileDoubleClick: (file: StirlingFileStub) => void;
onOpenFiles: () => void;
onSearchChange: (value: string) => void;
onFileInputChange: (event: React.ChangeEvent<HTMLInputElement>) => void;
onSelectAll: () => void;
onDeleteSelected: () => void;
onDownloadSelected: () => void;
onDownloadSingle: (file: FileMetadata) => void;
onDownloadSingle: (file: StirlingFileStub) => void;
onToggleExpansion: (fileId: FileId) => void;
onAddToRecents: (file: StirlingFileStub) => void;
onNewFilesSelect: (files: File[]) => void;
// External props
recentFiles: FileMetadata[];
recentFiles: StirlingFileStub[];
isFileSupported: (fileName: string) => boolean;
modalHeight: string;
}
@ -41,8 +49,8 @@ const FileManagerContext = createContext<FileManagerContextValue | null>(null);
// Provider component props
interface FileManagerProviderProps {
children: React.ReactNode;
recentFiles: FileMetadata[];
onFilesSelected: (files: FileMetadata[]) => void; // For selecting stored files
recentFiles: StirlingFileStub[];
onRecentFilesSelected: (files: StirlingFileStub[]) => void; // For selecting stored files
onNewFilesSelect: (files: File[]) => void; // For uploading new local files
onClose: () => void;
isFileSupported: (fileName: string) => boolean;
@ -55,7 +63,7 @@ interface FileManagerProviderProps {
export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
children,
recentFiles,
onFilesSelected,
onRecentFilesSelected,
onNewFilesSelect,
onClose,
isFileSupported,
@ -68,19 +76,44 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
const [selectedFileIds, setSelectedFileIds] = useState<FileId[]>([]);
const [searchTerm, setSearchTerm] = useState('');
const [lastClickedIndex, setLastClickedIndex] = useState<number | null>(null);
const [expandedFileIds, setExpandedFileIds] = useState<Set<FileId>>(new Set());
const [loadedHistoryFiles, setLoadedHistoryFiles] = useState<Map<FileId, StirlingFileStub[]>>(new Map()); // Cache for loaded history
const fileInputRef = useRef<HTMLInputElement>(null);
// Track blob URLs for cleanup
const createdBlobUrls = useRef<Set<string>>(new Set());
// Computed values (with null safety)
const selectedFilesSet = new Set(selectedFileIds);
const selectedFiles = selectedFileIds.length === 0 ? [] :
(recentFiles || []).filter(file => selectedFilesSet.has(file.id));
// Group files by original file ID for version management
const fileGroups = useMemo(() => {
if (!recentFiles || recentFiles.length === 0) return new Map();
const filteredFiles = !searchTerm ? recentFiles || [] :
(recentFiles || []).filter(file =>
// Convert StirlingFileStub to FileRecord-like objects for grouping utility
const recordsForGrouping = recentFiles.map(file => ({
...file,
originalFileId: file.originalFileId,
versionNumber: file.versionNumber || 1
}));
return groupFilesByOriginal(recordsForGrouping);
}, [recentFiles]);
// Get files to display with expansion logic
const displayFiles = useMemo(() => {
if (!recentFiles || recentFiles.length === 0) return [];
// Only return leaf files - history files will be handled by separate components
return recentFiles;
}, [recentFiles]);
const selectedFiles = selectedFileIds.length === 0 ? [] :
displayFiles.filter(file => selectedFilesSet.has(file.id));
const filteredFiles = !searchTerm ? displayFiles :
displayFiles.filter(file =>
file.name.toLowerCase().includes(searchTerm.toLowerCase())
);
@ -97,7 +130,7 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
fileInputRef.current?.click();
}, []);
const handleFileSelect = useCallback((file: FileMetadata, currentIndex: number, shiftKey?: boolean) => {
const handleFileSelect = useCallback((file: StirlingFileStub, currentIndex: number, shiftKey?: boolean) => {
const fileId = file.id;
if (!fileId) return;
@ -138,27 +171,214 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
}
}, [filteredFiles, lastClickedIndex]);
const handleFileRemove = useCallback((index: number) => {
// Helper function to safely determine which files can be deleted
const getSafeFilesToDelete = useCallback((
fileIds: FileId[],
allStoredStubs: StirlingFileStub[]
): FileId[] => {
const fileMap = new Map(allStoredStubs.map(f => [f.id, f]));
const filesToDelete = new Set<FileId>();
const filesToPreserve = new Set<FileId>();
// First, identify all files in the lineages of the leaf files being deleted
for (const leafFileId of fileIds) {
const currentFile = fileMap.get(leafFileId);
if (!currentFile) continue;
// Always include the leaf file itself for deletion
filesToDelete.add(leafFileId);
// If this is a processed file with history, trace back through its lineage
if (currentFile.versionNumber && currentFile.versionNumber > 1) {
const originalFileId = currentFile.originalFileId || currentFile.id;
// Find all files in this history chain
const chainFiles = allStoredStubs.filter((file: StirlingFileStub) =>
(file.originalFileId || file.id) === originalFileId
);
// Add all files in this lineage as candidates for deletion
chainFiles.forEach(file => filesToDelete.add(file.id));
}
}
// Now identify files that must be preserved because they're referenced by OTHER lineages
for (const file of allStoredStubs) {
const fileOriginalId = file.originalFileId || file.id;
// If this file is a leaf node (not being deleted) and its lineage overlaps with files we want to delete
if (file.isLeaf !== false && !fileIds.includes(file.id)) {
// Find all files in this preserved lineage
const preservedChainFiles = allStoredStubs.filter((chainFile: StirlingFileStub) =>
(chainFile.originalFileId || chainFile.id) === fileOriginalId
);
// Mark all files in this preserved lineage as must-preserve
preservedChainFiles.forEach(chainFile => filesToPreserve.add(chainFile.id));
}
}
// Final list: files to delete minus files that must be preserved
let safeToDelete = Array.from(filesToDelete).filter(fileId => !filesToPreserve.has(fileId));
// Check for orphaned non-leaf files after main deletion
const remainingFiles = allStoredStubs.filter(file => !safeToDelete.includes(file.id));
const orphanedNonLeafFiles: FileId[] = [];
for (const file of remainingFiles) {
// Only check non-leaf files (files that have been processed and have children)
if (file.isLeaf === false) {
const fileOriginalId = file.originalFileId || file.id;
// Check if this non-leaf file has any living descendants
const hasLivingDescendants = remainingFiles.some(otherFile => {
// Check if otherFile is a descendant of this file
const otherOriginalId = otherFile.originalFileId || otherFile.id;
return (
// Direct parent relationship
otherFile.parentFileId === file.id ||
// Same lineage but different from this file
(otherOriginalId === fileOriginalId && otherFile.id !== file.id)
);
});
if (!hasLivingDescendants) {
orphanedNonLeafFiles.push(file.id);
}
}
}
// Add orphaned non-leaf files to deletion list
safeToDelete = [...safeToDelete, ...orphanedNonLeafFiles];
return safeToDelete;
}, []);
// Shared internal delete logic
const performFileDelete = useCallback(async (fileToRemove: StirlingFileStub, fileIndex: number) => {
const deletedFileId = fileToRemove.id;
// Get all stored files to analyze lineages
const allStoredStubs = await fileStorage.getAllStirlingFileStubs();
// Get safe files to delete (respecting shared lineages)
const filesToDelete = getSafeFilesToDelete([deletedFileId], allStoredStubs);
// Clear from selection immediately
setSelectedFileIds(prev => prev.filter(id => !filesToDelete.includes(id)));
// Clear from expanded state to prevent ghost entries
setExpandedFileIds(prev => {
const newExpanded = new Set(prev);
filesToDelete.forEach(id => newExpanded.delete(id));
return newExpanded;
});
// Clear from history cache - remove all files in the chain
setLoadedHistoryFiles(prev => {
const newCache = new Map(prev);
// Remove cache entries for all deleted files
filesToDelete.forEach(id => newCache.delete(id as FileId));
// Also remove deleted files from any other file's history cache
for (const [mainFileId, historyFiles] of newCache.entries()) {
const filteredHistory = historyFiles.filter(histFile => !filesToDelete.includes(histFile.id));
if (filteredHistory.length !== historyFiles.length) {
newCache.set(mainFileId, filteredHistory);
}
}
return newCache;
});
// Delete safe files from IndexedDB
try {
for (const fileId of filesToDelete) {
await fileStorage.deleteStirlingFile(fileId as FileId);
}
} catch (error) {
console.error('Failed to delete files from chain:', error);
}
// Call the parent's deletion logic for the main file only
onFileRemove(fileIndex);
// Refresh to ensure consistent state
await refreshRecentFiles();
}, [getSafeFilesToDelete, setSelectedFileIds, setExpandedFileIds, setLoadedHistoryFiles, onFileRemove, refreshRecentFiles]);
const handleFileRemove = useCallback(async (index: number) => {
const fileToRemove = filteredFiles[index];
if (fileToRemove) {
setSelectedFileIds(prev => prev.filter(id => id !== fileToRemove.id));
await performFileDelete(fileToRemove, index);
}
onFileRemove(index);
}, [filteredFiles, onFileRemove]);
}, [filteredFiles, performFileDelete]);
const handleFileDoubleClick = useCallback((file: FileMetadata) => {
// Handle deletion by fileId (more robust than index-based)
const handleFileRemoveById = useCallback(async (fileId: FileId) => {
// Find the file and its index in filteredFiles
const fileIndex = filteredFiles.findIndex(file => file.id === fileId);
const fileToRemove = filteredFiles[fileIndex];
if (fileToRemove && fileIndex !== -1) {
await performFileDelete(fileToRemove, fileIndex);
}
}, [filteredFiles, performFileDelete]);
// Handle deletion of specific history files (not index-based)
const handleHistoryFileRemove = useCallback(async (fileToRemove: StirlingFileStub) => {
const deletedFileId = fileToRemove.id;
// Clear from expanded state to prevent ghost entries
setExpandedFileIds(prev => {
const newExpanded = new Set(prev);
newExpanded.delete(deletedFileId);
return newExpanded;
});
// Clear from history cache - remove all files in the chain
setLoadedHistoryFiles(prev => {
const newCache = new Map(prev);
// Remove cache entries for all deleted files
newCache.delete(deletedFileId);
// Also remove deleted files from any other file's history cache
for (const [mainFileId, historyFiles] of newCache.entries()) {
const filteredHistory = historyFiles.filter(histFile => deletedFileId != histFile.id);
if (filteredHistory.length !== historyFiles.length) {
newCache.set(mainFileId, filteredHistory);
}
}
return newCache;
});
// Delete safe files from IndexedDB
try {
await fileStorage.deleteStirlingFile(deletedFileId);
} catch (error) {
console.error('Failed to delete files from chain:', error);
}
// Refresh to ensure consistent state
await refreshRecentFiles();
}, [filteredFiles, onFileRemove, refreshRecentFiles, getSafeFilesToDelete]);
const handleFileDoubleClick = useCallback((file: StirlingFileStub) => {
if (isFileSupported(file.name)) {
onFilesSelected([file]);
onRecentFilesSelected([file]);
onClose();
}
}, [isFileSupported, onFilesSelected, onClose]);
}, [isFileSupported, onRecentFilesSelected, onClose]);
const handleOpenFiles = useCallback(() => {
if (selectedFiles.length > 0) {
onFilesSelected(selectedFiles);
onRecentFilesSelected(selectedFiles);
onClose();
}
}, [selectedFiles, onFilesSelected, onClose]);
}, [selectedFiles, onRecentFilesSelected, onClose]);
const handleSearchChange = useCallback((value: string) => {
setSearchTerm(value);
@ -196,25 +416,14 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
if (selectedFileIds.length === 0) return;
try {
// Get files to delete based on current filtered view
const filesToDelete = filteredFiles.filter(file =>
selectedFileIds.includes(file.id)
);
// Delete files from storage
for (const file of filesToDelete) {
await fileStorage.deleteFile(file.id);
// Delete each selected file using the proven single delete logic
for (const fileId of selectedFileIds) {
await handleFileRemoveById(fileId);
}
// Clear selection
setSelectedFileIds([]);
// Refresh the file list
await refreshRecentFiles();
} catch (error) {
console.error('Failed to delete selected files:', error);
}
}, [selectedFileIds, filteredFiles, refreshRecentFiles]);
}, [selectedFileIds, handleFileRemoveById]);
const handleDownloadSelected = useCallback(async () => {
@ -235,7 +444,7 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
}
}, [selectedFileIds, filteredFiles]);
const handleDownloadSingle = useCallback(async (file: FileMetadata) => {
const handleDownloadSingle = useCallback(async (file: StirlingFileStub) => {
try {
await downloadFiles([file]);
} catch (error) {
@ -243,6 +452,94 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
}
}, []);
const handleToggleExpansion = useCallback(async (fileId: FileId) => {
const isCurrentlyExpanded = expandedFileIds.has(fileId);
// Update expansion state
setExpandedFileIds(prev => {
const newSet = new Set(prev);
if (newSet.has(fileId)) {
newSet.delete(fileId);
} else {
newSet.add(fileId);
}
return newSet;
});
// Load complete history chain if expanding
if (!isCurrentlyExpanded) {
const currentFileMetadata = recentFiles.find(f => f.id === fileId);
if (currentFileMetadata && (currentFileMetadata.versionNumber || 1) > 1) {
try {
// Get all stored file metadata for chain traversal
const allStoredStubs = await fileStorage.getAllStirlingFileStubs();
const fileMap = new Map(allStoredStubs.map(f => [f.id, f]));
// Get the current file's IndexedDB data
const currentStoredStub = fileMap.get(fileId as FileId);
if (!currentStoredStub) {
console.warn(`No stored file found for ${fileId}`);
return;
}
// Build complete history chain using IndexedDB metadata
const historyFiles: StirlingFileStub[] = [];
// Find the original file
// Collect only files in this specific branch (ancestors of current file)
const chainFiles: StirlingFileStub[] = [];
const allFiles = Array.from(fileMap.values());
// Build a map for fast parent lookups
const fileIdMap = new Map<FileId, StirlingFileStub>();
allFiles.forEach(f => fileIdMap.set(f.id, f));
// Trace back from current file through parent chain
let currentFile = fileIdMap.get(fileId);
while (currentFile?.parentFileId) {
const parentFile = fileIdMap.get(currentFile.parentFileId);
if (parentFile) {
chainFiles.push(parentFile);
currentFile = parentFile;
} else {
break; // Parent not found, stop tracing
}
}
// Sort by version number (oldest first for history display)
chainFiles.sort((a, b) => (a.versionNumber || 1) - (b.versionNumber || 1));
// StirlingFileStubs already have all the data we need - no conversion required!
historyFiles.push(...chainFiles);
// Cache the loaded history files
setLoadedHistoryFiles(prev => new Map(prev.set(fileId as FileId, historyFiles)));
} catch (error) {
console.warn(`Failed to load history chain for file ${fileId}:`, error);
}
}
} else {
// Clear loaded history when collapsing
setLoadedHistoryFiles(prev => {
const newMap = new Map(prev);
newMap.delete(fileId as FileId);
return newMap;
});
}
}, [expandedFileIds, recentFiles]);
const handleAddToRecents = useCallback(async (file: StirlingFileStub) => {
try {
// Mark the file as a leaf node so it appears in recent files
await fileStorage.markFileAsLeaf(file.id);
// Refresh the recent files list to show updated state
await refreshRecentFiles();
} catch (error) {
console.error('Failed to add to recents:', error);
}
}, [refreshRecentFiles]);
// Cleanup blob URLs when component unmounts
useEffect(() => {
@ -274,12 +571,16 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
filteredFiles,
fileInputRef,
selectedFilesSet,
expandedFileIds,
fileGroups,
loadedHistoryFiles,
// Handlers
onSourceChange: handleSourceChange,
onLocalFileClick: handleLocalFileClick,
onFileSelect: handleFileSelect,
onFileRemove: handleFileRemove,
onHistoryFileRemove: handleHistoryFileRemove,
onFileDoubleClick: handleFileDoubleClick,
onOpenFiles: handleOpenFiles,
onSearchChange: handleSearchChange,
@ -288,6 +589,9 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
onDeleteSelected: handleDeleteSelected,
onDownloadSelected: handleDownloadSelected,
onDownloadSingle: handleDownloadSingle,
onToggleExpansion: handleToggleExpansion,
onAddToRecents: handleAddToRecents,
onNewFilesSelect,
// External props
recentFiles,
@ -300,10 +604,15 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
selectedFiles,
filteredFiles,
fileInputRef,
expandedFileIds,
fileGroups,
loadedHistoryFiles,
handleSourceChange,
handleLocalFileClick,
handleFileSelect,
handleFileRemove,
handleFileRemoveById,
performFileDelete,
handleFileDoubleClick,
handleOpenFiles,
handleSearchChange,
@ -311,6 +620,9 @@ export const FileManagerProvider: React.FC<FileManagerProviderProps> = ({
handleSelectAll,
handleDeleteSelected,
handleDownloadSelected,
handleToggleExpansion,
handleAddToRecents,
onNewFilesSelect,
recentFiles,
isFileSupported,
modalHeight,

71
frontend/src/contexts/FilesModalContext.tsx
View File

@ -1,15 +1,15 @@
import React, { createContext, useContext, useState, useCallback, useMemo } from 'react';
import { useFileHandler } from '../hooks/useFileHandler';
import { FileMetadata } from '../types/file';
import { FileId } from '../types/file';
import { useFileActions } from './FileContext';
import { StirlingFileStub } from '../types/fileContext';
import { fileStorage } from '../services/fileStorage';
interface FilesModalContextType {
isFilesModalOpen: boolean;
openFilesModal: (options?: { insertAfterPage?: number; customHandler?: (files: File[], insertAfterPage?: number) => void }) => void;
closeFilesModal: () => void;
onFileSelect: (file: File) => void;
onFilesSelect: (files: File[]) => void;
onStoredFilesSelect: (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>) => void;
onFileUpload: (files: File[]) => void;
onRecentFileSelect: (stirlingFileStubs: StirlingFileStub[]) => void;
onModalClose?: () => void;
setOnModalClose: (callback: () => void) => void;
}
@ -17,7 +17,8 @@ interface FilesModalContextType {
const FilesModalContext = createContext<FilesModalContextType | null>(null);
export const FilesModalProvider: React.FC<{ children: React.ReactNode }> = ({ children }) => {
const { addToActiveFiles, addMultipleFiles, addStoredFiles } = useFileHandler();
const { addFiles } = useFileHandler();
const { actions } = useFileActions();
const [isFilesModalOpen, setIsFilesModalOpen] = useState(false);
const [onModalClose, setOnModalClose] = useState<(() => void) | undefined>();
const [insertAfterPage, setInsertAfterPage] = useState<number | undefined>();
@ -36,39 +37,45 @@ export const FilesModalProvider: React.FC<{ children: React.ReactNode }> = ({ ch
onModalClose?.();
}, [onModalClose]);
const handleFileSelect = useCallback((file: File) => {
if (customHandler) {
// Use custom handler for special cases (like page insertion)
customHandler([file], insertAfterPage);
} else {
// Use normal file handling
addToActiveFiles(file);
}
closeFilesModal();
}, [addToActiveFiles, closeFilesModal, insertAfterPage, customHandler]);
const handleFilesSelect = useCallback((files: File[]) => {
const handleFileUpload = useCallback((files: File[]) => {
if (customHandler) {
// Use custom handler for special cases (like page insertion)
customHandler(files, insertAfterPage);
} else {
// Use normal file handling
addMultipleFiles(files);
addFiles(files);
}
closeFilesModal();
}, [addMultipleFiles, closeFilesModal, insertAfterPage, customHandler]);
}, [addFiles, closeFilesModal, insertAfterPage, customHandler]);
const handleStoredFilesSelect = useCallback((filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>) => {
const handleRecentFileSelect = useCallback(async (stirlingFileStubs: StirlingFileStub[]) => {
if (customHandler) {
// Use custom handler for special cases (like page insertion)
const files = filesWithMetadata.map(item => item.file);
customHandler(files, insertAfterPage);
// Load the actual files from storage for custom handler
try {
const loadedFiles: File[] = [];
for (const stub of stirlingFileStubs) {
const stirlingFile = await fileStorage.getStirlingFile(stub.id);
if (stirlingFile) {
loadedFiles.push(stirlingFile);
}
}
if (loadedFiles.length > 0) {
customHandler(loadedFiles, insertAfterPage);
}
} catch (error) {
console.error('Failed to load files for custom handler:', error);
}
} else {
// Use normal file handling
addStoredFiles(filesWithMetadata);
// Normal case - use addStirlingFileStubs to preserve metadata
if (actions.addStirlingFileStubs) {
actions.addStirlingFileStubs(stirlingFileStubs, { selectFiles: true });
} else {
console.error('addStirlingFileStubs action not available');
}
}
closeFilesModal();
}, [addStoredFiles, closeFilesModal, insertAfterPage, customHandler]);
}, [actions.addStirlingFileStubs, closeFilesModal, customHandler, insertAfterPage]);
const setModalCloseCallback = useCallback((callback: () => void) => {
setOnModalClose(() => callback);
@ -78,18 +85,16 @@ export const FilesModalProvider: React.FC<{ children: React.ReactNode }> = ({ ch
isFilesModalOpen,
openFilesModal,
closeFilesModal,
onFileSelect: handleFileSelect,
onFilesSelect: handleFilesSelect,
onStoredFilesSelect: handleStoredFilesSelect,
onFileUpload: handleFileUpload,
onRecentFileSelect: handleRecentFileSelect,
onModalClose,
setOnModalClose: setModalCloseCallback,
}), [
isFilesModalOpen,
openFilesModal,
closeFilesModal,
handleFileSelect,
handleFilesSelect,
handleStoredFilesSelect,
handleFileUpload,
handleRecentFileSelect,
onModalClose,
setModalCloseCallback,
]);

128
frontend/src/contexts/IndexedDBContext.tsx
View File

@ -4,28 +4,30 @@
*/
import React, { createContext, useContext, useCallback, useRef } from 'react';
const DEBUG = process.env.NODE_ENV === 'development';
import { fileStorage } from '../services/fileStorage';
import { FileId } from '../types/file';
import { FileMetadata } from '../types/file';
import { StirlingFileStub, createStirlingFile, createQuickKey } from '../types/fileContext';
import { generateThumbnailForFile } from '../utils/thumbnailUtils';
const DEBUG = process.env.NODE_ENV === 'development';
interface IndexedDBContextValue {
// Core CRUD operations
saveFile: (file: File, fileId: FileId, existingThumbnail?: string) => Promise<FileMetadata>;
saveFile: (file: File, fileId: FileId, existingThumbnail?: string) => Promise<StirlingFileStub>;
loadFile: (fileId: FileId) => Promise<File | null>;
loadMetadata: (fileId: FileId) => Promise<FileMetadata | null>;
loadMetadata: (fileId: FileId) => Promise<StirlingFileStub | null>;
deleteFile: (fileId: FileId) => Promise<void>;
// Batch operations
loadAllMetadata: () => Promise<FileMetadata[]>;
loadAllMetadata: () => Promise<StirlingFileStub[]>;
loadLeafMetadata: () => Promise<StirlingFileStub[]>; // Only leaf files for recent files list
deleteMultiple: (fileIds: FileId[]) => Promise<void>;
clearAll: () => Promise<void>;
// Utilities
getStorageStats: () => Promise<{ used: number; available: number; fileCount: number }>;
updateThumbnail: (fileId: FileId, thumbnail: string) => Promise<boolean>;
markFileAsProcessed: (fileId: FileId) => Promise<boolean>;
}
const IndexedDBContext = createContext<IndexedDBContextValue | null>(null);
@ -56,26 +58,42 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
if (DEBUG) console.log(`🗂️ Evicted ${toRemove.length} LRU cache entries`);
}, []);
const saveFile = useCallback(async (file: File, fileId: FileId, existingThumbnail?: string): Promise<FileMetadata> => {
const saveFile = useCallback(async (file: File, fileId: FileId, existingThumbnail?: string): Promise<StirlingFileStub> => {
// Use existing thumbnail or generate new one if none provided
const thumbnail = existingThumbnail || await generateThumbnailForFile(file);
// Store in IndexedDB
await fileStorage.storeFile(file, fileId, thumbnail);
// Store in IndexedDB (no history data - that's handled by direct fileStorage calls now)
const stirlingFile = createStirlingFile(file, fileId);
// Create minimal stub for storage
const stub: StirlingFileStub = {
id: fileId,
name: file.name,
size: file.size,
type: file.type,
lastModified: file.lastModified,
quickKey: createQuickKey(file),
thumbnailUrl: thumbnail,
isLeaf: true,
createdAt: Date.now(),
versionNumber: 1,
originalFileId: fileId,
toolHistory: []
};
await fileStorage.storeStirlingFile(stirlingFile, stub);
const storedFile = await fileStorage.getStirlingFileStub(fileId);
// Cache the file object for immediate reuse
fileCache.current.set(fileId, { file, lastAccessed: Date.now() });
evictLRUEntries();
// Return metadata
return {
id: fileId,
name: file.name,
type: file.type,
size: file.size,
lastModified: file.lastModified,
thumbnail
};
// Return StirlingFileStub from the stored file (no conversion needed)
if (!storedFile) {
throw new Error(`Failed to retrieve stored file after saving: ${file.name}`);
}
return storedFile;
}, []);
const loadFile = useCallback(async (fileId: FileId): Promise<File | null> => {
@ -88,14 +106,11 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
}
// Load from IndexedDB
const storedFile = await fileStorage.getFile(fileId);
const storedFile = await fileStorage.getStirlingFile(fileId);
if (!storedFile) return null;
// Reconstruct File object
const file = new File([storedFile.data], storedFile.name, {
type: storedFile.type,
lastModified: storedFile.lastModified
});
// StirlingFile is already a File object, no reconstruction needed
const file = storedFile;
// Cache for future use with LRU eviction
fileCache.current.set(fileId, { file, lastAccessed: Date.now() });
@ -104,34 +119,9 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
return file;
}, [evictLRUEntries]);
const loadMetadata = useCallback(async (fileId: FileId): Promise<FileMetadata | null> => {
// Try to get from cache first (no IndexedDB hit)
const cached = fileCache.current.get(fileId);
if (cached) {
const file = cached.file;
return {
id: fileId,
name: file.name,
type: file.type,
size: file.size,
lastModified: file.lastModified
};
}
// Load metadata from IndexedDB (efficient - no data field)
const metadata = await fileStorage.getAllFileMetadata();
const fileMetadata = metadata.find(m => m.id === fileId);
if (!fileMetadata) return null;
return {
id: fileMetadata.id,
name: fileMetadata.name,
type: fileMetadata.type,
size: fileMetadata.size,
lastModified: fileMetadata.lastModified,
thumbnail: fileMetadata.thumbnail
};
const loadMetadata = useCallback(async (fileId: FileId): Promise<StirlingFileStub | null> => {
// Load stub directly from storage service
return await fileStorage.getStirlingFileStub(fileId);
}, []);
const deleteFile = useCallback(async (fileId: FileId): Promise<void> => {
@ -139,20 +129,22 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
fileCache.current.delete(fileId);
// Remove from IndexedDB
await fileStorage.deleteFile(fileId);
await fileStorage.deleteStirlingFile(fileId);
}, []);
const loadAllMetadata = useCallback(async (): Promise<FileMetadata[]> => {
const metadata = await fileStorage.getAllFileMetadata();
const loadLeafMetadata = useCallback(async (): Promise<StirlingFileStub[]> => {
const metadata = await fileStorage.getLeafStirlingFileStubs(); // Only get leaf files
return metadata.map(m => ({
id: m.id,
name: m.name,
type: m.type,
size: m.size,
lastModified: m.lastModified,
thumbnail: m.thumbnail
}));
// All files are already StirlingFileStub objects, no processing needed
return metadata;
}, []);
const loadAllMetadata = useCallback(async (): Promise<StirlingFileStub[]> => {
const metadata = await fileStorage.getAllStirlingFileStubs();
// All files are already StirlingFileStub objects, no processing needed
return metadata;
}, []);
const deleteMultiple = useCallback(async (fileIds: FileId[]): Promise<void> => {
@ -160,7 +152,7 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
fileIds.forEach(id => fileCache.current.delete(id));
// Remove from IndexedDB in parallel
await Promise.all(fileIds.map(id => fileStorage.deleteFile(id)));
await Promise.all(fileIds.map(id => fileStorage.deleteStirlingFile(id)));
}, []);
const clearAll = useCallback(async (): Promise<void> => {
@ -179,16 +171,22 @@ export function IndexedDBProvider({ children }: IndexedDBProviderProps) {
return await fileStorage.updateThumbnail(fileId, thumbnail);
}, []);
const markFileAsProcessed = useCallback(async (fileId: FileId): Promise<boolean> => {
return await fileStorage.markFileAsProcessed(fileId);
}, []);
const value: IndexedDBContextValue = {
saveFile,
loadFile,
loadMetadata,
deleteFile,
loadAllMetadata,
loadLeafMetadata,
deleteMultiple,
clearAll,
getStorageStats,
updateThumbnail
updateThumbnail,
markFileAsProcessed
};
return (

10
frontend/src/contexts/file/FileReducer.ts
View File

@ -125,16 +125,18 @@ export function fileContextReducer(state: FileContextState, action: FileContextA
return state; // File doesn't exist, no-op
}
const updatedRecord = {
...existingRecord,
...updates
};
return {
...state,
files: {
...state.files,
byId: {
...state.files.byId,
[id]: {
...existingRecord,
...updates
}
[id]: updatedRecord
}
}
};

572
frontend/src/contexts/file/fileActions.ts
View File

@ -6,15 +6,18 @@ import {
StirlingFileStub,
FileContextAction,
FileContextState,
toStirlingFileStub,
createNewStirlingFileStub,
createFileId,
createQuickKey
createQuickKey,
createStirlingFile,
ProcessedFileMetadata,
} from '../../types/fileContext';
import { FileId, FileMetadata } from '../../types/file';
import { FileId } from '../../types/file';
import { generateThumbnailWithMetadata } from '../../utils/thumbnailUtils';
import { FileLifecycleManager } from './lifecycle';
import { buildQuickKeySet } from './fileSelectors';
import { StirlingFile } from '../../types/fileContext';
import { fileStorage } from '../../services/fileStorage';
const DEBUG = process.env.NODE_ENV === 'development';
/**
@ -69,345 +72,283 @@ export function createProcessedFile(pageCount: number, thumbnail?: string) {
}
/**
* File addition types
* Generate fresh ProcessedFileMetadata for a file
* Used when tools process files to ensure metadata matches actual file content
*/
type AddFileKind = 'raw' | 'processed' | 'stored';
export async function generateProcessedFileMetadata(file: File): Promise<ProcessedFileMetadata | undefined> {
// Only generate metadata for PDF files
if (!file.type.startsWith('application/pdf')) {
return undefined;
}
try {
const result = await generateThumbnailWithMetadata(file);
return createProcessedFile(result.pageCount, result.thumbnail);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate processedFileMetadata for ${file.name}:`, error);
}
return undefined;
}
/**
* Create a child StirlingFileStub from a parent stub with proper history management.
* Used when a tool processes an existing file to create a new version with incremented history.
*
* @param parentStub - The parent StirlingFileStub to create a child from
* @param operation - Tool operation information (toolName, timestamp)
* @param resultingFile - The processed File object
* @param thumbnail - Optional thumbnail for the child
* @param processedFileMetadata - Optional fresh metadata for the processed file
* @returns New child StirlingFileStub with proper version history
*/
export function createChildStub(
parentStub: StirlingFileStub,
operation: { toolName: string; timestamp: number },
resultingFile: File,
thumbnail?: string,
processedFileMetadata?: ProcessedFileMetadata
): StirlingFileStub {
const newFileId = createFileId();
// Build new tool history by appending to parent's history
const parentToolHistory = parentStub.toolHistory || [];
const newToolHistory = [...parentToolHistory, operation];
// Calculate new version number
const newVersionNumber = (parentStub.versionNumber || 1) + 1;
// Determine original file ID (root of the version chain)
const originalFileId = parentStub.originalFileId || parentStub.id;
// Copy parent metadata but exclude processedFile to prevent stale data
const { processedFile: _processedFile, ...parentMetadata } = parentStub;
return {
// Copy parent metadata (excluding processedFile)
...parentMetadata,
// Update identity and version info
id: newFileId,
versionNumber: newVersionNumber,
parentFileId: parentStub.id,
originalFileId: originalFileId,
toolHistory: newToolHistory,
createdAt: Date.now(),
isLeaf: true, // New child is the current leaf node
name: resultingFile.name,
size: resultingFile.size,
type: resultingFile.type,
lastModified: resultingFile.lastModified,
thumbnailUrl: thumbnail,
// Set fresh processedFile metadata (no inheritance from parent)
processedFile: processedFileMetadata
};
}
interface AddFileOptions {
// For 'raw' files
files?: File[];
// For 'processed' files
filesWithThumbnails?: Array<{ file: File; thumbnail?: string; pageCount?: number }>;
// For 'stored' files
filesWithMetadata?: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>;
// Insertion position
insertAfterPageId?: string;
}
export interface AddedFile {
file: File;
id: FileId;
thumbnail?: string;
// Auto-selection after adding
selectFiles?: boolean;
}
/**
* Unified file addition helper - replaces addFiles/addProcessedFiles/addStoredFiles
* Unified file addition helper - replaces addFiles
*/
export async function addFiles(
kind: AddFileKind,
options: AddFileOptions,
stateRef: React.MutableRefObject<FileContextState>,
filesRef: React.MutableRefObject<Map<FileId, File>>,
dispatch: React.Dispatch<FileContextAction>,
lifecycleManager: FileLifecycleManager
): Promise<AddedFile[]> {
lifecycleManager: FileLifecycleManager,
enablePersistence: boolean = false
): Promise<StirlingFile[]> {
// Acquire mutex to prevent race conditions
await addFilesMutex.lock();
try {
const stirlingFileStubs: StirlingFileStub[] = [];
const addedFiles: AddedFile[] = [];
const stirlingFiles: StirlingFile[] = [];
// Build quickKey lookup from existing files for deduplication
const existingQuickKeys = buildQuickKeySet(stateRef.current.files.byId);
if (DEBUG) console.log(`📄 addFiles(${kind}): Existing quickKeys for deduplication:`, Array.from(existingQuickKeys));
switch (kind) {
case 'raw': {
const { files = [] } = options;
if (DEBUG) console.log(`📄 addFiles(raw): Adding ${files.length} files with immediate thumbnail generation`);
const { files = [] } = options;
if (DEBUG) console.log(`📄 addFiles(raw): Adding ${files.length} files with immediate thumbnail generation`);
for (const file of files) {
const quickKey = createQuickKey(file);
for (const file of files) {
const quickKey = createQuickKey(file);
// Soft deduplication: Check if file already exists by metadata
if (existingQuickKeys.has(quickKey)) {
if (DEBUG) console.log(`📄 Skipping duplicate file: ${file.name} (quickKey: ${quickKey})`);
continue;
}
if (DEBUG) console.log(`📄 Adding new file: ${file.name} (quickKey: ${quickKey})`);
const fileId = createFileId();
filesRef.current.set(fileId, file);
// Generate thumbnail and page count immediately
let thumbnail: string | undefined;
let pageCount: number = 1;
// Route based on file type - PDFs through full metadata pipeline, non-PDFs through simple path
if (file.type.startsWith('application/pdf')) {
try {
if (DEBUG) console.log(`📄 Generating PDF metadata for ${file.name}`);
const result = await generateThumbnailWithMetadata(file);
thumbnail = result.thumbnail;
pageCount = result.pageCount;
if (DEBUG) console.log(`📄 Generated PDF metadata for ${file.name}: ${pageCount} pages, thumbnail: SUCCESS`);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate PDF metadata for ${file.name}:`, error);
}
} else {
// Non-PDF files: simple thumbnail generation, no page count
try {
if (DEBUG) console.log(`📄 Generating simple thumbnail for non-PDF file ${file.name}`);
const { generateThumbnailForFile } = await import('../../utils/thumbnailUtils');
thumbnail = await generateThumbnailForFile(file);
pageCount = 0; // Non-PDFs have no page count
if (DEBUG) console.log(`📄 Generated simple thumbnail for ${file.name}: no page count, thumbnail: SUCCESS`);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate simple thumbnail for ${file.name}:`, error);
}
}
// Create record with immediate thumbnail and page metadata
const record = toStirlingFileStub(file, fileId);
if (thumbnail) {
record.thumbnailUrl = thumbnail;
// Track blob URLs for cleanup (images return blob URLs that need revocation)
if (thumbnail.startsWith('blob:')) {
lifecycleManager.trackBlobUrl(thumbnail);
}
}
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Create initial processedFile metadata with page count
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, thumbnail);
if (DEBUG) console.log(`📄 addFiles(raw): Created initial processedFile metadata for ${file.name} with ${pageCount} pages`);
}
existingQuickKeys.add(quickKey);
stirlingFileStubs.push(record);
addedFiles.push({ file, id: fileId, thumbnail });
}
break;
// Soft deduplication: Check if file already exists by metadata
if (existingQuickKeys.has(quickKey)) {
if (DEBUG) console.log(`📄 Skipping duplicate file: ${file.name} (quickKey: ${quickKey})`);
continue;
}
if (DEBUG) console.log(`📄 Adding new file: ${file.name} (quickKey: ${quickKey})`);
case 'processed': {
const { filesWithThumbnails = [] } = options;
if (DEBUG) console.log(`📄 addFiles(processed): Adding ${filesWithThumbnails.length} processed files with pre-existing thumbnails`);
const fileId = createFileId();
filesRef.current.set(fileId, file);
for (const { file, thumbnail, pageCount = 1 } of filesWithThumbnails) {
const quickKey = createQuickKey(file);
// Generate processedFile metadata using centralized function
const processedFileMetadata = await generateProcessedFileMetadata(file);
if (existingQuickKeys.has(quickKey)) {
if (DEBUG) console.log(`📄 Skipping duplicate processed file: ${file.name}`);
continue;
}
const fileId = createFileId();
filesRef.current.set(fileId, file);
const record = toStirlingFileStub(file, fileId);
if (thumbnail) {
record.thumbnailUrl = thumbnail;
// Track blob URLs for cleanup (images return blob URLs that need revocation)
if (thumbnail.startsWith('blob:')) {
lifecycleManager.trackBlobUrl(thumbnail);
}
}
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Create processedFile with provided metadata
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, thumbnail);
if (DEBUG) console.log(`📄 addFiles(processed): Created initial processedFile metadata for ${file.name} with ${pageCount} pages`);
}
existingQuickKeys.add(quickKey);
stirlingFileStubs.push(record);
addedFiles.push({ file, id: fileId, thumbnail });
// Extract thumbnail for non-PDF files or use from processedFile for PDFs
let thumbnail: string | undefined;
if (processedFileMetadata) {
// PDF file - use thumbnail from processedFile metadata
thumbnail = processedFileMetadata.thumbnailUrl;
if (DEBUG) console.log(`📄 Generated PDF metadata for ${file.name}: ${processedFileMetadata.totalPages} pages, thumbnail: SUCCESS`);
} else if (!file.type.startsWith('application/pdf')) {
// Non-PDF files: simple thumbnail generation, no processedFile metadata
try {
if (DEBUG) console.log(`📄 Generating simple thumbnail for non-PDF file ${file.name}`);
const { generateThumbnailForFile } = await import('../../utils/thumbnailUtils');
thumbnail = await generateThumbnailForFile(file);
if (DEBUG) console.log(`📄 Generated simple thumbnail for ${file.name}: no page count, thumbnail: SUCCESS`);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate simple thumbnail for ${file.name}:`, error);
}
break;
}
case 'stored': {
const { filesWithMetadata = [] } = options;
if (DEBUG) console.log(`📄 addFiles(stored): Restoring ${filesWithMetadata.length} files from storage with existing metadata`);
for (const { file, originalId, metadata } of filesWithMetadata) {
const quickKey = createQuickKey(file);
if (existingQuickKeys.has(quickKey)) {
if (DEBUG) console.log(`📄 Skipping duplicate stored file: ${file.name} (quickKey: ${quickKey})`);
continue;
}
if (DEBUG) console.log(`📄 Adding stored file: ${file.name} (quickKey: ${quickKey})`);
// Try to preserve original ID, but generate new if it conflicts
let fileId = originalId;
if (filesRef.current.has(originalId)) {
fileId = createFileId();
if (DEBUG) console.log(`📄 ID conflict for stored file ${file.name}, using new ID: ${fileId}`);
}
filesRef.current.set(fileId, file);
const record = toStirlingFileStub(file, fileId);
// Generate processedFile metadata for stored files
let pageCount: number = 1;
// Only process PDFs through PDF worker manager, non-PDFs have no page count
if (file.type.startsWith('application/pdf')) {
try {
if (DEBUG) console.log(`📄 addFiles(stored): Generating PDF metadata for stored file ${file.name}`);
// Get page count from PDF
const arrayBuffer = await file.arrayBuffer();
const { pdfWorkerManager } = await import('../../services/pdfWorkerManager');
const pdf = await pdfWorkerManager.createDocument(arrayBuffer);
pageCount = pdf.numPages;
pdfWorkerManager.destroyDocument(pdf);
if (DEBUG) console.log(`📄 addFiles(stored): Found ${pageCount} pages in PDF ${file.name}`);
} catch (error) {
if (DEBUG) console.warn(`📄 addFiles(stored): Failed to generate PDF metadata for ${file.name}:`, error);
}
} else {
pageCount = 0; // Non-PDFs have no page count
if (DEBUG) console.log(`📄 addFiles(stored): Non-PDF file ${file.name}, no page count`);
}
// Restore metadata from storage
if (metadata.thumbnail) {
record.thumbnailUrl = metadata.thumbnail;
// Track blob URLs for cleanup (images return blob URLs that need revocation)
if (metadata.thumbnail.startsWith('blob:')) {
lifecycleManager.trackBlobUrl(metadata.thumbnail);
}
}
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Create processedFile metadata with correct page count
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, metadata.thumbnail);
if (DEBUG) console.log(`📄 addFiles(stored): Created processedFile metadata for ${file.name} with ${pageCount} pages`);
}
existingQuickKeys.add(quickKey);
stirlingFileStubs.push(record);
addedFiles.push({ file, id: fileId, thumbnail: metadata.thumbnail });
// Create new filestub with processedFile metadata
const fileStub = createNewStirlingFileStub(file, fileId, thumbnail, processedFileMetadata);
if (thumbnail) {
// Track blob URLs for cleanup (images return blob URLs that need revocation)
if (thumbnail.startsWith('blob:')) {
lifecycleManager.trackBlobUrl(thumbnail);
}
break;
}
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
fileStub.insertAfterPageId = options.insertAfterPageId;
}
existingQuickKeys.add(quickKey);
stirlingFileStubs.push(fileStub);
// Create StirlingFile directly
const stirlingFile = createStirlingFile(file, fileId);
stirlingFiles.push(stirlingFile);
}
// Persist to storage if enabled using fileStorage service
if (enablePersistence && stirlingFiles.length > 0) {
await Promise.all(stirlingFiles.map(async (stirlingFile, index) => {
try {
// Get corresponding stub with all metadata
const fileStub = stirlingFileStubs[index];
// Store using the cleaner signature - pass StirlingFile + StirlingFileStub directly
await fileStorage.storeStirlingFile(stirlingFile, fileStub);
if (DEBUG) console.log(`📄 addFiles: Stored file ${stirlingFile.name} with metadata:`, fileStub);
} catch (error) {
console.error('Failed to persist file to storage:', stirlingFile.name, error);
}
}));
}
// Dispatch ADD_FILES action if we have new files
if (stirlingFileStubs.length > 0) {
dispatch({ type: 'ADD_FILES', payload: { stirlingFileStubs } });
if (DEBUG) console.log(`📄 addFiles(${kind}): Successfully added ${stirlingFileStubs.length} files`);
}
return addedFiles;
return stirlingFiles;
} finally {
// Always release mutex even if error occurs
addFilesMutex.unlock();
}
}
/**
* Helper function to process files into records with thumbnails and metadata
*/
async function processFilesIntoRecords(
files: File[],
filesRef: React.MutableRefObject<Map<FileId, File>>
): Promise<Array<{ record: StirlingFileStub; file: File; fileId: FileId; thumbnail?: string }>> {
return Promise.all(
files.map(async (file) => {
const fileId = createFileId();
filesRef.current.set(fileId, file);
// Generate thumbnail and page count
let thumbnail: string | undefined;
let pageCount: number = 1;
try {
if (DEBUG) console.log(`📄 Generating thumbnail for file ${file.name}`);
const result = await generateThumbnailWithMetadata(file);
thumbnail = result.thumbnail;
pageCount = result.pageCount;
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to generate thumbnail for file ${file.name}:`, error);
}
const record = toStirlingFileStub(file, fileId);
if (thumbnail) {
record.thumbnailUrl = thumbnail;
}
if (pageCount > 0) {
record.processedFile = createProcessedFile(pageCount, thumbnail);
}
return { record, file, fileId, thumbnail };
})
);
}
/**
* Helper function to persist files to IndexedDB
*/
async function persistFilesToIndexedDB(
stirlingFileStubs: Array<{ file: File; fileId: FileId; thumbnail?: string }>,
indexedDB: { saveFile: (file: File, fileId: FileId, existingThumbnail?: string) => Promise<any> }
): Promise<void> {
await Promise.all(stirlingFileStubs.map(async ({ file, fileId, thumbnail }) => {
try {
await indexedDB.saveFile(file, fileId, thumbnail);
} catch (error) {
console.error('Failed to persist file to IndexedDB:', file.name, error);
}
}));
}
/**
* Consume files helper - replace unpinned input files with output files
* Now accepts pre-created StirlingFiles and StirlingFileStubs to preserve all metadata
*/
export async function consumeFiles(
inputFileIds: FileId[],
outputFiles: File[],
outputStirlingFiles: StirlingFile[],
outputStirlingFileStubs: StirlingFileStub[],
filesRef: React.MutableRefObject<Map<FileId, File>>,
dispatch: React.Dispatch<FileContextAction>,
indexedDB?: { saveFile: (file: File, fileId: FileId, existingThumbnail?: string) => Promise<any> } | null
dispatch: React.Dispatch<FileContextAction>
): Promise<FileId[]> {
if (DEBUG) console.log(`📄 consumeFiles: Processing ${inputFileIds.length} input files, ${outputFiles.length} output files`);
if (DEBUG) console.log(`📄 consumeFiles: Processing ${inputFileIds.length} input files, ${outputStirlingFiles.length} output files with pre-created stubs`);
// Process output files with thumbnails and metadata
const outputStirlingFileStubs = await processFilesIntoRecords(outputFiles, filesRef);
// Persist output files to IndexedDB if available
if (indexedDB) {
await persistFilesToIndexedDB(outputStirlingFileStubs, indexedDB);
// Validate that we have matching files and stubs
if (outputStirlingFiles.length !== outputStirlingFileStubs.length) {
throw new Error(`Mismatch between output files (${outputStirlingFiles.length}) and stubs (${outputStirlingFileStubs.length})`);
}
// Dispatch the consume action
// Store StirlingFiles in filesRef using their existing IDs (no ID generation needed)
for (let i = 0; i < outputStirlingFiles.length; i++) {
const stirlingFile = outputStirlingFiles[i];
const stub = outputStirlingFileStubs[i];
if (stirlingFile.fileId !== stub.id) {
console.warn(`📄 consumeFiles: ID mismatch between StirlingFile (${stirlingFile.fileId}) and stub (${stub.id})`);
}
filesRef.current.set(stirlingFile.fileId, stirlingFile);
if (DEBUG) console.log(`📄 consumeFiles: Stored StirlingFile ${stirlingFile.name} with ID ${stirlingFile.fileId}`);
}
// Mark input files as processed in storage (no longer leaf nodes)
if(!outputStirlingFileStubs.reduce((areAllV1, stub) => areAllV1 && (stub.versionNumber == 1), true)) {
await Promise.all(
inputFileIds.map(async (fileId) => {
try {
await fileStorage.markFileAsProcessed(fileId);
if (DEBUG) console.log(`📄 Marked file ${fileId} as processed (no longer leaf)`);
} catch (error) {
if (DEBUG) console.warn(`📄 Failed to mark file ${fileId} as processed:`, error);
}
})
);
}
// Save output files directly to fileStorage with complete metadata
for (let i = 0; i < outputStirlingFiles.length; i++) {
const stirlingFile = outputStirlingFiles[i];
const stub = outputStirlingFileStubs[i];
try {
// Use fileStorage directly with complete metadata from stub
await fileStorage.storeStirlingFile(stirlingFile, stub);
if (DEBUG) console.log(`📄 Saved StirlingFile ${stirlingFile.name} directly to storage with complete metadata:`, {
fileId: stirlingFile.fileId,
versionNumber: stub.versionNumber,
originalFileId: stub.originalFileId,
parentFileId: stub.parentFileId,
toolChainLength: stub.toolHistory?.length || 0
});
} catch (error) {
console.error('Failed to persist output file to fileStorage:', stirlingFile.name, error);
}
}
// Dispatch the consume action with pre-created stubs (no processing needed)
dispatch({
type: 'CONSUME_FILES',
payload: {
inputFileIds,
outputStirlingFileStubs: outputStirlingFileStubs.map(({ record }) => record)
outputStirlingFileStubs: outputStirlingFileStubs
}
});
if (DEBUG) console.log(`📄 consumeFiles: Successfully consumed files - removed ${inputFileIds.length} inputs, added ${outputStirlingFileStubs.length} outputs`);
// Return the output file IDs for undo tracking
return outputStirlingFileStubs.map(({ fileId }) => fileId);
return outputStirlingFileStubs.map(stub => stub.id);
}
/**
@ -518,6 +459,97 @@ export async function undoConsumeFiles(
/**
* Action factory functions
*/
/**
* Add files using existing StirlingFileStubs from storage - preserves all metadata
* Use this when loading files that already exist in storage (FileManager, etc.)
* StirlingFileStubs come with proper thumbnails, history, processing state
*/
export async function addStirlingFileStubs(
stirlingFileStubs: StirlingFileStub[],
options: { insertAfterPageId?: string; selectFiles?: boolean } = {},
stateRef: React.MutableRefObject<FileContextState>,
filesRef: React.MutableRefObject<Map<FileId, File>>,
dispatch: React.Dispatch<FileContextAction>,
_lifecycleManager: FileLifecycleManager
): Promise<StirlingFile[]> {
await addFilesMutex.lock();
try {
if (DEBUG) console.log(`📄 addStirlingFileStubs: Adding ${stirlingFileStubs.length} StirlingFileStubs preserving metadata`);
const existingQuickKeys = buildQuickKeySet(stateRef.current.files.byId);
const validStubs: StirlingFileStub[] = [];
const loadedFiles: StirlingFile[] = [];
for (const stub of stirlingFileStubs) {
// Check for duplicates using quickKey
if (existingQuickKeys.has(stub.quickKey || '')) {
if (DEBUG) console.log(`📄 Skipping duplicate StirlingFileStub: ${stub.name}`);
continue;
}
// Load the actual StirlingFile from storage
const stirlingFile = await fileStorage.getStirlingFile(stub.id);
if (!stirlingFile) {
console.warn(`📄 Failed to load StirlingFile for stub: ${stub.name} (${stub.id})`);
continue;
}
// Store the loaded file in filesRef
filesRef.current.set(stub.id, stirlingFile);
// Use the original stub (preserves thumbnails, history, metadata!)
const record = { ...stub };
// Store insertion position if provided
if (options.insertAfterPageId !== undefined) {
record.insertAfterPageId = options.insertAfterPageId;
}
// Check if processedFile data needs regeneration for proper Page Editor support
if (stirlingFile.type.startsWith('application/pdf')) {
const needsProcessing = !record.processedFile ||
!record.processedFile.pages ||
record.processedFile.pages.length === 0 ||
record.processedFile.totalPages !== record.processedFile.pages.length;
if (needsProcessing) {
if (DEBUG) console.log(`📄 addStirlingFileStubs: Regenerating processedFile for ${record.name}`);
// Use centralized metadata generation function
const processedFileMetadata = await generateProcessedFileMetadata(stirlingFile);
if (processedFileMetadata) {
record.processedFile = processedFileMetadata;
record.thumbnailUrl = processedFileMetadata.thumbnailUrl; // Update thumbnail if needed
if (DEBUG) console.log(`📄 addStirlingFileStubs: Regenerated processedFile for ${record.name} with ${processedFileMetadata.totalPages} pages`);
} else {
// Fallback for files that couldn't be processed
if (DEBUG) console.warn(`📄 addStirlingFileStubs: Failed to regenerate processedFile for ${record.name}`);
if (!record.processedFile) {
record.processedFile = createProcessedFile(1); // Fallback to 1 page
}
}
}
}
existingQuickKeys.add(stub.quickKey || '');
validStubs.push(record);
loadedFiles.push(stirlingFile);
}
// Dispatch ADD_FILES action if we have new files
if (validStubs.length > 0) {
dispatch({ type: 'ADD_FILES', payload: { stirlingFileStubs: validStubs } });
if (DEBUG) console.log(`📄 addStirlingFileStubs: Successfully added ${validStubs.length} files with preserved metadata`);
}
return loadedFiles;
} finally {
addFilesMutex.unlock();
}
}
export const createFileActions = (dispatch: React.Dispatch<FileContextAction>) => ({
setSelectedFiles: (fileIds: FileId[]) => dispatch({ type: 'SET_SELECTED_FILES', payload: { fileIds } }),
setSelectedPages: (pageNumbers: number[]) => dispatch({ type: 'SET_SELECTED_PAGES', payload: { pageNumbers } }),

1
frontend/src/hooks/tools/addPassword/useAddPasswordOperation.test.ts
View File

@ -119,7 +119,6 @@ describe('useAddPasswordOperation', () => {
test.each([
{ property: 'toolType' as const, expectedValue: ToolType.singleFile },
{ property: 'endpoint' as const, expectedValue: '/api/v1/security/add-password' },
{ property: 'filePrefix' as const, expectedValue: 'translated-addPassword.filenamePrefix_' },
{ property: 'operationType' as const, expectedValue: 'addPassword' }
])('should configure $property correctly', ({ property, expectedValue }) => {
renderHook(() => useAddPasswordOperation());

2
frontend/src/hooks/tools/addPassword/useAddPasswordOperation.ts
View File

@ -30,7 +30,6 @@ export const addPasswordOperationConfig = {
buildFormData: buildAddPasswordFormData,
operationType: 'addPassword',
endpoint: '/api/v1/security/add-password',
filePrefix: 'encrypted_', // Will be overridden in hook with translation
defaultParameters: fullDefaultParameters,
} as const;
@ -39,7 +38,6 @@ export const useAddPasswordOperation = () => {
return useToolOperation<AddPasswordFullParameters>({
...addPasswordOperationConfig,
filePrefix: t('addPassword.filenamePrefix', 'encrypted') + '_',
getErrorMessage: createStandardErrorHandler(t('addPassword.error.failed', 'An error occurred while encrypting the PDF.'))
});
};

2
frontend/src/hooks/tools/addWatermark/useAddWatermarkOperation.ts
View File

@ -39,7 +39,6 @@ export const addWatermarkOperationConfig = {
buildFormData: buildAddWatermarkFormData,
operationType: 'watermark',
endpoint: '/api/v1/security/add-watermark',
filePrefix: 'watermarked_', // Will be overridden in hook with translation
defaultParameters,
} as const;
@ -48,7 +47,6 @@ export const useAddWatermarkOperation = () => {
return useToolOperation<AddWatermarkParameters>({
...addWatermarkOperationConfig,
filePrefix: t('watermark.filenamePrefix', 'watermarked') + '_',
getErrorMessage: createStandardErrorHandler(t('watermark.error.failed', 'An error occurred while adding watermark to the PDF.'))
});
};

1
frontend/src/hooks/tools/adjustPageScale/useAdjustPageScaleOperation.ts
View File

@ -16,7 +16,6 @@ export const adjustPageScaleOperationConfig = {
buildFormData: buildAdjustPageScaleFormData,
operationType: 'adjustPageScale',
endpoint: '/api/v1/general/scale-pages',
filePrefix: 'scaled_',
defaultParameters,
} as const;

1
frontend/src/hooks/tools/autoRename/useAutoRenameOperation.ts
View File

@ -28,7 +28,6 @@ export const autoRenameOperationConfig = {
buildFormData: buildAutoRenameFormData,
operationType: 'autoRename',
endpoint: '/api/v1/misc/auto-rename',
filePrefix: 'autoRename_',
preserveBackendFilename: true, // Use filename from backend response headers
defaultParameters,
} as const;

1
frontend/src/hooks/tools/automate/useAutomateOperation.ts
View File

@ -42,6 +42,5 @@ export function useAutomateOperation() {
toolType: ToolType.custom,
operationType: 'automate',
customProcessor,
filePrefix: '' // No prefix needed since automation handles naming internally
});
}

1
frontend/src/hooks/tools/changePermissions/useChangePermissionsOperation.test.ts
View File

@ -113,7 +113,6 @@ describe('useChangePermissionsOperation', () => {
test.each([
{ property: 'toolType' as const, expectedValue: ToolType.singleFile },
{ property: 'endpoint' as const, expectedValue: '/api/v1/security/add-password' },
{ property: 'filePrefix' as const, expectedValue: 'permissions_' },
{ property: 'operationType' as const, expectedValue: 'change-permissions' }
])('should configure $property correctly', ({ property, expectedValue }) => {
renderHook(() => useChangePermissionsOperation());

1
frontend/src/hooks/tools/changePermissions/useChangePermissionsOperation.ts
View File

@ -28,7 +28,6 @@ export const changePermissionsOperationConfig = {
buildFormData: buildChangePermissionsFormData,
operationType: 'change-permissions',
endpoint: '/api/v1/security/add-password', // Change Permissions is a fake endpoint for the Add Password tool
filePrefix: 'permissions_',
defaultParameters,
} as const;

1
frontend/src/hooks/tools/compress/useCompressOperation.ts
View File

@ -28,7 +28,6 @@ export const compressOperationConfig = {
buildFormData: buildCompressFormData,
operationType: 'compress',
endpoint: '/api/v1/misc/compress-pdf',
filePrefix: 'compressed_',
defaultParameters,
} as const;

3
frontend/src/hooks/tools/convert/useConvertOperation.ts
View File

@ -83,7 +83,7 @@ export const createFileFromResponse = (
targetExtension = 'pdf';
}
const fallbackFilename = `${originalName}_converted.${targetExtension}`;
const fallbackFilename = `${originalName}.${targetExtension}`;
return createFileFromApiResponse(responseData, headers, fallbackFilename);
};
@ -136,7 +136,6 @@ export const convertOperationConfig = {
toolType: ToolType.custom,
customProcessor: convertProcessor, // Can't use callback version here
operationType: 'convert',
filePrefix: 'converted_',
defaultParameters,
} as const;

4
frontend/src/hooks/tools/flatten/useFlattenOperation.ts
View File

@ -17,7 +17,6 @@ export const flattenOperationConfig = {
buildFormData: buildFlattenFormData,
operationType: 'flatten',
endpoint: '/api/v1/misc/flatten',
filePrefix: 'flattened_', // Will be overridden in hook with translation
multiFileEndpoint: false,
defaultParameters,
} as const;
@ -27,7 +26,6 @@ export const useFlattenOperation = () => {
return useToolOperation<FlattenParameters>({
...flattenOperationConfig,
filePrefix: t('flatten.filenamePrefix', 'flattened') + '_',
getErrorMessage: createStandardErrorHandler(t('flatten.error.failed', 'An error occurred while flattening the PDF.'))
});
};
};

5
frontend/src/hooks/tools/ocr/useOCROperation.ts
View File

@ -88,8 +88,8 @@ export const ocrResponseHandler = async (blob: Blob, originalFiles: File[], extr
throw new Error(`Response is not a valid PDF. Header: "${head}"`);
}
const base = stripExt(originalFiles[0].name);
return [new File([blob], `ocr_${base}.pdf`, { type: 'application/pdf' })];
const originalName = originalFiles[0].name;
return [new File([blob], originalName, { type: 'application/pdf' })];
};
// Static configuration object (without t function dependencies)
@ -98,7 +98,6 @@ export const ocrOperationConfig = {
buildFormData: buildOCRFormData,
operationType: 'ocr',
endpoint: '/api/v1/misc/ocr-pdf',
filePrefix: 'ocr_',
defaultParameters,
} as const;

1
frontend/src/hooks/tools/redact/useRedactOperation.ts
View File

@ -37,7 +37,6 @@ export const redactOperationConfig = {
throw new Error('Manual redaction not yet implemented');
}
},
filePrefix: 'redacted_',
defaultParameters,
} as const;

2
frontend/src/hooks/tools/removeCertificateSign/useRemoveCertificateSignOperation.ts
View File

@ -16,7 +16,6 @@ export const removeCertificateSignOperationConfig = {
buildFormData: buildRemoveCertificateSignFormData,
operationType: 'remove-certificate-sign',
endpoint: '/api/v1/security/remove-cert-sign',
filePrefix: 'unsigned_', // Will be overridden in hook with translation
defaultParameters,
} as const;
@ -25,7 +24,6 @@ export const useRemoveCertificateSignOperation = () => {
return useToolOperation<RemoveCertificateSignParameters>({
...removeCertificateSignOperationConfig,
filePrefix: t('removeCertSign.filenamePrefix', 'unsigned') + '_',
getErrorMessage: createStandardErrorHandler(t('removeCertSign.error.failed', 'An error occurred while removing certificate signatures.'))
});
};

1
frontend/src/hooks/tools/removePassword/useRemovePasswordOperation.test.ts
View File

@ -97,7 +97,6 @@ describe('useRemovePasswordOperation', () => {
test.each([
{ property: 'toolType' as const, expectedValue: ToolType.singleFile },
{ property: 'endpoint' as const, expectedValue: '/api/v1/security/remove-password' },
{ property: 'filePrefix' as const, expectedValue: 'translated-removePassword.filenamePrefix_' },
{ property: 'operationType' as const, expectedValue: 'removePassword' }
])('should configure $property correctly', ({ property, expectedValue }) => {
renderHook(() => useRemovePasswordOperation());

2
frontend/src/hooks/tools/removePassword/useRemovePasswordOperation.ts
View File

@ -17,7 +17,6 @@ export const removePasswordOperationConfig = {
buildFormData: buildRemovePasswordFormData,
operationType: 'removePassword',
endpoint: '/api/v1/security/remove-password',
filePrefix: 'decrypted_', // Will be overridden in hook with translation
defaultParameters,
} as const;
@ -26,7 +25,6 @@ export const useRemovePasswordOperation = () => {
return useToolOperation<RemovePasswordParameters>({
...removePasswordOperationConfig,
filePrefix: t('removePassword.filenamePrefix', 'decrypted') + '_',
getErrorMessage: createStandardErrorHandler(t('removePassword.error.failed', 'An error occurred while removing the password from the PDF.'))
});
};

2
frontend/src/hooks/tools/repair/useRepairOperation.ts
View File

@ -16,7 +16,6 @@ export const repairOperationConfig = {
buildFormData: buildRepairFormData,
operationType: 'repair',
endpoint: '/api/v1/misc/repair',
filePrefix: 'repaired_', // Will be overridden in hook with translation
defaultParameters,
} as const;
@ -25,7 +24,6 @@ export const useRepairOperation = () => {
return useToolOperation<RepairParameters>({
...repairOperationConfig,
filePrefix: t('repair.filenamePrefix', 'repaired') + '_',
getErrorMessage: createStandardErrorHandler(t('repair.error.failed', 'An error occurred while repairing the PDF.'))
});
};

2
frontend/src/hooks/tools/sanitize/useSanitizeOperation.ts
View File

@ -25,7 +25,6 @@ export const sanitizeOperationConfig = {
buildFormData: buildSanitizeFormData,
operationType: 'sanitize',
endpoint: '/api/v1/security/sanitize-pdf',
filePrefix: 'sanitized_', // Will be overridden in hook with translation
multiFileEndpoint: false,
defaultParameters,
} as const;
@ -35,7 +34,6 @@ export const useSanitizeOperation = () => {
return useToolOperation<SanitizeParameters>({
...sanitizeOperationConfig,
filePrefix: t('sanitize.filenamePrefix', 'sanitized') + '_',
getErrorMessage: createStandardErrorHandler(t('sanitize.error.failed', 'An error occurred while sanitising the PDF.'))
});
};

2
frontend/src/hooks/tools/shared/useToolApiCalls.ts
View File

@ -6,7 +6,7 @@ import type { ProcessingProgress } from './useToolState';
export interface ApiCallsConfig<TParams = void> {
endpoint: string | ((params: TParams) => string);
buildFormData: (params: TParams, file: File) => FormData;
filePrefix: string;
filePrefix?: string;
responseHandler?: ResponseHandler;
preserveBackendFilename?: boolean;
}

60
frontend/src/hooks/tools/shared/useToolOperation.ts
View File

@ -6,8 +6,9 @@ import { useToolState, type ProcessingProgress } from './useToolState';
import { useToolApiCalls, type ApiCallsConfig } from './useToolApiCalls';
import { useToolResources } from './useToolResources';
import { extractErrorMessage } from '../../../utils/toolErrorHandler';
import { StirlingFile, extractFiles, FileId, StirlingFileStub } from '../../../types/fileContext';
import { StirlingFile, extractFiles, FileId, StirlingFileStub, createStirlingFile, createNewStirlingFileStub } from '../../../types/fileContext';
import { ResponseHandler } from '../../../utils/toolResponseProcessor';
import { createChildStub, generateProcessedFileMetadata } from '../../../contexts/file/fileActions';
// Re-export for backwards compatibility
export type { ProcessingProgress, ResponseHandler };
@ -31,7 +32,7 @@ interface BaseToolOperationConfig<TParams> {
operationType: string;
/** Prefix added to processed filenames (e.g., 'compressed_', 'split_') */
filePrefix: string;
filePrefix?: string;
/**
* Whether to preserve the filename provided by the backend in response headers.
@ -165,18 +166,20 @@ export const useToolOperation = <TParams>(
return;
}
// Reset state
actions.setLoading(true);
actions.setError(null);
actions.resetResults();
cleanupBlobUrls();
// Prepare files with history metadata injection (for PDFs)
actions.setStatus('Processing files...');
try {
let processedFiles: File[];
// Convert StirlingFile to regular File objects for API processing
const validRegularFiles = extractFiles(validFiles);
// Use original files directly (no PDF metadata injection - history stored in IndexedDB)
const filesForAPI = extractFiles(validFiles);
switch (config.toolType) {
case ToolType.singleFile: {
@ -190,18 +193,17 @@ export const useToolOperation = <TParams>(
};
processedFiles = await processFiles(
params,
validRegularFiles,
filesForAPI,
apiCallsConfig,
actions.setProgress,
actions.setStatus
);
break;
}
case ToolType.multiFile: {
// Multi-file processing - single API call with all files
actions.setStatus('Processing files...');
const formData = config.buildFormData(params, validRegularFiles);
const formData = config.buildFormData(params, filesForAPI);
const endpoint = typeof config.endpoint === 'function' ? config.endpoint(params) : config.endpoint;
const response = await axios.post(endpoint, formData, { responseType: 'blob' });
@ -209,11 +211,11 @@ export const useToolOperation = <TParams>(
// Multi-file responses are typically ZIP files that need extraction, but some may return single PDFs
if (config.responseHandler) {
// Use custom responseHandler for multi-file (handles ZIP extraction)
processedFiles = await config.responseHandler(response.data, validRegularFiles);
processedFiles = await config.responseHandler(response.data, filesForAPI);
} else if (response.data.type === 'application/pdf' ||
(response.headers && response.headers['content-type'] === 'application/pdf')) {
// Single PDF response (e.g. split with merge option) - use original filename
const originalFileName = validRegularFiles[0]?.name || 'document.pdf';
const originalFileName = filesForAPI[0]?.name || 'document.pdf';
const singleFile = new File([response.data], originalFileName, { type: 'application/pdf' });
processedFiles = [singleFile];
} else {
@ -230,13 +232,14 @@ export const useToolOperation = <TParams>(
case ToolType.custom:
actions.setStatus('Processing files...');
processedFiles = await config.customProcessor(params, validRegularFiles);
processedFiles = await config.customProcessor(params, filesForAPI);
break;
}
if (processedFiles.length > 0) {
actions.setFiles(processedFiles);
// Generate thumbnails and download URL concurrently
actions.setGeneratingThumbnails(true);
const [thumbnails, downloadInfo] = await Promise.all([
@ -264,7 +267,40 @@ export const useToolOperation = <TParams>(
}
}
const outputFileIds = await consumeFiles(inputFileIds, processedFiles);
// Create new tool operation
const newToolOperation = {
toolName: config.operationType,
timestamp: Date.now()
};
// Generate fresh processedFileMetadata for all processed files to ensure accuracy
actions.setStatus('Generating metadata for processed files...');
const processedFileMetadataArray = await Promise.all(
processedFiles.map(file => generateProcessedFileMetadata(file))
);
const shouldBranchHistory = processedFiles.length != inputStirlingFileStubs.length;
// Create output stubs with fresh metadata (no inheritance of stale processedFile data)
const outputStirlingFileStubs = shouldBranchHistory
? processedFiles.map((file, index) =>
createNewStirlingFileStub(file, undefined, thumbnails[index], processedFileMetadataArray[index])
)
: processedFiles.map((resultingFile, index) =>
createChildStub(
inputStirlingFileStubs[index],
newToolOperation,
resultingFile,
thumbnails[index],
processedFileMetadataArray[index]
)
);
// Create StirlingFile objects from processed files and child stubs
const outputStirlingFiles = processedFiles.map((file, index) => {
const childStub = outputStirlingFileStubs[index];
return createStirlingFile(file, childStub.id);
});
const outputFileIds = await consumeFiles(inputFileIds, outputStirlingFiles, outputStirlingFileStubs);
// Store operation data for undo (only store what we need to avoid memory bloat)
lastOperationRef.current = {

2
frontend/src/hooks/tools/singleLargePage/useSingleLargePageOperation.ts
View File

@ -16,7 +16,6 @@ export const singleLargePageOperationConfig = {
buildFormData: buildSingleLargePageFormData,
operationType: 'single-large-page',
endpoint: '/api/v1/general/pdf-to-single-page',
filePrefix: 'single_page_', // Will be overridden in hook with translation
defaultParameters,
} as const;
@ -25,7 +24,6 @@ export const useSingleLargePageOperation = () => {
return useToolOperation<SingleLargePageParameters>({
...singleLargePageOperationConfig,
filePrefix: t('pdfToSinglePage.filenamePrefix', 'single_page') + '_',
getErrorMessage: createStandardErrorHandler(t('pdfToSinglePage.error.failed', 'An error occurred while converting to single page.'))
});
};

1
frontend/src/hooks/tools/split/useSplitOperation.ts
View File

@ -73,7 +73,6 @@ export const splitOperationConfig = {
buildFormData: buildSplitFormData,
operationType: 'splitPdf',
endpoint: getSplitEndpoint,
filePrefix: 'split_',
defaultParameters,
} as const;

2
frontend/src/hooks/tools/unlockPdfForms/useUnlockPdfFormsOperation.ts
View File

@ -16,7 +16,6 @@ export const unlockPdfFormsOperationConfig = {
buildFormData: buildUnlockPdfFormsFormData,
operationType: 'unlock-pdf-forms',
endpoint: '/api/v1/misc/unlock-pdf-forms',
filePrefix: 'unlocked_forms_', // Will be overridden in hook with translation
defaultParameters,
} as const;
@ -25,7 +24,6 @@ export const useUnlockPdfFormsOperation = () => {
return useToolOperation<UnlockPdfFormsParameters>({
...unlockPdfFormsOperationConfig,
filePrefix: t('unlockPDFForms.filenamePrefix', 'unlocked_forms') + '_',
getErrorMessage: createStandardErrorHandler(t('unlockPDFForms.error.failed', 'An error occurred while unlocking PDF forms.'))
});
};

34
frontend/src/hooks/useFileHandler.ts
View File

@ -1,39 +1,17 @@
import { useCallback } from 'react';
import { useFileState, useFileActions } from '../contexts/FileContext';
import { FileMetadata } from '../types/file';
import { FileId } from '../types/file';
import { useFileActions } from '../contexts/FileContext';
export const useFileHandler = () => {
const { state } = useFileState(); // Still needed for addStoredFiles
const { actions } = useFileActions();
const addToActiveFiles = useCallback(async (file: File) => {
const addFiles = useCallback(async (files: File[], options: { insertAfterPageId?: string; selectFiles?: boolean } = {}) => {
// Merge default options with passed options - passed options take precedence
const mergedOptions = { selectFiles: true, ...options };
// Let FileContext handle deduplication with quickKey logic
await actions.addFiles([file], { selectFiles: true });
await actions.addFiles(files, mergedOptions);
}, [actions.addFiles]);
const addMultipleFiles = useCallback(async (files: File[]) => {
// Let FileContext handle deduplication with quickKey logic
await actions.addFiles(files, { selectFiles: true });
}, [actions.addFiles]);
// Add stored files preserving their original IDs to prevent session duplicates
const addStoredFiles = useCallback(async (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>) => {
// Filter out files that already exist with the same ID (exact match)
const newFiles = filesWithMetadata.filter(({ originalId }) => {
return state.files.byId[originalId] === undefined;
});
if (newFiles.length > 0) {
await actions.addStoredFiles(newFiles, { selectFiles: true });
}
console.log(`📁 Added ${newFiles.length} stored files (${filesWithMetadata.length - newFiles.length} skipped as duplicates)`);
}, [state.files.byId, actions.addStoredFiles]);
return {
addToActiveFiles,
addMultipleFiles,
addStoredFiles,
addFiles,
};
};

81
frontend/src/hooks/useFileManager.ts
View File

@ -1,72 +1,40 @@
import { useState, useCallback } from 'react';
import { useIndexedDB } from '../contexts/IndexedDBContext';
import { FileMetadata } from '../types/file';
import { fileStorage } from '../services/fileStorage';
import { StirlingFileStub, StirlingFile } from '../types/fileContext';
import { FileId } from '../types/fileContext';
export const useFileManager = () => {
const [loading, setLoading] = useState(false);
const indexedDB = useIndexedDB();
const convertToFile = useCallback(async (fileMetadata: FileMetadata): Promise<File> => {
const convertToFile = useCallback(async (fileStub: StirlingFileStub): Promise<File> => {
if (!indexedDB) {
throw new Error('IndexedDB context not available');
}
// Handle drafts differently from regular files
if (fileMetadata.isDraft) {
// Load draft from the drafts database
try {
const { indexedDBManager, DATABASE_CONFIGS } = await import('../services/indexedDBManager');
const db = await indexedDBManager.openDatabase(DATABASE_CONFIGS.DRAFTS);
return new Promise((resolve, reject) => {
const transaction = db.transaction(['drafts'], 'readonly');
const store = transaction.objectStore('drafts');
const request = store.get(fileMetadata.id);
request.onsuccess = () => {
const draft = request.result;
if (draft && draft.pdfData) {
const file = new File([draft.pdfData], fileMetadata.name, {
type: 'application/pdf',
lastModified: fileMetadata.lastModified
});
resolve(file);
} else {
reject(new Error('Draft data not found'));
}
};
request.onerror = () => reject(request.error);
});
} catch (error) {
throw new Error(`Failed to load draft: ${fileMetadata.name} (${error})`);
}
}
// Regular file loading
if (fileMetadata.id) {
const file = await indexedDB.loadFile(fileMetadata.id);
if (fileStub.id) {
const file = await indexedDB.loadFile(fileStub.id);
if (file) {
return file;
}
}
throw new Error(`File not found in storage: ${fileMetadata.name} (ID: ${fileMetadata.id})`);
throw new Error(`File not found in storage: ${fileStub.name} (ID: ${fileStub.id})`);
}, [indexedDB]);
const loadRecentFiles = useCallback(async (): Promise<FileMetadata[]> => {
const loadRecentFiles = useCallback(async (): Promise<StirlingFileStub[]> => {
setLoading(true);
try {
if (!indexedDB) {
return [];
}
// Load regular files metadata only
const storedFileMetadata = await indexedDB.loadAllMetadata();
// Load only leaf files metadata (processed files that haven't been used as input for other tools)
const stirlingFileStubs = await fileStorage.getLeafStirlingFileStubs();
// For now, only regular files - drafts will be handled separately in the future
const allFiles = storedFileMetadata;
const sortedFiles = allFiles.sort((a, b) => (b.lastModified || 0) - (a.lastModified || 0));
const sortedFiles = stirlingFileStubs.sort((a, b) => (b.lastModified || 0) - (a.lastModified || 0));
return sortedFiles;
} catch (error) {
@ -77,7 +45,7 @@ export const useFileManager = () => {
}
}, [indexedDB]);
const handleRemoveFile = useCallback(async (index: number, files: FileMetadata[], setFiles: (files: FileMetadata[]) => void) => {
const handleRemoveFile = useCallback(async (index: number, files: StirlingFileStub[], setFiles: (files: StirlingFileStub[]) => void) => {
const file = files[index];
if (!file.id) {
throw new Error('File ID is required for removal');
@ -102,10 +70,10 @@ export const useFileManager = () => {
// Store file with provided UUID from FileContext (thumbnail generated internally)
const metadata = await indexedDB.saveFile(file, fileId);
// Convert file to ArrayBuffer for StoredFile interface compatibility
// Convert file to ArrayBuffer for storage compatibility
const arrayBuffer = await file.arrayBuffer();
// Return StoredFile format for compatibility with old API
// This method is deprecated - use FileStorage directly instead
return {
id: fileId,
name: file.name,
@ -113,7 +81,7 @@ export const useFileManager = () => {
size: file.size,
lastModified: file.lastModified,
data: arrayBuffer,
thumbnail: metadata.thumbnail
thumbnail: metadata.thumbnailUrl
};
} catch (error) {
console.error('Failed to store file:', error);
@ -137,23 +105,24 @@ export const useFileManager = () => {
setSelectedFiles([]);
};
const selectMultipleFiles = async (files: FileMetadata[], onStoredFilesSelect: (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>) => void) => {
const selectMultipleFiles = async (files: StirlingFileStub[], onStirlingFilesSelect: (stirlingFiles: StirlingFile[]) => void) => {
if (selectedFiles.length === 0) return;
try {
// Filter by UUID and convert to File objects
// Filter by UUID and load full StirlingFile objects directly
const selectedFileObjects = files.filter(f => selectedFiles.includes(f.id));
// Use stored files flow that preserves IDs
const filesWithMetadata = await Promise.all(
selectedFileObjects.map(async (metadata) => ({
file: await convertToFile(metadata),
originalId: metadata.id,
metadata
}))
const stirlingFiles = await Promise.all(
selectedFileObjects.map(async (stub) => {
const stirlingFile = await fileStorage.getStirlingFile(stub.id);
if (!stirlingFile) {
throw new Error(`File not found in storage: ${stub.name}`);
}
return stirlingFile;
})
);
onStoredFilesSelect(filesWithMetadata);
onStirlingFilesSelect(stirlingFiles);
clearSelection();
} catch (error) {
console.error('Failed to load selected files:', error);

10
frontend/src/hooks/useIndexedDBThumbnail.ts
View File

@ -1,5 +1,5 @@
import { useState, useEffect } from "react";
import { FileMetadata } from "../types/file";
import { StirlingFileStub } from "../types/fileContext";
import { useIndexedDB } from "../contexts/IndexedDBContext";
import { generateThumbnailForFile } from "../utils/thumbnailUtils";
import { FileId } from "../types/fileContext";
@ -9,7 +9,7 @@ import { FileId } from "../types/fileContext";
* Hook for IndexedDB-aware thumbnail loading
* Handles thumbnail generation for files not in IndexedDB
*/
export function useIndexedDBThumbnail(file: FileMetadata | undefined | null): {
export function useIndexedDBThumbnail(file: StirlingFileStub | undefined | null): {
thumbnail: string | null;
isGenerating: boolean
} {
@ -27,8 +27,8 @@ export function useIndexedDBThumbnail(file: FileMetadata | undefined | null): {
}
// First priority: use stored thumbnail
if (file.thumbnail) {
setThumb(file.thumbnail);
if (file.thumbnailUrl) {
setThumb(file.thumbnailUrl);
return;
}
@ -77,7 +77,7 @@ export function useIndexedDBThumbnail(file: FileMetadata | undefined | null): {
loadThumbnail();
return () => { cancelled = true; };
}, [file, file?.thumbnail, file?.id, indexedDB, generating]);
}, [file, file?.thumbnailUrl, file?.id, indexedDB, generating]);
return { thumbnail: thumb, isGenerating: generating };
}

731
frontend/src/services/fileStorage.ts
View File

@ -1,19 +1,21 @@
/**
* IndexedDB File Storage Service
* Provides high-capacity file storage for PDF processing
* Now uses centralized IndexedDB manager
* Stirling File Storage Service
* Single-table architecture with typed query methods
* Forces correct usage patterns through service API design
*/
import { FileId } from '../types/file';
import { FileId, BaseFileMetadata } from '../types/file';
import { StirlingFile, StirlingFileStub, createStirlingFile } from '../types/fileContext';
import { indexedDBManager, DATABASE_CONFIGS } from './indexedDBManager';
export interface StoredFile {
id: FileId;
name: string;
type: string;
size: number;
lastModified: number;
/**
* Storage record - single source of truth
* Contains all data needed for both StirlingFile and StirlingFileStub
*/
export interface StoredStirlingFileRecord extends BaseFileMetadata {
data: ArrayBuffer;
fileId: FileId; // Matches runtime StirlingFile.fileId exactly
quickKey: string; // Matches runtime StirlingFile.quickKey exactly
thumbnail?: string;
url?: string; // For compatibility with existing components
}
@ -37,47 +39,49 @@ class FileStorageService {
}
/**
* Store a file in IndexedDB with external UUID
* Store a StirlingFile with its metadata from StirlingFileStub
*/
async storeFile(file: File, fileId: FileId, thumbnail?: string): Promise<StoredFile> {
async storeStirlingFile(stirlingFile: StirlingFile, stub: StirlingFileStub): Promise<void> {
const db = await this.getDatabase();
const arrayBuffer = await stirlingFile.arrayBuffer();
const arrayBuffer = await file.arrayBuffer();
const storedFile: StoredFile = {
id: fileId, // Use provided UUID
name: file.name,
type: file.type,
size: file.size,
lastModified: file.lastModified,
const record: StoredStirlingFileRecord = {
id: stirlingFile.fileId,
fileId: stirlingFile.fileId, // Explicit field for clarity
quickKey: stirlingFile.quickKey,
name: stirlingFile.name,
type: stirlingFile.type,
size: stirlingFile.size,
lastModified: stirlingFile.lastModified,
data: arrayBuffer,
thumbnail
thumbnail: stub.thumbnailUrl,
isLeaf: stub.isLeaf ?? true,
// History data from stub
versionNumber: stub.versionNumber ?? 1,
originalFileId: stub.originalFileId ?? stirlingFile.fileId,
parentFileId: stub.parentFileId ?? undefined,
toolHistory: stub.toolHistory ?? []
};
return new Promise((resolve, reject) => {
try {
// Verify store exists before creating transaction
if (!db.objectStoreNames.contains(this.storeName)) {
throw new Error(`Object store '${this.storeName}' not found. Available stores: ${Array.from(db.objectStoreNames).join(', ')}`);
}
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
// Debug logging
console.log('Object store keyPath:', store.keyPath);
console.log('Storing file with UUID:', {
id: storedFile.id, // Now a UUID from FileContext
name: storedFile.name,
hasData: !!storedFile.data,
dataSize: storedFile.data.byteLength
});
const request = store.add(storedFile);
const request = store.add(record);
request.onerror = () => {
console.error('IndexedDB add error:', request.error);
console.error('Failed object:', storedFile);
reject(request.error);
};
request.onsuccess = () => {
console.log('File stored successfully with ID:', storedFile.id);
resolve(storedFile);
resolve();
};
} catch (error) {
console.error('Transaction error:', error);
@ -87,9 +91,9 @@ class FileStorageService {
}
/**
* Retrieve a file from IndexedDB
* Get StirlingFile with full data - for loading into workbench
*/
async getFile(id: FileId): Promise<StoredFile | null> {
async getStirlingFile(id: FileId): Promise<StirlingFile | null> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
@ -97,77 +101,167 @@ class FileStorageService {
const store = transaction.objectStore(this.storeName);
const request = store.get(id);
request.onerror = () => reject(request.error);
request.onsuccess = () => resolve(request.result || null);
});
}
/**
* Get all stored files (WARNING: loads all data into memory)
*/
async getAllFiles(): Promise<StoredFile[]> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.getAll();
request.onerror = () => reject(request.error);
request.onsuccess = () => {
// Filter out null/corrupted entries
const files = request.result.filter(file =>
file &&
file.data &&
file.name &&
typeof file.size === 'number'
);
resolve(files);
const record = request.result as StoredStirlingFileRecord | undefined;
if (!record) {
resolve(null);
return;
}
// Create File from stored data
const blob = new Blob([record.data], { type: record.type });
const file = new File([blob], record.name, {
type: record.type,
lastModified: record.lastModified
});
// Convert to StirlingFile with preserved IDs
const stirlingFile = createStirlingFile(file, record.fileId);
resolve(stirlingFile);
};
});
}
/**
* Get metadata of all stored files (without loading data into memory)
* Get multiple StirlingFiles - for batch loading
*/
async getAllFileMetadata(): Promise<Omit<StoredFile, 'data'>[]> {
async getStirlingFiles(ids: FileId[]): Promise<StirlingFile[]> {
const results = await Promise.all(ids.map(id => this.getStirlingFile(id)));
return results.filter((file): file is StirlingFile => file !== null);
}
/**
* Get StirlingFileStub (metadata only) - for UI browsing
*/
async getStirlingFileStub(id: FileId): Promise<StirlingFileStub | null> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.get(id);
request.onerror = () => reject(request.error);
request.onsuccess = () => {
const record = request.result as StoredStirlingFileRecord | undefined;
if (!record) {
resolve(null);
return;
}
// Create StirlingFileStub from metadata (no file data)
const stub: StirlingFileStub = {
id: record.id,
name: record.name,
type: record.type,
size: record.size,
lastModified: record.lastModified,
quickKey: record.quickKey,
thumbnailUrl: record.thumbnail,
isLeaf: record.isLeaf,
versionNumber: record.versionNumber,
originalFileId: record.originalFileId,
parentFileId: record.parentFileId,
toolHistory: record.toolHistory,
createdAt: Date.now() // Current session
};
resolve(stub);
};
});
}
/**
* Get all StirlingFileStubs (metadata only) - for FileManager browsing
*/
async getAllStirlingFileStubs(): Promise<StirlingFileStub[]> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.openCursor();
const files: Omit<StoredFile, 'data'>[] = [];
const stubs: StirlingFileStub[] = [];
request.onerror = () => reject(request.error);
request.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result;
if (cursor) {
const storedFile = cursor.value;
// Only extract metadata, skip the data field
if (storedFile && storedFile.name && typeof storedFile.size === 'number') {
files.push({
id: storedFile.id,
name: storedFile.name,
type: storedFile.type,
size: storedFile.size,
lastModified: storedFile.lastModified,
thumbnail: storedFile.thumbnail
const record = cursor.value as StoredStirlingFileRecord;
if (record && record.name && typeof record.size === 'number') {
// Extract metadata only - no file data
stubs.push({
id: record.id,
name: record.name,
type: record.type,
size: record.size,
lastModified: record.lastModified,
quickKey: record.quickKey,
thumbnailUrl: record.thumbnail,
isLeaf: record.isLeaf,
versionNumber: record.versionNumber || 1,
originalFileId: record.originalFileId || record.id,
parentFileId: record.parentFileId,
toolHistory: record.toolHistory || [],
createdAt: Date.now()
});
}
cursor.continue();
} else {
// Metadata loaded efficiently without file data
resolve(files);
resolve(stubs);
}
};
});
}
/**
* Delete a file from IndexedDB
* Get leaf StirlingFileStubs only - for unprocessed files
*/
async deleteFile(id: FileId): Promise<void> {
async getLeafStirlingFileStubs(): Promise<StirlingFileStub[]> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.openCursor();
const leafStubs: StirlingFileStub[] = [];
request.onerror = () => reject(request.error);
request.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result;
if (cursor) {
const record = cursor.value as StoredStirlingFileRecord;
// Only include leaf files (default to true if undefined)
if (record && record.name && typeof record.size === 'number' && record.isLeaf !== false) {
leafStubs.push({
id: record.id,
name: record.name,
type: record.type,
size: record.size,
lastModified: record.lastModified,
quickKey: record.quickKey,
thumbnailUrl: record.thumbnail,
isLeaf: record.isLeaf,
versionNumber: record.versionNumber || 1,
originalFileId: record.originalFileId || record.id,
parentFileId: record.parentFileId,
toolHistory: record.toolHistory || [],
createdAt: Date.now()
});
}
cursor.continue();
} else {
resolve(leafStubs);
}
};
});
}
/**
* Delete StirlingFile - single operation, no sync issues
*/
async deleteStirlingFile(id: FileId): Promise<void> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
@ -181,317 +275,7 @@ class FileStorageService {
}
/**
* Update the lastModified timestamp of a file (for most recently used sorting)
*/
async touchFile(id: FileId): Promise<boolean> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const getRequest = store.get(id);
getRequest.onsuccess = () => {
const file = getRequest.result;
if (file) {
// Update lastModified to current timestamp
file.lastModified = Date.now();
const updateRequest = store.put(file);
updateRequest.onsuccess = () => resolve(true);
updateRequest.onerror = () => reject(updateRequest.error);
} else {
resolve(false); // File not found
}
};
getRequest.onerror = () => reject(getRequest.error);
});
}
/**
* Clear all stored files
*/
async clearAll(): Promise<void> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const request = store.clear();
request.onerror = () => reject(request.error);
request.onsuccess = () => resolve();
});
}
/**
* Get storage statistics (only our IndexedDB usage)
*/
async getStorageStats(): Promise<StorageStats> {
let used = 0;
let available = 0;
let quota: number | undefined;
let fileCount = 0;
try {
// Get browser quota for context
if ('storage' in navigator && 'estimate' in navigator.storage) {
const estimate = await navigator.storage.estimate();
quota = estimate.quota;
available = estimate.quota || 0;
}
// Calculate our actual IndexedDB usage from file metadata
const files = await this.getAllFileMetadata();
used = files.reduce((total, file) => total + (file?.size || 0), 0);
fileCount = files.length;
// Adjust available space
if (quota) {
available = quota - used;
}
} catch (error) {
console.warn('Could not get storage stats:', error);
// If we can't read metadata, database might be purged
used = 0;
fileCount = 0;
}
return {
used,
available,
fileCount,
quota
};
}
/**
* Get file count quickly without loading metadata
*/
async getFileCount(): Promise<number> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.count();
request.onerror = () => reject(request.error);
request.onsuccess = () => resolve(request.result);
});
}
/**
* Check all IndexedDB databases to see if files are in another version
*/
async debugAllDatabases(): Promise<void> {
console.log('=== Checking All IndexedDB Databases ===');
if ('databases' in indexedDB) {
try {
const databases = await indexedDB.databases();
console.log('Found databases:', databases);
for (const dbInfo of databases) {
if (dbInfo.name?.includes('stirling') || dbInfo.name?.includes('pdf')) {
console.log(`Checking database: ${dbInfo.name} (version: ${dbInfo.version})`);
try {
const db = await new Promise<IDBDatabase>((resolve, reject) => {
const request = indexedDB.open(dbInfo.name!, dbInfo.version);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
console.log(`Database ${dbInfo.name} object stores:`, Array.from(db.objectStoreNames));
db.close();
} catch (error) {
console.error(`Failed to open database ${dbInfo.name}:`, error);
}
}
}
} catch (error) {
console.error('Failed to list databases:', error);
}
} else {
console.log('indexedDB.databases() not supported');
}
// Also check our specific database with different versions
for (let version = 1; version <= 3; version++) {
try {
console.log(`Trying to open ${this.dbConfig.name} version ${version}...`);
const db = await new Promise<IDBDatabase>((resolve, reject) => {
const request = indexedDB.open(this.dbConfig.name, version);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
request.onupgradeneeded = () => {
// Don't actually upgrade, just check
request.transaction?.abort();
};
});
console.log(`Version ${version} object stores:`, Array.from(db.objectStoreNames));
if (db.objectStoreNames.contains('files')) {
const transaction = db.transaction(['files'], 'readonly');
const store = transaction.objectStore('files');
const countRequest = store.count();
countRequest.onsuccess = () => {
console.log(`Version ${version} files store has ${countRequest.result} entries`);
};
}
db.close();
} catch (error) {
if (error instanceof Error) {
console.log(`Version ${version} not accessible:`, error.message);
}
}
}
}
/**
* Debug method to check what's actually in the database
*/
async debugDatabaseContents(): Promise<void> {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
// First try getAll to see if there's anything
const getAllRequest = store.getAll();
getAllRequest.onsuccess = () => {
console.log('=== Raw getAll() result ===');
console.log('Raw entries found:', getAllRequest.result.length);
getAllRequest.result.forEach((item, index) => {
console.log(`Raw entry ${index}:`, {
keys: Object.keys(item || {}),
id: item?.id,
name: item?.name,
size: item?.size,
type: item?.type,
hasData: !!item?.data,
dataSize: item?.data?.byteLength,
fullObject: item
});
});
};
// Then try cursor
const cursorRequest = store.openCursor();
console.log('=== IndexedDB Cursor Debug ===');
let count = 0;
cursorRequest.onerror = () => {
console.error('Cursor error:', cursorRequest.error);
reject(cursorRequest.error);
};
cursorRequest.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result;
if (cursor) {
count++;
const value = cursor.value;
console.log(`Cursor File ${count}:`, {
id: value?.id,
name: value?.name,
size: value?.size,
type: value?.type,
hasData: !!value?.data,
dataSize: value?.data?.byteLength,
hasThumbnail: !!value?.thumbnail,
allKeys: Object.keys(value || {})
});
cursor.continue();
} else {
console.log(`=== End Cursor Debug - Found ${count} files ===`);
resolve();
}
};
});
}
/**
* Convert StoredFile back to pure File object without mutations
* Returns a clean File object - use FileContext.addStoredFiles() for proper metadata handling
*/
createFileFromStored(storedFile: StoredFile): File {
if (!storedFile || !storedFile.data) {
throw new Error('Invalid stored file: missing data');
}
if (!storedFile.name || typeof storedFile.size !== 'number') {
throw new Error('Invalid stored file: missing metadata');
}
const blob = new Blob([storedFile.data], { type: storedFile.type });
const file = new File([blob], storedFile.name, {
type: storedFile.type,
lastModified: storedFile.lastModified
});
// Use FileContext.addStoredFiles() to properly associate with metadata
return file;
}
/**
* Convert StoredFile to the format expected by FileContext.addStoredFiles()
* This is the recommended way to load stored files into FileContext
*/
createFileWithMetadata(storedFile: StoredFile): { file: File; originalId: FileId; metadata: { thumbnail?: string } } {
const file = this.createFileFromStored(storedFile);
return {
file,
originalId: storedFile.id,
metadata: {
thumbnail: storedFile.thumbnail
}
};
}
/**
* Create blob URL for stored file
*/
createBlobUrl(storedFile: StoredFile): string {
const blob = new Blob([storedFile.data], { type: storedFile.type });
return URL.createObjectURL(blob);
}
/**
* Get file data as ArrayBuffer for streaming/chunked processing
*/
async getFileData(id: FileId): Promise<ArrayBuffer | null> {
try {
const storedFile = await this.getFile(id);
return storedFile ? storedFile.data : null;
} catch (error) {
console.warn(`Failed to get file data for ${id}:`, error);
return null;
}
}
/**
* Create a temporary blob URL that gets revoked automatically
*/
async createTemporaryBlobUrl(id: FileId): Promise<string | null> {
const data = await this.getFileData(id);
if (!data) return null;
const blob = new Blob([data], { type: 'application/pdf' });
const url = URL.createObjectURL(blob);
// Auto-revoke after a short delay to free memory
setTimeout(() => {
URL.revokeObjectURL(url);
}, 10000); // 10 seconds
return url;
}
/**
* Update thumbnail for an existing file
* Update thumbnail for existing file
*/
async updateThumbnail(id: FileId, thumbnail: string): Promise<boolean> {
const db = await this.getDatabase();
@ -503,13 +287,12 @@ class FileStorageService {
const getRequest = store.get(id);
getRequest.onsuccess = () => {
const storedFile = getRequest.result;
if (storedFile) {
storedFile.thumbnail = thumbnail;
const updateRequest = store.put(storedFile);
const record = getRequest.result as StoredStirlingFileRecord;
if (record) {
record.thumbnail = thumbnail;
const updateRequest = store.put(record);
updateRequest.onsuccess = () => {
console.log('Thumbnail updated for file:', id);
resolve(true);
};
updateRequest.onerror = () => {
@ -533,31 +316,161 @@ class FileStorageService {
}
/**
* Check if storage quota is running low
* Clear all stored files
*/
async isStorageLow(): Promise<boolean> {
const stats = await this.getStorageStats();
if (!stats.quota) return false;
async clearAll(): Promise<void> {
const db = await this.getDatabase();
const usagePercent = stats.used / stats.quota;
return usagePercent > 0.8; // Consider low if over 80% used
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const request = store.clear();
request.onerror = () => reject(request.error);
request.onsuccess = () => resolve();
});
}
/**
* Clean up old files if storage is low
* Get storage statistics
*/
async cleanupOldFiles(maxFiles: number = 50): Promise<void> {
const files = await this.getAllFileMetadata();
async getStorageStats(): Promise<StorageStats> {
let used = 0;
let available = 0;
let quota: number | undefined;
let fileCount = 0;
if (files.length <= maxFiles) return;
try {
// Get browser quota for context
if ('storage' in navigator && 'estimate' in navigator.storage) {
const estimate = await navigator.storage.estimate();
quota = estimate.quota;
available = estimate.quota || 0;
}
// Sort by last modified (oldest first)
files.sort((a, b) => a.lastModified - b.lastModified);
// Calculate our actual IndexedDB usage from file metadata
const stubs = await this.getAllStirlingFileStubs();
used = stubs.reduce((total, stub) => total + (stub?.size || 0), 0);
fileCount = stubs.length;
// Delete oldest files
const filesToDelete = files.slice(0, files.length - maxFiles);
for (const file of filesToDelete) {
await this.deleteFile(file.id);
// Adjust available space
if (quota) {
available = quota - used;
}
} catch (error) {
console.warn('Could not get storage stats:', error);
used = 0;
fileCount = 0;
}
return {
used,
available,
fileCount,
quota
};
}
/**
* Create blob URL for stored file data
*/
async createBlobUrl(id: FileId): Promise<string | null> {
try {
const db = await this.getDatabase();
return new Promise((resolve, reject) => {
const transaction = db.transaction([this.storeName], 'readonly');
const store = transaction.objectStore(this.storeName);
const request = store.get(id);
request.onerror = () => reject(request.error);
request.onsuccess = () => {
const record = request.result as StoredStirlingFileRecord | undefined;
if (record) {
const blob = new Blob([record.data], { type: record.type });
const url = URL.createObjectURL(blob);
resolve(url);
} else {
resolve(null);
}
};
});
} catch (error) {
console.warn(`Failed to create blob URL for ${id}:`, error);
return null;
}
}
/**
* Mark a file as processed (no longer a leaf file)
* Used when a file becomes input to a tool operation
*/
async markFileAsProcessed(fileId: FileId): Promise<boolean> {
try {
const db = await this.getDatabase();
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const record = await new Promise<StoredStirlingFileRecord | undefined>((resolve, reject) => {
const request = store.get(fileId);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
if (!record) {
return false; // File not found
}
// Update the isLeaf flag to false
record.isLeaf = false;
await new Promise<void>((resolve, reject) => {
const request = store.put(record);
request.onsuccess = () => resolve();
request.onerror = () => reject(request.error);
});
return true;
} catch (error) {
console.error('Failed to mark file as processed:', error);
return false;
}
}
/**
* Mark a file as leaf (opposite of markFileAsProcessed)
* Used when promoting a file back to "recent" status
*/
async markFileAsLeaf(fileId: FileId): Promise<boolean> {
try {
const db = await this.getDatabase();
const transaction = db.transaction([this.storeName], 'readwrite');
const store = transaction.objectStore(this.storeName);
const record = await new Promise<StoredStirlingFileRecord | undefined>((resolve, reject) => {
const request = store.get(fileId);
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
if (!record) {
return false; // File not found
}
// Update the isLeaf flag to true
record.isLeaf = true;
await new Promise<void>((resolve, reject) => {
const request = store.put(record);
request.onsuccess = () => resolve();
request.onerror = () => reject(request.error);
});
return true;
} catch (error) {
console.error('Failed to mark file as leaf:', error);
return false;
}
}
}

148
frontend/src/services/indexedDBManager.ts
View File

@ -87,45 +87,135 @@ class IndexedDBManager {
request.onupgradeneeded = (event) => {
const db = request.result;
const oldVersion = event.oldVersion;
const transaction = request.transaction;
console.log(`Upgrading ${config.name} from v${oldVersion} to v${config.version}`);
// Create or update object stores
config.stores.forEach(storeConfig => {
let store: IDBObjectStore | undefined;
if (db.objectStoreNames.contains(storeConfig.name)) {
// Store exists - for now, just continue (could add migration logic here)
// Store exists - get reference for migration
console.log(`Object store '${storeConfig.name}' already exists`);
return;
store = transaction?.objectStore(storeConfig.name);
// Add new indexes if they don't exist
if (storeConfig.indexes && store) {
storeConfig.indexes.forEach(indexConfig => {
if (!store?.indexNames.contains(indexConfig.name)) {
store?.createIndex(
indexConfig.name,
indexConfig.keyPath,
{ unique: indexConfig.unique }
);
console.log(`Created index '${indexConfig.name}' on '${storeConfig.name}'`);
}
});
}
} else {
// Create new object store
const options: IDBObjectStoreParameters = {};
if (storeConfig.keyPath) {
options.keyPath = storeConfig.keyPath;
}
if (storeConfig.autoIncrement) {
options.autoIncrement = storeConfig.autoIncrement;
}
store = db.createObjectStore(storeConfig.name, options);
console.log(`Created object store '${storeConfig.name}'`);
// Create indexes
if (storeConfig.indexes) {
storeConfig.indexes.forEach(indexConfig => {
store?.createIndex(
indexConfig.name,
indexConfig.keyPath,
{ unique: indexConfig.unique }
);
console.log(`Created index '${indexConfig.name}' on '${storeConfig.name}'`);
});
}
}
// Create new object store
const options: IDBObjectStoreParameters = {};
if (storeConfig.keyPath) {
options.keyPath = storeConfig.keyPath;
}
if (storeConfig.autoIncrement) {
options.autoIncrement = storeConfig.autoIncrement;
}
const store = db.createObjectStore(storeConfig.name, options);
console.log(`Created object store '${storeConfig.name}'`);
// Create indexes
if (storeConfig.indexes) {
storeConfig.indexes.forEach(indexConfig => {
store.createIndex(
indexConfig.name,
indexConfig.keyPath,
{ unique: indexConfig.unique }
);
console.log(`Created index '${indexConfig.name}' on '${storeConfig.name}'`);
});
// Perform data migration for files database
if (config.name === 'stirling-pdf-files' && storeConfig.name === 'files' && store) {
this.migrateFileHistoryFields(store, oldVersion);
}
});
};
});
}
/**
* Migrate existing file records to include new file history fields
*/
private migrateFileHistoryFields(store: IDBObjectStore, oldVersion: number): void {
// Only migrate if upgrading from a version before file history was added (version < 3)
if (oldVersion >= 3) {
return;
}
console.log('Starting file history migration for existing records...');
const cursor = store.openCursor();
let migratedCount = 0;
cursor.onsuccess = (event) => {
const cursor = (event.target as IDBRequest).result;
if (cursor) {
const record = cursor.value;
let needsUpdate = false;
// Add missing file history fields with sensible defaults
if (record.isLeaf === undefined) {
record.isLeaf = true; // Existing files are unprocessed, should appear in recent files
needsUpdate = true;
}
if (record.versionNumber === undefined) {
record.versionNumber = 1; // Existing files are first version
needsUpdate = true;
}
if (record.originalFileId === undefined) {
record.originalFileId = record.id; // Existing files are their own root
needsUpdate = true;
}
if (record.parentFileId === undefined) {
record.parentFileId = undefined; // No parent for existing files
needsUpdate = true;
}
if (record.toolHistory === undefined) {
record.toolHistory = []; // No history for existing files
needsUpdate = true;
}
// Update the record if any fields were missing
if (needsUpdate) {
try {
cursor.update(record);
migratedCount++;
} catch (error) {
console.error('Failed to migrate record:', record.id, error);
}
}
cursor.continue();
} else {
// Migration complete
console.log(`File history migration completed. Migrated ${migratedCount} records.`);
}
};
cursor.onerror = (event) => {
console.error('File history migration failed:', (event.target as IDBRequest).error);
};
}
/**
* Get database connection (must be already opened)
*/
@ -201,13 +291,16 @@ class IndexedDBManager {
export const DATABASE_CONFIGS = {
FILES: {
name: 'stirling-pdf-files',
version: 2,
version: 3,
stores: [{
name: 'files',
keyPath: 'id',
indexes: [
{ name: 'name', keyPath: 'name', unique: false },
{ name: 'lastModified', keyPath: 'lastModified', unique: false }
{ name: 'lastModified', keyPath: 'lastModified', unique: false },
{ name: 'originalFileId', keyPath: 'originalFileId', unique: false },
{ name: 'parentFileId', keyPath: 'parentFileId', unique: false },
{ name: 'versionNumber', keyPath: 'versionNumber', unique: false }
]
}]
} as DatabaseConfig,
@ -219,7 +312,8 @@ export const DATABASE_CONFIGS = {
name: 'drafts',
keyPath: 'id'
}]
} as DatabaseConfig
} as DatabaseConfig,
} as const;
export const indexedDBManager = IndexedDBManager.getInstance();

4
frontend/src/services/pdfExportService.ts
View File

@ -29,7 +29,7 @@ export class PDFExportService {
// Load original PDF and create new document
const originalPDFBytes = await pdfDocument.file.arrayBuffer();
const sourceDoc = await PDFLibDocument.load(originalPDFBytes);
const sourceDoc = await PDFLibDocument.load(originalPDFBytes, { ignoreEncryption: true });
const blob = await this.createSingleDocument(sourceDoc, pagesToExport);
const exportFilename = this.generateFilename(filename || pdfDocument.name);
@ -86,7 +86,7 @@ export class PDFExportService {
for (const [fileId, file] of sourceFiles) {
try {
const arrayBuffer = await file.arrayBuffer();
const doc = await PDFLibDocument.load(arrayBuffer);
const doc = await PDFLibDocument.load(arrayBuffer, { ignoreEncryption: true });
loadedDocs.set(fileId, doc);
} catch (error) {
console.warn(`Failed to load source file ${fileId}:`, error);

4
frontend/src/tests/convert/ConvertIntegration.test.tsx
View File

@ -141,7 +141,7 @@ describe('Convert Tool Integration Tests', () => {
// Verify hook state updates
expect(result.current.downloadUrl).toBeTruthy();
expect(result.current.downloadFilename).toBe('test_converted.png');
expect(result.current.downloadFilename).toBe('test.png');
expect(result.current.isLoading).toBe(false);
expect(result.current.errorMessage).toBe(null);
});
@ -363,7 +363,7 @@ describe('Convert Tool Integration Tests', () => {
// Verify hook state updates correctly
expect(result.current.downloadUrl).toBeTruthy();
expect(result.current.downloadFilename).toBe('test_converted.csv');
expect(result.current.downloadFilename).toBe('test.csv');
expect(result.current.isLoading).toBe(false);
expect(result.current.errorMessage).toBe(null);
});

69
frontend/src/types/file.ts
View File

@ -7,55 +7,34 @@ declare const tag: unique symbol;
export type FileId = string & { readonly [tag]: 'FileId' };
/**
* File metadata for efficient operations without loading full file data
* Used by IndexedDBContext and FileContext for lazy file loading
* Tool operation metadata for history tracking
* Note: Parameters removed for security - sensitive data like passwords should not be stored in history
*/
export interface FileMetadata {
export interface ToolOperation {
toolName: string;
timestamp: number;
}
/**
* Base file metadata shared between storage and runtime layers
* Contains all common file properties and history tracking
*/
export interface BaseFileMetadata {
id: FileId;
name: string;
type: string;
size: number;
lastModified: number;
thumbnail?: string;
isDraft?: boolean; // Marks files as draft versions
createdAt?: number; // When file was added to system
// File history tracking
isLeaf: boolean; // True if this file hasn't been processed yet
originalFileId: string; // Root file ID for grouping versions
versionNumber: number; // Version number in chain
parentFileId?: FileId; // Immediate parent file ID
toolHistory?: Array<{
toolName: string;
timestamp: number;
}>; // Tool chain for history tracking
}
export interface StorageConfig {
useIndexedDB: boolean;
maxFileSize: number; // Maximum size per file in bytes
maxTotalStorage: number; // Maximum total storage in bytes
warningThreshold: number; // Warning threshold (percentage 0-1)
}
export const defaultStorageConfig: StorageConfig = {
useIndexedDB: true,
maxFileSize: 100 * 1024 * 1024, // 100MB per file
maxTotalStorage: 1024 * 1024 * 1024, // 1GB default, will be updated dynamically
warningThreshold: 0.8, // Warn at 80% capacity
};
// Calculate and update storage limit: half of available storage or 10GB, whichever is smaller
export const initializeStorageConfig = async (): Promise<StorageConfig> => {
const tenGB = 10 * 1024 * 1024 * 1024; // 10GB in bytes
const oneGB = 1024 * 1024 * 1024; // 1GB fallback
let maxTotalStorage = oneGB; // Default fallback
// Try to estimate available storage
if ('storage' in navigator && 'estimate' in navigator.storage) {
try {
const estimate = await navigator.storage.estimate();
if (estimate.quota) {
const halfQuota = estimate.quota / 2;
maxTotalStorage = Math.min(halfQuota, tenGB);
}
} catch (error) {
console.warn('Could not estimate storage quota, using 1GB default:', error);
}
}
return {
...defaultStorageConfig,
maxTotalStorage
};
};

42
frontend/src/types/fileContext.ts
View File

@ -3,7 +3,7 @@
*/
import { PageOperation } from './pageEditor';
import { FileId, FileMetadata } from './file';
import { FileId, BaseFileMetadata } from './file';
// Re-export FileId for convenience
export type { FileId };
@ -40,7 +40,6 @@ export interface ProcessedFilePage {
export interface ProcessedFileMetadata {
pages: ProcessedFilePage[];
totalPages?: number;
thumbnailUrl?: string;
lastProcessed?: number;
[key: string]: any;
}
@ -52,16 +51,17 @@ export interface ProcessedFileMetadata {
* separately in refs for memory efficiency. Supports multi-tool workflows
* where files persist across tool operations.
*/
export interface StirlingFileStub {
id: FileId; // UUID primary key for collision-free operations
name: string; // Display name for UI
size: number; // File size for progress indicators
type: string; // MIME type for format validation
lastModified: number; // Original timestamp for deduplication
/**
* StirlingFileStub - Runtime UI metadata for files in the active workbench session
*
* Contains UI display data and processing state. Actual File objects stored
* separately in refs for memory efficiency. Supports multi-tool workflows
* where files persist across tool operations.
*/
export interface StirlingFileStub extends BaseFileMetadata {
quickKey?: string; // Fast deduplication key: name|size|lastModified
thumbnailUrl?: string; // Generated thumbnail blob URL for visual display
blobUrl?: string; // File access blob URL for downloads/processing
createdAt?: number; // When added to workbench for sorting
processedFile?: ProcessedFileMetadata; // PDF page data and processing results
insertAfterPageId?: string; // Page ID after which this file should be inserted
isPinned?: boolean; // Protected from tool consumption (replace/remove)
@ -107,6 +107,11 @@ export function isStirlingFile(file: File): file is StirlingFile {
// Create a StirlingFile from a regular File object
export function createStirlingFile(file: File, id?: FileId): StirlingFile {
// Check if file is already a StirlingFile to avoid property redefinition
if (isStirlingFile(file)) {
return file; // Already has fileId and quickKey properties
}
const fileId = id || createFileId();
const quickKey = createQuickKey(file);
@ -151,9 +156,11 @@ export function isFileObject(obj: any): obj is File | StirlingFile {
export function toStirlingFileStub(
export function createNewStirlingFileStub(
file: File,
id?: FileId
id?: FileId,
thumbnail?: string,
processedFileMetadata?: ProcessedFileMetadata
): StirlingFileStub {
const fileId = id || createFileId();
return {
@ -162,8 +169,13 @@ export function toStirlingFileStub(
size: file.size,
type: file.type,
lastModified: file.lastModified,
originalFileId: fileId,
quickKey: createQuickKey(file),
createdAt: Date.now()
createdAt: Date.now(),
isLeaf: true, // New files are leaf nodes by default
versionNumber: 1, // New files start at version 1
thumbnailUrl: thumbnail,
processedFile: processedFileMetadata
};
}
@ -209,7 +221,6 @@ export interface FileOperation {
metadata?: {
originalFileName?: string;
outputFileNames?: string[];
parameters?: Record<string, any>;
fileSize?: number;
pageCount?: number;
error?: string;
@ -286,8 +297,7 @@ export type FileContextAction =
export interface FileContextActions {
// File management - lightweight actions only
addFiles: (files: File[], options?: { insertAfterPageId?: string; selectFiles?: boolean }) => Promise<StirlingFile[]>;
addProcessedFiles: (filesWithThumbnails: Array<{ file: File; thumbnail?: string; pageCount?: number }>) => Promise<StirlingFile[]>;
addStoredFiles: (filesWithMetadata: Array<{ file: File; originalId: FileId; metadata: FileMetadata }>, options?: { selectFiles?: boolean }) => Promise<StirlingFile[]>;
addStirlingFileStubs: (stirlingFileStubs: StirlingFileStub[], options?: { insertAfterPageId?: string; selectFiles?: boolean }) => Promise<StirlingFile[]>;
removeFiles: (fileIds: FileId[], deleteFromStorage?: boolean) => Promise<void>;
updateStirlingFileStub: (id: FileId, updates: Partial<StirlingFileStub>) => void;
reorderFiles: (orderedFileIds: FileId[]) => void;
@ -299,7 +309,7 @@ export interface FileContextActions {
unpinFile: (file: StirlingFile) => void;
// File consumption (replace unpinned files with outputs)
consumeFiles: (inputFileIds: FileId[], outputFiles: File[]) => Promise<FileId[]>;
consumeFiles: (inputFileIds: FileId[], outputStirlingFiles: StirlingFile[], outputStirlingFileStubs: StirlingFileStub[]) => Promise<FileId[]>;
undoConsumeFiles: (inputFiles: File[], inputStirlingFileStubs: StirlingFileStub[], outputFileIds: FileId[]) => Promise<void>;
// Selection management
setSelectedFiles: (fileIds: FileId[]) => void;

57
frontend/src/utils/downloadUtils.ts
View File

@ -1,4 +1,4 @@
import { FileMetadata } from '../types/file';
import { StirlingFileStub } from '../types/fileContext';
import { fileStorage } from '../services/fileStorage';
import { zipFileService } from '../services/zipFileService';
@ -9,14 +9,14 @@ import { zipFileService } from '../services/zipFileService';
*/
export function downloadBlob(blob: Blob, filename: string): void {
const url = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = url;
link.download = filename;
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
// Clean up the blob URL
URL.revokeObjectURL(url);
}
@ -26,23 +26,23 @@ export function downloadBlob(blob: Blob, filename: string): void {
* @param file - The file object with storage information
* @throws Error if file cannot be retrieved from storage
*/
export async function downloadFileFromStorage(file: FileMetadata): Promise<void> {
export async function downloadFileFromStorage(file: StirlingFileStub): Promise<void> {
const lookupKey = file.id;
const storedFile = await fileStorage.getFile(lookupKey);
if (!storedFile) {
const stirlingFile = await fileStorage.getStirlingFile(lookupKey);
if (!stirlingFile) {
throw new Error(`File "${file.name}" not found in storage`);
}
const blob = new Blob([storedFile.data], { type: storedFile.type });
downloadBlob(blob, storedFile.name);
// StirlingFile is already a File object, just download it
downloadBlob(stirlingFile, stirlingFile.name);
}
/**
* Downloads multiple files as individual downloads
* @param files - Array of files to download
*/
export async function downloadMultipleFiles(files: FileMetadata[]): Promise<void> {
export async function downloadMultipleFiles(files: StirlingFileStub[]): Promise<void> {
for (const file of files) {
await downloadFileFromStorage(file);
}
@ -53,36 +53,33 @@ export async function downloadMultipleFiles(files: FileMetadata[]): Promise<void
* @param files - Array of files to include in ZIP
* @param zipFilename - Optional custom ZIP filename (defaults to timestamped name)
*/
export async function downloadFilesAsZip(files: FileMetadata[], zipFilename?: string): Promise<void> {
export async function downloadFilesAsZip(files: StirlingFileStub[], zipFilename?: string): Promise<void> {
if (files.length === 0) {
throw new Error('No files provided for ZIP download');
}
// Convert stored files to File objects
const fileObjects: File[] = [];
const filesToZip: File[] = [];
for (const fileWithUrl of files) {
const lookupKey = fileWithUrl.id;
const storedFile = await fileStorage.getFile(lookupKey);
if (storedFile) {
const file = new File([storedFile.data], storedFile.name, {
type: storedFile.type,
lastModified: storedFile.lastModified
});
fileObjects.push(file);
const stirlingFile = await fileStorage.getStirlingFile(lookupKey);
if (stirlingFile) {
// StirlingFile is already a File object!
filesToZip.push(stirlingFile);
}
}
if (fileObjects.length === 0) {
if (filesToZip.length === 0) {
throw new Error('No valid files found in storage for ZIP download');
}
// Generate default filename if not provided
const finalZipFilename = zipFilename ||
const finalZipFilename = zipFilename ||
`files-${new Date().toISOString().slice(0, 19).replace(/[:-]/g, '')}.zip`;
// Create and download ZIP
const { zipFile } = await zipFileService.createZipFromFiles(fileObjects, finalZipFilename);
const { zipFile } = await zipFileService.createZipFromFiles(filesToZip, finalZipFilename);
downloadBlob(zipFile, finalZipFilename);
}
@ -94,7 +91,7 @@ export async function downloadFilesAsZip(files: FileMetadata[], zipFilename?: st
* @param options - Download options
*/
export async function downloadFiles(
files: FileMetadata[],
files: StirlingFileStub[],
options: {
forceZip?: boolean;
zipFilename?: string;
@ -133,8 +130,8 @@ export function downloadFileObject(file: File, filename?: string): void {
* @param mimeType - MIME type (defaults to text/plain)
*/
export function downloadTextAsFile(
content: string,
filename: string,
content: string,
filename: string,
mimeType: string = 'text/plain'
): void {
const blob = new Blob([content], { type: mimeType });
@ -149,4 +146,4 @@ export function downloadTextAsFile(
export function downloadJsonAsFile(data: any, filename: string): void {
const content = JSON.stringify(data, null, 2);
downloadTextAsFile(content, filename, 'application/json');
}
}

78
frontend/src/utils/fileHistoryUtils.ts Normal file
View File

@ -0,0 +1,78 @@
/**
* File History Utilities
*
* Helper functions for IndexedDB-based file history management.
* Handles file history operations and lineage tracking.
*/
import { StirlingFileStub } from '../types/fileContext';
/**
* Group files by processing branches - each branch ends in a leaf file
* Returns Map<fileId, lineagePath[]> where fileId is the leaf and lineagePath is the path back to original
*/
export function groupFilesByOriginal(StirlingFileStubs: StirlingFileStub[]): Map<string, StirlingFileStub[]> {
const groups = new Map<string, StirlingFileStub[]>();
// Create a map for quick lookups
const fileMap = new Map<string, StirlingFileStub>();
for (const record of StirlingFileStubs) {
fileMap.set(record.id, record);
}
// Find leaf files (files that are not parents of any other files AND have version history)
// Original files (v0) should only be leaves if they have no processed versions at all
const leafFiles = StirlingFileStubs.filter(stub => {
const isParentOfOthers = StirlingFileStubs.some(otherStub => otherStub.parentFileId === stub.id);
const isOriginalOfOthers = StirlingFileStubs.some(otherStub => otherStub.originalFileId === stub.id);
// A file is a leaf if:
// 1. It's not a parent of any other files, AND
// 2. It has processing history (versionNumber > 0) OR it's not referenced as original by others
return !isParentOfOthers && (stub.versionNumber && stub.versionNumber > 0 || !isOriginalOfOthers);
});
// For each leaf file, build its complete lineage path back to original
for (const leafFile of leafFiles) {
const lineagePath: StirlingFileStub[] = [];
let currentFile: StirlingFileStub | undefined = leafFile;
// Trace back through parentFileId chain to build this specific branch
while (currentFile) {
lineagePath.push(currentFile);
// Move to parent file in this branch
let nextFile: StirlingFileStub | undefined = undefined;
if (currentFile.parentFileId) {
nextFile = fileMap.get(currentFile.parentFileId);
} else if (currentFile.originalFileId && currentFile.originalFileId !== currentFile.id) {
// For v1 files, the original file might be referenced by originalFileId
nextFile = fileMap.get(currentFile.originalFileId);
}
// Check for infinite loops before moving to next
if (nextFile && lineagePath.some(file => file.id === nextFile!.id)) {
break;
}
currentFile = nextFile;
}
// Sort lineage with latest version first (leaf at top)
lineagePath.sort((a, b) => (b.versionNumber || 0) - (a.versionNumber || 0));
// Use leaf file ID as the group key - each branch gets its own group
groups.set(leafFile.id, lineagePath);
}
return groups;
}
/**
* Check if a file has version history
*/
export function hasVersionHistory(fileStub: StirlingFileStub): boolean {
return !!(fileStub.originalFileId && fileStub.versionNumber && fileStub.versionNumber > 0);
}

3
frontend/src/utils/toolOperationTracker.ts
View File

@ -6,7 +6,7 @@ import { FileOperation } from '../types/fileContext';
*/
export const createOperation = <TParams = void>(
operationType: string,
params: TParams,
_params: TParams,
selectedFiles: File[]
): { operation: FileOperation; operationId: string; fileId: FileId } => {
const operationId = `${operationType}-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
@ -20,7 +20,6 @@ export const createOperation = <TParams = void>(
status: 'pending',
metadata: {
originalFileName: selectedFiles[0]?.name,
parameters: params,
fileSize: selectedFiles.reduce((sum, f) => sum + f.size, 0)
}
} as any /* FIX ME*/;

13
frontend/src/utils/toolResponseProcessor.ts
View File

@ -8,11 +8,12 @@ export type ResponseHandler = (blob: Blob, originalFiles: File[]) => Promise<Fil
* - If a tool-specific responseHandler is provided, it is used.
* - If responseHeaders provided and contains Content-Disposition, uses that filename.
* - Otherwise, create a single file using the filePrefix + original name.
* - If filePrefix is empty, preserves the original filename.
*/
export async function processResponse(
blob: Blob,
originalFiles: File[],
filePrefix: string,
filePrefix?: string,
responseHandler?: ResponseHandler,
responseHeaders?: Record<string, any>
): Promise<File[]> {
@ -36,7 +37,13 @@ export async function processResponse(
// Default behavior: use filePrefix + original name
const original = originalFiles[0]?.name ?? 'result.pdf';
const name = `${filePrefix}${original}`;
// Only add prefix if it's not empty - this preserves original filenames for file history
const name = filePrefix ? `${filePrefix}${original}` : original;
const type = blob.type || 'application/octet-stream';
return [new File([blob], name, { type })];
// File was modified by tool processing - set lastModified to current time
return [new File([blob], name, {
type,
lastModified: Date.now()
})];
}