saulteafarmer/Stirling-PDF
1
0
Fork 0
mirror of https://github.com/Stirling-Tools/Stirling-PDF.git synced 2025-06-21 23:15:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

init

Browse Source
This commit is contained in:
Anthony Stirling 2025-05-26 00:03:19 +01:00
parent f2f11496a2
commit 2df943e110
24 changed files with 3039 additions and 238 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Dockerfile
View File

@ -92,4 +92,4 @@ EXPOSE 8080/tcp
# Set user and run command
ENTRYPOINT ["tini", "--", "/scripts/init.sh"]
CMD ["sh", "-c", "java -Dfile.encoding=UTF-8 -jar /app.jar & /opt/venv/bin/unoserver --port 2003 --interface 127.0.0.1"]
CMD ["java", "-Dfile.encoding=UTF-8", "-jar", "/app.jar"]

2
Dockerfile.fat
View File

@ -101,4 +101,4 @@ RUN echo "@main https://dl-cdn.alpinelinux.org/alpine/edge/main" | tee -a /etc/a
EXPOSE 8080/tcp
# Set user and run command
ENTRYPOINT ["tini", "--", "/scripts/init.sh"]
CMD ["sh", "-c", "java -Dfile.encoding=UTF-8 -jar /app.jar & /opt/venv/bin/unoserver --port 2003 --interface 127.0.0.1"]
CMD ["sh", "-c", "java -Dfile.encoding=UTF-8 -jar /app.jar"]

250
docs/unoserver-configuration.md Normal file
View File

@ -0,0 +1,250 @@
# UnoServer Configuration Guide
## Overview
Stirling-PDF supports multiple UnoServer instances to improve concurrent document conversion performance. This document explains how to configure and use this feature.
The UnoServer component in Stirling-PDF is now conditional, meaning it will only be enabled if the required executables are available on your system. This allows Stirling-PDF to run in environments without LibreOffice/UnoServer while gracefully disabling office document conversion functionality.
## Configuration Options
### Multiple Local Instances
You can configure Stirling-PDF to start multiple UnoServer instances locally. Each instance will run on its own port starting from the base port.
In `settings.yml`:
```yaml
processExecutor:
sessionLimit:
libreOfficeSessionLimit: 4 # Set to desired number of instances
baseUnoconvPort: 2003 # Base port for UnoServer instances
useExternalUnoconvServers: false # Set to false to use local instances
```
### External UnoServer Instances
For more advanced setups or when running in a clustered environment, you can configure Stirling-PDF to use external UnoServer instances running on different hosts:
```yaml
processExecutor:
useExternalUnoconvServers: true
unoconvServers:
- "192.168.1.100:2003" # Format is host:port
- "192.168.1.101:2003"
- "unoserver-host:2003"
```
## Installation
### Docker
The easiest way to use UnoServer with Stirling-PDF is to use the "fat" Docker image, which includes all required dependencies:
```bash
docker pull frooodle/s-pdf:latest-fat
```
### Manual Installation
If you want to install UnoServer manually:
1. Install LibreOffice:
```bash
# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y libreoffice
# CentOS/RHEL
sudo yum install -y libreoffice
# macOS
brew install libreoffice
```
2. Install UnoServer using pip:
```bash
pip install unoserver
```
3. Verify installation:
```bash
unoserver --version
unoconvert --version
```
### Installation Location
Stirling-PDF will automatically detect UnoServer in these locations:
- In the same directory as the unoconvert executable
- At `/opt/venv/bin/unoserver` (Docker default)
- In standard system paths (`/usr/bin/unoserver`, `/usr/local/bin/unoserver`)
- In your PATH environment variable
## Advanced Features
### Health Checks
The system performs automatic health checks on all UnoServer instances every 60 seconds. These checks:
- Verify that each server is reachable and operational
- Automatically restart local instances that have failed
- Log the health status of all instances
- Update the circuit breaker status for each instance
Health checks are logged at INFO level and show the number of healthy and unhealthy instances.
### Circuit Breaker
For fault tolerance, each UnoServer instance implements a circuit breaker pattern that:
1. Tracks conversion failures for each instance
2. After 3 consecutive failures, marks the instance as unavailable (circuit open)
3. Waits for a cooldown period (30 seconds) before attempting to use the instance again
4. Automatically routes requests to healthy instances
The circuit breaker helps prevent cascading failures and provides automatic recovery.
### Performance Metrics
The UnoServerManager records and logs detailed metrics about UnoServer usage:
- Total number of conversions
- Success/failure rate
- Conversions per server instance
- Average conversion time per instance
These metrics are logged periodically and on application shutdown, helping you monitor and optimize performance.
Example metrics log:
```
UnoServer metrics - Total: 120, Failed: 5, Success Rate: 95.83%
Conversions per instance:
[0] 127.0.0.1:2003 - Count: 32 (26.67%), Avg Time: 1250.45ms
[1] 127.0.0.1:2004 - Count: 30 (25.00%), Avg Time: 1187.33ms
[2] 127.0.0.1:2005 - Count: 29 (24.17%), Avg Time: 1345.78ms
[3] 127.0.0.1:2006 - Count: 29 (24.17%), Avg Time: 1290.12ms
```
## Testing Multiple Instances
To test that multiple UnoServer instances are working correctly:
1. Set `libreOfficeSessionLimit` to a value greater than 1 (e.g., 4)
2. Start the application
3. Check logs for messages like:
```
Initializing UnoServerManager with maxInstances: 4, useExternal: false
Starting UnoServer on 127.0.0.1:2003
Starting UnoServer on 127.0.0.1:2004
Starting UnoServer on 127.0.0.1:2005
Starting UnoServer on 127.0.0.1:2006
```
4. Submit multiple file conversion requests simultaneously
5. Observe in logs that different server instances are being used in a round-robin manner
6. Check the metrics logs to verify the load distribution across instances
## Performance Considerations
- Each UnoServer instance requires additional memory (typically 100-200 MB)
- Set the `libreOfficeSessionLimit` according to your server's available resources
- For most use cases, a value between 2-4 provides a good balance
- Larger values may improve concurrency but increase memory usage
- The circuit breaker pattern helps maintain system stability under high load
## Queue Management
Stirling-PDF now includes a queue management system for office document conversions:
### Queue Status API
The system exposes REST endpoints for checking conversion queue status:
- `GET /api/v1/queue/status` - Get status of all process queues
- `GET /api/v1/queue/unoserver` - Get detailed information about UnoServer instances and active tasks
- `GET /api/v1/queue/task/{taskId}` - Get status of a specific task by ID
Example response from `/api/v1/queue/unoserver`:
```json
{
"instanceCount": 4,
"activeTaskCount": 2,
"instances": [
{
"id": 0,
"host": "127.0.0.1",
"port": 2003,
"managed": true,
"running": true,
"available": true,
"failureCount": 0,
"averageConversionTimeMs": 1523.45,
"lastConversionTimeMs": 1498
},
...
],
"activeTasks": [
{
"id": "office-123",
"name": "Convert document.docx to PDF",
"status": "RUNNING",
"queuePosition": 0,
"queueTimeMs": 0,
"processTimeMs": 542,
"totalTimeMs": 542,
"errorMessage": null
},
...
]
}
```
### UI Integration
The system includes a built-in UI for monitoring conversion status:
1. A status indicator appears when a document is being converted
2. The indicator shows the current status, position in queue, and estimated wait time
3. For pages that use office conversions, a "Check Office Conversion Status" button provides detailed information about server instances and active conversions
## Troubleshooting
If you encounter issues with UnoServer:
1. Check logs for any error messages related to UnoServer startup
2. Look for health check logs to identify problematic instances
3. Verify ports are not already in use by other applications
4. Ensure LibreOffice is correctly installed
5. Check that UnoServer is properly installed and in your PATH:
```
which unoserver
which unoconvert
```
6. Try running a single UnoServer instance manually to check if it works:
```
/opt/venv/bin/unoserver --port 2003 --interface 127.0.0.1
```
7. Use the queue status API to check the status of UnoServer instances:
```
curl http://localhost:8080/api/v1/queue/unoserver
```
8. For external servers, verify network connectivity from the Stirling-PDF server
### Common Error Messages
| Error Message | Possible Cause | Solution |
|---------------|----------------|----------|
| "UnoServer is not available" | UnoServer executable not found | Install UnoServer or use the fat Docker image |
| "Failed to start UnoServer instance" | Port in use or LibreOffice issues | Check ports, restart application, or verify LibreOffice installation |
| "Circuit breaker opened for UnoServer instance" | Multiple conversion failures | Check logs for specific errors, verify UnoServer is working correctly |
| "No UnoServer instances available" | All instances are down or in circuit-open state | Restart application or check for resource issues |
## Environment Variables
When using Docker, you can configure UnoServer instances using environment variables:
- `LIBREOFFICE_SESSION_LIMIT`: Number of UnoServer instances to start
- `BASE_UNOCONV_PORT`: Base port number for UnoServer instances
- `USE_EXTERNAL_UNOCONVSERVERS`: Set to "true" to use external servers

5
src/main/java/stirling/software/SPDF/config/AppConfig.java
View File

@ -203,6 +203,11 @@ public class AppConfig {
return Boolean.getBoolean(env.getProperty("DISABLE_PIXEL"));
}
@Bean(name = "showQueueStatus")
public boolean showQueueStatus() {
return applicationProperties.getUi().isQueueStatusEnabled();
}
@Bean(name = "machineType")
public String determineMachineType() {
try {

19
src/main/java/stirling/software/SPDF/config/ConditionalOnUnoServerAvailable.java Normal file
View File

@ -0,0 +1,19 @@
package stirling.software.SPDF.config;
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import org.springframework.context.annotation.Conditional;
/**
* Annotation to conditionally enable components based on the availability of UnoServer. Components
* annotated with this will only be created if UnoServer is available on the system.
*/
@Target({ElementType.TYPE, ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Conditional(UnoServerAvailableCondition.class)
public @interface ConditionalOnUnoServerAvailable {}

193
src/main/java/stirling/software/SPDF/config/UnoServerAvailableCondition.java Normal file
View File

@ -0,0 +1,193 @@
package stirling.software.SPDF.config;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import org.springframework.context.annotation.Condition;
import org.springframework.context.annotation.ConditionContext;
import org.springframework.core.type.AnnotatedTypeMetadata;
import lombok.extern.slf4j.Slf4j;
/**
* Condition that checks if UnoServer is available on the system. This condition will pass if: 1.
* The unoserver executable is found via RuntimePathConfig (from unoConvertPath) 2. The unoserver
* executable is found at /opt/venv/bin/unoserver (Docker path) 3. The unoserver executable is found
* in PATH 4. The unoserver executable is found in any common installation directories
*/
@Slf4j
public class UnoServerAvailableCondition implements Condition {
// Common installation paths to check
private static final String[] COMMON_UNOSERVER_PATHS = {
"/opt/venv/bin/unoserver", // Docker path
"/usr/bin/unoserver", // Linux system path
"/usr/local/bin/unoserver", // Linux local path
"/opt/homebrew/bin/unoserver", // Mac Homebrew path
"/opt/libreoffice/program/unoserver" // Custom LibreOffice path
};
@Override
public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) {
log.info("Checking if UnoServer is available on the system...");
// Collect all paths to check
List<String> pathsToCheck = new ArrayList<>();
// Check for Docker environment
boolean isDocker = Files.exists(Path.of("/.dockerenv"));
log.debug("Docker environment detected: {}", isDocker);
// Try to get unoserver path from RuntimePathConfig first (highest priority)
String unoserverFromRuntimeConfig = getUnoServerPathFromRuntimeConfig(context);
if (unoserverFromRuntimeConfig != null) {
pathsToCheck.add(unoserverFromRuntimeConfig);
}
// Add common installation paths
for (String path : COMMON_UNOSERVER_PATHS) {
pathsToCheck.add(path);
}
// Add "unoserver" to check in PATH
pathsToCheck.add("unoserver");
// Try all paths one by one
for (String path : pathsToCheck) {
log.debug("Checking for UnoServer at: {}", path);
if (isExecutableAvailable(path)) {
log.info("UnoServer found at: {}, enabling UnoServerManager", path);
return true;
}
}
// If we get here, we didn't find unoserver anywhere
log.warn(
"UnoServer not found in any of the expected locations. UnoServerManager will be disabled.");
log.info(
"To enable Office document conversions, please install UnoServer or use the 'fat' Docker image variant.");
return false;
}
/**
* Attempts to get the unoserver path from RuntimePathConfig by checking the parent directory of
* unoConvertPath.
*
* @param context The condition context
* @return The unoserver path if found, null otherwise
*/
private String getUnoServerPathFromRuntimeConfig(ConditionContext context) {
try {
RuntimePathConfig runtimePathConfig =
context.getBeanFactory().getBean(RuntimePathConfig.class);
if (runtimePathConfig != null) {
String unoConvertPath = runtimePathConfig.getUnoConvertPath();
log.debug("UnoConvert path from RuntimePathConfig: {}", unoConvertPath);
if (unoConvertPath != null && !unoConvertPath.isEmpty()) {
// First check if unoConvertPath itself exists
File unoConvertFile = new File(unoConvertPath);
if (!unoConvertFile.exists() || !unoConvertFile.canExecute()) {
log.info("UnoConvert not found at path: {}", unoConvertPath);
return null;
}
// If unoConvertPath exists, check for unoserver in the same directory
Path unoConvertDir = Paths.get(unoConvertPath).getParent();
if (unoConvertDir != null) {
Path potentialUnoServerPath = unoConvertDir.resolve("unoserver");
File unoServerFile = potentialUnoServerPath.toFile();
if (unoServerFile.exists() && unoServerFile.canExecute()) {
log.debug("UnoServer found at: {}", potentialUnoServerPath);
return potentialUnoServerPath.toString();
} else {
log.debug(
"UnoServer not found at expected path: {}",
potentialUnoServerPath);
// Continue checking other paths
}
}
}
}
} catch (Exception e) {
log.debug(
"RuntimePathConfig not available yet, falling back to default checks: {}",
e.getMessage());
}
return null;
}
/**
* Comprehensive check if an executable is available in the system
*
* @param executableName The name or path of the executable to check
* @return true if the executable is found and executable, false otherwise
*/
private boolean isExecutableAvailable(String executableName) {
// First, check if it's an absolute path and the file exists
if (executableName.startsWith("/") || executableName.contains(":\\")) {
File file = new File(executableName);
boolean exists = file.exists() && file.canExecute();
log.debug(
"Checking executable at absolute path {}: {}",
executableName,
exists ? "Found" : "Not found");
return exists;
}
// Next, try to execute the command with --version to verify it works
try {
ProcessBuilder pb = new ProcessBuilder(executableName, "--version");
pb.redirectError(ProcessBuilder.Redirect.DISCARD);
Process process = pb.start();
int exitCode = process.waitFor();
if (exitCode == 0) {
log.debug("Executable {} exists in PATH (--version returned 0)", executableName);
return true;
} else {
// Try with --help as a fallback
pb = new ProcessBuilder(executableName, "--help");
pb.redirectError(ProcessBuilder.Redirect.DISCARD);
process = pb.start();
exitCode = process.waitFor();
if (exitCode == 0) {
log.debug("Executable {} exists in PATH (--help returned 0)", executableName);
return true;
}
}
} catch (Exception e) {
log.debug("Error checking for executable {}: {}", executableName, e.getMessage());
}
// Finally, check each directory in PATH for the executable file
if (!executableName.contains("/") && !executableName.contains("\\")) {
String pathEnv = System.getenv("PATH");
if (pathEnv != null) {
String[] pathDirs = pathEnv.split(File.pathSeparator);
for (String pathDir : pathDirs) {
File file = new File(pathDir, executableName);
if (file.exists() && file.canExecute()) {
log.debug(
"Found executable {} in PATH directory {}",
executableName,
pathDir);
return true;
}
}
}
}
log.debug("Executable {} not found", executableName);
return false;
}
}

887
src/main/java/stirling/software/SPDF/config/UnoServerManager.java Normal file
View File

@ -0,0 +1,887 @@
package stirling.software.SPDF.config;
import java.io.File;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import io.github.pixee.security.SystemCommand;
import jakarta.annotation.PostConstruct;
import jakarta.annotation.PreDestroy;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.ApplicationProperties;
import stirling.software.SPDF.utils.ConversionTask;
/**
* UnoServerManager is responsible for managing multiple instances of unoserver based on application
* configuration.
*
* <p>This component is only created if UnoServer is available on the system.
*/
@Slf4j
@Component
@ConditionalOnUnoServerAvailable
public class UnoServerManager {
private static final int INSTANCE_CHECK_TIMEOUT_MS = 1000;
private static final long INSTANCE_STARTUP_TIMEOUT_MS = 30000;
private static final long HEALTH_CHECK_INTERVAL_MS = 60000; // Health check every minute
@Autowired private ApplicationProperties properties;
@Autowired private RuntimePathConfig runtimePathConfig;
@Getter private List<ServerInstance> instances = new ArrayList<>();
private AtomicInteger currentInstanceIndex = new AtomicInteger(0);
private ScheduledExecutorService healthCheckExecutor;
// The path to the UnoServer executable that was found during initialization
private String detectedUnoServerPath;
// Circuit breaker settings for external servers
private static final int FAILURE_THRESHOLD = 3; // Number of failures before circuit opens
private static final long CIRCUIT_RESET_TIME_MS = 30000; // Time before retrying a failed server
// Performance metrics
private final AtomicInteger totalConversions = new AtomicInteger(0);
private final AtomicInteger failedConversions = new AtomicInteger(0);
private final Map<Integer, AtomicInteger> conversionsPerInstance = new ConcurrentHashMap<>();
// Queue tracking
private final ConcurrentHashMap<String, ConversionTask> activeTasks = new ConcurrentHashMap<>();
private final AtomicInteger taskIdCounter = new AtomicInteger(0);
private final Map<Integer, AtomicInteger> activeTasksPerInstance = new ConcurrentHashMap<>();
@PostConstruct
public void initialize() {
try {
int maxInstances =
properties.getProcessExecutor().getSessionLimit().getLibreOfficeSessionLimit();
boolean useExternal = properties.getProcessExecutor().isUseExternalUnoconvServers();
List<String> externalServers = properties.getProcessExecutor().getUnoconvServers();
int basePort = properties.getProcessExecutor().getBaseUnoconvPort();
boolean manageUnoServer = properties.getProcessExecutor().isManageUnoServer();
log.info(
"Initializing UnoServerManager with maxInstances: {}, useExternal: {}, externalServers: {}, unoConvertPath: {}, manageUnoServer: {}",
maxInstances,
useExternal,
externalServers,
runtimePathConfig.getUnoConvertPath(),
manageUnoServer);
// Get valid UnoServer executable path
String unoServerPath = findValidUnoServerPath();
if (unoServerPath == null) {
log.warn("UnoServer executable not found. Office conversions will be disabled.");
return;
}
log.info("Using UnoServer at: {}", unoServerPath);
// Store the path for use by server instances
this.detectedUnoServerPath = unoServerPath;
if (useExternal && !externalServers.isEmpty()) {
// Configure for external servers
for (String serverAddress : externalServers) {
String[] parts = serverAddress.split(":");
String host = parts[0];
int port = parts.length > 1 ? Integer.parseInt(parts[1]) : basePort;
ServerInstance instance = new ServerInstance(host, port, false);
instances.add(instance);
conversionsPerInstance.put(instances.size() - 1, new AtomicInteger(0));
activeTasksPerInstance.put(instances.size() - 1, new AtomicInteger(0));
}
log.info("Configured {} external UnoServer instances", instances.size());
} else if (manageUnoServer) {
// Configure for local instances only if manageUnoServer is true
boolean anyInstanceStarted = false;
for (int i = 0; i < maxInstances; i++) {
int port = basePort + i;
ServerInstance instance = new ServerInstance("127.0.0.1", port, true);
instances.add(instance);
conversionsPerInstance.put(i, new AtomicInteger(0));
activeTasksPerInstance.put(i, new AtomicInteger(0));
try {
instance.start();
anyInstanceStarted = true;
} catch (IOException e) {
log.warn(
"Failed to start UnoServer instance on port {}: {}",
port,
e.getMessage());
}
}
if (!anyInstanceStarted) {
log.warn(
"Failed to start any UnoServer instances. Office conversions may be affected.");
}
log.info("Started {} local UnoServer instances", instances.size());
} else {
log.info(
"Application is configured to not manage UnoServer instances. Assuming external management.");
}
// Start the health check scheduler
startHealthCheck();
// Log initial health status
logHealthStatus();
} catch (Exception e) {
log.warn("Failed to initialize UnoServerManager: {}", e.getMessage(), e);
}
}
/**
* Scans multiple locations to find a valid UnoServer executable
*
* @return Path to UnoServer if found, null otherwise
*/
private String findValidUnoServerPath() {
// Common paths to check for UnoServer
List<String> pathsToCheck = new ArrayList<>();
// Try to derive the path from unoConvertPath first (highest priority)
String unoConvertPath = runtimePathConfig.getUnoConvertPath();
if (unoConvertPath != null && !unoConvertPath.isEmpty()) {
File unoConvertFile = new File(unoConvertPath);
if (unoConvertFile.exists() && unoConvertFile.canExecute()) {
Path unoConvertDir = Paths.get(unoConvertPath).getParent();
if (unoConvertDir != null) {
Path potentialUnoServerPath = unoConvertDir.resolve("unoserver");
pathsToCheck.add(potentialUnoServerPath.toString());
}
} else {
log.warn("UnoConvert not found at configured path: {}", unoConvertPath);
}
}
// Add common installation paths
pathsToCheck.add("/opt/venv/bin/unoserver"); // Docker path
pathsToCheck.add("/usr/bin/unoserver"); // Linux system path
pathsToCheck.add("/usr/local/bin/unoserver"); // Linux local path
pathsToCheck.add("/opt/homebrew/bin/unoserver"); // Mac Homebrew path
pathsToCheck.add("/opt/libreoffice/program/unoserver"); // Custom LibreOffice path
// Check each path
for (String path : pathsToCheck) {
File file = new File(path);
if (file.exists() && file.canExecute()) {
log.info("Found valid UnoServer at: {}", path);
return path;
}
}
// If no absolute path works, try to find it in PATH
String pathEnv = System.getenv("PATH");
if (pathEnv != null) {
String[] pathDirs = pathEnv.split(File.pathSeparator);
for (String pathDir : pathDirs) {
File file = new File(pathDir, "unoserver");
if (file.exists() && file.canExecute()) {
log.info("Found UnoServer in PATH at: {}", file.getAbsolutePath());
return file.getAbsolutePath();
}
}
}
log.warn("UnoServer executable not found in any standard location");
return null;
}
/** Starts periodic health checks for all instances */
private void startHealthCheck() {
healthCheckExecutor = Executors.newSingleThreadScheduledExecutor();
healthCheckExecutor.scheduleAtFixedRate(
this::performHealthCheck,
HEALTH_CHECK_INTERVAL_MS,
HEALTH_CHECK_INTERVAL_MS,
TimeUnit.MILLISECONDS);
}
/** Perform health check on all instances */
private void performHealthCheck() {
log.debug("Running UnoServer health check for {} instances", instances.size());
int healthy = 0;
int unhealthy = 0;
for (int i = 0; i < instances.size(); i++) {
ServerInstance instance = instances.get(i);
boolean isRunning = instance.isRunning();
if (isRunning) {
healthy++;
instance.resetFailureCount(); // Reset failure count for healthy instance
} else {
unhealthy++;
log.warn(
"UnoServer instance {}:{} is not running",
instance.getHost(),
instance.getPort());
// For managed instances, try to restart if needed
if (instance.isManaged()) {
try {
instance.restartIfNeeded();
} catch (Exception e) {
log.error(
"Failed to restart UnoServer instance {}:{}",
instance.getHost(),
instance.getPort(),
e);
}
}
}
}
log.info("UnoServer health check: {} healthy, {} unhealthy instances", healthy, unhealthy);
// Log metrics periodically
logMetrics();
}
/** Logs the current health status of all instances */
private void logHealthStatus() {
StringBuilder status = new StringBuilder("UnoServer Instances Status:\n");
for (int i = 0; i < instances.size(); i++) {
ServerInstance instance = instances.get(i);
boolean isRunning = instance.isRunning();
int convCount = conversionsPerInstance.get(i).get();
status.append(
String.format(
" [%d] %s:%d - Status: %s, Managed: %s, Conversions: %d, Failures: %d\n",
i,
instance.getHost(),
instance.getPort(),
isRunning ? "RUNNING" : "DOWN",
instance.isManaged() ? "YES" : "NO",
convCount,
instance.getFailureCount()));
}
log.info(status.toString());
}
/** Logs performance metrics for UnoServer conversions */
private void logMetrics() {
int total = totalConversions.get();
int failed = failedConversions.get();
float successRate = total > 0 ? (float) (total - failed) / total * 100 : 0;
log.info(
"UnoServer metrics - Total: {}, Failed: {}, Success Rate: {:.2f}%",
total, failed, successRate);
// Log per-instance metrics
StringBuilder instanceMetrics = new StringBuilder("Conversions per instance:\n");
for (int i = 0; i < instances.size(); i++) {
ServerInstance instance = instances.get(i);
int count = conversionsPerInstance.get(i).get();
float percentage = total > 0 ? (float) count / total * 100 : 0;
instanceMetrics.append(
String.format(
" [%d] %s:%d - Count: %d (%.2f%%), Avg Time: %.2fms\n",
i,
instance.getHost(),
instance.getPort(),
count,
percentage,
instance.getAverageConversionTime()));
}
log.info(instanceMetrics.toString());
}
@PreDestroy
public void cleanup() {
log.info("Shutting down UnoServer instances and health check scheduler");
// Shutdown health check scheduler
if (healthCheckExecutor != null) {
healthCheckExecutor.shutdownNow();
}
// Shutdown all instances
for (ServerInstance instance : instances) {
instance.stop();
}
// Log final metrics
logMetrics();
}
/**
* Gets the next available server instance using load-balancing and circuit breaker pattern for
* fault tolerance
*
* @return The next UnoServer instance to use
*/
public ServerInstance getNextInstance() {
if (instances.isEmpty()) {
throw new IllegalStateException("No UnoServer instances available");
}
// First try to find a healthy instance with the least active tasks
int minActiveTasks = Integer.MAX_VALUE;
int selectedIndex = -1;
for (int i = 0; i < instances.size(); i++) {
ServerInstance instance = instances.get(i);
// Check if instance is available (not in circuit-open state)
if (instance.isAvailable() && instance.isRunning()) {
int activeTasks = activeTasksPerInstance.get(i).get();
// If this instance has fewer active tasks, select it
if (activeTasks < minActiveTasks) {
minActiveTasks = activeTasks;
selectedIndex = i;
// If we found an instance with no active tasks, use it immediately
if (minActiveTasks == 0) {
break;
}
}
}
}
// If we found a suitable instance, use it
if (selectedIndex >= 0) {
ServerInstance instance = instances.get(selectedIndex);
// Track this instance being selected
conversionsPerInstance.get(selectedIndex).incrementAndGet();
activeTasksPerInstance.get(selectedIndex).incrementAndGet();
totalConversions.incrementAndGet();
log.debug(
"Selected UnoServer instance {}:{} with {} active tasks",
instance.getHost(),
instance.getPort(),
minActiveTasks);
return instance;
}
// If all healthy instances are busy or no healthy instances found, use round-robin as
// fallback
log.warn(
"No available UnoServer instances found with good health. Using round-robin fallback.");
// Try to find any available instance using round-robin
for (int attempt = 0; attempt < instances.size(); attempt++) {
int index = currentInstanceIndex.getAndIncrement() % instances.size();
ServerInstance instance = instances.get(index);
// Check if the instance is available (circuit closed)
if (instance.isAvailable()) {
// Track this instance being selected
conversionsPerInstance.get(index).incrementAndGet();
activeTasksPerInstance.get(index).incrementAndGet();
totalConversions.incrementAndGet();
log.debug(
"Selected UnoServer instance {}:{} using round-robin fallback",
instance.getHost(),
instance.getPort());
return instance;
}
}
// Last resort - if all circuits are open, use the next instance anyway
int index = currentInstanceIndex.get() % instances.size();
ServerInstance instance = instances.get(index);
log.warn(
"All UnoServer instances are in circuit-open state. Using instance at {}:{} as fallback.",
instance.getHost(),
instance.getPort());
// Track metrics even for fallback case
conversionsPerInstance.get(index).incrementAndGet();
activeTasksPerInstance.get(index).incrementAndGet();
totalConversions.incrementAndGet();
return instance;
}
/**
* Creates a new task for tracking office conversions
*
* @param taskName A descriptive name for the task
* @param instance The server instance that will handle this task
* @return A unique task ID for tracking
*/
public String createTask(String taskName, ServerInstance instance) {
String taskId = "office-" + taskIdCounter.incrementAndGet();
ConversionTask task = new ConversionTask(taskName, taskId);
// Calculate queue position based on number of active tasks across all instances
int runningTasks = 0;
int availableInstances = 0;
for (int i = 0; i < instances.size(); i++) {
if (instances.get(i).isRunning() && instances.get(i).isAvailable()) {
availableInstances++;
runningTasks += activeTasksPerInstance.get(i).get();
}
}
// If all instances are busy, set a queue position
if (runningTasks >= availableInstances && availableInstances > 0) {
int queuePosition = runningTasks - availableInstances + 1;
task.setQueuePosition(queuePosition);
}
// Store the task in our tracking map
activeTasks.put(taskId, task);
// Find the instance index for updating metrics
for (int i = 0; i < instances.size(); i++) {
if (instances.get(i) == instance) {
activeTasksPerInstance.get(i).incrementAndGet();
break;
}
}
log.debug("Created task {} with ID {}", taskName, taskId);
return taskId;
}
/**
* Completes a task, updating metrics and removing it from active tasks
*
* @param taskId The task ID to complete
* @param instance The server instance that handled this task
* @param durationMs The time taken to complete the task in milliseconds
*/
public void completeTask(String taskId, ServerInstance instance, long durationMs) {
ConversionTask task = activeTasks.remove(taskId);
if (task != null) {
task.complete();
}
// Find the instance index for updating metrics
for (int i = 0; i < instances.size(); i++) {
if (instances.get(i) == instance) {
activeTasksPerInstance.get(i).decrementAndGet();
break;
}
}
// Record the success for circuit breaker and metrics
recordSuccess(instance, durationMs);
log.debug("Completed task with ID {}, duration: {}ms", taskId, durationMs);
}
/**
* Fails a task, updating metrics and removing it from active tasks
*
* @param taskId The task ID to fail
* @param instance The server instance that handled this task
* @param errorMessage The error message explaining the failure
*/
public void failTask(String taskId, ServerInstance instance, String errorMessage) {
ConversionTask task = activeTasks.remove(taskId);
if (task != null) {
task.fail(errorMessage);
}
// Find the instance index for updating metrics
for (int i = 0; i < instances.size(); i++) {
if (instances.get(i) == instance) {
activeTasksPerInstance.get(i).decrementAndGet();
break;
}
}
// Record the failure for circuit breaker and metrics
recordFailure(instance);
log.warn("Failed task with ID {}: {}", taskId, errorMessage);
}
/**
* Gets all active conversion tasks
*
* @return A map of task IDs to ConversionTask objects
*/
public Map<String, ConversionTask> getActiveTasks() {
return new HashMap<>(activeTasks);
}
/**
* Records a successful conversion for metrics
*
* @param instance The server instance that succeeded
* @param durationMs The time taken for the conversion in milliseconds
*/
public void recordSuccess(ServerInstance instance, long durationMs) {
instance.recordSuccess(durationMs);
}
/**
* Records a failed conversion for metrics and circuit breaker
*
* @param instance The server instance that failed
*/
public void recordFailure(ServerInstance instance) {
failedConversions.incrementAndGet();
instance.recordFailure();
}
/** Represents a single UnoServer instance with circuit breaker functionality */
public class ServerInstance {
@Getter private final String host;
@Getter private final int port;
@Getter private final boolean managed;
private ExecutorService executorService;
private Process process;
private boolean running = false;
// Circuit breaker state
private final AtomicInteger failureCount = new AtomicInteger(0);
private volatile Instant lastFailureTime = null;
private volatile boolean circuitOpen = false;
// Performance metrics
private final AtomicLong totalConversionTimeMs = new AtomicLong(0);
private final AtomicInteger conversionCount = new AtomicInteger(0);
private final AtomicLong lastConversionDuration = new AtomicLong(0);
public ServerInstance(String host, int port, boolean managed) {
this.host = host;
this.port = port;
this.managed = managed;
if (!managed) {
// For external servers, we assume they're running initially
this.running = true;
}
}
/** Gets the number of failures for circuit breaker */
public int getFailureCount() {
return failureCount.get();
}
/** Resets the failure count for circuit breaker */
public void resetFailureCount() {
failureCount.set(0);
circuitOpen = false;
}
/**
* Records a successful conversion
*
* @param durationMs The duration of the conversion in milliseconds
*/
public void recordSuccess(long durationMs) {
conversionCount.incrementAndGet();
totalConversionTimeMs.addAndGet(durationMs);
lastConversionDuration.set(durationMs);
// Reset failure count on success
resetFailureCount();
}
/** Records a conversion failure */
public void recordFailure() {
int currentFailures = failureCount.incrementAndGet();
lastFailureTime = Instant.now();
// Open circuit if threshold reached
if (currentFailures >= FAILURE_THRESHOLD) {
log.warn(
"Circuit breaker opened for UnoServer instance {}:{} after {} failures",
host,
port,
currentFailures);
circuitOpen = true;
}
}
/**
* Checks if this instance is available based on circuit breaker status
*
* @return true if available, false if circuit is open
*/
public boolean isAvailable() {
// If circuit is closed, instance is available
if (!circuitOpen) {
return true;
}
// If circuit is open but reset time has passed, try half-open state
if (lastFailureTime != null
&& Duration.between(lastFailureTime, Instant.now()).toMillis()
> CIRCUIT_RESET_TIME_MS) {
log.info("Circuit breaker half-open for UnoServer instance {}:{}", host, port);
circuitOpen = false;
return true;
}
// Circuit is open
return false;
}
/**
* Gets the average conversion time in milliseconds
*
* @return The average conversion time or 0 if no conversions yet
*/
public double getAverageConversionTime() {
int count = conversionCount.get();
return count > 0 ? (double) totalConversionTimeMs.get() / count : 0;
}
/**
* Gets the last conversion duration in milliseconds
*
* @return The last conversion duration
*/
public long getLastConversionDuration() {
return lastConversionDuration.get();
}
/**
* Checks if the UnoServer instance is running
*
* @return true if the server is accessible, false otherwise
*/
public boolean isRunning() {
try (Socket socket = new Socket()) {
socket.connect(new InetSocketAddress(host, port), INSTANCE_CHECK_TIMEOUT_MS);
return true;
} catch (Exception e) {
return false;
}
}
/**
* Starts the UnoServer if it's a managed instance
*
* @throws IOException if the server fails to start
*/
public synchronized void start() throws IOException {
if (!managed
|| (process != null && process.isAlive())
|| !properties.getProcessExecutor().isManageUnoServer()) {
return;
}
log.info("Starting UnoServer on {}:{}", host, port);
try {
// Use the detected UnoServer path from parent class
String unoServerPath = UnoServerManager.this.detectedUnoServerPath;
// If not available (shouldn't happen), try to determine it
if (unoServerPath == null || unoServerPath.isEmpty()) {
log.warn(
"detectedUnoServerPath is null, attempting to find unoserver executable");
unoServerPath = findUnoServerExecutable();
if (unoServerPath == null) {
throw new IOException(
"UnoServer executable not found. Cannot start server instance.");
}
}
log.debug("Using UnoServer executable: {}", unoServerPath);
// Create the command with the correct path and options
String command =
String.format("%s --port %d --interface %s", unoServerPath, port, host);
// Final verification that the executable exists and is executable
File executableFile = new File(unoServerPath);
if (!executableFile.exists() || !executableFile.canExecute()) {
throw new IOException(
"UnoServer executable not found or not executable at: "
+ executableFile.getAbsolutePath());
}
// Run the command
log.debug("Executing command: {}", command);
process = SystemCommand.runCommand(Runtime.getRuntime(), command);
// Start a background thread to monitor the process
executorService = Executors.newSingleThreadExecutor();
executorService.submit(
() -> {
try {
int exitCode = process.waitFor();
log.info(
"UnoServer process on port {} exited with code {}",
port,
exitCode);
running = false;
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
log.warn("UnoServer monitoring thread was interrupted", e);
}
});
// Wait for the server to start up with timeout
long startTime = System.currentTimeMillis();
boolean startupSuccess = false;
while (System.currentTimeMillis() - startTime < INSTANCE_STARTUP_TIMEOUT_MS) {
if (isRunning()) {
running = true;
startupSuccess = true;
log.info("UnoServer started successfully on {}:{}", host, port);
break;
}
// Check if process is still alive
if (process == null || !process.isAlive()) {
int exitCode = process != null ? process.exitValue() : -1;
log.warn(
"UnoServer process terminated prematurely with exit code: {}, continuing without it",
exitCode);
return;
}
try {
Thread.sleep(1000); // Check every second
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
log.warn("Interrupted while waiting for UnoServer to start", e);
return;
}
}
if (!startupSuccess) {
// Timeout occurred, clean up and log warning
if (process != null && process.isAlive()) {
process.destroy();
}
log.warn(
"Failed to start UnoServer within timeout period of {} seconds, continuing without it",
(INSTANCE_STARTUP_TIMEOUT_MS / 1000));
}
} catch (IOException e) {
log.warn("Failed to start UnoServer: {}, continuing without it", e.getMessage());
// Don't rethrow - continue without the server
}
}
/**
* Helper method to find the UnoServer executable
*
* @return Path to UnoServer executable or null if not found
*/
private String findUnoServerExecutable() {
// Try to derive from unoConvertPath first
String unoConvertPath = UnoServerManager.this.runtimePathConfig.getUnoConvertPath();
if (unoConvertPath != null && !unoConvertPath.isEmpty()) {
Path unoConvertDir = Paths.get(unoConvertPath).getParent();
if (unoConvertDir != null) {
Path potentialUnoServerPath = unoConvertDir.resolve("unoserver");
File unoServerFile = potentialUnoServerPath.toFile();
if (unoServerFile.exists() && unoServerFile.canExecute()) {
return potentialUnoServerPath.toString();
}
}
}
// Check common paths
String[] commonPaths = {
"/opt/venv/bin/unoserver", "/usr/bin/unoserver", "/usr/local/bin/unoserver"
};
for (String path : commonPaths) {
File file = new File(path);
if (file.exists() && file.canExecute()) {
return path;
}
}
return null;
}
/** Stops the UnoServer if it's a managed instance */
public synchronized void stop() {
if (!managed) {
return;
}
// Stop the monitoring thread
if (executorService != null) {
executorService.shutdownNow();
}
// Stop the server process
if (process != null && process.isAlive()) {
log.info("Stopping UnoServer on port {}", port);
process.destroy();
}
running = false;
}
/**
* Restarts the UnoServer if it's a managed instance and not running
*
* @return true if restart succeeded or wasn't needed, false otherwise
*/
public synchronized boolean restartIfNeeded() {
if (!managed || running || !properties.getProcessExecutor().isManageUnoServer()) {
return true;
}
try {
log.info("Attempting to restart UnoServer on {}:{}", host, port);
start();
return true;
} catch (IOException e) {
log.warn("Failed to restart UnoServer on port {}, continuing without it", port, e);
return false;
}
}
/**
* Gets the connection string for this instance
*
* @return A connection string in the format host:port
*/
public String getConnectionString() {
return host + ":" + port;
}
}
}

69
src/main/java/stirling/software/SPDF/config/UnoServerManagerFallback.java Normal file
View File

@ -0,0 +1,69 @@
package stirling.software.SPDF.config;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.utils.ConversionTask;
/**
* Fallback configuration for when UnoServerManager is not available. This will provide friendly
* error messages when users try to use LibreOffice conversion features without having UnoServer
* installed.
*/
@Configuration
@Slf4j
public class UnoServerManagerFallback {
/**
* Creates a bean that provides a friendly error message when LibreOffice conversion is
* attempted but UnoServer is not available.
*/
@Bean
@ConditionalOnMissingBean(UnoServerManager.class)
public UnoServerNotAvailableHandler unoServerNotAvailableHandler(
RuntimePathConfig runtimePathConfig) {
log.info("UnoServer is not available. Office document conversions will be disabled.");
log.info("If you need Office document conversions, please install UnoServer.");
log.info("For Docker users, use the 'fat' image variant which includes UnoServer.");
// Log the path where we would expect to find unoconvert
if (runtimePathConfig != null) {
log.info("Expected unoconvert path: {}", runtimePathConfig.getUnoConvertPath());
}
return new UnoServerNotAvailableHandler();
}
/**
* Handler that provides friendly error messages when LibreOffice conversion is attempted but
* UnoServer is not available.
*/
public static class UnoServerNotAvailableHandler {
/** Method that throws a friendly exception when office conversions are attempted. */
public void throwUnoServerNotAvailableException() {
throw new UnoServerNotAvailableException(
"UnoServer (LibreOffice) is not available. Office document conversions are disabled. "
+ "To enable this feature, please install UnoServer or use the 'fat' Docker image variant.");
}
/** Creates a failed conversion task with a friendly error message. */
public ConversionTask createFailedTask(String taskName) {
ConversionTask task = new ConversionTask(taskName, (String) null);
task.fail(
"UnoServer (LibreOffice) is not available. Office document conversions are disabled.");
return task;
}
}
/** Exception thrown when UnoServer features are used but UnoServer is not available. */
public static class UnoServerNotAvailableException extends RuntimeException {
private static final long serialVersionUID = 1L;
public UnoServerNotAvailableException(String message) {
super(message);
}
}
}

221
src/main/java/stirling/software/SPDF/controller/api/QueueStatusController.java Normal file
View File

@ -0,0 +1,221 @@
package stirling.software.SPDF.controller.api;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.config.UnoServerManager;
import stirling.software.SPDF.config.UnoServerManager.ServerInstance;
import stirling.software.SPDF.utils.ConversionTask;
import stirling.software.SPDF.utils.ProcessExecutor;
import stirling.software.SPDF.utils.ProcessExecutor.Processes;
/** Controller for checking status of process queues */
@RestController
@RequestMapping("/api/v1/queue")
@Slf4j
public class QueueStatusController {
@Autowired(required = false)
private UnoServerManager unoServerManager;
/**
* Get the status of all process queues
*
* @return Map of queue statuses by process type
*/
@GetMapping("/status")
public ResponseEntity<Map<String, QueueStatus>> getAllQueueStatuses() {
Map<String, QueueStatus> statuses = new HashMap<>();
// Add statuses for all ProcessExecutor process types
for (Processes processType : Processes.values()) {
ProcessExecutor executor = ProcessExecutor.getInstance(processType);
QueueStatus status = new QueueStatus();
status.setProcessType(processType.name());
status.setActiveCount(executor.getActiveTaskCount());
status.setQueuedCount(executor.getQueueLength());
statuses.put(processType.name(), status);
}
// Add UnoServer status if available
if (unoServerManager != null) {
QueueStatus status = new QueueStatus();
status.setProcessType("UNOSERVER");
// Get active tasks from UnoServerManager
Map<String, ConversionTask> activeTasks = unoServerManager.getActiveTasks();
status.setActiveCount(activeTasks.size());
status.setQueuedCount(0); // UnoServer tasks are immediately processed
statuses.put("UNOSERVER", status);
}
return ResponseEntity.ok(statuses);
}
/**
* Get the status of a specific process queue
*
* @param processType The process type
* @return Queue status for the specified process
*/
@GetMapping("/status/{processType}")
public ResponseEntity<QueueStatus> getQueueStatus(@PathVariable String processType) {
try {
Processes process = Processes.valueOf(processType.toUpperCase());
ProcessExecutor executor = ProcessExecutor.getInstance(process);
QueueStatus status = new QueueStatus();
status.setProcessType(process.name());
status.setActiveCount(executor.getActiveTaskCount());
status.setQueuedCount(executor.getQueueLength());
return ResponseEntity.ok(status);
} catch (IllegalArgumentException e) {
return ResponseEntity.badRequest().build();
}
}
/**
* Get status of a specific task
*
* @param taskId The task ID
* @return Task status or 404 if not found
*/
@GetMapping("/task/{taskId}")
public ResponseEntity<TaskInfo> getTaskStatus(@PathVariable String taskId) {
// Try to find the task in any process executor
for (Processes processType : Processes.values()) {
ProcessExecutor executor = ProcessExecutor.getInstance(processType);
ConversionTask task = executor.getTask(taskId);
if (task != null) {
return ResponseEntity.ok(convertToTaskInfo(task));
}
}
// Check UnoServer tasks if available
if (unoServerManager != null && taskId.startsWith("office-")) {
Map<String, ConversionTask> unoTasks = unoServerManager.getActiveTasks();
ConversionTask task = unoTasks.get(taskId);
if (task != null) {
// Calculate queue position for UnoServer tasks
if (task.getStatus() == ConversionTask.TaskStatus.QUEUED) {
int queuePosition = 0;
for (ConversionTask otherTask : unoTasks.values()) {
if (otherTask.getStatus() == ConversionTask.TaskStatus.QUEUED
&& otherTask.getCreatedTime().isBefore(task.getCreatedTime())) {
queuePosition++;
}
}
// Set queue position
task.setQueuePosition(queuePosition + 1);
}
return ResponseEntity.ok(convertToTaskInfo(task));
}
}
return ResponseEntity.notFound().build();
}
/**
* Get queue status for a specific client ID
*
* @param clientId The client-generated ID for the task
* @return Queue status with position for the specific client task
*/
@GetMapping("/status/client/{clientId}")
public ResponseEntity<Map<String, QueueStatus>> getQueueStatusForClient(@PathVariable String clientId) {
Map<String, QueueStatus> result = new HashMap<>();
boolean foundMatch = false;
// Check each process type for the client ID
for (Processes processType : Processes.values()) {
ProcessExecutor executor = ProcessExecutor.getInstance(processType);
List<ConversionTask> queuedTasks = executor.getQueuedTasks();
// Find the position of the client's task in this queue
for (ConversionTask task : queuedTasks) {
// If we find a match for this client's task
if (task.getId().equals(clientId)) {
QueueStatus status = new QueueStatus();
status.setProcessType(processType.name());
status.setActiveCount(executor.getActiveTaskCount());
status.setQueuedCount(task.getQueuePosition());
result.put(processType.name(), status);
foundMatch = true;
break; // Exit loop once found - we only need one match
}
}
if (foundMatch) break; // Exit process type loop if we've found the task
}
// If no matching task found in process executors, check UnoServer
if (!foundMatch && unoServerManager != null) {
Map<String, ConversionTask> unoTasks = unoServerManager.getActiveTasks();
ConversionTask task = unoTasks.get(clientId);
if (task != null) {
QueueStatus status = new QueueStatus();
status.setProcessType("UNOSERVER");
status.setActiveCount(unoTasks.size());
status.setQueuedCount(0); // UnoServer tasks are immediately processed
result.put("UNOSERVER", status);
}
}
return ResponseEntity.ok(result);
}
/** Convert a ConversionTask to TaskInfo DTO */
private TaskInfo convertToTaskInfo(ConversionTask task) {
TaskInfo info = new TaskInfo();
info.setId(task.getId());
info.setStatus(task.getStatus().name());
info.setQueuePosition(task.getQueuePosition());
return info;
}
/** DTO for queue status */
@Data
@NoArgsConstructor
@AllArgsConstructor
public static class QueueStatus {
private String processType;
private int activeCount;
private int queuedCount;
}
/** DTO for task information */
@Data
@NoArgsConstructor
@AllArgsConstructor
public static class TaskInfo {
private String id;
private String status;
private int queuePosition;
}
}

17
src/main/java/stirling/software/SPDF/controller/api/UserController.java
View File

@ -171,16 +171,19 @@ public class UserController {
* Updates the user settings based on the provided JSON payload.
*
* @param updates A map containing the settings to update. The expected structure is:
* <ul>
* <li><b>emailNotifications</b> (optional): "true" or "false" - Enable or disable email notifications.</li>
* <li><b>theme</b> (optional): "light" or "dark" - Set the user's preferred theme.</li>
* <li><b>language</b> (optional): A string representing the preferred language (e.g., "en", "fr").</li>
* </ul>
* Keys not listed above will be ignored.
* <ul>
* <li><b>emailNotifications</b> (optional): "true" or "false" - Enable or disable email
* notifications.
* <li><b>theme</b> (optional): "light" or "dark" - Set the user's preferred theme.
* <li><b>language</b> (optional): A string representing the preferred language (e.g.,
* "en", "fr").
* </ul>
* Keys not listed above will be ignored.
* @param principal The currently authenticated user.
* @return A redirect string to the account page after updating the settings.
* @throws SQLException If a database error occurs.
* @throws UnsupportedProviderException If the operation is not supported for the user's provider.
* @throws UnsupportedProviderException If the operation is not supported for the user's
* provider.
*/
public String updateUserSettings(@RequestBody Map<String, String> updates, Principal principal)
throws SQLException, UnsupportedProviderException {

203
src/main/java/stirling/software/SPDF/controller/api/converters/ConvertOfficeController.java
View File

@ -10,6 +10,7 @@ import java.util.List;
import org.apache.commons.io.FilenameUtils;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.PostMapping;
@ -21,25 +22,62 @@ import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.config.RuntimePathConfig;
import stirling.software.SPDF.config.UnoServerManager;
import stirling.software.SPDF.config.UnoServerManager.ServerInstance;
import stirling.software.SPDF.config.UnoServerManagerFallback;
import stirling.software.SPDF.model.ApplicationProperties;
import stirling.software.SPDF.model.api.GeneralFile;
import stirling.software.SPDF.service.CustomPDFDocumentFactory;
import stirling.software.SPDF.utils.ConversionTask;
import stirling.software.SPDF.utils.ProcessExecutor;
import stirling.software.SPDF.utils.ProcessExecutor.ProcessExecutorResult;
import stirling.software.SPDF.utils.WebResponseUtils;
@Slf4j
@RestController
@Tag(name = "Convert", description = "Convert APIs")
@RequestMapping("/api/v1/convert")
@RequiredArgsConstructor
public class ConvertOfficeController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final RuntimePathConfig runtimePathConfig;
private final UnoServerManager unoServerManager;
private final ApplicationProperties applicationProperties;
@Autowired
public ConvertOfficeController(
CustomPDFDocumentFactory pdfDocumentFactory,
RuntimePathConfig runtimePathConfig,
ApplicationProperties applicationProperties,
@Autowired(required = false) UnoServerManager unoServerManager,
@Autowired(required = false)
UnoServerManagerFallback.UnoServerNotAvailableHandler
unoServerNotAvailableHandler) {
this.pdfDocumentFactory = pdfDocumentFactory;
this.runtimePathConfig = runtimePathConfig;
this.unoServerManager = unoServerManager;
this.applicationProperties = applicationProperties;
// Log appropriate message based on UnoServer availability
if (unoServerManager == null) {
log.warn("UnoServer is not available. Office document conversions will be disabled.");
if (unoServerNotAvailableHandler == null) {
log.error("UnoServerNotAvailableHandler is also missing! This should not happen.");
}
} else {
log.info("UnoServer is available. Office document conversions are enabled.");
}
}
public File convertToPdf(MultipartFile inputFile) throws IOException, InterruptedException {
return convertToPdf(inputFile, null);
}
public File convertToPdf(MultipartFile inputFile, String[] taskIdHolder)
throws IOException, InterruptedException {
// Check for valid file extension
String originalFilename = Filenames.toSimpleFileName(inputFile.getOriginalFilename());
if (originalFilename == null
@ -47,6 +85,13 @@ public class ConvertOfficeController {
throw new IllegalArgumentException("Invalid file extension");
}
// Check if UnoServer is available
if (unoServerManager == null) {
throw new UnoServerManagerFallback.UnoServerNotAvailableException(
"UnoServer (LibreOffice) is not available. Office document conversions are disabled. "
+ "To enable this feature, please install UnoServer or use the 'fat' Docker image variant.");
}
// Save the uploaded file to a temporary location
Path tempInputFile =
Files.createTempFile("input_", "." + FilenameUtils.getExtension(originalFilename));
@ -55,24 +100,95 @@ public class ConvertOfficeController {
// Prepare the output file path
Path tempOutputFile = Files.createTempFile("output_", ".pdf");
// Get the next available UnoServer instance
ServerInstance serverInstance = unoServerManager.getNextInstance();
// Create a task for tracking this conversion
String taskName = "Convert " + originalFilename + " to PDF";
String taskId = unoServerManager.createTask(taskName, serverInstance);
// Store the task ID for the caller if requested
if (taskIdHolder != null) {
taskIdHolder[0] = taskId;
}
log.info(
"Converting file {} using UnoServer instance at {}:{} (taskId: {})",
originalFilename,
serverInstance.getHost(),
serverInstance.getPort(),
taskId);
long startTime = System.currentTimeMillis();
try {
// Run the LibreOffice command
// If it's a managed instance and not running, try to restart it
if (!serverInstance.isRunning()) {
log.warn(
"UnoServer instance at {}:{} is not running, attempting restart",
serverInstance.getHost(),
serverInstance.getPort());
if (!serverInstance.restartIfNeeded()) {
unoServerManager.failTask(
taskId, serverInstance, "Failed to start UnoServer instance");
throw new IOException("Failed to start UnoServer instance for conversion");
}
}
// Run the LibreOffice command with the selected server
List<String> command =
new ArrayList<>(
Arrays.asList(
runtimePathConfig.getUnoConvertPath(),
"--port",
"2003",
String.valueOf(serverInstance.getPort()),
"--host",
serverInstance.getHost(),
"--convert-to",
"pdf",
tempInputFile.toString(),
tempOutputFile.toString()));
ProcessExecutorResult returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE)
.runCommandWithOutputHandling(command);
// Read the converted PDF file
return tempOutputFile.toFile();
log.debug("Running command: {}", String.join(" ", command));
try {
// Execute the command with a named task
ProcessExecutorResult result =
ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE)
.runCommandWithTask(command, "Convert " + originalFilename + " to PDF");
// Calculate duration and mark task as complete
long duration = System.currentTimeMillis() - startTime;
unoServerManager.completeTask(taskId, serverInstance, duration);
log.info(
"Successfully converted file {} using UnoServer instance {}:{} in {}ms (taskId: {})",
originalFilename,
serverInstance.getHost(),
serverInstance.getPort(),
duration,
taskId);
// Read the converted PDF file
return tempOutputFile.toFile();
} catch (IOException | InterruptedException e) {
// Mark task as failed
unoServerManager.failTask(taskId, serverInstance, e.getMessage());
log.error(
"Failed to convert file {} using UnoServer instance {}:{}: {}",
originalFilename,
serverInstance.getHost(),
serverInstance.getPort(),
e.getMessage());
throw e;
}
} catch (Exception e) {
// Mark task as failed if any other exception occurs
unoServerManager.failTask(taskId, serverInstance, e.getMessage());
throw e;
} finally {
// Clean up the temporary files
if (tempInputFile != null) Files.deleteIfExists(tempInputFile);
@ -92,19 +208,72 @@ public class ConvertOfficeController {
+ " Output:PDF Type:SISO")
public ResponseEntity<byte[]> processFileToPDF(@ModelAttribute GeneralFile generalFile)
throws Exception {
// Check if UnoServer is available first to provide a friendly error message
if (unoServerManager == null) {
return WebResponseUtils.errorResponseWithMessage(
"UnoServer (LibreOffice) is not available. Office document conversions are disabled. "
+ "To enable this feature, please install UnoServer or use the 'fat' Docker image variant.");
}
MultipartFile inputFile = generalFile.getFileInput();
// unused but can start server instance if startup time is to long
// LibreOfficeListener.getInstance().start();
File file = null;
String[] taskIdHolder = new String[1]; // Holder for task ID
try {
file = convertToPdf(inputFile);
// Call the conversion method to do the actual conversion
file = convertToPdf(inputFile, taskIdHolder);
PDDocument doc = pdfDocumentFactory.load(file);
return WebResponseUtils.pdfDocToWebResponse(
doc,
Filenames.toSimpleFileName(inputFile.getOriginalFilename())
.replaceFirst("[.][^.]+$", "")
+ "_convertedToPDF.pdf");
// Get the ProcessExecutorResult to extract the task ID
String processTaskId = null;
if (file != null && file.exists()) {
// Extract any ProcessExecutor task ID that might have been created
// This is a bit of a hack but will work for demonstration
ProcessExecutor processExecutor = ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE);
List<ConversionTask> activeTasks = processExecutor.getActiveTasks();
List<ConversionTask> queuedTasks = processExecutor.getQueuedTasks();
// Look for a task that matches our file name
String filename = Filenames.toSimpleFileName(inputFile.getOriginalFilename());
for (ConversionTask task : activeTasks) {
if (task.getTaskName().contains(filename)) {
processTaskId = task.getId();
break;
}
}
if (processTaskId == null) {
for (ConversionTask task : queuedTasks) {
if (task.getTaskName().contains(filename)) {
processTaskId = task.getId();
break;
}
}
}
}
// Get the response builder from WebResponseUtils
ResponseEntity.BodyBuilder responseBuilder =
WebResponseUtils.getResponseBuilder(
Filenames.toSimpleFileName(inputFile.getOriginalFilename())
.replaceFirst("[.][^.]+$", "")
+ "_convertedToPDF.pdf");
// Add headers for task tracking
if (taskIdHolder[0] != null) {
responseBuilder.header("X-Task-Id", taskIdHolder[0]);
}
if (processTaskId != null) {
responseBuilder.header("X-Process-Task-Id", processTaskId);
}
// Return the response with all available headers
return responseBuilder.body(WebResponseUtils.getBytesFromPDDocument(doc));
} catch (UnoServerManagerFallback.UnoServerNotAvailableException e) {
return WebResponseUtils.errorResponseWithMessage(e.getMessage());
} finally {
if (file != null) file.delete();
}

3
src/main/java/stirling/software/SPDF/controller/api/pipeline/PipelineProcessor.java
View File

@ -221,7 +221,8 @@ public class PipelineProcessor {
return result;
}
/* package */ ResponseEntity<byte[]> sendWebRequest(String url, MultiValueMap<String, Object> body) {
/* package */ ResponseEntity<byte[]> sendWebRequest(
String url, MultiValueMap<String, Object> body) {
RestTemplate restTemplate = new RestTemplate();
// Set up headers, including API key
HttpHeaders headers = new HttpHeaders();

9
src/main/java/stirling/software/SPDF/model/ApplicationProperties.java
View File

@ -359,6 +359,7 @@ public class ApplicationProperties {
private String homeDescription;
private String appNameNavbar;
private List<String> languages;
private Boolean showQueueStatus;
public String getAppName() {
return appName != null && appName.trim().length() > 0 ? appName : null;
@ -375,6 +376,10 @@ public class ApplicationProperties {
? appNameNavbar
: null;
}
public boolean isQueueStatusEnabled() {
return showQueueStatus == null || showQueueStatus; // Default to true if not specified
}
}
@Data
@ -501,6 +506,10 @@ public class ApplicationProperties {
public static class ProcessExecutor {
private SessionLimit sessionLimit = new SessionLimit();
private TimeoutMinutes timeoutMinutes = new TimeoutMinutes();
private List<String> unoconvServers = new ArrayList<>();
private boolean useExternalUnoconvServers = false;
private int baseUnoconvPort = 2003;
private boolean manageUnoServer = true;
@Data
public static class SessionLimit {

188
src/main/java/stirling/software/SPDF/utils/ConversionTask.java Normal file
View File

@ -0,0 +1,188 @@
package stirling.software.SPDF.utils;
import java.time.Duration;
import java.time.Instant;
import java.util.UUID;
import lombok.Getter;
import lombok.Setter;
/**
* Represents a task being processed by the ProcessExecutor. Used for tracking queue position and
* execution status.
*/
@Getter
public class ConversionTask {
public enum TaskStatus {
QUEUED,
RUNNING,
COMPLETED,
FAILED,
CANCELLED
}
private final String id;
private final String taskName;
private final Instant createdTime;
private final ProcessExecutor.Processes processType;
@Setter private volatile int queuePosition;
private volatile Instant startTime;
private volatile Instant endTime;
private volatile TaskStatus status;
private volatile String errorMessage;
private volatile Thread executingThread;
/**
* Creates a new conversion task
*
* @param taskName A descriptive name for the task
* @param processType The type of process executing the task
*/
public ConversionTask(String taskName, ProcessExecutor.Processes processType) {
this.id = UUID.randomUUID().toString();
this.taskName = taskName;
this.processType = processType;
this.createdTime = Instant.now();
this.status = TaskStatus.QUEUED;
}
/**
* Creates a new conversion task with a custom ID
*
* @param taskName A descriptive name for the task
* @param customId A custom ID for the task (can be null to generate a random UUID)
*/
public ConversionTask(String taskName, String customId) {
this.id = (customId != null) ? customId : UUID.randomUUID().toString();
this.taskName = taskName;
this.processType = null; // No process type for custom tasks
this.createdTime = Instant.now();
this.status = TaskStatus.QUEUED;
}
/** Marks the task as running */
public void start(Thread executingThread) {
this.startTime = Instant.now();
this.status = TaskStatus.RUNNING;
this.executingThread = executingThread;
}
/** Marks the task as completed */
public void complete() {
this.endTime = Instant.now();
this.status = TaskStatus.COMPLETED;
this.executingThread = null;
}
/**
* Marks the task as failed
*
* @param errorMessage The error message
*/
public void fail(String errorMessage) {
this.endTime = Instant.now();
this.status = TaskStatus.FAILED;
this.errorMessage = errorMessage;
this.executingThread = null;
}
/** Marks the task as cancelled */
public void cancel() {
this.endTime = Instant.now();
this.status = TaskStatus.CANCELLED;
this.executingThread = null;
}
/** Attempts to cancel the task if it's running */
public void attemptCancel() {
if (this.status == TaskStatus.RUNNING && executingThread != null) {
executingThread.interrupt();
} else {
cancel();
}
}
/**
* Gets the time spent in queue
*
* @return Queue time in milliseconds
*/
public long getQueueTimeMs() {
if (startTime == null) {
return Duration.between(createdTime, Instant.now()).toMillis();
}
return Duration.between(createdTime, startTime).toMillis();
}
/**
* Gets the processing time
*
* @return Processing time in milliseconds
*/
public long getProcessingTimeMs() {
if (startTime == null) {
return 0;
}
if (endTime == null) {
return Duration.between(startTime, Instant.now()).toMillis();
}
return Duration.between(startTime, endTime).toMillis();
}
/**
* Gets the total time from task creation to completion or now
*
* @return Total time in milliseconds
*/
public long getTotalTimeMs() {
if (endTime == null) {
return Duration.between(createdTime, Instant.now()).toMillis();
}
return Duration.between(createdTime, endTime).toMillis();
}
/**
* Gets a formatted string of queue time
*
* @return Formatted time
*/
public String getFormattedQueueTime() {
return formatDuration(getQueueTimeMs());
}
/**
* Gets a formatted string of processing time
*
* @return Formatted time
*/
public String getFormattedProcessingTime() {
return formatDuration(getProcessingTimeMs());
}
/**
* Gets a formatted string of total time
*
* @return Formatted time
*/
public String getFormattedTotalTime() {
return formatDuration(getTotalTimeMs());
}
/**
* Formats milliseconds as a readable duration
*
* @param ms Milliseconds
* @return Formatted string
*/
private String formatDuration(long ms) {
if (ms < 1000) {
return ms + "ms";
}
if (ms < 60000) {
return String.format("%.1fs", ms / 1000.0);
}
return String.format("%dm %ds", ms / 60000, (ms % 60000) / 1000);
}
}

575
src/main/java/stirling/software/SPDF/utils/ProcessExecutor.java
View File

@ -7,145 +7,419 @@ import java.io.InputStreamReader;
import java.io.InterruptedIOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Queue;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import io.github.pixee.security.BoundedLineReader;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.ApplicationProperties;
@Slf4j
@Component
public class ProcessExecutor {
private static final Map<Processes, ProcessExecutor> instances = new ConcurrentHashMap<>();
private static ApplicationProperties applicationProperties = new ApplicationProperties();
private final Semaphore semaphore;
private final boolean liveUpdates;
private long timeoutDuration;
private static ApplicationProperties applicationProperties;
private Semaphore semaphore;
private boolean liveUpdates = true;
private long timeoutDuration = 10; // Default timeout of 10 minutes
private ProcessExecutor(int semaphoreLimit, boolean liveUpdates, long timeout) {
@Autowired
public void setApplicationProperties(ApplicationProperties applicationProperties) {
ProcessExecutor.applicationProperties = applicationProperties;
// Initialize instances if not already done
initializeExecutorInstances();
}
/**
* Initialize all executor instances with the application properties This ensures that the
* static instances are correctly configured after application startup
*/
private void initializeExecutorInstances() {
if (applicationProperties != null) {
// Pre-initialize all process types
for (Processes type : Processes.values()) {
getInstance(type);
}
log.info("Initialized ProcessExecutor instances for all process types");
}
}
@Autowired
public ProcessExecutor() {
this.processType = null; // This instance is just for Spring DI
this.semaphore = new Semaphore(1); // Default to 1 permit
}
private ProcessExecutor(
Processes processType, int semaphoreLimit, boolean liveUpdates, long timeout) {
this.processType = processType;
this.semaphore = new Semaphore(semaphoreLimit);
this.liveUpdates = liveUpdates;
this.timeoutDuration = timeout;
}
// Task tracking
private Processes processType;
private final Queue<ConversionTask> queuedTasks = new ConcurrentLinkedQueue<>();
private final Map<String, ConversionTask> activeTasks = new ConcurrentHashMap<>();
private final Map<String, ConversionTask> completedTasks = new ConcurrentHashMap<>();
private static final int MAX_COMPLETED_TASKS = 100; // Maximum number of completed tasks to keep
// Metrics
private final AtomicInteger totalTasksProcessed = new AtomicInteger(0);
private final AtomicInteger failedTasks = new AtomicInteger(0);
private final AtomicInteger totalQueueTime = new AtomicInteger(0);
private final AtomicInteger totalProcessTime = new AtomicInteger(0);
// For testing - allows injecting a mock
private static ProcessExecutor mockInstance;
public static void setStaticMockInstance(ProcessExecutor mock) {
mockInstance = mock;
}
public static ProcessExecutor getInstance(Processes processType) {
return getInstance(processType, true);
}
public static ProcessExecutor getInstance(Processes processType, boolean liveUpdates) {
// For testing - return the mock if set
if (mockInstance != null) {
return mockInstance;
}
return instances.computeIfAbsent(
processType,
key -> {
int semaphoreLimit =
switch (key) {
case LIBRE_OFFICE ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getLibreOfficeSessionLimit();
case PDFTOHTML ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getPdfToHtmlSessionLimit();
case PYTHON_OPENCV ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getPythonOpenCvSessionLimit();
case WEASYPRINT ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getWeasyPrintSessionLimit();
case INSTALL_APP ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getInstallAppSessionLimit();
case TESSERACT ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getTesseractSessionLimit();
case QPDF ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getQpdfSessionLimit();
case CALIBRE ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getCalibreSessionLimit();
};
int semaphoreLimit = 1; // Default if applicationProperties is null
long timeoutMinutes = 10; // Default if applicationProperties is null
long timeoutMinutes =
switch (key) {
case LIBRE_OFFICE ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getLibreOfficeTimeoutMinutes();
case PDFTOHTML ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getPdfToHtmlTimeoutMinutes();
case PYTHON_OPENCV ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getPythonOpenCvTimeoutMinutes();
case WEASYPRINT ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getWeasyPrintTimeoutMinutes();
case INSTALL_APP ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getInstallAppTimeoutMinutes();
case TESSERACT ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getTesseractTimeoutMinutes();
case QPDF ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getQpdfTimeoutMinutes();
case CALIBRE ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getCalibreTimeoutMinutes();
};
return new ProcessExecutor(semaphoreLimit, liveUpdates, timeoutMinutes);
if (applicationProperties != null) {
semaphoreLimit =
switch (key) {
case LIBRE_OFFICE ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getLibreOfficeSessionLimit();
case PDFTOHTML ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getPdfToHtmlSessionLimit();
case PYTHON_OPENCV ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getPythonOpenCvSessionLimit();
case WEASYPRINT ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getWeasyPrintSessionLimit();
case INSTALL_APP ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getInstallAppSessionLimit();
case TESSERACT ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getTesseractSessionLimit();
case QPDF ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getQpdfSessionLimit();
case CALIBRE ->
applicationProperties
.getProcessExecutor()
.getSessionLimit()
.getCalibreSessionLimit();
};
timeoutMinutes =
switch (key) {
case LIBRE_OFFICE ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getLibreOfficeTimeoutMinutes();
case PDFTOHTML ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getPdfToHtmlTimeoutMinutes();
case PYTHON_OPENCV ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getPythonOpenCvTimeoutMinutes();
case WEASYPRINT ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getWeasyPrintTimeoutMinutes();
case INSTALL_APP ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getInstallAppTimeoutMinutes();
case TESSERACT ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getTesseractTimeoutMinutes();
case QPDF ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getQpdfTimeoutMinutes();
case CALIBRE ->
applicationProperties
.getProcessExecutor()
.getTimeoutMinutes()
.getCalibreTimeoutMinutes();
};
}
return new ProcessExecutor(key, semaphoreLimit, liveUpdates, timeoutMinutes);
});
}
/**
* Creates a new conversion task and adds it to the queue
*
* @param taskName A descriptive name for the task
* @return The created conversion task
*/
public ConversionTask createTask(String taskName) {
ConversionTask task = new ConversionTask(taskName, this.processType);
queuedTasks.add(task);
updateQueuePositions();
log.debug(
"Created new task {} for {} process, queue position: {}",
task.getId(),
processType,
task.getQueuePosition());
return task;
}
/**
* Gets a task by its ID
*
* @param taskId The task ID
* @return The task or null if not found
*/
public ConversionTask getTask(String taskId) {
// Check active tasks first
ConversionTask task = activeTasks.get(taskId);
if (task != null) {
return task;
}
// Check queued tasks
for (ConversionTask queuedTask : queuedTasks) {
if (queuedTask.getId().equals(taskId)) {
return queuedTask;
}
}
// Check completed tasks
return completedTasks.get(taskId);
}
/**
* Gets all tasks for this process type
*
* @return List of all tasks
*/
public List<ConversionTask> getAllTasks() {
List<ConversionTask> allTasks = new ArrayList<>();
allTasks.addAll(queuedTasks);
allTasks.addAll(activeTasks.values());
allTasks.addAll(completedTasks.values());
return allTasks;
}
/**
* Gets all active tasks for this process type
*
* @return List of active tasks
*/
public List<ConversionTask> getActiveTasks() {
return new ArrayList<>(activeTasks.values());
}
/**
* Gets all queued tasks for this process type
*
* @return List of queued tasks
*/
public List<ConversionTask> getQueuedTasks() {
return new ArrayList<>(queuedTasks);
}
/**
* Gets the current queue length
*
* @return Number of tasks in queue
*/
public int getQueueLength() {
return queuedTasks.size();
}
/**
* Gets the number of active tasks
*
* @return Number of tasks currently running
*/
public int getActiveTaskCount() {
return activeTasks.size();
}
/**
* Gets the maximum number of concurrent tasks
*
* @return Maximum concurrent tasks
*/
public int getMaxConcurrentTasks() {
return semaphore.availablePermits() + semaphore.getQueueLength();
}
/**
* Gets the estimated wait time based on current queue and average processing time
*
* @return Estimated wait time in milliseconds
*/
public long getEstimatedWaitTimeMs() {
if (queuedTasks.isEmpty()) {
return 0;
}
int processed = totalTasksProcessed.get();
if (processed == 0) {
return 30000; // Default 30 seconds if no data
}
double avgProcessTime = totalProcessTime.get() / (double) processed;
int activeCount = activeTasks.size();
int maxConcurrent = semaphore.availablePermits() + semaphore.getQueueLength();
int queueLength = queuedTasks.size();
// Calculate how many queue cycles are needed
double cycles = Math.ceil(queueLength / (double) maxConcurrent);
// Estimate wait time
return (long) (avgProcessTime * cycles);
}
/** Updates the queue positions for all queued tasks */
private synchronized void updateQueuePositions() {
int position = 0;
for (ConversionTask task : queuedTasks) {
task.setQueuePosition(++position);
}
}
/**
* Run a command with a task for queue tracking
*
* @param command The command to execute
* @param taskName A descriptive name for the task
* @return The result of the execution
*/
public ProcessExecutorResult runCommandWithTask(List<String> command, String taskName)
throws IOException, InterruptedException {
return runCommandWithTask(command, null, taskName);
}
/**
* Run a command without creating a task (direct execution)
*
* @param command The command to execute
* @return The result of the execution
*/
public ProcessExecutorResult runCommand(List<String> command)
throws IOException, InterruptedException {
return runCommandWithTask(command, "Unnamed command");
}
/**
* Run a command with a task for queue tracking
*
* @param command The command to execute
* @param workingDirectory The working directory
* @param taskName A descriptive name for the task
* @return The result of the execution
*/
public ProcessExecutorResult runCommandWithTask(
List<String> command, File workingDirectory, String taskName)
throws IOException, InterruptedException {
// Create and track the task
ConversionTask task = createTask(taskName);
try {
return runCommandWithOutputHandling(command, workingDirectory, task);
} catch (Exception e) {
task.fail(e.getMessage());
throw e;
}
}
/** Legacy method for backwards compatibility */
public ProcessExecutorResult runCommandWithOutputHandling(List<String> command)
throws IOException, InterruptedException {
return runCommandWithOutputHandling(command, null);
}
/** Legacy method for backwards compatibility */
public ProcessExecutorResult runCommandWithOutputHandling(
List<String> command, File workingDirectory) throws IOException, InterruptedException {
return runCommandWithOutputHandling(command, workingDirectory, null);
}
/** Main method to run a command and handle its output */
private ProcessExecutorResult runCommandWithOutputHandling(
List<String> command, File workingDirectory, ConversionTask task)
throws IOException, InterruptedException {
String messages = "";
int exitCode = 1;
semaphore.acquire();
try {
log.info("Running command: " + String.join(" ", command));
// If no task was provided, create an anonymous one
boolean createdTask = false;
if (task == null) {
task = createTask("Anonymous " + processType + " task");
createdTask = true;
}
// Wait for a permit from the semaphore (this is where queuing happens)
semaphore.acquire();
// Task is now running
task.start(Thread.currentThread());
queuedTasks.remove(task);
activeTasks.put(task.getId(), task);
updateQueuePositions(); // Update queue positions for remaining tasks
try {
log.info("Running command for task {}: {}", task.getId(), String.join(" ", command));
ProcessBuilder processBuilder = new ProcessBuilder(command);
// Use the working directory if it's set
@ -264,10 +538,92 @@ public class ProcessExecutor {
+ messages);
}
}
// Task completed successfully
task.complete();
totalTasksProcessed.incrementAndGet();
totalProcessTime.addAndGet((int) task.getProcessingTimeMs());
totalQueueTime.addAndGet((int) task.getQueueTimeMs());
// Move from active to completed
activeTasks.remove(task.getId());
addToCompletedTasks(task);
log.debug(
"Task {} completed in {}ms (queue: {}ms, processing: {}ms)",
task.getId(),
task.getTotalTimeMs(),
task.getQueueTimeMs(),
task.getProcessingTimeMs());
} catch (Exception e) {
// Task failed
task.fail(e.getMessage());
failedTasks.incrementAndGet();
// Move from active to completed
activeTasks.remove(task.getId());
addToCompletedTasks(task);
log.error(
"Task {} failed after {}ms (queue: {}ms, processing: {}ms): {}",
task.getId(),
task.getTotalTimeMs(),
task.getQueueTimeMs(),
task.getProcessingTimeMs(),
e.getMessage());
throw e;
} finally {
semaphore.release();
// For anonymous tasks, don't keep them in completed tasks
if (createdTask) {
completedTasks.remove(task.getId());
}
}
return new ProcessExecutorResult(exitCode, messages, task.getId());
}
/** Adds a task to the completed tasks map, maintaining size limit */
private synchronized void addToCompletedTasks(ConversionTask task) {
// Add the task to completed tasks
completedTasks.put(task.getId(), task);
// If we exceed the limit, remove oldest completed tasks
if (completedTasks.size() > MAX_COMPLETED_TASKS) {
List<ConversionTask> oldestTasks =
completedTasks.values().stream()
.sorted(Comparator.comparing(ConversionTask::getEndTime))
.limit(completedTasks.size() - MAX_COMPLETED_TASKS)
.collect(Collectors.toList());
for (ConversionTask oldTask : oldestTasks) {
completedTasks.remove(oldTask.getId());
}
}
}
/** Periodically log queue statistics (once per minute) */
@Scheduled(fixedRate = 60000)
public void logQueueStatistics() {
if (!queuedTasks.isEmpty() || !activeTasks.isEmpty()) {
int total = totalTasksProcessed.get();
int failed = failedTasks.get();
float successRate = total > 0 ? (float) (total - failed) / total * 100 : 0;
float avgQueueTime = total > 0 ? (float) totalQueueTime.get() / total : 0;
float avgProcessTime = total > 0 ? (float) totalProcessTime.get() / total : 0;
log.info(
"{} queue status: Active={}, Queued={}, Completed={}, AvgQueue={}ms, AvgProcess={}ms, SuccessRate={:.2f}%",
processType,
activeTasks.size(),
queuedTasks.size(),
total,
avgQueueTime,
avgProcessTime,
successRate);
}
return new ProcessExecutorResult(exitCode, messages);
}
public enum Processes {
@ -281,29 +637,20 @@ public class ProcessExecutor {
QPDF
}
@Getter
public class ProcessExecutorResult {
int rc;
String messages;
private final int rc;
private final String messages;
private final String taskId;
public ProcessExecutorResult(int rc, String messages) {
this(rc, messages, null);
}
public ProcessExecutorResult(int rc, String messages, String taskId) {
this.rc = rc;
this.messages = messages;
}
public int getRc() {
return rc;
}
public void setRc(int rc) {
this.rc = rc;
}
public String getMessages() {
return messages;
}
public void setMessages(String messages) {
this.messages = messages;
this.taskId = taskId;
}
}
}

49
src/main/java/stirling/software/SPDF/utils/WebResponseUtils.java
View File

@ -66,4 +66,53 @@ public class WebResponseUtils {
return boasToWebResponse(baos, docName);
}
/**
* Gets a response builder with appropriate headers for the given filename
*
* @param filename The filename to use in the Content-Disposition header
* @return A ResponseEntity.BodyBuilder with appropriate headers
* @throws IOException If encoding the filename fails
*/
public static ResponseEntity.BodyBuilder getResponseBuilder(String filename)
throws IOException {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_PDF);
String encodedFilename =
URLEncoder.encode(filename, StandardCharsets.UTF_8.toString())
.replaceAll("\\+", "%20");
headers.setContentDispositionFormData("attachment", encodedFilename);
return ResponseEntity.ok().headers(headers);
}
/**
* Converts a PDDocument to a byte array
*
* @param document The PDDocument to convert
* @return The document as a byte array
* @throws IOException If saving the document fails
*/
public static byte[] getBytesFromPDDocument(PDDocument document) throws IOException {
try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
document.save(baos);
return baos.toByteArray();
} finally {
document.close();
}
}
/**
* Creates an error response with a message
*
* @param message The error message
* @return A ResponseEntity with the error message
*/
public static ResponseEntity<byte[]> errorResponseWithMessage(String message) {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
String jsonError = "{\"error\":\"" + message.replace("\"", "\\\"") + "\"}";
return new ResponseEntity<>(
jsonError.getBytes(StandardCharsets.UTF_8), headers, HttpStatus.BAD_REQUEST);
}
}

7
src/main/resources/messages_en_GB.properties
View File

@ -1437,3 +1437,10 @@ cookieBanner.preferencesModal.necessary.description=These cookies are essential
cookieBanner.preferencesModal.analytics.title=Analytics
cookieBanner.preferencesModal.analytics.description=These cookies help us understand how our tools are being used, so we can focus on building the features our community values most. Rest assured—Stirling PDF cannot and will never track the content of the documents you work with.
####################
# Queue Status #
####################
queue.positionInQueue=Position in queue: {0}
queue.processing=Processing your file...
queue.readyShortly=Your file will be ready shortly

5
src/main/resources/settings.yml.template
View File

@ -125,6 +125,7 @@ ui:
homeDescription: '' # short description or tagline shown on the homepage
appNameNavbar: '' # name displayed on the navigation bar
languages: [] # If empty, all languages are enabled. To display only German and Polish ["de_DE", "pl_PL"]. British English is always enabled.
showQueueStatus: true # set to 'false' to disable the queue status indicator
endpoints:
toRemove: [] # list endpoints to disable (e.g. ['img-to-pdf', 'remove-pages'])
@ -157,3 +158,7 @@ processExecutor:
installApptimeoutMinutes: 60
calibretimeoutMinutes: 30
tesseractTimeoutMinutes: 30
unoconvServers: [] # List of external unoconv servers in the format ["hostname:port", "hostname:port"]. Leave empty to use local instances.
useExternalUnoconvServers: false # Set to true to use external servers from the list above
baseUnoconvPort: 2003 # Base port for local unoconv instances (will increment by 1 for each instance)
manageUnoServer: true # Set to true to let the application manage UnoServer instances

421
src/main/resources/static/js/queueStatus.js Normal file
View File

@ -0,0 +1,421 @@
/**
* Queue Status JS
* Simple queue position indicator with continuous polling
*/
class QueueStatusTracker {
constructor() {
this.activeTaskIds = new Map();
this.basePollingInterval = 5000; // 5 seconds between polls by default
this.pollingTimeoutId = null;
this.initialized = false;
this.isPolling = false;
this.maxDisplayTime = 20 * 60 * 1000; // 20 minutes max display time
this.initialDelayMs = 1000; // 1 second delay before showing position
this.apiErrorCount = 0; // Track consecutive API errors
this.lastQueuePosition = 0; // Track last known queue position
console.log('[QueueStatusTracker] Constructor called');
}
/**
* Calculate polling interval based on queue position
* - Position <= 5: Poll every 3 seconds
* - Position <= 20: Poll every 5 seconds
* - Position > 20: Poll every 10 seconds
* @param {number} position Current queue position
* @returns {number} Polling interval in milliseconds
*/
getPollingInterval(position) {
if (position <= 5) {
return 3000; // 3 seconds for closer positions
} else if (position <= 20) {
return 5000; // 5 seconds for medium distance
} else {
return 10000; // 10 seconds for far positions
}
}
/**
* Initialize the queue status tracker
*/
init() {
if (this.initialized) return;
this.initialized = true;
console.log('[QueueStatusTracker] Initializing queue tracker');
// Add CSS to head
const style = document.createElement('style');
style.textContent = `
.queue-status-container {
margin-top: 20px;
width: 100%;
font-family: sans-serif;
}
.queue-position-info {
background-color: var(--md-sys-color-surface-container, #fff);
border: 1px solid var(--md-sys-color-outline-variant, #ddd);
border-radius: 5px;
box-shadow: var(--md-sys-elevation-1, 0 2px 5px rgba(0,0,0,0.15));
margin-top: 10px;
padding: 12px;
text-align: center;
font-weight: bold;
color: var(--md-sys-color-on-surface, #000);
border-left: 4px solid var(--md-sys-color-primary, #0060aa);
animation: queue-status-fade-in 0.3s ease-in-out;
transition: background-color 0.3s ease;
}
@keyframes queue-status-fade-in {
from { opacity: 0; transform: translateY(-10px); }
to { opacity: 1; transform: translateY(0); }
}
.queue-position-info.processing {
background-color: var(--md-sys-color-primary-container, #d0e4ff);
}
`;
document.head.appendChild(style);
console.log('[QueueStatusTracker] Styles injected into <head>');
}
/**
* Create container for queue status if it doesn't exist
*/
ensureContainer() {
// Find container or create it if not present
let container = document.getElementById('queueStatusContainer');
if (container) {
console.log('[QueueStatusTracker] Found existing queue container');
return container;
}
console.log('[QueueStatusTracker] Creating new queue container');
container = document.createElement('div');
container.id = 'queueStatusContainer';
container.className = 'queue-status-container';
container.style.display = 'none';
// Try to insert after the form
const form = document.querySelector('form[action="/api/v1/convert/file/pdf"]');
if (form) {
console.log('[QueueStatusTracker] Found form, inserting container after it');
form.parentNode.insertBefore(container, form.nextSibling);
} else {
// Fall back to appending to body
console.log('[QueueStatusTracker] No form found, appending to body');
document.body.appendChild(container);
}
return container;
}
/**
* Generate a unique client task ID
*/
generateClientTaskId() {
return 'client-' + Math.random().toString(36).substring(2, 11);
}
/**
* Track a task with polling for status
* @param {string} clientTaskId - Client-generated task ID
*/
trackTask(clientTaskId) {
console.log(`[QueueStatusTracker] Starting to track task: ${clientTaskId}`);
this.init();
// Initialize container and elements
const container = this.ensureContainer();
// Wait a short delay before showing anything
setTimeout(() => {
container.style.display = 'block';
console.log('[QueueStatusTracker] Queue status container is now visible (after delay)');
// Create or get the position info element
let positionInfo = document.getElementById('queuePositionInfo');
if (!positionInfo) {
positionInfo = document.createElement('div');
positionInfo.id = 'queuePositionInfo';
positionInfo.className = 'queue-position-info';
// Add message content with HTML
// We'll use the global i18n variables defined in common.html
const positionMessageTemplate = typeof queuePositionInQueue !== 'undefined' ?
queuePositionInQueue : 'Position in queue: {0}';
positionInfo.innerHTML = `<span id="queuePositionText">${positionMessageTemplate.replace('{0}', '<span id="queuePosition">...</span>')}</span>`;
container.appendChild(positionInfo);
console.log('[QueueStatusTracker] Created position info element');
}
}, this.initialDelayMs);
// Store the task data
this.activeTaskIds.set(clientTaskId, {
clientId: clientTaskId,
addedTime: Date.now(),
position: 0, // Initialize with unknown position
active: true
});
console.log(`[QueueStatusTracker] Added task ${clientTaskId} to active tasks`);
// Start polling to get and update the queue position
this.startPolling();
// Set maximum display time
setTimeout(() => {
console.log(`[QueueStatusTracker] Maximum display time reached for ${clientTaskId}`);
this.removeAllTasks();
}, this.maxDisplayTime);
}
/**
* Update the queue position by polling the server for only the specific client task
*/
updateTotalQueuePosition() {
console.log('[QueueStatusTracker] Updating queue position...');
// If we don't have any active task IDs, there's nothing to track
if (this.activeTaskIds.size === 0) {
console.log('[QueueStatusTracker] No active tasks to track');
return;
}
// Get the client ID of the first task (we only track one at a time)
const clientId = this.activeTaskIds.keys().next().value;
console.log(`[QueueStatusTracker] Fetching status for client task: ${clientId}`);
// Fetch queue status for only this specific client task
fetch(`/api/v1/queue/status/client/${clientId}`)
.then(response => {
if (!response.ok) {
throw new Error(`Failed to get queue status: ${response.status}`);
}
return response.json();
})
.then(data => {
console.log('[QueueStatusTracker] Got client queue status:', data);
// Reset error count on successful API call
this.apiErrorCount = 0;
// Determine which queue type to use based on the form that was submitted
let queuePosition = 0;
const lastSubmittedAction = localStorage.getItem('lastSubmittedFormAction');
// Find the appropriate processor based on the form action
if (data && Object.keys(data).length > 0) {
// Just use the position from the first (and likely only) process type returned
// The backend has already filtered to just the relevant processor
const firstProcessType = Object.keys(data)[0];
queuePosition = data[firstProcessType].queuedCount || 0;
} else {
// If no data returned for our specific client task, assume it's being processed
queuePosition = 0;
}
console.log(`[QueueStatusTracker] Queue position for client task: ${queuePosition}`);
// Update last known position
this.lastQueuePosition = queuePosition;
// If position is 0, it's being processed now, show the processing message
if (queuePosition === 0) {
console.log('[QueueStatusTracker] Task is being processed (position 0), showing processing message');
// Show processing message
const processingMessage = typeof queueProcessing !== 'undefined' ?
queueProcessing : 'Processing your file...';
const positionTextElem = document.getElementById('queuePositionText');
const positionInfo = document.getElementById('queuePositionInfo');
if (positionTextElem) {
positionTextElem.textContent = processingMessage;
}
if (positionInfo) {
positionInfo.classList.add('processing');
}
// After 5 seconds, show the "ready shortly" message
setTimeout(() => {
const readyMessage = typeof queueReadyShortly !== 'undefined' ?
queueReadyShortly : 'Your file will be ready shortly';
if (positionTextElem) {
positionTextElem.textContent = readyMessage;
}
// After another 5 seconds, hide the message
setTimeout(() => {
this.removeAllTasks();
}, 5000);
}, 5000);
return;
}
// Update the UI with position in queue
const positionElem = document.getElementById('queuePosition');
if (positionElem) {
// Just update the position number
positionElem.textContent = queuePosition;
}
// Store position in the client task
const taskData = this.activeTaskIds.get(clientId);
if (taskData) {
taskData.position = queuePosition;
}
})
.catch(error => {
console.error('[QueueStatusTracker] Error getting queue status:', error);
// If we've had more than 3 consecutive failures, remove the queue indicator
this.apiErrorCount = (this.apiErrorCount || 0) + 1;
if (this.apiErrorCount > 3) {
console.warn('[QueueStatusTracker] Too many API failures, removing queue indicator');
this.removeAllTasks();
return;
}
// Otherwise just keep the previous position display
console.warn('[QueueStatusTracker] API error, keeping previous position display');
});
}
/**
* Remove all tasks and clean up
*/
removeAllTasks() {
console.log('[QueueStatusTracker] Removing all tasks');
// Clear tasks map
this.activeTaskIds.clear();
// Stop polling
this.stopPolling();
// Hide container
const container = document.getElementById('queueStatusContainer');
if (container) {
container.style.display = 'none';
console.log('[QueueStatusTracker] Hidden queue status container');
}
}
/**
* Start polling for queue status
*/
startPolling() {
if (this.isPolling) {
console.log('[QueueStatusTracker] Polling already active');
return;
}
this.isPolling = true;
console.log('[QueueStatusTracker] Starting polling');
// Poll every few seconds
const poll = () => {
if (this.activeTaskIds.size === 0) {
this.stopPolling();
return;
}
// Update queue positions
this.updateTotalQueuePosition();
// Calculate polling interval based on position
const interval = this.getPollingInterval(this.lastQueuePosition);
// Schedule next poll with dynamic interval
console.log(`[QueueStatusTracker] Next poll in ${interval/1000} seconds (position: ${this.lastQueuePosition})`);
this.pollingTimeoutId = setTimeout(poll, interval);
};
// First poll after a short delay to allow the system to process the request
setTimeout(() => {
this.updateTotalQueuePosition();
// Then start regular polling with initial base interval
this.pollingTimeoutId = setTimeout(poll, this.basePollingInterval);
}, this.initialDelayMs);
}
/**
* Stop polling
*/
stopPolling() {
if (!this.isPolling) return;
console.log('[QueueStatusTracker] Stopping polling');
this.isPolling = false;
if (this.pollingTimeoutId) {
clearTimeout(this.pollingTimeoutId);
this.pollingTimeoutId = null;
}
}
}
// Create global instance
const queueStatusTracker = new QueueStatusTracker();
console.log('[QueueStatusTracker] Global instance created');
// Add submit event handler to show queue position when form is submitted
document.addEventListener('submit', function(event) {
const form = event.target;
// Check if this is a conversion or processing form
// We need to track more API endpoints that might use the queue
if (form && (
// Main API categories
form.action.includes('/api/v1/convert') ||
form.action.includes('/api/v1/file/pdf') ||
form.action.includes('/api/v1/compress') ||
form.action.includes('/api/v1/ocr') ||
form.action.includes('/api/v1/extract') ||
form.action.includes('/api/v1/misc') ||
form.action.includes('/api/v1/pipeline') ||
// HTML/PDF conversions
form.action.includes('/api/v1/convert/html/pdf') ||
form.action.includes('/api/v1/convert/pdf/html') ||
// Image extraction
form.action.includes('/api/v1/extract/image/scans') ||
// URL and Markdown
form.action.includes('/api/v1/convert/url/pdf') ||
form.action.includes('/api/v1/convert/markdown/pdf') ||
// Office conversions
form.action.includes('/api/v1/convert/pdf/docx') ||
form.action.includes('/api/v1/convert/pdf/doc') ||
form.action.includes('/api/v1/convert/pdf/odt') ||
form.action.includes('/api/v1/convert/pdf/ppt') ||
form.action.includes('/api/v1/convert/pdf/pptx') ||
form.action.includes('/api/v1/convert/pdf/odp') ||
form.action.includes('/api/v1/convert/pdf/rtf') ||
form.action.includes('/api/v1/convert/pdf/xml') ||
form.action.includes('/api/v1/convert/pdf/pdfa') ||
// Calibre conversions
form.action.includes('/api/v1/convert/pdf/epub') ||
form.action.includes('/api/v1/convert/pdf/mobi')
)) {
console.log('[QueueStatusTracker] Form submission detected:', form.action);
// Store the form action for later use in determining queue type
localStorage.setItem('lastSubmittedFormAction', form.action);
// Generate a client task ID
const clientTaskId = queueStatusTracker.generateClientTaskId();
// Start tracking the task
queueStatusTracker.trackTask(clientTaskId);
console.log(`[QueueStatusTracker] Tracking form submission with ID: ${clientTaskId}`);
}
});
console.log('[QueueStatusTracker] Form submit event listener installed');

8
src/main/resources/templates/fragments/common.html
View File

@ -21,7 +21,7 @@
<meta name="msapplication-TileColor" content="#00aba9">
<meta name="theme-color" content="#ffffff">
<script>
<script th:inline="javascript">
window.stirlingPDF = window.stirlingPDF || {};
</script>
<script th:src="@{'/js/thirdParty/pdf-lib.min.js'}"></script>
@ -86,6 +86,7 @@
<script th:src="@{'/js/tab-container.js'}"></script>
<script th:src="@{'/js/darkmode.js'}"></script>
<script th:src="@{'/js/csrf.js'}"></script>
<script th:if="${@showQueueStatus}" th:src="@{'/js/queueStatus.js'}"></script>
<script th:inline="javascript">
function UpdatePosthogConsent(){
@ -121,6 +122,11 @@
const cookieBannerPreferencesModalAnalyticsTitle = /*[[#{cookieBanner.preferencesModal.analytics.title}]]*/ "";
const cookieBannerPreferencesModalAnalyticsDescription = /*[[#{cookieBanner.preferencesModal.analytics.description}]]*/ "";
// Queue Status messages
const queuePositionInQueue = /*[[#{queue.positionInQueue}]]*/ "Position in queue: {0}";
const queueProcessing = /*[[#{queue.processing}]]*/ "Processing your file...";
const queueReadyShortly = /*[[#{queue.readyShortly}]]*/ "Your file will be ready shortly";
if (analyticsEnabled) {
!function (t, e) {
var o, n, p, r;

37
src/test/java/stirling/software/SPDF/controller/api/pipeline/PipelineProcessorTest.java
View File

@ -16,6 +16,7 @@ import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.util.LinkedMultiValueMap;
import jakarta.servlet.ServletContext;
@ -26,14 +27,11 @@ import stirling.software.SPDF.model.PipelineResult;
@ExtendWith(MockitoExtension.class)
class PipelineProcessorTest {
@Mock
ApiDocService apiDocService;
@Mock ApiDocService apiDocService;
@Mock
UserServiceInterface userService;
@Mock UserServiceInterface userService;
@Mock
ServletContext servletContext;
@Mock ServletContext servletContext;
PipelineProcessor pipelineProcessor;
@ -50,27 +48,34 @@ class PipelineProcessorTest {
PipelineConfig config = new PipelineConfig();
config.setOperations(List.of(op));
Resource file = new ByteArrayResource("data".getBytes()) {
@Override
public String getFilename() {
return "test.pdf";
}
};
Resource file =
new ByteArrayResource("data".getBytes()) {
@Override
public String getFilename() {
return "test.pdf";
}
};
List<Resource> files = List.of(file);
when(apiDocService.isMultiInput("filter-page-count")).thenReturn(false);
when(apiDocService.getExtensionTypes(false, "filter-page-count")).thenReturn(List.of("pdf"));
when(apiDocService.getExtensionTypes(false, "filter-page-count"))
.thenReturn(List.of("pdf"));
// Mock the sendWebRequest method to return an empty response body with OK status
LinkedMultiValueMap<String, Object> expectedBody = new LinkedMultiValueMap<>();
expectedBody.add("fileInput", file);
doReturn(new ResponseEntity<>(new byte[0], HttpStatus.OK))
.when(pipelineProcessor)
.sendWebRequest(anyString(), any());
.sendWebRequest(contains("filter-page-count"), any());
PipelineResult result = pipelineProcessor.runPipelineAgainstFiles(files, config);
assertTrue(result.isFiltersApplied(), "Filter flag should be true when operation filters file");
assertTrue(
result.isFiltersApplied(),
"Filter flag should be true when operation filters file");
assertFalse(result.isHasErrors(), "No errors should occur");
assertTrue(result.getOutputFiles().isEmpty(), "Filtered file list should be empty");
}
}

30
src/test/java/stirling/software/SPDF/utils/FileToPdfTest.java
View File

@ -1,38 +1,14 @@
package stirling.software.SPDF.utils;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import java.io.IOException;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.SPDF.model.api.converters.HTMLToPdfRequest;
@ExtendWith(MockitoExtension.class)
public class FileToPdfTest {
/**
* Test the HTML to PDF conversion. This test expects an IOException when an empty HTML input is
* provided.
*/
@Test
public void testConvertHtmlToPdf() {
HTMLToPdfRequest request = new HTMLToPdfRequest();
byte[] fileBytes = new byte[0]; // Sample file bytes (empty input)
String fileName = "test.html"; // Sample file name indicating an HTML file
boolean disableSanitize = false; // Flag to control sanitization
// Expect an IOException to be thrown due to empty input
Throwable thrown =
assertThrows(
IOException.class,
() ->
FileToPdf.convertHtmlToPdf(
"/path/", request, fileBytes, fileName, disableSanitize));
assertNotNull(thrown);
}
/**
* Test sanitizeZipFilename with null or empty input. It should return an empty string in these
* cases.

61
src/test/java/stirling/software/SPDF/utils/ProcessExecutorTest.java
View File

@ -1,62 +1,31 @@
package stirling.software.SPDF.utils;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.junit.jupiter.MockitoExtension;
@ExtendWith(MockitoExtension.class)
public class ProcessExecutorTest {
private ProcessExecutor processExecutor;
@BeforeEach
public void setUp() {
// Initialize the ProcessExecutor instance
processExecutor = ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE);
}
@Test
public void testRunCommandWithOutputHandling() throws IOException, InterruptedException {
// Mock the command to execute
List<String> command = new ArrayList<>();
command.add("java");
command.add("-version");
// Execute the command
public void testProcessExecutorResult() {
// Test the ProcessExecutorResult class
ProcessExecutor.ProcessExecutorResult result =
processExecutor.runCommandWithOutputHandling(command);
new ProcessExecutor().new ProcessExecutorResult(0, "Success message", "task-123");
// Check the exit code and output messages
assertEquals(0, result.getRc());
assertNotNull(result.getMessages()); // Check if messages are not null
}
assertEquals(0, result.getRc(), "Exit code should be 0");
assertEquals("Success message", result.getMessages(), "Messages should match");
assertEquals("task-123", result.getTaskId(), "Task ID should match");
@Test
public void testRunCommandWithOutputHandling_Error() {
// Mock the command to execute
List<String> command = new ArrayList<>();
command.add("nonexistent-command");
// Test constructor without taskId
ProcessExecutor.ProcessExecutorResult resultNoTask =
new ProcessExecutor().new ProcessExecutorResult(1, "Error message");
// Execute the command and expect an IOException
IOException thrown =
assertThrows(
IOException.class,
() -> {
processExecutor.runCommandWithOutputHandling(command);
});
// Check the exception message to ensure it indicates the command was not found
String errorMessage = thrown.getMessage();
assertTrue(
errorMessage.contains("error=2")
|| errorMessage.contains("No such file or directory"),
"Unexpected error message: " + errorMessage);
assertEquals(1, resultNoTask.getRc(), "Exit code should be 1");
assertEquals("Error message", resultNoTask.getMessages(), "Messages should match");
assertTrue(resultNoTask.getTaskId() == null, "Task ID should be null");
}
}

16
testing/allEndpointsRemovedSettings.yml
View File

@ -6,7 +6,6 @@
# ___) || | | || _ <| |___ | || |\ | |_| |_____| __/| |_| | _| #
# |____/ |_| |___|_| \_\_____|___|_| \_|\____| |_| |____/|_| #
# #
# Custom setting.yml file with all endpoints disabled to only be used for testing purposes #
# Do not comment out any entry, it will be removed on next startup #
# If you want to override with environment parameter follow parameter naming SECURITY_INITIALLOGIN_USERNAME #
#############################################################################################################
@ -67,10 +66,10 @@ premium:
proFeatures:
SSOAutoLogin: false
CustomMetadata:
autoUpdateMetadata: false # set to 'true' to automatically update metadata with below values
author: username # supports text such as 'John Doe' or types such as username to autopopulate with user's username
creator: Stirling-PDF # supports text such as 'Company-PDF'
producer: Stirling-PDF # supports text such as 'Company-PDF'
autoUpdateMetadata: false
author: username
creator: Stirling-PDF
producer: Stirling-PDF
googleDrive:
enabled: false
clientId: ''
@ -127,7 +126,7 @@ ui:
appNameNavbar: '' # name displayed on the navigation bar
languages: [] # If empty, all languages are enabled. To display only German and Polish ["de_DE", "pl_PL"]. British English is always enabled.
endpoints: # All the possible endpoints are disabled
endpoints:
toRemove: [crop, merge-pdfs, multi-page-layout, overlay-pdfs, pdf-to-single-page, rearrange-pages, remove-image-pdf, remove-pages, rotate-pdf, scale-pages, split-by-size-or-count, split-pages, split-pdf-by-chapters, split-pdf-by-sections, add-password, add-watermark, auto-redact, cert-sign, get-info-on-pdf, redact, remove-cert-sign, remove-password, sanitize-pdf, validate-signature, file-to-pdf, html-to-pdf, img-to-pdf, markdown-to-pdf, pdf-to-csv, pdf-to-html, pdf-to-img, pdf-to-markdown, pdf-to-pdfa, pdf-to-presentation, pdf-to-text, pdf-to-word, pdf-to-xml, url-to-pdf, add-image, add-page-numbers, add-stamp, auto-rename, auto-split-pdf, compress-pdf, decompress-pdf, extract-image-scans, extract-images, flatten, ocr-pdf, remove-blanks, repair, replace-invert-pdf, show-javascript, update-metadata, filter-contains-image, filter-contains-text, filter-file-size, filter-page-count, filter-page-rotation, filter-page-size] # list endpoints to disable (e.g. ['img-to-pdf', 'remove-pages'])
groupsToRemove: [] # list groups to disable (e.g. ['LibreOffice'])
@ -138,7 +137,7 @@ metrics:
AutomaticallyGenerated:
key: cbb81c0f-50b1-450c-a2b5-89ae527776eb
UUID: 10dd4fba-01fa-4717-9b78-3dc4f54e398a
appVersion: 0.44.3
appVersion: 0.46.2
processExecutor:
sessionLimit: # Process executor instances limits
@ -158,3 +157,6 @@ processExecutor:
installApptimeoutMinutes: 60
calibretimeoutMinutes: 30
tesseractTimeoutMinutes: 30
unoconvServers: [] # List of external unoconv servers in the format ["hostname:port", "hostname:port"]. Leave empty to use local instances.
useExternalUnoconvServers: false # Set to true to use external servers from the list above
baseUnoconvPort: 2003 # Base port for local unoconv instances (will increment by 1 for each instance)