Commit graph

631 commits

Author SHA1 Message Date
Richard R
fe59685454 feat(worker): add granular PDF progress reporting and abort error handling
Introduce per-page progress events for PDF parsing by adding onPageStarted
hooks throughout the compute worker and core PDF pipeline. Add
compute/worker/src/pdf-progress.ts to encapsulate progress event builders.
Update server logic to emit page start and page parsed events, improving
client feedback during long-running PDF jobs.

Add src/lib/server/compute/abort-like-error.ts and integrate isAbortLikeError
to robustly detect and suppress expected aborts in API event streaming.
Update API route to ignore abort-like errors on worker proxy crash.

Add unit tests for abort-like error detection and PDF progress event
generation. Refactor PDF parse logic to allow empty merged regions without
throwing, ensuring downstream flows remain resilient to edge cases.

These changes improve observability, error handling, and reliability for
document parsing and event streaming.
2026-05-29 20:13:59 -06:00
Richard R
dcec74a4a3 refactor(package): remove unused compute worker watch script 2026-05-29 19:58:56 -06:00
Richard R
5428a21b67 refactor(onboarding): modularize onboarding flow logic and improve testability
Extract onboarding flow step resolution and async runner logic into a
dedicated module for better separation of concerns and maintainability.
Refactor context/provider to use new helpers and simplify settings modal
control. Add unit tests for onboarding step resolution and async runner
coalescing. Update e2e helpers and selectors for onboarding modals.
Adjust environment file handling for CI and Docker contexts.

- Add .dockerignore for build hygiene
- Refactor onboarding context and modal logic
- Add src/lib/client/onboarding-flow.ts with core onboarding helpers
- Add onboarding-flow unit tests
- Update Playwright config and scripts for CI env loading
- Improve test selectors for claim modal
- Update .gitignore to exclude all .env* except .env.example

This change increases onboarding logic modularity and reliability, and improves CI/test environment handling.
2026-05-29 18:03:17 -06:00
Richard R
c6047ffaa4 fix(onboarding): prevent no-auth auto settings modal races 2026-05-29 09:20:34 -06:00
Richard R
c5b39f5312 test(e2e): dismiss blocking overlays after folder reload 2026-05-28 23:13:34 -06:00
Richard R
8566206911 test(e2e): disambiguate settings and uploader selectors 2026-05-28 22:46:55 -06:00
Richard R
dca1a9b35d feat(doclist): unify document identity handling and improve drag-and-drop robustness
Introduce a document identity key combining type and id for consistent
identification across document operations, including selection, drag-and-drop,
and folder management. Refactor drag item structures to use identity objects
instead of plain ids, preventing cross-type collisions and improving merge
accuracy. Update Dexie recently opened map to use identity keys. Enhance
document size formatting for small files and improve error feedback in the
uploader. Clean up redundant props and standardize icon SVG handling.
2026-05-28 22:27:15 -06:00
Richard R
2f1eafeb79 test(e2e): scope settings Documents tab selector to modal 2026-05-28 22:26:49 -06:00
Richard R
5a684214c8 fix(doclist): mobile drawer close behavior and recents toolbar polish 2026-05-28 17:45:05 -06:00
Richard R
98736bf6e3 refactor(doclist): remove columns view and update text labeling
Eliminate the deprecated "columns" view mode from the document list, toolbar,
icons, and skeleton components. Update all related types and logic to only
support "icons", "list", and "gallery" views. Standardize user-facing labels
from "HTML" to "Text" for improved clarity across the sidebar, segment
locator labels, and document counts. Clean up associated code and types to
reflect these changes.
2026-05-28 17:20:02 -06:00
Richard R
c12f795026 feat(ui): move app actions to sidebar and add sidebar user menu variant
Shift app-level actions (settings and user menu) from the top bar to the bottom
of the sidebar for improved navigation consistency. Introduce a new "sidebar"
variant for the UserMenu component with tailored styling and behavior for
sidebar placement. Update SettingsModal to support a customizable trigger label
and styling for sidebar integration. Refactor FinderSidebar to accept an
optional bottomSlot, enabling flexible placement of sidebar controls.
2026-05-28 14:58:03 -06:00
Richard R
eea143abbf refactor(doclist): centralize icons grid layout logic into reusable utility
Move grid style and template calculation for icon views into a shared
iconsGrid.ts utility. Replace inline grid logic in IconsView and DocumentListSkeleton
with calls to the new iconsGridStyle function for consistency and maintainability.
2026-05-28 14:51:48 -06:00
Richard R
e4b0c8358e test(documents): update selectors and assertions for document actions and accessibility
Revise test helpers and specs to use stricter role-based selectors for document
actions, including delete buttons and document links. Improve accessibility and
upload tests by asserting document presence via explicit file lists instead of
generic text checks. Remove redundant summary assertion from delete flow.
2026-05-28 12:34:29 -06:00
Richard R
b9490a53ac refactor(doclist): remove loading state and unify navigation prefetch behavior
Eliminate unused loading state and related effects from document tile and list
row components. Standardize navigation by disabling prefetch on all document
open links in doclist views for consistent client-side routing. Cache
document list state on load to improve state restoration and initialization.
2026-05-28 09:15:18 -06:00
Richard R
eff25ad8df feat(doclist): add bulk folder removal with sidebar menu and confirmation
Introduce a "Remove All Folders" action accessible via a new sidebar menu,
allowing users to clear all folders at once. Add a confirmation dialog to
prevent accidental removal. Update the FinderSidebar to include a menu button
with the new action, and implement supporting state and handlers in the
document list. Add DotsHorizontalIcon for menu UI. Expand folder tests to
cover the new bulk removal flow.
2026-05-28 05:18:53 -06:00
Richard R
f569c8ded2 style(doclist): improve gallery view tile visuals and image preview readiness
Update gallery view document tiles with refined border, shadow, and background
styles for better visual feedback and accessibility. Enhance image preview
component to immediately reveal cached images by checking decode state on mount,
preventing opacity glitches on remount. Adjust gallery rail scrolling to reset
on document list changes for consistent navigation.
2026-05-28 04:51:04 -06:00
Richard R
85aff2c1be refactor(doclist): flatten foldered document structure and remove folder UI
Eliminate folder grouping logic and related UI from all document list views.
Switch to a flat document array model for rendering, selection, and filtering.
Remove folder collapse/expand state and folder drag-and-drop handling. Update
sidebar and toolbar components to remove folder creation and management
controls. Simplify props and internal state across all views for consistency.
2026-05-28 04:36:08 -06:00
Richard R
d59e911ca2 Fix flaky document preview blob URL reuse 2026-05-28 04:23:12 -06:00
Richard R
495c197c4c feat(doclist): add recently opened sorting and metadata to documents list
Introduce recentlyOpenedAt field to document types and fetch per-document
recently opened timestamps from local cache. Update the "Recently Opened"
sidebar filter to sort by this value and show only documents with recent
activity. Adjust grid layout logic for icons view to better handle small
numbers of documents. Update sidebar label for clarity.
2026-05-27 23:13:53 -06:00
Richard R
13e0647b47 refactor(doclist): convert icon size controls to dropdown for icons view toggle
Replace inline icon size buttons with a dropdown menu that appears when
the icons view toggle is active. This change improves toolbar clarity by
grouping related controls and enhances accessibility with better focus
handling. The update also streamlines the button structure for view mode
selection.
2026-05-27 21:10:01 -06:00
Richard R
8104afa3e3 feat(doclist): implement dynamic grid layout for icons view using CSS variables
Replace fixed Tailwind grid column classes with a responsive CSS grid
that leverages tile width variables per icon size. Grid columns now
adapt automatically via `auto-fit` and `minmax`, and grid gaps are
centralized for consistency. This update enhances layout flexibility,
maintainability, and responsiveness in the icons view.
2026-05-27 20:52:52 -06:00
Richard R
53993aa3c6 feat(doclist): improve sidebar responsiveness and toolbar interaction in document list
Enhance mobile and desktop sidebar behavior by introducing a separate state for mobile sidebar visibility and ensuring it does not persist from desktop state. Update FinderToolbar to visually indicate sidebar open state and make the toolbar sticky for better accessibility. Refine FinderSidebar width handling for improved responsiveness. Remove redundant border in mobile sidebar dialog for a cleaner appearance. These changes provide a more consistent and intuitive sidebar experience across viewports.
2026-05-27 20:11:43 -06:00
Richard R
0470f044ef refactor(doclist): restructure loading and empty states, update skeletons and columns UI
Simplify document list state handling by centralizing loading and empty logic into DocumentList.
Remove redundant state management from HomeContent. Refactor DocumentListSkeleton to support all view modes and icon sizes for improved visual consistency during loading. Enhance ColumnsView styling for folder and document rows, improving selection, hover, and drag feedback. Use useMemo for visible order calculation to optimize rendering.

These changes streamline the document list rendering flow, unify skeleton loading visuals, and improve the clarity and responsiveness of the columns view interface.
2026-05-27 19:55:23 -06:00
Richard R
46d5bab647 cleanup doclist formatting helpers and view props 2026-05-27 19:40:12 -06:00
Richard R
eb8820d888 feat(doclist): enhance document list UI with overlay uploader and improved file size formatting
Integrate DocumentUploader in overlay mode to wrap all document views, enabling drag-and-drop uploads across the entire list area. Refine file size formatting to support KB, MB, and GB units in all views and tiles. Standardize button scaling and padding for consistent UI rhythm. Update DocumentPreview and detail panels to display both type and human-readable size. Add overlay drag visual feedback for uploads.

These changes improve usability, visual clarity, and consistency in the document list interface.
2026-05-27 19:35:03 -06:00
Richard R
7a88dabc53 refactor(doclist): redesign document list with new views, DnD, and windowed UI
Revamp the document list experience by introducing a Finder-style window
interface with multiple views (icons, list, columns, gallery) and a new
sidebar filter system. Remove legacy folder and list item components in
favor of modular, windowed views. Integrate react-dnd-touch-backend and
custom DnD context for improved drag-and-drop, including mobile support.
Update uploader and dialog styles for consistency. Extend document types
to support new view modes, icon sizing, and sidebar state.
2026-05-27 18:59:13 -06:00
Richard R
3db193e012 feat(hooks): introduce useUnmountCleanupRef for effect cleanup abstraction
Add useUnmountCleanupRef custom hook to centralize and simplify unmount
cleanup logic in document page components. Refactor EPUB, HTML, and PDF
page components to utilize this hook, replacing repetitive ref and effect
patterns for invoking cleanup functions on unmount. This reduces code
duplication and improves maintainability.
2026-05-27 16:22:14 -06:00
Richard R
139a227aa4 fix(html): use ref to ensure latest clearCurrDoc on unmount 2026-05-27 16:12:09 -06:00
Richard R
f27b5e6504 fix(epub): prevent stale clearCurrDoc reference on unmount in EPUB page 2026-05-27 16:03:34 -06:00
Richard R
a00caa9052 refactor(ui): streamline document cleanup and navigation logic in doc pages
Remove redundant clearCurrDoc calls from navigation links and ensure document cleanup occurs on component unmount for EPUB, HTML, and PDF pages. Simplify PDF back navigation by eliminating sidebar delay logic. Refactor DocumentListItem to avoid unnecessary router usage and consolidate document link handling. These changes improve maintainability and consistency in document lifecycle management across the UI.
2026-05-27 15:48:46 -06:00
Richard R
5305b896c6 refactor: complete phase 7 error-contract rollout 2026-05-27 08:02:31 -06:00
Richard R
00ce39ed74 refactor: roll out phase 7 server error contract 2026-05-27 07:24:07 -06:00
Richard R
3da13c2970 chore(logging): remove redundant errorCode fields from server logs and error contracts
Eliminate the errorCode property from server logger calls, error contract types, and API error bodies throughout the codebase. This reduces duplication and streamlines error reporting, relying on structured event names and error classes for identification. All affected API routes, server logic, and logging utilities have been updated for consistency. Documentation directory added for future reference.
2026-05-27 06:33:14 -06:00
Richard R
5de9069c20 refactor(logging): finalize phase 6 error contract and pretty output 2026-05-27 06:17:25 -06:00
Richard R
f78d066434 refactor(api): enhance tts segment error handling and logging patterns
Update TTS segment ensure route to distinguish aborted and failed segment generation, providing more granular logging and context. Replace ambiguous message variables with clearer detail fields, and improve log event naming for abort scenarios. Adjust ESLint logger call selectors for maintainability and consistency, consolidating selector logic and enforcing stricter server logger usage patterns. These changes improve observability and error traceability in TTS segment operations.
2026-05-27 05:58:48 -06:00
Richard R
034c1f03a4 logging: complete phase 5 contract and enforce via eslint rules 2026-05-27 04:54:52 -06:00
Richard R
29cc4fa42d logging: apply phase 4 warn/error level tuning 2026-05-27 04:28:35 -06:00
Richard R
59d0101c0c feat(server): implement unified structured logging with pino and enforce usage
Introduce `serverLogger` utility based on pino for consistent, structured logging across all server and API modules. Replace direct console logging with `serverLogger` and add request-scoped logging helpers. Update environment variable handling, documentation, and deployment guides to reflect new logging configuration (`LOG_FORMAT`, `LOG_LEVEL`, `COMPUTE_LOG_LEVEL`). Enforce no-console in server code via ESLint and add pino/pino-pretty dependencies.

This change standardizes log output, improves observability, and prepares the codebase for ingestion by log platforms.
2026-05-27 04:20:17 -06:00
Richard R
7732426963 feat(pdf): enable opId tracking and propagation for layout parsing operations
Integrate opId handling throughout PDF layout parsing by updating client hooks, API routes, and event streams to support operation ID propagation. Add server utility for worker operation creation to centralize opId management. Enhance polling and SSE subscription utilities to accept and return opId, improving traceability and coordination of parse operations across the client and server.
2026-05-27 02:35:46 -06:00
Richard R
1ec9632d4b feat(api): set maxDuration for document parsed events route
Define maxDuration as 300 seconds for the parsed events API route to explicitly
control execution timeouts during long-running PDF layout processing. This
ensures more predictable serverless behavior for streaming operations.
2026-05-27 01:41:29 -06:00
Richard R
956ecf5f4a refactor(api): add structured logging for worker op and SSE error states
Introduce detailed console logging across document parsing API routes and worker
operation state fetches. Logs now capture stream openings, error conditions,
worker state unavailability, and invalid responses, improving observability and
debuggability for PDF parsing and ONNX layout processing flows.
2026-05-27 01:32:14 -06:00
Richard R
5c24ea5763 docs(env): clarify env var ownership split for external compute worker mode
Expand documentation and example env files to explicitly distinguish which
environment variables belong in the app/root `.env` versus the `compute/worker/.env*`
files when deploying in external worker mode. This clarifies routing/auth
settings versus worker runtime configuration, reducing misconfiguration risk.
2026-05-27 01:19:37 -06:00
Richard R
93ae9f3082 orchestrate privacy modal in onboarding flow 2026-05-27 00:50:10 -06:00
Richard R
22f132bbc8 hard-cut onboarding modal flow orchestration 2026-05-27 00:40:12 -06:00
Richard R
14b1b5602f fix(parse-progress): persist opId only after worker op exists 2026-05-27 00:08:42 -06:00
Richard R
c717f7e33e fix(storage): restore main-like upload path without pre-presign bucket scan 2026-05-26 20:08:02 -06:00
Richard R
ebc1dbcda0 ci(playwright): install nats-server from linux tarball asset 2026-05-26 19:48:55 -06:00
Richard R
16e94a1587 build(docker): include compute/core and worker package manifests for deps
Add compute/core/package.json and compute/worker/package.json to Docker build
context to ensure all workspace dependencies are correctly installed during
image creation. This change prevents missing dependency issues for subpackages
when building in isolated environments.
2026-05-26 19:33:41 -06:00
Richard R
cd530a365d refactor(whisper): migrate default ONNX model references to q4 variant
Update all configuration, manifest, and documentation references to use the q4
Whisper ONNX model files instead of int8. Adjust expected file names, hashes,
and environment variable descriptions to reflect this new default. This aligns
runtime, deployment, and developer documentation with the updated model artifact
expectations for improved consistency.
2026-05-26 19:22:07 -06:00
Richard R
b2cf4aa7aa ci(playwright): install nats for embedded worker startup 2026-05-26 19:00:07 -06:00