Commit graph

653 commits

Author SHA1 Message Date
Richard R
36fe5f66a3 ci(workflow): add migration step to vitest CI pipeline
include drizzle migration before running Vitest to ensure database schema
is up to date during test execution
2026-05-30 15:45:42 -06:00
Richard R
8060ae9944 fix(config): update vitest reporter for CI to github-actions 2026-05-30 15:40:34 -06:00
Richard R
0448835892 style(ui): improve bottom padding for safe area compatibility
update bottom padding on banners, status bars, and player controls to use
env(safe-area-inset-bottom) for better support on devices with display
cutouts or home indicators. also add viewportFit: cover to layout for
full-viewport rendering on mobile browsers.
2026-05-30 15:35:02 -06:00
Richard R
60c1540744 refactor(ui): decouple settings modal trigger from modal logic
extract SettingsTrigger component to separate modal open control from SettingsModal.
update HomeContent to manage modal state and pass open/onOpenChange props.
remove redundant trigger button from SettingsModal for improved composability.

removes embedded trigger logic from SettingsModal, enabling external state
management and better integration with various UI layouts. also cleans up
playwright config by centralizing testIgnore for unit tests.
2026-05-30 15:13:15 -06:00
Richard R
8b45a4f6cd refactor(admin): group rate limit and upload settings in dedicated section
move TTS and PDF parsing rate limit toggles and max upload size input into a new
"Rate limiting" section in the admin panel for improved organization. update
documentation to reflect the new grouping and clarify the relationship between
these settings.
2026-05-30 14:59:45 -06:00
Richard R
3de33e2f1e docs(config): update compute rate limit defaults to disabled for first boot
change RUNTIME_SEED_DISABLE_COMPUTE_LIMIT default to true in env example,
documentation, and runtime config schema to align with TTS rate limit behavior.
clarify admin panel as the source of truth for enabling PDF parsing rate limits.
2026-05-30 14:39:04 -06:00
Richard R
fc3d05d65b feat(rate-limit): add per-user PDF parsing rate limiting and job event ledger
implement a generic user_job_events table for tracking compute job creation
enforce configurable burst and sustained limits for PDF layout parsing
add admin panel controls for compute rate limiting and max upload size
update API routes to apply and record rate checks for PDF parse jobs
document new environment variables and admin settings for compute limits
improve IP extraction logic for rate limiting accuracy
add tests for request IP extraction and test namespace gating

This change introduces a robust mechanism to throttle expensive compute operations, such as PDF parsing, on a per-user basis. It provides both burst and sustained rate controls, with admin-tunable parameters and clear user feedback on throttling. The job event ledger enables accurate concurrency and rate enforcement, while new documentation and tests ensure maintainability and clarity.
2026-05-30 14:17:49 -06:00
Richard R
3307477548 test(unit): complete vitest migration for remaining unit suites 2026-05-30 11:17:18 -06:00
Richard R
22a1b5de57 test(unit): migrate batch2 document and parser suites to vitest 2026-05-30 11:15:19 -06:00
Richard R
7570181b8a test(unit): migrate batch1 core contracts to vitest 2026-05-30 11:11:44 -06:00
Richard R
b16328abe9 fix(vitest): update test command from 'test:vitest' to 'test:unit' 2026-05-30 11:01:44 -06:00
Richard R
e1483ab80a refactor(worker): modularize worker entrypoint and add vitest infra
move worker startup logic from server.ts to runtime.ts for better modularity
add worker-loop-policy.ts for loop control abstraction
delete legacy unit tests under tests/unit/ related to compute worker and control-plane
add vitest config and test directories for compute-core and compute-worker
add vitest workflow for CI
update package.json scripts for new test commands and add vitest as dev dependency

BREAKING CHANGE: worker entrypoint is now compute/worker/src/runtime.ts instead of server.ts; legacy Playwright-based unit tests for compute worker are removed in favor of Vitest
2026-05-30 11:00:01 -06:00
Richard R
b0a4bccf4c refactor(admin): remove unused EditableRow and EditIcon components
- Delete EditableRow from formPrimitives and its usage in AdminFeaturesPanel
- Remove EditIcon from icon set as it is no longer referenced
- Simplify AdminFeaturesPanel state and draft handling
- Update AdminProvidersPanel to use distinct query key for default provider
- Ensure query invalidation covers new default provider key

This streamlines the admin codebase by eliminating obsolete editing UI and related icon, and clarifies provider query logic.
2026-05-30 10:33:11 -06:00
Richard R
267e2ce411 fix admin features tab reset/flash on tab switches 2026-05-30 10:05:24 -06:00
Richard R
a0f2d008e0 refactor(tts): transition rate limiting to runtime config and admin panel
- Remove legacy TTS rate limit environment variables and migrate all related configuration to runtime settings managed via the Admin UI
- Add runtime config keys for TTS rate limiting enable/disable and per-user/IP daily quotas
- Refactor rate limiter and API routes to use runtime config for thresholds and enablement
- Update AdminFeaturesPanel to allow editing TTS rate limiting and quota values in the UI
- Add shared-provider-selection utility for consistent provider selection logic
- Update documentation to reflect new runtime/admin configuration and remove obsolete env var docs
- Add unit tests for rate limit runtime settings and provider selection

BREAKING CHANGE: TTS rate limiting is now controlled via Admin → Site features; environment variables for TTS rate limiting are no longer supported and will be ignored.
2026-05-30 09:55:16 -06:00
Richard R
e262f93f90
Merge pull request #91 from richardr1126/redesign/app-view-documents-list 2026-05-29 22:28:09 -06:00
Richard R
2695c7055b Fix narrow-screen initial render in FinderWindow 2026-05-29 22:27:30 -06:00
Richard R
d120719abd refactor(doclist): implement upload batch progress tracking in sidebar
Add SidebarUploadLoader component to DocumentList for displaying real-time
upload progress in the sidebar. Extend DocumentUploader to emit detailed
batch state updates via onUploadBatchChange, enabling granular feedback
on current file, phase, and completion status. Improves user awareness of
document upload and conversion processes.
2026-05-29 22:11:27 -06:00
Richard R
b295d2f38d feat(doclist): add mobile sidebar close handler to FinderWindow and DocumentList
Introduce onRequestSidebarClose prop to FinderWindow and implement its
usage in DocumentList to enable closing the sidebar on mobile via
backdrop tap or Esc key. Enhances mobile usability by allowing users to
dismiss the sidebar more intuitively.
2026-05-29 21:58:45 -06:00
Richard R
503319d46e style(ui): adjust sidebar hover effects and document skeleton timeout
Update FinderSidebar to remove background color on inactive hover for
improved visual clarity. Extend the DocumentSkeleton warning timeout to
10 seconds to allow more time before displaying the alert. Refine test
description for icons grid layout to clarify auto-fit behavior.
2026-05-29 21:49:13 -06:00
Richard R
e62923fb91 feat(ui): improve icon grid layout responsiveness and add single row stretch suppression
Enhance the document icons grid to better handle single-row layouts by
dynamically suppressing column stretching when the number of documents fits
in a single row. Introduce maxColumnsForIconGrid and related logic to
calculate optimal column counts based on container width and icon size.
Update grid style computation to allow optional suppression of single row
stretch, improving alignment and appearance for small document sets.

Add unit tests for icon grid layout calculation and column logic to ensure
correctness across different icon sizes and container widths.
2026-05-29 21:33:01 -06:00
Richard R
fe59685454 feat(worker): add granular PDF progress reporting and abort error handling
Introduce per-page progress events for PDF parsing by adding onPageStarted
hooks throughout the compute worker and core PDF pipeline. Add
compute/worker/src/pdf-progress.ts to encapsulate progress event builders.
Update server logic to emit page start and page parsed events, improving
client feedback during long-running PDF jobs.

Add src/lib/server/compute/abort-like-error.ts and integrate isAbortLikeError
to robustly detect and suppress expected aborts in API event streaming.
Update API route to ignore abort-like errors on worker proxy crash.

Add unit tests for abort-like error detection and PDF progress event
generation. Refactor PDF parse logic to allow empty merged regions without
throwing, ensuring downstream flows remain resilient to edge cases.

These changes improve observability, error handling, and reliability for
document parsing and event streaming.
2026-05-29 20:13:59 -06:00
Richard R
dcec74a4a3 refactor(package): remove unused compute worker watch script 2026-05-29 19:58:56 -06:00
Richard R
5428a21b67 refactor(onboarding): modularize onboarding flow logic and improve testability
Extract onboarding flow step resolution and async runner logic into a
dedicated module for better separation of concerns and maintainability.
Refactor context/provider to use new helpers and simplify settings modal
control. Add unit tests for onboarding step resolution and async runner
coalescing. Update e2e helpers and selectors for onboarding modals.
Adjust environment file handling for CI and Docker contexts.

- Add .dockerignore for build hygiene
- Refactor onboarding context and modal logic
- Add src/lib/client/onboarding-flow.ts with core onboarding helpers
- Add onboarding-flow unit tests
- Update Playwright config and scripts for CI env loading
- Improve test selectors for claim modal
- Update .gitignore to exclude all .env* except .env.example

This change increases onboarding logic modularity and reliability, and improves CI/test environment handling.
2026-05-29 18:03:17 -06:00
Richard R
c6047ffaa4 fix(onboarding): prevent no-auth auto settings modal races 2026-05-29 09:20:34 -06:00
Richard R
c5b39f5312 test(e2e): dismiss blocking overlays after folder reload 2026-05-28 23:13:34 -06:00
Richard R
8566206911 test(e2e): disambiguate settings and uploader selectors 2026-05-28 22:46:55 -06:00
Richard R
dca1a9b35d feat(doclist): unify document identity handling and improve drag-and-drop robustness
Introduce a document identity key combining type and id for consistent
identification across document operations, including selection, drag-and-drop,
and folder management. Refactor drag item structures to use identity objects
instead of plain ids, preventing cross-type collisions and improving merge
accuracy. Update Dexie recently opened map to use identity keys. Enhance
document size formatting for small files and improve error feedback in the
uploader. Clean up redundant props and standardize icon SVG handling.
2026-05-28 22:27:15 -06:00
Richard R
2f1eafeb79 test(e2e): scope settings Documents tab selector to modal 2026-05-28 22:26:49 -06:00
Richard R
5a684214c8 fix(doclist): mobile drawer close behavior and recents toolbar polish 2026-05-28 17:45:05 -06:00
Richard R
98736bf6e3 refactor(doclist): remove columns view and update text labeling
Eliminate the deprecated "columns" view mode from the document list, toolbar,
icons, and skeleton components. Update all related types and logic to only
support "icons", "list", and "gallery" views. Standardize user-facing labels
from "HTML" to "Text" for improved clarity across the sidebar, segment
locator labels, and document counts. Clean up associated code and types to
reflect these changes.
2026-05-28 17:20:02 -06:00
Richard R
c12f795026 feat(ui): move app actions to sidebar and add sidebar user menu variant
Shift app-level actions (settings and user menu) from the top bar to the bottom
of the sidebar for improved navigation consistency. Introduce a new "sidebar"
variant for the UserMenu component with tailored styling and behavior for
sidebar placement. Update SettingsModal to support a customizable trigger label
and styling for sidebar integration. Refactor FinderSidebar to accept an
optional bottomSlot, enabling flexible placement of sidebar controls.
2026-05-28 14:58:03 -06:00
Richard R
eea143abbf refactor(doclist): centralize icons grid layout logic into reusable utility
Move grid style and template calculation for icon views into a shared
iconsGrid.ts utility. Replace inline grid logic in IconsView and DocumentListSkeleton
with calls to the new iconsGridStyle function for consistency and maintainability.
2026-05-28 14:51:48 -06:00
Richard R
e4b0c8358e test(documents): update selectors and assertions for document actions and accessibility
Revise test helpers and specs to use stricter role-based selectors for document
actions, including delete buttons and document links. Improve accessibility and
upload tests by asserting document presence via explicit file lists instead of
generic text checks. Remove redundant summary assertion from delete flow.
2026-05-28 12:34:29 -06:00
Richard R
b9490a53ac refactor(doclist): remove loading state and unify navigation prefetch behavior
Eliminate unused loading state and related effects from document tile and list
row components. Standardize navigation by disabling prefetch on all document
open links in doclist views for consistent client-side routing. Cache
document list state on load to improve state restoration and initialization.
2026-05-28 09:15:18 -06:00
Richard R
eff25ad8df feat(doclist): add bulk folder removal with sidebar menu and confirmation
Introduce a "Remove All Folders" action accessible via a new sidebar menu,
allowing users to clear all folders at once. Add a confirmation dialog to
prevent accidental removal. Update the FinderSidebar to include a menu button
with the new action, and implement supporting state and handlers in the
document list. Add DotsHorizontalIcon for menu UI. Expand folder tests to
cover the new bulk removal flow.
2026-05-28 05:18:53 -06:00
Richard R
f569c8ded2 style(doclist): improve gallery view tile visuals and image preview readiness
Update gallery view document tiles with refined border, shadow, and background
styles for better visual feedback and accessibility. Enhance image preview
component to immediately reveal cached images by checking decode state on mount,
preventing opacity glitches on remount. Adjust gallery rail scrolling to reset
on document list changes for consistent navigation.
2026-05-28 04:51:04 -06:00
Richard R
85aff2c1be refactor(doclist): flatten foldered document structure and remove folder UI
Eliminate folder grouping logic and related UI from all document list views.
Switch to a flat document array model for rendering, selection, and filtering.
Remove folder collapse/expand state and folder drag-and-drop handling. Update
sidebar and toolbar components to remove folder creation and management
controls. Simplify props and internal state across all views for consistency.
2026-05-28 04:36:08 -06:00
Richard R
d59e911ca2 Fix flaky document preview blob URL reuse 2026-05-28 04:23:12 -06:00
Richard R
495c197c4c feat(doclist): add recently opened sorting and metadata to documents list
Introduce recentlyOpenedAt field to document types and fetch per-document
recently opened timestamps from local cache. Update the "Recently Opened"
sidebar filter to sort by this value and show only documents with recent
activity. Adjust grid layout logic for icons view to better handle small
numbers of documents. Update sidebar label for clarity.
2026-05-27 23:13:53 -06:00
Richard R
13e0647b47 refactor(doclist): convert icon size controls to dropdown for icons view toggle
Replace inline icon size buttons with a dropdown menu that appears when
the icons view toggle is active. This change improves toolbar clarity by
grouping related controls and enhances accessibility with better focus
handling. The update also streamlines the button structure for view mode
selection.
2026-05-27 21:10:01 -06:00
Richard R
8104afa3e3 feat(doclist): implement dynamic grid layout for icons view using CSS variables
Replace fixed Tailwind grid column classes with a responsive CSS grid
that leverages tile width variables per icon size. Grid columns now
adapt automatically via `auto-fit` and `minmax`, and grid gaps are
centralized for consistency. This update enhances layout flexibility,
maintainability, and responsiveness in the icons view.
2026-05-27 20:52:52 -06:00
Richard R
53993aa3c6 feat(doclist): improve sidebar responsiveness and toolbar interaction in document list
Enhance mobile and desktop sidebar behavior by introducing a separate state for mobile sidebar visibility and ensuring it does not persist from desktop state. Update FinderToolbar to visually indicate sidebar open state and make the toolbar sticky for better accessibility. Refine FinderSidebar width handling for improved responsiveness. Remove redundant border in mobile sidebar dialog for a cleaner appearance. These changes provide a more consistent and intuitive sidebar experience across viewports.
2026-05-27 20:11:43 -06:00
Richard R
0470f044ef refactor(doclist): restructure loading and empty states, update skeletons and columns UI
Simplify document list state handling by centralizing loading and empty logic into DocumentList.
Remove redundant state management from HomeContent. Refactor DocumentListSkeleton to support all view modes and icon sizes for improved visual consistency during loading. Enhance ColumnsView styling for folder and document rows, improving selection, hover, and drag feedback. Use useMemo for visible order calculation to optimize rendering.

These changes streamline the document list rendering flow, unify skeleton loading visuals, and improve the clarity and responsiveness of the columns view interface.
2026-05-27 19:55:23 -06:00
Richard R
46d5bab647 cleanup doclist formatting helpers and view props 2026-05-27 19:40:12 -06:00
Richard R
eb8820d888 feat(doclist): enhance document list UI with overlay uploader and improved file size formatting
Integrate DocumentUploader in overlay mode to wrap all document views, enabling drag-and-drop uploads across the entire list area. Refine file size formatting to support KB, MB, and GB units in all views and tiles. Standardize button scaling and padding for consistent UI rhythm. Update DocumentPreview and detail panels to display both type and human-readable size. Add overlay drag visual feedback for uploads.

These changes improve usability, visual clarity, and consistency in the document list interface.
2026-05-27 19:35:03 -06:00
Richard R
7a88dabc53 refactor(doclist): redesign document list with new views, DnD, and windowed UI
Revamp the document list experience by introducing a Finder-style window
interface with multiple views (icons, list, columns, gallery) and a new
sidebar filter system. Remove legacy folder and list item components in
favor of modular, windowed views. Integrate react-dnd-touch-backend and
custom DnD context for improved drag-and-drop, including mobile support.
Update uploader and dialog styles for consistency. Extend document types
to support new view modes, icon sizing, and sidebar state.
2026-05-27 18:59:13 -06:00
Richard R
218e248cc2
Merge pull request #90: Refactor compute architecture with workerized ONNX PDF/Whisper pipeline
- Monorepo Architecture: Separates processing into compute/core and compute/worker Fastify service.
- NATS JetStream: Migrates processing queues to NATS JetStream with lazy connection and 120s idle sleep for Railway.
- ONNX PDF Layout: Upgrades to PP-DocLayout-V3 with block overlay UI, force reparse, and block-level skip playback rules.
- ONNX Whisper Alignment: Replaces whisper.cpp with a pure JS ONNX alignment decoder.
- Onboarding Coordinator: Integrates OnboardingFlowContext to manage sequential modals (Privacy, Claim Guest Data, Dexie Cloud Migration).
- Database Schema: Adds documentSettings table and tracks parsing stages (parseState, parsedJsonKey) in the documents table.
- Error Contracts: Standardizes API endpoints on ServerAppError responses.
2026-05-27 18:00:01 -06:00
Richard R
3db193e012 feat(hooks): introduce useUnmountCleanupRef for effect cleanup abstraction
Add useUnmountCleanupRef custom hook to centralize and simplify unmount
cleanup logic in document page components. Refactor EPUB, HTML, and PDF
page components to utilize this hook, replacing repetitive ref and effect
patterns for invoking cleanup functions on unmount. This reduces code
duplication and improves maintainability.
2026-05-27 16:22:14 -06:00
Richard R
139a227aa4 fix(html): use ref to ensure latest clearCurrDoc on unmount 2026-05-27 16:12:09 -06:00