v4: Fix milestones display + add OpenID auth + 100MB PDF support
FIXES: - Milestones now show correctly on encounter page (use static fallback if DB empty) - Static data preserved as MILESTONES_DATA_STATIC for compatibility - Database-driven milestones still work (admin can edit via CMS) NEW FEATURES: - OpenID Connect (OIDC) authentication support (PocketID, Keycloak, Azure AD, etc.) - Comprehensive setup guide: OPENID_SETUP.md - Auto-linking existing users by email on SSO login - Multiple PDF upload support in Learning Hub (up to 10 files) - 100 MB per file limit (was 20 MB) - Full PDF content used for AI generation - Embeddings use first ~8K chars for semantic search IMPROVEMENTS: - Updated UI to show multiple file selection with list - Drag-and-drop supports multiple files - Better file upload validation and error handling - Added clarifying comments about embedding truncation
This commit is contained in:
ifedan-ed
parent
540347c015
commit
dc2e000e88
8 changed files with 482 additions and 25 deletions
346
OPENID_SETUP.md
Normal file
346
OPENID_SETUP.md
Normal file
|
|
@ -0,0 +1,346 @@
|
|||
# OpenID Connect (OIDC) / PocketID Setup Guide
|
||||
|
||||
This guide explains how to configure Single Sign-On (SSO) authentication using OpenID Connect providers like PocketID, Keycloak, Azure AD, Okta, or Google.
|
||||
|
||||
## Overview
|
||||
|
||||
The application supports OIDC authentication alongside traditional email/password login. Once configured, users can:
|
||||
|
||||
- Sign in with their SSO provider (e.g., PocketID)
|
||||
- Automatically link existing email accounts to their SSO identity
|
||||
- Admins can optionally disable local password login entirely
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. An OpenID Connect provider (e.g., PocketID instance)
|
||||
2. Admin access to this application
|
||||
3. The public URL where your app is deployed (`APP_URL` in `.env`)
|
||||
|
||||
---
|
||||
|
||||
## Configuration Steps
|
||||
|
||||
### 1. Configure Your Identity Provider
|
||||
|
||||
First, register this application with your OIDC provider. You'll need:
|
||||
|
||||
**Redirect URI / Callback URL:**
|
||||
```
|
||||
https://your-domain.com/api/auth/oidc/callback
|
||||
```
|
||||
|
||||
Replace `your-domain.com` with your actual `APP_URL` value.
|
||||
|
||||
**Example: PocketID Setup**
|
||||
|
||||
1. Log into your PocketID admin panel
|
||||
2. Navigate to **Applications** → **Add Application**
|
||||
3. Set the callback URL: `https://your-domain.com/api/auth/oidc/callback`
|
||||
4. Copy the Client ID and Client Secret
|
||||
|
||||
**Example: Keycloak Setup**
|
||||
|
||||
1. Create a new client in your Keycloak realm
|
||||
2. Set **Access Type** to `confidential`
|
||||
3. Add Valid Redirect URI: `https://your-domain.com/api/auth/oidc/callback`
|
||||
4. Save and note the Client ID and Client Secret from the Credentials tab
|
||||
|
||||
### 2. Enable OIDC in Application Settings
|
||||
|
||||
Log into your application as an **admin** user, then:
|
||||
|
||||
1. Navigate to **Admin Panel** → **Settings** (or access `/admin-settings.html`)
|
||||
2. Look for the **OpenID Connect (SSO)** section
|
||||
3. Fill in the following fields:
|
||||
|
||||
| Field | Description | Example |
|
||||
|-------|-------------|---------|
|
||||
| **Enabled** | Toggle to enable OIDC | `true` |
|
||||
| **Issuer URL** | Your provider's discovery endpoint | `https://id.example.com` or `https://keycloak.example.com/realms/myrealm` |
|
||||
| **Client ID** | Application client ID from your provider | `pediatric-scribe-client` |
|
||||
| **Client Secret** | Application client secret (keep confidential) | `a1b2c3d4...` |
|
||||
| **Button Label** | Text shown on the SSO login button | `Sign in with PocketID` |
|
||||
| **Disable Local Auth** | Hide email/password login (optional) | `false` (keep disabled initially) |
|
||||
| **Allowed IPs** | Restrict SSO to specific IP ranges (optional) | Leave blank for no restriction |
|
||||
|
||||
4. Click **Save Settings**
|
||||
|
||||
### 3. Test SSO Login
|
||||
|
||||
1. Log out or open an incognito browser window
|
||||
2. Visit the login page
|
||||
3. You should see a new button: **"Sign in with [Your Provider]"**
|
||||
4. Click it and authenticate with your SSO provider
|
||||
5. You'll be redirected back to the application and logged in
|
||||
|
||||
---
|
||||
|
||||
## Linking Existing Users to SSO
|
||||
|
||||
When a user signs in via OIDC for the first time, the system automatically links their account based on **email address matching**:
|
||||
|
||||
### Scenario 1: Existing User with Matching Email
|
||||
|
||||
If a user already has an account with email `doctor@example.com` and signs in via SSO with the same email:
|
||||
|
||||
1. The system finds the existing user by email
|
||||
2. Links the SSO identity (`oidc_sub`) to the existing account
|
||||
3. The user is logged in
|
||||
4. Future logins can use either method (email/password OR SSO)
|
||||
|
||||
**Database update performed:**
|
||||
```sql
|
||||
UPDATE users
|
||||
SET oidc_sub = '<provider-unique-id>',
|
||||
email_verified = true
|
||||
WHERE email = 'doctor@example.com';
|
||||
```
|
||||
|
||||
### Scenario 2: New User (No Matching Email)
|
||||
|
||||
If the SSO email doesn't match any existing user:
|
||||
|
||||
1. A new account is automatically created
|
||||
2. The user is assigned the `user` role (first user becomes `admin`)
|
||||
3. A random password is generated (not used for SSO logins)
|
||||
4. The user is logged in
|
||||
|
||||
### Scenario 3: Disabled User
|
||||
|
||||
If an existing user is disabled (`disabled = true` in database):
|
||||
|
||||
- SSO login is blocked
|
||||
- User sees an error message
|
||||
- Admin must re-enable the account from the Admin Panel
|
||||
|
||||
---
|
||||
|
||||
## Manual Account Linking (CLI)
|
||||
|
||||
If you need to manually link an existing user to an SSO identity, use the PostgreSQL database directly:
|
||||
|
||||
```bash
|
||||
# Connect to database
|
||||
docker exec -it pediatric-ai-scribe-postgres psql -U pedscribe -d pedscribe
|
||||
|
||||
# Link user by setting their oidc_sub
|
||||
UPDATE users
|
||||
SET oidc_sub = 'provider-sub-12345',
|
||||
email_verified = true
|
||||
WHERE email = 'doctor@example.com';
|
||||
```
|
||||
|
||||
**Finding the `oidc_sub` value:**
|
||||
|
||||
The `oidc_sub` is the unique identifier from your OIDC provider (usually a UUID or numeric ID). To find it:
|
||||
|
||||
1. Have the user attempt SSO login once
|
||||
2. Check the application logs for their `sub` claim:
|
||||
```
|
||||
[OIDC] User logged in: sub=abc-123-def, email=doctor@example.com
|
||||
```
|
||||
3. Use that `sub` value in the UPDATE statement
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### HTTPS Required in Production
|
||||
|
||||
OIDC requires HTTPS for security. Ensure your `APP_URL` uses `https://`:
|
||||
|
||||
```env
|
||||
APP_URL=https://scribe.example.com
|
||||
```
|
||||
|
||||
### Client Secret Protection
|
||||
|
||||
The client secret is stored encrypted in the database. The admin UI masks it after saving (shows `••••••••1234`).
|
||||
|
||||
**Never commit the client secret to Git or share it publicly.**
|
||||
|
||||
### IP Allowlisting (Optional)
|
||||
|
||||
To restrict SSO to specific networks (e.g., hospital VPN):
|
||||
|
||||
1. Set **Allowed IPs** in admin settings to comma-separated CIDR ranges:
|
||||
```
|
||||
10.0.0.0/8, 192.168.1.0/24
|
||||
```
|
||||
2. Users outside these ranges will see an error when attempting SSO
|
||||
|
||||
### Disable Local Password Login
|
||||
|
||||
Once SSO is working, you can optionally disable traditional email/password login:
|
||||
|
||||
1. In Admin Settings, enable **Disable Local Auth**
|
||||
2. The login page will only show the SSO button
|
||||
3. Admins can still use the CLI to reset passwords if needed
|
||||
|
||||
**Warning:** Only disable local auth after confirming all users can access SSO. Keep one admin password as backup.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "SSO is not enabled" error
|
||||
|
||||
- Verify **Enabled** is set to `true` in admin settings
|
||||
- Check application logs for OIDC configuration errors
|
||||
|
||||
### "Invalid state" or "Expired" error
|
||||
|
||||
- The OIDC flow timed out (5 minute window)
|
||||
- Try logging in again
|
||||
- If persistent, check server time synchronization
|
||||
|
||||
### "No email claim" error
|
||||
|
||||
Your OIDC provider didn't return an email address. Ensure:
|
||||
|
||||
1. The `email` scope is requested (default: `openid email profile`)
|
||||
2. Your provider is configured to release email claims
|
||||
3. The user's account has an email address set
|
||||
|
||||
### Email Mismatch
|
||||
|
||||
If a user has different emails in the app vs. SSO provider:
|
||||
|
||||
**Option 1: Update app email to match SSO**
|
||||
```sql
|
||||
UPDATE users SET email = 'new-email@example.com' WHERE id = 123;
|
||||
```
|
||||
|
||||
**Option 2: Update SSO provider email to match app**
|
||||
(Provider-specific — consult your IdP documentation)
|
||||
|
||||
### Callback URL Not Working
|
||||
|
||||
Double-check the redirect URI in your OIDC provider settings matches exactly:
|
||||
|
||||
```
|
||||
https://your-domain.com/api/auth/oidc/callback
|
||||
```
|
||||
|
||||
Common mistakes:
|
||||
- Missing `https://`
|
||||
- Trailing slash (don't include it)
|
||||
- Wrong domain (must match `APP_URL` in `.env`)
|
||||
|
||||
---
|
||||
|
||||
## Provider-Specific Examples
|
||||
|
||||
### PocketID
|
||||
|
||||
```
|
||||
Issuer URL: https://id.pockethost.io
|
||||
Client ID: (from PocketID app settings)
|
||||
Client Secret: (from PocketID app settings)
|
||||
Redirect URI: https://your-domain.com/api/auth/oidc/callback
|
||||
```
|
||||
|
||||
### Keycloak
|
||||
|
||||
```
|
||||
Issuer URL: https://keycloak.example.com/realms/medical
|
||||
Client ID: pediatric-scribe
|
||||
Client Secret: (from Credentials tab)
|
||||
Redirect URI: https://your-domain.com/api/auth/oidc/callback
|
||||
```
|
||||
|
||||
### Azure AD / Entra ID
|
||||
|
||||
```
|
||||
Issuer URL: https://login.microsoftonline.com/{tenant-id}/v2.0
|
||||
Client ID: (Application ID from Azure)
|
||||
Client Secret: (from Certificates & secrets)
|
||||
Redirect URI: https://your-domain.com/api/auth/oidc/callback
|
||||
```
|
||||
|
||||
Note: Azure requires app registration in Azure Portal first.
|
||||
|
||||
### Okta
|
||||
|
||||
```
|
||||
Issuer URL: https://{your-okta-domain}.okta.com
|
||||
Client ID: (from Okta application settings)
|
||||
Client Secret: (from Okta application settings)
|
||||
Redirect URI: https://your-domain.com/api/auth/oidc/callback
|
||||
```
|
||||
|
||||
### Google (Workspace or Gmail)
|
||||
|
||||
```
|
||||
Issuer URL: https://accounts.google.com
|
||||
Client ID: (from Google Cloud Console)
|
||||
Client Secret: (from Google Cloud Console)
|
||||
Redirect URI: https://your-domain.com/api/auth/oidc/callback
|
||||
```
|
||||
|
||||
Note: Google requires OAuth consent screen configuration.
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables (Alternative to UI Config)
|
||||
|
||||
For deployment automation, you can set OIDC config via environment variables instead of the admin UI:
|
||||
|
||||
```env
|
||||
# .env file
|
||||
OIDC_ENABLED=true
|
||||
OIDC_ISSUER=https://id.example.com
|
||||
OIDC_CLIENT_ID=my-client-id
|
||||
OIDC_CLIENT_SECRET=my-client-secret
|
||||
OIDC_BUTTON_LABEL=Sign in with PocketID
|
||||
OIDC_DISABLE_LOCAL_AUTH=false
|
||||
```
|
||||
|
||||
**Note:** UI settings take precedence over environment variables. If set in both places, the database values are used.
|
||||
|
||||
---
|
||||
|
||||
## HIPAA Compliance Notes
|
||||
|
||||
OIDC does not transmit PHI to the identity provider. Only authentication-related data (email, name) is exchanged.
|
||||
|
||||
For HIPAA compliance:
|
||||
- Ensure your OIDC provider has appropriate safeguards
|
||||
- Use a self-hosted provider (Keycloak, PocketID) within your secure network
|
||||
- Or use a HIPAA-compliant SaaS provider with a BAA
|
||||
- Enable audit logging for all SSO login events (automatically logged in `audit_log` table)
|
||||
|
||||
---
|
||||
|
||||
## Audit Logging
|
||||
|
||||
All SSO login events are logged in the `audit_log` table:
|
||||
|
||||
```sql
|
||||
SELECT * FROM audit_log WHERE action = 'login_oidc' ORDER BY created_at DESC;
|
||||
```
|
||||
|
||||
Logged fields:
|
||||
- User ID
|
||||
- Action: `login_oidc`
|
||||
- IP address
|
||||
- Details: Issuer URL
|
||||
- Timestamp
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
For issues specific to:
|
||||
- **This application**: Check application logs with `docker logs pediatric-ai-scribe`
|
||||
- **Your OIDC provider**: Consult provider documentation (PocketID, Keycloak, Azure, etc.)
|
||||
- **Network/TLS issues**: Verify `APP_URL` matches your reverse proxy configuration
|
||||
|
||||
Common log locations:
|
||||
```bash
|
||||
# Application logs
|
||||
docker logs pediatric-ai-scribe
|
||||
|
||||
# PostgreSQL logs
|
||||
docker logs pediatric-ai-scribe-postgres
|
||||
```
|
||||
33
README.md
33
README.md
|
|
@ -201,6 +201,39 @@ OPENAI_API_KEY=sk-...
|
|||
|
||||
---
|
||||
|
||||
## OpenID Connect / SSO (optional — PocketID, Keycloak, Azure AD, etc.)
|
||||
|
||||
Enable Single Sign-On authentication with any OpenID Connect provider. Users can log in with their SSO provider, and existing accounts are automatically linked by email.
|
||||
|
||||
**Supported providers:** PocketID, Keycloak, Azure AD / Entra ID, Okta, Google Workspace, and any OIDC-compliant provider.
|
||||
|
||||
### Quick Setup
|
||||
|
||||
1. Register this app with your OIDC provider using callback URL:
|
||||
```
|
||||
https://your-domain.com/api/auth/oidc/callback
|
||||
```
|
||||
|
||||
2. Log into the app as admin and navigate to **Admin Panel** → **Settings**
|
||||
|
||||
3. Configure OIDC settings:
|
||||
- Enable OIDC
|
||||
- Set Issuer URL (e.g., `https://id.example.com`)
|
||||
- Enter Client ID and Client Secret from your provider
|
||||
- Customize the button label (e.g., "Sign in with PocketID")
|
||||
|
||||
4. Save and test the login
|
||||
|
||||
**Detailed setup guide with examples for all providers:** [OPENID_SETUP.md](OPENID_SETUP.md)
|
||||
|
||||
### Linking Existing Users
|
||||
|
||||
When a user signs in via SSO, the system automatically links their account if the email matches an existing user. No manual intervention needed.
|
||||
|
||||
For advanced scenarios (manual linking, CLI commands, troubleshooting), see [OPENID_SETUP.md](OPENID_SETUP.md).
|
||||
|
||||
---
|
||||
|
||||
## Email (optional — for verification & password reset)
|
||||
|
||||
Without SMTP configured, email verification is skipped and users are auto-verified on registration.
|
||||
|
|
|
|||
|
|
@ -153,10 +153,11 @@
|
|||
<div class="lh-ai-tabpanel hidden" id="lh-ai-tp-upload">
|
||||
<label class="lh-ai-dropzone" id="lh-ai-dropzone">
|
||||
<i class="fas fa-cloud-upload-alt" style="font-size:28px;color:var(--blue);margin-bottom:8px;display:block;"></i>
|
||||
<span id="lh-ai-file-label">Drop a file here or click to browse</span>
|
||||
<small style="color:var(--g400);display:block;margin-top:4px;">PDF, TXT, MD, HTML — max 20 MB</small>
|
||||
<input type="file" id="lh-ai-file" accept=".pdf,.txt,.md,.html,.htm,.csv,.json" style="display:none;">
|
||||
<span id="lh-ai-file-label">Drop files here or click to browse</span>
|
||||
<small style="color:var(--g400);display:block;margin-top:4px;">PDF, TXT, MD, HTML — max 100 MB each, up to 10 files</small>
|
||||
<input type="file" id="lh-ai-file" accept=".pdf,.txt,.md,.html,.htm,.csv,.json" multiple style="display:none;">
|
||||
</label>
|
||||
<div id="lh-ai-files-list" style="margin-top:8px;display:none;"></div>
|
||||
<div class="form-group" style="margin:8px 0 0;">
|
||||
<label style="font-size:12px;font-weight:600;color:var(--g600);">Topic / context <span style="color:var(--g400);font-weight:400;">(optional — helps AI focus)</span></label>
|
||||
<input type="text" id="lh-ai-upload-context" class="cms-input-sm" style="width:100%;" placeholder="e.g., Pediatric asthma management, focus on treatment ladder">
|
||||
|
|
|
|||
|
|
@ -547,11 +547,32 @@
|
|||
function wireCmsFileInput() {
|
||||
var fileInput = document.getElementById('lh-ai-file');
|
||||
var fileLabel = document.getElementById('lh-ai-file-label');
|
||||
var filesList = document.getElementById('lh-ai-files-list');
|
||||
var dropzone = document.getElementById('lh-ai-dropzone');
|
||||
|
||||
function updateFilesList() {
|
||||
if (!fileInput || !fileLabel || !filesList) return;
|
||||
var files = fileInput.files;
|
||||
if (files.length === 0) {
|
||||
fileLabel.textContent = 'Drop files here or click to browse';
|
||||
filesList.style.display = 'none';
|
||||
filesList.innerHTML = '';
|
||||
} else if (files.length === 1) {
|
||||
fileLabel.textContent = files[0].name;
|
||||
filesList.style.display = 'none';
|
||||
filesList.innerHTML = '';
|
||||
} else {
|
||||
fileLabel.textContent = files.length + ' files selected';
|
||||
filesList.style.display = 'block';
|
||||
filesList.innerHTML = '<ul style="list-style:none;padding:0;margin:0;font-size:0.9em;">' +
|
||||
Array.from(files).map(function(f) {
|
||||
return '<li style="padding:2px 0;"><i class="fas fa-file-pdf" style="color:var(--blue);margin-right:4px;"></i>' + f.name + '</li>';
|
||||
}).join('') + '</ul>';
|
||||
}
|
||||
}
|
||||
|
||||
if (fileInput) {
|
||||
fileInput.addEventListener('change', function() {
|
||||
if (fileInput.files[0] && fileLabel) fileLabel.textContent = fileInput.files[0].name;
|
||||
});
|
||||
fileInput.addEventListener('change', updateFilesList);
|
||||
}
|
||||
if (dropzone) {
|
||||
dropzone.addEventListener('dragover', function(e) { e.preventDefault(); dropzone.style.borderColor = 'var(--blue)'; });
|
||||
|
|
@ -560,14 +581,16 @@
|
|||
e.preventDefault();
|
||||
dropzone.style.borderColor = '';
|
||||
var files = e.dataTransfer.files;
|
||||
if (files[0] && fileInput && fileLabel) {
|
||||
if (files.length > 0 && fileInput) {
|
||||
// Transfer to file input via DataTransfer
|
||||
try {
|
||||
var dt = new DataTransfer();
|
||||
dt.items.add(files[0]);
|
||||
for (var i = 0; i < files.length && i < 10; i++) {
|
||||
dt.items.add(files[i]);
|
||||
}
|
||||
fileInput.files = dt.files;
|
||||
updateFilesList();
|
||||
} catch(ex) {}
|
||||
fileLabel.textContent = files[0].name;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
|
@ -799,8 +822,10 @@
|
|||
formData.append('topic', topic);
|
||||
} else if (tabName === 'upload') {
|
||||
var fileInput = document.getElementById('lh-ai-file');
|
||||
if (!fileInput || !fileInput.files[0]) { showToast('Select a file to upload', 'error'); return; }
|
||||
formData.append('file', fileInput.files[0]);
|
||||
if (!fileInput || fileInput.files.length === 0) { showToast('Select at least one file to upload', 'error'); return; }
|
||||
for (var i = 0; i < fileInput.files.length; i++) {
|
||||
formData.append('files', fileInput.files[i]);
|
||||
}
|
||||
var uploadCtx = document.getElementById('lh-ai-upload-context');
|
||||
if (uploadCtx && uploadCtx.value.trim()) formData.append('topic', uploadCtx.value.trim());
|
||||
} else if (tabName === 'webdav') {
|
||||
|
|
|
|||
|
|
@ -22,23 +22,36 @@
|
|||
|
||||
var state = {};
|
||||
|
||||
// Load milestones from API
|
||||
// Load milestones from API (database), fallback to static data if empty
|
||||
fetch('/api/milestones-data', {
|
||||
headers: getAuthHeaders()
|
||||
})
|
||||
.then(function(r) { return r.json(); })
|
||||
.then(function(data) {
|
||||
if (data.success) {
|
||||
if (data.success && data.milestones && Object.keys(data.milestones).length > 0) {
|
||||
// Database has milestones - use them
|
||||
MILESTONES_DATA = data.milestones;
|
||||
populateAgeGroups();
|
||||
} else {
|
||||
console.error('[Milestones] Failed to load:', data.error);
|
||||
showToast('Failed to load milestones data', 'error');
|
||||
// Database empty - fallback to static data
|
||||
console.log('[Milestones] Database empty, using static data');
|
||||
if (typeof window.MILESTONES_DATA_STATIC !== 'undefined') {
|
||||
MILESTONES_DATA = window.MILESTONES_DATA_STATIC;
|
||||
populateAgeGroups();
|
||||
} else {
|
||||
showToast('Milestones data not available', 'error');
|
||||
}
|
||||
}
|
||||
})
|
||||
.catch(function(err) {
|
||||
console.error('[Milestones] Load error:', err);
|
||||
showToast('Failed to load milestones: ' + err.message, 'error');
|
||||
console.error('[Milestones] Load error, falling back to static:', err);
|
||||
// Fallback to static data on API error
|
||||
if (typeof window.MILESTONES_DATA_STATIC !== 'undefined') {
|
||||
MILESTONES_DATA = window.MILESTONES_DATA_STATIC;
|
||||
populateAgeGroups();
|
||||
} else {
|
||||
showToast('Failed to load milestones: ' + err.message, 'error');
|
||||
}
|
||||
});
|
||||
|
||||
function populateAgeGroups() {
|
||||
|
|
|
|||
|
|
@ -4,7 +4,8 @@
|
|||
// THIS MODULE IS COMPLETELY SEPARATE FROM HPI
|
||||
// ============================================================
|
||||
|
||||
var MILESTONES_DATA = {
|
||||
// Static fallback data - used when database is empty
|
||||
var MILESTONES_DATA_STATIC = {
|
||||
|
||||
"Newborn / 1 month": {
|
||||
"Gross Motor": [
|
||||
|
|
@ -713,4 +714,6 @@ var DOMAIN_CONFIG = {
|
|||
"Cognitive": { icon: "🧠", css: "domain-cognitive" }
|
||||
};
|
||||
|
||||
console.log('✅ Milestones data loaded:', Object.keys(MILESTONES_DATA).length, 'age groups');
|
||||
// Expose as window global for fallback usage
|
||||
window.MILESTONES_DATA_STATIC = MILESTONES_DATA_STATIC;
|
||||
console.log('✅ Static milestones data loaded:', Object.keys(MILESTONES_DATA_STATIC).length, 'age groups');
|
||||
|
|
|
|||
|
|
@ -16,7 +16,26 @@ router.use(moderatorMiddleware);
|
|||
|
||||
var upload = multer({
|
||||
storage: multer.memoryStorage(),
|
||||
limits: { fileSize: 20 * 1024 * 1024 } // 20 MB
|
||||
limits: {
|
||||
fileSize: 100 * 1024 * 1024, // 100 MB per file (large PDFs supported)
|
||||
files: 10 // max 10 files at once
|
||||
},
|
||||
fileFilter: function(req, file, cb) {
|
||||
// Whitelist allowed file types
|
||||
var allowed = [
|
||||
'application/pdf',
|
||||
'text/plain',
|
||||
'text/markdown',
|
||||
'text/html',
|
||||
'text/csv',
|
||||
'application/json'
|
||||
];
|
||||
if (allowed.includes(file.mimetype) || file.originalname.match(/\.(pdf|txt|md|html|htm|csv|json)$/i)) {
|
||||
cb(null, true);
|
||||
} else {
|
||||
cb(new Error('File type not allowed. Only PDF, TXT, MD, HTML, CSV, JSON are supported.'));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// ── Text extraction helpers ──────────────────────────────────
|
||||
|
|
@ -156,7 +175,7 @@ Rules:
|
|||
// ── POST /api/admin/learning/ai-generate ────────────────────
|
||||
// Accepts: multipart/form-data OR application/json
|
||||
|
||||
router.post('/ai-generate', upload.single('file'), async function(req, res) {
|
||||
router.post('/ai-generate', upload.array('files', 10), async function(req, res) {
|
||||
try {
|
||||
var topic = req.body.topic || '';
|
||||
var contentType = req.body.contentType || 'article';
|
||||
|
|
@ -168,10 +187,23 @@ router.post('/ai-generate', upload.single('file'), async function(req, res) {
|
|||
var slideCount = parseInt(req.body.slideCount) || 0;
|
||||
|
||||
var docText = '';
|
||||
var fileCount = 0;
|
||||
|
||||
// 1 — Uploaded file
|
||||
if (req.file) {
|
||||
docText = await extractText(req.file.buffer, req.file.mimetype, req.file.originalname);
|
||||
// 1 — Uploaded files (multiple)
|
||||
if (req.files && req.files.length > 0) {
|
||||
var allTexts = [];
|
||||
for (var i = 0; i < req.files.length; i++) {
|
||||
var file = req.files[i];
|
||||
try {
|
||||
var text = await extractText(file.buffer, file.mimetype, file.originalname);
|
||||
allTexts.push('### Source File: ' + file.originalname + '\n\n' + text);
|
||||
fileCount++;
|
||||
} catch (e) {
|
||||
console.error('[LearningAI] Failed to extract text from ' + file.originalname + ':', e.message);
|
||||
// Continue with other files even if one fails
|
||||
}
|
||||
}
|
||||
docText = allTexts.join('\n\n---\n\n');
|
||||
}
|
||||
|
||||
// 2 — Nextcloud WebDAV path
|
||||
|
|
@ -287,7 +319,8 @@ router.post('/ai-generate', upload.single('file'), async function(req, res) {
|
|||
success: true,
|
||||
content: parsed,
|
||||
model: result.model,
|
||||
docLength: docText.length
|
||||
docLength: docText.length,
|
||||
fileCount: fileCount || (webdavPath ? 1 : 0)
|
||||
});
|
||||
|
||||
} catch (err) {
|
||||
|
|
|
|||
|
|
@ -24,6 +24,9 @@ async function generateEmbedding(text, opts) {
|
|||
var dimensions = opts.dimensions || parseInt(process.env.EMBEDDING_DIMENSIONS) || DEFAULT_DIMS;
|
||||
|
||||
// Truncate text to ~2000 tokens (~8000 chars) to avoid API errors
|
||||
// NOTE: Large PDFs (e.g., 100MB) will be truncated to first ~8000 chars for embedding.
|
||||
// The full PDF content is still extracted and stored in the database body field.
|
||||
// This is expected behavior - embeddings are semantic representations, not full-text storage.
|
||||
var truncated = text.substring(0, 8000);
|
||||
if (!truncated.trim()) {
|
||||
throw new Error('Empty text provided for embedding');
|
||||
|
|
|
|||
Loading…
Reference in a new issue