Commit graph

9 commits

Author SHA1 Message Date
Sanjeev Kumar
89991bcc4a
Merge pull request #4 from ksanjeev284/feature/observability-enhancements
fix: --limit flag now works correctly + video download with audio
2025-12-14 07:10:00 +05:30
Sanjeev Kumar
076404906d fix: --limit flag now works correctly + video download with audio
- Fixed --limit flag being ignored (was always requesting 100 posts)
- Added ffmpeg-based audio merging for Reddit videos
- Updated README with ffmpeg requirement
- Applied fixes to both main.py and async_scraper.py
2025-12-14 07:09:19 +05:30
Sanjeev Kumar
a3f8030934
Merge pull request #1 from ksanjeev284/feature/observability-enhancements
feat: Add long-running observability and integration features
2025-12-14 03:43:01 +05:30
Sanjeev Kumar
f65b35f881 feat: Add long-running observability and integration features
- Job history tracking with SQLite table
- Dry-run mode (--dry-run) to test scrape rules
- Plugin system with 3 built-in plugins (sentiment, dedupe, keywords)
- REST API server (--api) for Metabase/Grafana integration
- Parquet export (--export-parquet) for DuckDB/warehouses
- SQLite maintenance (--backup, --vacuum)
- Dashboard Integrations tab with external tools guides
- Updated Dockerfile and docker-compose.yml for cloud deployment
- Comprehensive README documentation
2025-12-14 03:42:24 +05:30
Sanjeev Kumar
facd3a50af
Update README.md 2025-12-13 22:58:54 +05:30
Sanjeev Kumar
0a1a604f49 v3.0: Full suite - Dashboard, Analytics, Scheduling, Notifications, Search 2025-12-13 22:17:32 +05:30
Sanjeev Kumar
2eb9ef9a54 v2.0: Full media & comment scraping support 2025-12-13 22:06:03 +05:30
Sanjeev Kumar
d1a475a5dc Add GitHub Actions CI/CD pipeline with build badge 2025-12-13 21:55:06 +05:30
Sanjeev Kumar
db4e363130 Initial release of Universal Scraper 2025-12-13 21:51:02 +05:30