Skip to content

FenLiu (分流) Documentation

Welcome to the FenLiu documentation! FenLiu is a Fediverse content stream filtering system that applies ancient Chinese water management principles to modern social media content distribution.

📚 Live Documentation: https://marvinsmastodontools.codeberg.page/dujiangyan/fenliu/ 📦 Repository: https://codeberg.org/marvinsmastodontools/fenliu

What is FenLiu?

FenLiu helps you: - 🌊 Monitor multiple hashtag streams from the Fediverse - 📊 Score posts using rule-based spam detection - ✅ Review and approve quality content before sharing - 📤 Export curated content reliably via Curated Queue API - 🚫 Control which posts are eligible for export with fine-grained filters

Quick Navigation

Start here if you're new to FenLiu:

Learn how to use FenLiu's features:

Integrate FenLiu with external systems:

Contribute to or extend FenLiu:

Key Features

📊 Intelligent Spam Scoring

Automatically score posts using rule-based detection with 7 intelligent detection rules. Achieves >80% accuracy on obvious spam detection.

🎯 Fine-Grained Export Controls

  • Block specific users from being boosted
  • Exclude posts containing blocked hashtags
  • Require media attachments for export
  • Automatically reject blocked content before review

📈 Real-Time Statistics

  • Monitor post flow across all streams
  • Track approval rates and review progress
  • Visualize top hashtags and posting patterns

🔄 Curated Queue Integration

  • Reliable Queue API with ack/nack/error handling
  • State Machine with pending → reserved → delivered | error transitions
  • Timeout Management for in-flight posts (5 minutes)
  • Random Selection: ?random=true for variety, with author-fairness rule
  • Queue Preview Dashboard showing queue health and all-time statistics
  • Lifecycle Management: auto-cleanup of old delivered posts, trim excess pending

🚀 Container Support

  • Podman-based containerization via Containerfile
  • Environment-variable configuration via .env

👥 Review Workflow

  • Manually approve or reject posts
  • Adjust spam scores based on context
  • Track review decisions and history

⏰ Automated Scheduling

  • Configurable fetch intervals (5-1440 minutes)
  • Per-stream scheduling control
  • Bulk fetch from all active streams

System Architecture

FenLiu consists of:

  • Backend: Python 3.12+ with PyView (Starlette-based LiveView) for web serving
  • Database: SQLite with SQLAlchemy 2.0 ORM and Alembic migrations
  • Frontend: Jinja2 templates with Tailwind CSS, fully responsive
  • APIs: RESTful endpoints for all operations with API key authentication

See System Design for detailed architecture information.

Version Information

  • Current Version: 0.7.1
  • Status: Phase 4 In Progress
  • Tests: 411 passing
  • Code Quality: All checks passing

Latest features: - Random queue selection: GET /api/v1/curated/next?random=true with author-fairness rule (v0.7.1) - Review page pagination, bulk approve/reject, back-to-top link (v0.7.0) - ML training data snapshots on every review action (v0.7.0) - Queue Lifecycle Management: auto-cleanup and trim-pending (v0.6.0) - Containerization with Podman (v0.6.0) - Pattern-based user blocking (exact, suffix, prefix, contains) (v0.5.3)

Getting Help

  • 📖 Check the FAQ for common questions
  • 🐛 Report issues on Codeberg
  • 💬 Discuss on Codeberg
  • 📚 Read the Roadmap for upcoming features

Inspired by the Dujiangyan irrigation system (256 BC) - ancient water management principles applied to modern content distribution.