⚡ Updated .gitignore to include additional Python, IDE, OS, and application-specific artifacts for better project cleanliness.

- Added a startup event in `api_server.py` to verify database connectivity and provide warnings if no workflows are found. - Implemented new API endpoints for fetching service categories and searching workflows by category, improving the API's usability and functionality. - Removed deprecated scripts and files to streamline the codebase and focus on the new FastAPI system.
2025-06-22 01:01:58 +02:00
parent 964665bd13
commit 25cfefe303
13 changed files with 686 additions and 3259 deletions
--- a/README.md
+++ b/README.md
@@ -12,21 +12,22 @@ A professionally organized collection of **2,053 n8n workflows** with a lightnin
 pip install -r requirements.txt

 # Start the fast API server
-python3 api_server.py
+python run.py

 # Open in browser
 http://localhost:8000
 ```

 **Features:**
- ⚡ **Sub-100ms response times** (vs 10+ seconds before)
- 🔍 **Instant full-text search** with ranking and filters
+- ⚡ **Sub-100ms response times** with SQLite FTS5 search
+- 🔍 **Instant full-text search** with advanced filtering
 - 📱 **Responsive design** - works perfectly on mobile
 - 🌙 **Dark/light themes** with system preference detection
- 📊 **Live statistics** and workflow insights
+- 📊 **Live statistics** - 365 unique integrations, 29,445 total nodes
 - 🎯 **Smart categorization** by trigger type and complexity
 - 📄 **On-demand JSON viewing** and download
 - 🔗 **Mermaid diagram generation** for workflow visualization
+- 🔄 **Real-time workflow naming** with intelligent formatting

 ### Performance Comparison

@@ -44,15 +45,16 @@ http://localhost:8000

 ### Workflow Collection
 - **2,053 workflows** with meaningful, searchable names
- **Professional naming convention** - `[ID]_[Service]_[Purpose]_[Trigger].json`
- **Comprehensive coverage** - 100+ services and use cases
+- **365 unique integrations** across popular platforms
+- **29,445 total nodes** with professional categorization
 - **Quality assurance** - All workflows analyzed and categorized

-### Recent Improvements
- ✅ **858 generic workflows renamed** from meaningless "workflow_XXX" patterns
- ✅ **36 overly long names shortened** while preserving meaning
- ✅ **9 broken filenames fixed** with proper extensions
- ✅ **100% success rate** with zero data loss during transformation
+### Advanced Naming System ✨
+Our intelligent naming system converts technical filenames into readable titles:
+- **Before**: `2051_Telegram_Webhook_Automation_Webhook.json`
+- **After**: `Telegram Webhook Automation`
+- **100% meaningful names** with smart capitalization
+- **Automatic integration detection** from node analysis

 ---

@@ -60,23 +62,29 @@ http://localhost:8000

 ### Option 1: Modern Fast System (Recommended)
 ```bash
+# Clone repository
+git clone <repo-url>
+cd n8n-workflows
+
 # Install Python dependencies
-pip install fastapi uvicorn
+pip install -r requirements.txt

 # Start the documentation server
-python3 api_server.py
+python run.py

 # Browse workflows at http://localhost:8000
-# - Instant search and filtering
+# - Instant search across 2,053 workflows
 # - Professional responsive interface
 # - Real-time workflow statistics
 ```

-### Option 2: Legacy System (Deprecated)
+### Option 2: Development Mode
 ```bash
-# ⚠️ WARNING: Generates 71MB file, very slow
-python3 generate_documentation.py
-# Then open workflow-documentation.html
+# Start with auto-reload for development
+python api_server.py --reload
+
+# Or specify custom host/port
+python api_server.py --host 0.0.0.0 --port 3000
 ```

 ### Import Workflows into n8n
@@ -85,84 +93,95 @@ python3 generate_documentation.py
 3. Choose any `.json` file from the `workflows/` folder
 4. Update credentials/webhook URLs before running

-### Bulk Import All Workflows
-```bash
-./import-workflows.sh
-```
-
 ---

 ## 📊 Workflow Statistics

+### Current Collection Stats
 - **Total Workflows**: 2,053 automation workflows
- **Naming Quality**: 100% meaningful names (improved from 58%)
- **Categories**: Data sync, notifications, integrations, monitoring
- **Services**: 100+ platforms (Gmail, Slack, Notion, Stripe, etc.)
- **Complexity Range**: Simple 2-node to complex 50+ node automations
- **File Format**: Standard n8n-compatible JSON exports
+- **Active Workflows**: 215 (10.5% active rate)
+- **Total Nodes**: 29,445 (avg 14.3 nodes per workflow)
+- **Unique Integrations**: 365 different services and APIs
+- **Database**: SQLite with FTS5 full-text search

 ### Trigger Distribution
- **Manual**: ~40% - User-initiated workflows
- **Webhook**: ~25% - API-triggered automations  
- **Scheduled**: ~20% - Time-based executions
- **Complex**: ~15% - Multi-trigger systems
+- **Complex**: 831 workflows (40.5%) - Multi-trigger systems
+- **Webhook**: 519 workflows (25.3%) - API-triggered automations  
+- **Manual**: 477 workflows (23.2%) - User-initiated workflows
+- **Scheduled**: 226 workflows (11.0%) - Time-based executions

-### Complexity Levels
- **Low (≤5 nodes)**: ~45% - Simple automations
- **Medium (6-15 nodes)**: ~35% - Standard workflows
- **High (16+ nodes)**: ~20% - Complex systems
+### Complexity Analysis
+- **Low (≤5 nodes)**: ~35% - Simple automations
+- **Medium (6-15 nodes)**: ~45% - Standard workflows
+- **High (16+ nodes)**: ~20% - Complex enterprise systems
+
+### Popular Integrations
+Top services by usage frequency:
+- **Communication**: Telegram, Discord, Slack, WhatsApp
+- **Cloud Storage**: Google Drive, Google Sheets, Dropbox
+- **Databases**: PostgreSQL, MySQL, MongoDB, Airtable
+- **AI/ML**: OpenAI, Anthropic, Hugging Face
+- **Development**: HTTP Request, Webhook, GraphQL

 ---

-## 📋 Naming Convention
+## 🔍 Advanced Search Features

-### Standard Format
-```
-[ID]_[Service1]_[Service2]_[Purpose]_[Trigger].json
-```
+### Smart Search Categories
+Our system automatically categorizes workflows into 12 service categories:

-### Examples
+#### Available Categories:
+- **messaging**: Telegram, Discord, Slack, WhatsApp, Teams
+- **ai_ml**: OpenAI, Anthropic, Hugging Face 
+- **database**: PostgreSQL, MySQL, MongoDB, Redis, Airtable
+- **email**: Gmail, Mailjet, Outlook, SMTP/IMAP
+- **cloud_storage**: Google Drive, Google Docs, Dropbox, OneDrive
+- **project_management**: Jira, GitHub, GitLab, Trello, Asana
+- **social_media**: LinkedIn, Twitter/X, Facebook, Instagram
+- **ecommerce**: Shopify, Stripe, PayPal
+- **analytics**: Google Analytics, Mixpanel
+- **calendar_tasks**: Google Calendar, Cal.com, Calendly
+- **forms**: Typeform, Google Forms, Form Triggers
+- **development**: Webhook, HTTP Request, GraphQL, SSE
+
+### API Usage Examples
 ```bash
-# Good naming examples:
-100_Gmail_Slack_Notification_Webhook.json
-250_Stripe_Hubspot_Invoice_Sync.json
-375_Airtable_Data_Backup_Scheduled.json
+# Search workflows by text
+curl "http://localhost:8000/api/workflows?q=telegram+automation"

-# Service mappings:
-n8n-nodes-base.gmail → Gmail
-n8n-nodes-base.slack → Slack
-n8n-nodes-base.webhook → Webhook
+# Filter by trigger type and complexity
+curl "http://localhost:8000/api/workflows?trigger=Webhook&complexity=high"
+
+# Find all messaging workflows
+curl "http://localhost:8000/api/workflows/category/messaging"
+
+# Get database statistics
+curl "http://localhost:8000/api/stats"
+
+# Browse available categories
+curl "http://localhost:8000/api/categories"
 ```

-### Purpose Categories
- **Create** - Creating new records/content
- **Update** - Updating existing data
- **Sync** - Synchronizing between systems
- **Send** - Sending notifications/messages
- **Monitor** - Monitoring and alerting
- **Process** - Data processing/transformation
- **Import/Export** - Data migration tasks
-
 ---

 ## 🏗 Technical Architecture

-### Modern Stack (New System)
- **SQLite Database** - FTS5 full-text search, indexed metadata
- **FastAPI Backend** - REST API with automatic documentation
- **Responsive Frontend** - Single-file HTML with embedded assets
- **Smart Analysis** - Automatic workflow categorization
+### Modern Stack
+- **SQLite Database** - FTS5 full-text search with 365 indexed integrations
+- **FastAPI Backend** - RESTful API with automatic OpenAPI documentation
+- **Responsive Frontend** - Modern HTML5 with embedded CSS/JavaScript
+- **Smart Analysis** - Automatic workflow categorization and naming

 ### Key Features
- **Change Detection** - Only reprocess modified workflows
- **Background Indexing** - Non-blocking workflow analysis
- **Compressed Responses** - Gzip middleware for speed
- **Virtual Scrolling** - Handle thousands of workflows smoothly
- **Lazy Loading** - Diagrams and JSON loaded on demand
+- **Change Detection** - MD5 hashing for efficient re-indexing
+- **Background Processing** - Non-blocking workflow analysis
+- **Compressed Responses** - Gzip middleware for optimal speed
+- **Error Handling** - Graceful degradation and comprehensive logging
+- **Mobile Optimization** - Touch-friendly interface design

-### Database Schema
+### Database Performance
 ```sql
-- Optimized for search and filtering
+-- Optimized schema for lightning-fast queries
 CREATE TABLE workflows (
    id INTEGER PRIMARY KEY,
    filename TEXT UNIQUE,
@@ -171,14 +190,16 @@ CREATE TABLE workflows (
    trigger_type TEXT,
    complexity TEXT,
    node_count INTEGER,
-    integrations TEXT,  -- JSON array
-    tags TEXT,         -- JSON array
-    file_hash TEXT     -- For change detection
+    integrations TEXT,  -- JSON array of 365 unique services
+    description TEXT,
+    file_hash TEXT,     -- MD5 for change detection
+    analyzed_at TIMESTAMP
 );

-- Full-text search capability
+-- Full-text search with ranking
 CREATE VIRTUAL TABLE workflows_fts USING fts5(
-    filename, name, description, integrations, tags
+    filename, name, description, integrations, tags,
+    content='workflows', content_rowid='id'
 );
 ```

@@ -189,6 +210,7 @@ CREATE VIRTUAL TABLE workflows_fts USING fts5(
 ### System Requirements
 - **Python 3.7+** - For running the documentation system
 - **Modern Browser** - Chrome, Firefox, Safari, Edge
+- **50MB Storage** - For SQLite database and indexes
 - **n8n Instance** - For importing and running workflows

 ### Installation
@@ -197,14 +219,13 @@ CREATE VIRTUAL TABLE workflows_fts USING fts5(
 git clone <repo-url>
 cd n8n-workflows

-# Install dependencies (for fast system)
+# Install dependencies
 pip install -r requirements.txt

 # Start documentation server
-python3 api_server.py --port 8000
+python run.py

-# Or use legacy system (not recommended)
-python3 generate_documentation.py
+# Access at http://localhost:8000
 ```

 ### Development Setup
@@ -215,10 +236,75 @@ source .venv/bin/activate  # Linux/Mac
 # or .venv\Scripts\activate  # Windows

 # Install dependencies
-pip install fastapi uvicorn
+pip install -r requirements.txt

 # Run with auto-reload for development
-python3 api_server.py --reload
+python api_server.py --reload
+
+# Force database reindexing
+python workflow_db.py --index --force
+```
+
+---
+
+## 📋 Naming Convention
+
+### Intelligent Formatting System
+Our system automatically converts technical filenames to user-friendly names:
+
+```bash
+# Automatic transformations:
+2051_Telegram_Webhook_Automation_Webhook.json → "Telegram Webhook Automation"
+0250_HTTP_Discord_Import_Scheduled.json → "HTTP Discord Import Scheduled"  
+0966_OpenAI_Data_Processing_Manual.json → "OpenAI Data Processing Manual"
+```
+
+### Technical Format
+```
+[ID]_[Service1]_[Service2]_[Purpose]_[Trigger].json
+```
+
+### Smart Capitalization Rules
+- **HTTP** → HTTP (not Http)
+- **API** → API (not Api)  
+- **webhook** → Webhook
+- **automation** → Automation
+- **scheduled** → Scheduled
+
+---
+
+## 🚀 API Documentation
+
+### Core Endpoints
+- `GET /` - Main workflow browser interface
+- `GET /api/stats` - Database statistics and metrics
+- `GET /api/workflows` - Search with filters and pagination
+- `GET /api/workflows/{filename}` - Detailed workflow information
+- `GET /api/workflows/{filename}/download` - Download workflow JSON
+- `GET /api/workflows/{filename}/diagram` - Generate Mermaid diagram
+
+### Advanced Search
+- `GET /api/workflows/category/{category}` - Search by service category
+- `GET /api/categories` - List all available categories
+- `GET /api/integrations` - Get integration statistics
+- `POST /api/reindex` - Trigger background reindexing
+
+### Response Examples
+```json
+// GET /api/stats
+{
+  "total": 2053,
+  "active": 215,
+  "inactive": 1838,
+  "triggers": {
+    "Complex": 831,
+    "Webhook": 519,
+    "Manual": 477,
+    "Scheduled": 226
+  },
+  "total_nodes": 29445,
+  "unique_integrations": 365
+}
 ```

 ---
@@ -227,100 +313,79 @@ python3 api_server.py --reload

 ### Adding New Workflows
 1. **Export workflow** as JSON from n8n
-2. **Name descriptively** following the naming convention
+2. **Name descriptively** following the established pattern
 3. **Add to workflows/** directory
-4. **Test the workflow** before contributing
-5. **Remove sensitive data** (credentials, personal URLs)
-
-### Naming Guidelines
- Use clear, descriptive names
- Follow the established format: `[ID]_[Service]_[Purpose].json`
- Maximum 80 characters when possible
- Use underscores instead of spaces
+4. **Remove sensitive data** (credentials, personal URLs)
+5. **Run reindexing** to update the database

 ### Quality Standards
- ✅ Workflow must be functional
+- ✅ Workflow must be functional and tested
 - ✅ Remove all credentials and sensitive data
- ✅ Add meaningful description in workflow name
- ✅ Test in clean n8n instance
- ✅ Follow naming convention
-
---
-
-## 📚 Workflow Sources
-
-This collection includes workflows from:
- **Official n8n.io** - Website and community forum
- **GitHub repositories** - Public community contributions
- **Blog posts & tutorials** - Real-world examples
- **User submissions** - Tested automation patterns
- **Documentation examples** - Official n8n guides
+- ✅ Follow naming convention for consistency
+- ✅ Verify compatibility with recent n8n versions
+- ✅ Include meaningful description or comments

 ---

 ## ⚠️ Important Notes

 ### Security & Privacy
- **Review before use** - All workflows shared as-is
- **Update credentials** - Remove/replace API keys and tokens
+- **Review before use** - All workflows shared as-is for educational purposes
+- **Update credentials** - Replace API keys, tokens, and webhooks
 - **Test safely** - Verify in development environment first
- **Check permissions** - Ensure proper access rights
+- **Check permissions** - Ensure proper access rights for integrations

 ### Compatibility
- **n8n Version** - Most workflows compatible with recent versions
- **Community Nodes** - Some may require additional node installations
- **API Changes** - External services may have updated their APIs
- **Dependencies** - Check required integrations before importing
+- **n8n Version** - Compatible with n8n 1.0+ (most workflows)
+- **Community Nodes** - Some workflows may require additional node installations
+- **API Changes** - External services may have updated their APIs since creation
+- **Dependencies** - Verify required integrations before importing

 ---

-## 🎯 Quick Start Guide
+## 📚 Resources & References

-1. **Clone Repository**
-   ```bash
-   git clone <repo-url>
-   cd n8n-workflows
-   ```
+### Workflow Sources
+This comprehensive collection includes workflows from:
+- **Official n8n.io** - Documentation and community examples
+- **GitHub repositories** - Open source community contributions  
+- **Blog posts & tutorials** - Real-world automation patterns
+- **User submissions** - Tested and verified workflows
+- **Enterprise use cases** - Business process automations

-2. **Start Fast Documentation**
-   ```bash
-   pip install fastapi uvicorn
-   python3 api_server.py
-   ```
-
-3. **Browse Workflows**
-   - Open http://localhost:8000
-   - Use instant search and filters
-   - Explore workflow categories
-
-4. **Import & Use**
-   - Find interesting workflows
-   - Download JSON files
-   - Import into your n8n instance
-   - Update credentials and test
+### Learn More
+- [n8n Documentation](https://docs.n8n.io/) - Official documentation
+- [n8n Community](https://community.n8n.io/) - Community forum and support
+- [Workflow Templates](https://n8n.io/workflows/) - Official template library
+- [Integration Docs](https://docs.n8n.io/integrations/) - Service-specific guides

 ---

 ## 🏆 Project Achievements

 ### Repository Transformation
- **903 workflows renamed** with intelligent content analysis
- **100% meaningful names** (improved from 58% well-named)
- **Professional organization** with consistent standards
- **Zero data loss** during renaming process
+- **2,053 workflows** professionally organized and named
+- **365 unique integrations** automatically detected and categorized
+- **100% meaningful names** (improved from basic filename patterns)
+- **Zero data loss** during intelligent renaming process
+- **Advanced search** with 12 service categories

 ### Performance Revolution
- **71MB → <100KB** documentation size (700x improvement)
- **10+ seconds → <1 second** load time (10x faster)
- **Client-side → Server-side** search (infinite scalability)
- **Static → Dynamic** interface (modern user experience)
+- **Sub-100ms search** with SQLite FTS5 full-text indexing
+- **Instant filtering** across 29,445 workflow nodes
+- **Mobile-optimized** responsive design for all devices
+- **Real-time statistics** with live database queries
+- **Professional interface** with modern UX principles

-### Quality Improvements
- **Intelligent categorization** - Automatic trigger and complexity detection
- **Enhanced searchability** - Full-text search with ranking
- **Mobile optimization** - Responsive design for all devices
- **Professional presentation** - Clean, modern interface
+### System Reliability
+- **Robust error handling** with graceful degradation
+- **Change detection** for efficient database updates
+- **Background processing** for non-blocking operations
+- **Comprehensive logging** for debugging and monitoring
+- **Production-ready** with proper middleware and security

 ---

-*This repository represents the most comprehensive and well-organized collection of n8n workflows available, with cutting-edge documentation technology that makes workflow discovery and usage a delightful experience.*
+*This repository represents the most comprehensive and well-organized collection of n8n workflows available, featuring cutting-edge search technology and professional documentation that makes workflow discovery and usage a delightful experience.*
+
+**🎯 Perfect for**: Developers, automation engineers, business analysts, and anyone looking to streamline their workflows with proven n8n automations.