Logs Collector: Secure Tool for Log File Collection

Logs Collector is a highly reliable and functional server application designed as a convenient tool for the controlled and secure collection of diagnostic data (log files) from remote client infrastructure. The project aims to optimize the process of technical support and error diagnostics when direct access to client logs is restricted or impossible due to confidentiality concerns.

The system's architecture is built upon modern DevOps and security principles, offering deployment flexibility and ease of integration.

⚙️ Key Features and Support Value

Structured and Controlled Data Collection Process

The system ensures a transparent workflow where every log upload session is tied to a specific support ticket:

• Ticket-Centric Workflow: Creation of tickets and association with unique access tokens. This ensures full traceability: it is always clear which ticket the uploaded file belongs to.
• Centralized Storage: Centralized, secure storage of all archives.
• Data Lifecycle Management: Deleting a ticket automatically results in the secure physical deletion of associated log files from storage.
• Capacity Monitoring: Built-in mechanism to track free space in storage to proactively prevent disk space issues.

Client Convenience: Console Uploader

To maximize client convenience and ease of use, an additional Shell script (uploader.sh) has been developed.

• Purpose: The script can be shared with the client to perform log archive uploads directly from their server's console using curl.
• Advantage: This eliminates the need for a web interface for uploading and allows for the automation of large file transfers using a simple command execution. The script requires only three parameters: domain, token, and file path.

🔒 Security and Reliability (Trust and Reliability)

The project employs best practices to ensure secure data transfer:

💻 Technology Stack and Architecture

Stack and API

• Backend: Python
• Web Framework: Django
• API: Full-featured RestFull API v1 for managing the lifecycle of tickets and uploads.
• Databases: Flexible support for PostgreSQL 15+ (for Production) and SQLite3.

Deployment and Operation (DevOps Focus)

1. Full Containerization: Optimized, lightweight Docker image (less than 150 MB) with fast build times.
2. Flexible Configuration (12 Factor App): All settings are managed via environment variables, ensuring simple and rapid deployment in any environment.
3. Gunicorn Optimization: Gunicorn WSGI server settings (timeout up to 3600 seconds) are calibrated to reliably handle potentially long HTTP sessions during large log archive transfers.
4. Reverse Proxy Compliance: The application is designed for mandatory operation behind a Reverse Proxy. Logging is configured to read the X-Forwarded-For header for correct real IP address display.
5. Health Check: A standard endpoint is provided for automated availability monitoring.

🔗 My Role in the Project and Open Source

My Contribution

• Backend Development: Implementation of all API logic, ORM, business logic, and file handling in Python/DRF.
• Frontend Development: Implementation of responsive design based on Bootstrap 5 and JS.
• DevOps & Deployment: Design of containerization architecture (Dockerfile, Docker Compose), optimization of Gunicorn settings.
• Security Implementation: Integration of token mechanisms, 2FA setup, and framework-level security assurance.
• Tooling: Development of the auxiliary uploader.sh script to enhance the user experience.

Licensing

• The project is distributed under the GNU GPL 3.0 license, demonstrating a commitment to open-source principles.