melhin

Deploying Django applications with background tasks typically requires managing multiple processes and queues. But what if you could run everything in a single process without sacrificing isolation or performance? This talk shows how Python 3.14's subinterpreters enable a practical pattern for self-contained Django applications: The Pattern: - Web server and task workers run in separate subinterpreters within one process - True isolation prevents memory leaks and state contamination between components - Simplified deployment: one process, one server, one application What This Solves: - Deployment simplicity: No separate worker processes, no queue infrastructure - Resource efficiency: Shared memory space with isolated execution - Self-contained applications: Perfect for internal tools, small services, or edge deployments We'll walk through code showing: - Setting up Django web server and task workers in InterpreterPoolExecutor - Inter-interpreter communication using shared queues between webserver and task worker - We will also look at a version using PostgreSQL as a backend and how to run tasks - Running CPU-intensive tasks (image processing, PDF parsing) without blocking web responses - Gotchas and errors that occur, how to monitor them, and how to build in a mechanism for recovery - Error recovery mechanisms: - Worker subinterpreter crash detection and automatic restart - Graceful shutdown handling for in-flight tasks We will also discuss: - Limited third-party library support - Difficulties in exception handling and retry mechanisms This is a practical pattern for real Django applications that need background processing without the complexity of traditional distributed task queues. You'll leave with concrete examples you can adapt for your own single-server Django deployments. Target audience: Django developers who deploy their own applications and want simpler background task processing.