Posted on 20 Sep 2026

Introduction

In the previous article we established the theory: requirements produce domain entities, domain entities become REST resources, and HTTP methods map to the lifecycle operations of those resources. Kubernetes was the illustration — large enough to make the pattern clear, but too complex to implement in a blog post.

This article moves from theory to practice. The project is FastURL, a URL shortener and link inspector built with FastAPI. It is small enough to fit comfortably in a few articles, but real enough to demonstrate every concept that matters: domain modeling, Pydantic validation as a form of type-safe Value Objects, a standardized error envelope, and a resource hierarchy derived directly from the domain.

You should read this article if:

• You have read the previous article and want to see the methodology applied to real code.
• You want to understand how Pydantic v2 validators enforce domain rules at the HTTP boundary.
• You want to see how a consistent error contract is designed and implemented from scratch.

What FastURL Does

FastURL is a REST API that shortens long URLs into compact 7-character codes and resolves them back via HTTP redirects. Beyond basic shortening, it runs an asynchronous background inspection on every newly created link: it fetches the target page, records the HTTP status code and latency, and extracts the HTML title and OpenGraph metadata.

Three actors interact with the system:

This actor breakdown comes directly from the requirements document — before any code was written. The nouns in the job stories (Link, ShortCode, TargetUrl, Inspection, Metrics) became the domain entities. The verbs (shorten, resolve, inspect, delete) became the operations.

From Requirements to Domain Entities

Every requirement in FastURL can be traced to one of eight job stories documented in the requirements document. Three of them are worth looking at closely because they drive the entire domain design.

JS-001 — When I provide a valid target URL, I want a unique shortened URL with a random alphanumeric code. This story introduces two domain concepts: a Link (the association between a short code and a target URL) and a ShortCode (a validated 7-character Base62 string). The story also establishes a business rule: the code must be unique and collision-resistant.

JS-006 — When I navigate to a short URL, I want to be redirected immediately. This story adds two more rules: the link must be active (is_active = true) and must not have passed its expiration date (expires_at). If either condition fails, the redirect must not happen.

JS-007 — When a new link is registered, I want a non-blocking background check on the target URL. This story introduces the LinkInspection concept: a snapshot of target URL health, latency, and page metadata. Crucially, it must not block the creation response — the system acknowledges the creation immediately and inspects asynchronously. This is the async I/O pattern we covered earlier in the series.

These three stories alone define the core of the domain model.

The Domain Model

FastURL has a single Aggregate Root: Link. It owns all the data and enforces all invariants. The Value Objects — ShortCode, TargetUrl, LinkMetrics, and LinkInspection — are part of the Link aggregate, not separate entities with their own identity.

Link (Aggregate Root)
├── code          ← ShortCode (Base62, 7–16 chars, unique)
├── target_url    ← TargetUrl (HTTP/HTTPS only, no self-redirect)
├── is_active     ← soft-delete flag
├── expires_at    ← optional expiry
├── inspection    ← LinkInspection (status, latency, title, og:*)
└── metrics       ← LinkMetrics (clicks_count, last_clicked_at)

There are two bounded contexts: LinkManagement (CRUD and redirect resolution) and LinkInspection (async health check and metadata extraction). They communicate via a LinkCreated event that triggers the background task.

A single aggregate with Value Objects as members keeps the domain model flat and readable. LinkInspection and LinkMetrics have no identity of their own — they only make sense as part of a Link.

Mapping Entities to Resources

The mapping from domain to REST is direct:

Notice that /{code} carries no /api/v1/ prefix. This is intentional: it is a consumer-facing URL that must be short and stable, not a versioned management endpoint.

The POST /api/v1/links/{code}/inspect endpoint does not map to a CRUD operation — it triggers an action. As discussed in the previous article, POST is the correct method for non-CRUD actions on a resource.

Pydantic as Value Object Implementation

In Domain-Driven Design, a Value Object encapsulates validation rules and equality semantics. In FastAPI, Pydantic models serve exactly this role at the HTTP boundary. The LinkCreateRequest schema enforces every business rule on inbound data before the service layer is ever called.

class LinkCreateRequest(BaseModel):
    target_url: HttpUrl = Field(
        description="Destination URL to shorten. Must be HTTP or HTTPS.",
    )
    custom_code: str | None = Field(
        default=None,
        min_length=7,
        max_length=16,
    )
    expires_at: datetime | None = Field(default=None)

    @field_validator("custom_code")
    @classmethod
    def validate_custom_code(cls, v: str | None) -> str | None:
        if v is not None and not _BASE62_PATTERN.match(v):
            raise ValueError("custom_code must contain only alphanumeric characters [a-zA-Z0-9]")
        return v

    @model_validator(mode="after")
    def validate_expires_at_future(self) -> LinkCreateRequest:
        if self.expires_at is not None:
            now = datetime.now(UTC).replace(tzinfo=None)
            expires = self.expires_at.replace(tzinfo=None)
            if expires <= now:
                raise ValueError("expires_at -> Expiration date must be in the future")
        return self

Breaking down what each element enforces:

• HttpUrl — Pydantic’s built-in type rejects any URL that is not HTTP or HTTPS. No custom validator needed for the scheme.
• @field_validator — Enforces the Base62 alphabet rule: only [a-zA-Z0-9]. This is the ShortCode Value Object’s constraint, expressed as a Pydantic field-level validator.
• @model_validator(mode="after") — Runs after all fields are validated, so it can access self.expires_at. It rejects expiration dates in the past. This is a cross-field invariant that requires the full model context.

The response schema mirrors the domain structure: LinkResponse embeds InspectionResponse and MetricsResponse, reflecting the LinkInspection and LinkMetrics Value Objects in the aggregate.

class LinkResponse(BaseModel):
    code: str
    short_url: str
    target_url: str
    is_active: bool
    expires_at: datetime | None = None
    inspection: InspectionResponse
    metrics: MetricsResponse
    created_at: datetime
    updated_at: datetime

Note that id — the internal surrogate key — is absent from the response. The code is the sole public identifier. This is a deliberate design choice: a sequential integer PK is enumerable and guessable; a Base62 7-character code is not.

The Error Envelope

One of the most important consistency decisions in any API is the error contract. FastURL uses a single envelope for every error response:

{
  "success": false,
  "error": "LINK_NOT_FOUND",
  "details": [
    "code -> No link found with code 'aB3x9zK'"
  ]
}

The error field is a machine-readable short code. The details list contains human-readable, field-level explanations. This structure lets clients branch on error programmatically and display details to a developer or end user.

Domain Exceptions

Domain exceptions are pure Python classes — no FastAPI imports, no HTTP knowledge. They carry only the data needed to build the error response.

class LinkNotFoundError(AppError):
    def __init__(self, code: str) -> None:
        super().__init__(
            message="LINK_NOT_FOUND",
            status_code=404,
            details=[f"code -> No link found with code '{code}'"],
        )

class AliasAlreadyTakenError(AppError):
    def __init__(self, alias: str) -> None:
        super().__init__(
            message="ALIAS_ALREADY_TAKEN",
            status_code=409,
            details=[f"custom_code -> The alias '{alias}' is already in use"],
        )

class LinkExpiredError(AppError):
    def __init__(self, code: str, expired_at: str) -> None:
        super().__init__(
            message="LINK_EXPIRED",
            status_code=410,
            details=[f"code -> Link '{code}' expired on {expired_at}"],
        )

The AppError base class holds the message, status_code, and details. Subclasses just set the right values in __init__. The service layer raises these exceptions; the HTTP layer never needs to know how they are structured.

Global Exception Handlers

Three handlers registered in main.py cover every failure path:

def register_exception_handlers(app: FastAPI) -> None:

    @app.exception_handler(RequestValidationError)
    async def validation_exception_handler(request, exc) -> JSONResponse:
        error_details = []
        for e in exc.errors():
            loc = " -> ".join(str(x) for x in e["loc"])
            msg = e["msg"]
            error_details.append(f"{loc}: {msg}")
        return JSONResponse(
            status_code=400,
            content={"success": False, "error": "VALIDATION_ERROR", "details": error_details},
        )

    @app.exception_handler(AppError)
    async def app_error_handler(request, exc) -> JSONResponse:
        return JSONResponse(
            status_code=exc.status_code,
            content={"success": False, "error": exc.message, "details": exc.details},
        )

    @app.exception_handler(Exception)
    async def global_exception_handler(request, exc) -> JSONResponse:
        # Full traceback logged server-side; generic message returned to client
        return JSONResponse(
            status_code=500,
            content={"success": False, "error": "INTERNAL_SERVER_ERROR",
                     "details": ["An unexpected error occurred. Please contact support."]},
        )

The three handlers map precisely to the three failure categories:

The third handler is the safety net. It logs the full traceback server-side but returns only a generic message to the client — no stack traces, no internal details, no information disclosure.

The Full Endpoint Table

Bringing everything together, here is the complete API surface of FastURL:

A few design decisions worth noting:

• DELETE returns 204 No Content — no body, because there is nothing meaningful to return after a deletion.
• POST /api/v1/links/{code}/inspect returns 202 Accepted — not 200 OK. The inspection runs asynchronously; the client receives the current snapshot immediately and polls GET /api/v1/links/{code} for the result.
• GET /{code} can return 410 Gone for expired links. 410 is semantically different from 404: it signals that the resource existed but is permanently unavailable, not that it was never found.

Conclusion

In this article we covered:

• From job stories to domain entities: how the nouns and verbs in requirements directly produce the Link aggregate, the ShortCode and TargetUrl Value Objects, and the LinkInspection and LinkMetrics embedded snapshots.
• Pydantic as Value Object implementation: using HttpUrl, @field_validator, and @model_validator to enforce domain invariants at the HTTP boundary — before the service layer is called.
• A consistent error envelope: a single {success, error, details} structure for every failure, backed by domain exceptions that carry no HTTP knowledge and three global handlers that translate them.
• Resource hierarchy derived from the domain: five management endpoints plus a public redirect endpoint, each with a clear ownership and a precise set of status codes.

The next article walks through the layered code structure that makes all of this work: how routers, services, repositories, and the dependency injection system are organized to keep each concern in its own place.

If you enjoyed this article, don’t forget to give it a clap 👏, share it with your friends 🔗, and follow me for more tips and tutorials on software development 📘. Your support helps me create more content like this — thank you! 🙌

Posted on 16 Sep 2026

Introduction

In the previous installments of this series, we explored CLI application patterns with Click and Typer. While CLI interfaces offer a convenient way for developers to interact with services locally, building distributed applications requires exposing services over the network via structured HTTP interfaces.

When designing a REST API, engineers often focus on choosing the “right” framework — FastAPI, Spring Boot, and so on. Tools matter, but they solve only the implementation detail. The real challenge is designing an API that reflects the business domain, exposes domain entities as resources, and uses HTTP semantics consistently, including errors and status codes.

Think of an API as a contract between two systems. As the designer of this contract, you act like a notary, carefully drafting the terms of interaction. However, this contract must follow the rules of HTTP, ensuring it adheres to the standard conventions for communication.

The first step is to identify the entities involved in the contract. Once these entities are clear, the next step is to define the rules: what operations can be performed on each entity, and under what conditions, all while respecting the HTTP framework.

A good API should be intuitive, self-describing, and coherent with the domain. This requires identifying business entities, their lifecycle, and their relationships before thinking about technology. Once the domain model is clear, the API becomes almost a direct consequence of it.

To avoid remaining purely theoretical, we will use Kubernetes as a case study. It provides a concrete example where functional requirements naturally lead to entities and operations that are directly exposed through an HTTP interface.

You should read this article if:

• You are designing or refactoring a REST API and want a rigorous, entity-driven methodology.
• You want to understand how domain concepts translate into REST resources, HTTP methods, and status codes.
• You want to see how Kubernetes models its complex architecture cleanly using standard REST conventions.

From requirements to entities: the logical model of Kubernetes

In a Clean Architecture perspective, the design process begins from the inside: entities at the center, then use cases, then the external interfaces.

Clean Architecture Main Concepts

User stories play a fundamental role here. They describe what the system must accomplish from a business perspective, and in doing so they implicitly define the domain language: the nouns in the stories become the entities of the domain, while the verbs describe the operations and relationships between them.

Kubernetes is no exception. Its functional requirements — scheduling workloads, distributing containers, maintaining stable network access, scaling services, enforcing security boundaries — naturally give rise to a set of domain entities. These entities exist independently of any API design choice; the API merely exposes them.

In this section, I assume the requirements of Kubernetes are already known and focus directly on their consequences: the entities that form the system and the relationships between them.

Resource entities

Kubernetes Entities with their Relationships

In Kubernetes everything is a Resource. Conceptually, Kubernetes exposes two main categories of resources:

• Namespaced resources — exist within a specific namespace
• Cluster-scoped resources — global across the entire cluster

Although the “cluster” itself is not a first-class API object, we can treat it as the top-level element for reasoning about relationships.

• A cluster consists of one or more nodes or workers.
• Each node runs one or more *Pods+.
• Each Pod contains one or more containers.

Pods are ephemeral and their network identity is unstable. For this reason, Pods are typically exposed through a Service (namespaced resource). Services come in three main variants:

• ClusterIP
• NodePort
• LoadBalancer

Workloads are created and managed by Workload (this concept doesn’t explicitly exist in Kubernetes but it helps us to model the resources). The most common ones are:

• Deployments, used for stateless applications
• StatefulSets, used for stateful applications
• DaemonSets, used to run one Pod instance on every (or selected) node
• Jobs, used for finite, one-off batch operations
• CronJobs, used for scheduled batch workloads

Pods may rely on persistent storage. They reference one or more PersistentVolumeClaims (PVCs), each of which binds to a PersistentVolume (PV) in a one-to-one manner.

Finally, users are not represented by Kubernetes YAML objects but are defined in the Kubernetes configuration (commonly via the kubeconfig file). Users acquire permissions through Roles (namespaced) and ClusterRoles (cluster-scoped). Roles are connected to users through RoleBindings or ClusterRoleBindings, depending on the scope.

Once this entity model is clear, designing a RESTful API becomes almost mechanical. Each entity maps directly to a resource, operations correspond to the entity’s lifecycle, and access control follows from the relationships defined in the domain model.

Defining the Rules of an API

Once we have identified the entities in the system, the next step is to define the rules of interaction between these entities. These rules will dictate how resources can be accessed, manipulated, or deleted via the API. Each rule can be broken down into four main components:

Defining the Rules of an API

1. Endpoint

The endpoint is the URL pattern that maps to the entity and defines the resource’s location. It is the path that the client will use to access the resource or perform an action on it.

A few important principles:

Use plural names for entities:

Even when retrieving a single entity, the resource name in the endpoint should be plural to denote the collection.

Example:

/api/v1/namespaces/{namespace}/pods          # collection of pods
/api/v1/namespaces/{namespace}/pods/{pod_id}  # single pod

Using the plural form consistently improves readability, makes endpoints predictable, and clearly distinguishes between collections and individual entities.

Limit relationship depth to two entities:

When representing relationships between entities in the endpoint, it is recommended to limit the path to two entities (e.g., /api/v1/customers/{customer_id}/employees). Going beyond this introduces unnecessary complexity and can lead to performance issues, as deeper relationships make the endpoint more difficult to manage and can impact scalability. Keeping the representation simple ensures better performance and maintainability.

2. Operation

The operation specifies the action to be performed on the resource. This is determined by the HTTP method used in the request. The most common HTTP methods include:

Idempotent by spec reflects what the HTTP specification (RFC 9110) mandates for the method. Idempotent by design marks methods that are not idempotent by spec but can be made so through deliberate implementation choices:

• POST — A payment system must not charge a customer twice on retried requests. This is solved with an Idempotency-Key header: the server stores the result of the first call and returns it unchanged on subsequent calls with the same key.
• PATCH — A partial update that sets an absolute value (e.g., {"status": "active"}) is idempotent; one that applies a relative delta (e.g., {"counter": "+1"}) is not.
• DELETE — A soft delete (setting a deleted_at timestamp or a status=deleted flag) makes the operation idempotent: a second call finds the resource already marked as deleted and returns the same successful response.

Some actions, like restarting a Pod, are not CRUD operations but can be modeled using POST, as they trigger an action on a resource without modifying it directly.

3. Parameters

Parameters define the information needed to execute the operation. There are three types of parameters that can be used:

• URL Parameters: These are part of the endpoint and are typically used to specify the resource’s identity, such as a specific resource ID (e.g., /pods/{pod_id}).
• Query Parameters: These are appended to the URL to refine or filter the results. They can specify things like pagination, sorting, or specific filters (e.g., ?limit=10 or ?status=active).
• Body Parameters: These are included in the body of the request and are typically used with methods like POST, PUT, and PATCH. Body parameters allow you to send complex data structures like JSON or XML to define the resource’s state or request changes.

4. Status Code

The status code is the HTTP response code that indicates the result of the operation. Common status codes include:

• 200 OK: The request was successful (used for GET, PUT, DELETE).
• 201 Created: The resource was successfully created (used for POST).
• 400 Bad Request: The request was malformed or invalid.
• 404 Not Found: The resource was not found.
• 405 Method Not Allowed: The requested method (i.e. GET, POST, etc.) has not been implemented for the specified resource.
• 409 Conflict: Conflicting state or rate limit reached.
• 500 Internal Server Error: There was an error on the server side.
• 503 Bad Gateway / Service Unavailable: Usually when a 3rd party subsystem is unavailable.

5. Response

The response is the data returned from the server after processing the request. This can include:

• A representation of the resource (for GET requests).
• A confirmation message (for POST, PUT, or DELETE requests, indicating success or failure).
• A failure message (for errors, explaining what went wrong).

The response should be formatted in a structured and predictable way (typically JSON or XML) so that the client can easily parse and understand the result.

These components — endpoint, operation, parameters, status code, and response — form the backbone of any API rule. By clearly defining these elements, you ensure that the API is intuitive, consistent, and easy to use for clients. These rules are the foundation that will guide how entities interact with each other in a system, ensuring that the API remains predictable, maintainable, and aligned with business needs.

Managing a Pod via API

Since it’s impossible in an article cover the API for all the above mentioned resources, let’s see how to define the API for the Pod resource.

In Kubernetes, these are the typical operations you can perform on a Pod, including both CRUD operations and a non-CRUD action like restarting a Pod.

Managing Pod via API

1. CREATE (POST)

• Endpoint: /api/v1/namespaces/{namespace}/pods
• Operation: POST
• Parameters: Body (JSON representation of the Pod specification, equivalent to what would be defined in a YAML file)
• Description: Create a new Pod in the specified namespace. The JSON body contains the Pod definition, including metadata and specification (containers, images, etc.).
• Status Code: 201 Created
• Response: Details of the created Pod.

2. READ (GET)

• Endpoint: /api/v1/namespaces/{namespace}/pods (to list all Pods in a namespace) or /api/v1/namespaces/{namespace}/pods/{pod_id} (to retrieve a specific Pod)
• Operation: GET
• Parameters: URL parameters (namespace, pod_id), Query parameters (e.g., ?status=running)
• Description: Retrieve information about a specific Pod or a list of Pods.
• Status Code: 200 OK
• Response: The Pod’s details (e.g., metadata, status, containers).

3. UPDATE (PUT)

• Endpoint: /api/v1/namespaces/{namespace}/pods/{pod_id}
• Operation: PUT
• Parameters: Body (Updated Pod specification in JSON)
• Description: Replace an existing Pod’s specification with a new one (e.g., updating resources).
• Status Code: 200 OK
• Response: The updated Pod details.

4. PARTIAL UPDATE (PATCH)

• Endpoint: /api/v1/namespaces/{namespace}/pods/{pod_id}
• Operation: PATCH
• Parameters: Body (Partial update, e.g., only changing the labels or annotations).
• Description: Apply partial updates to a Pod’s specification, such as modifying labels, annotations, or other attributes without replacing the entire Pod.
• Status Code: 200 OK
• Response: The updated Pod details.

5. DELETE (DELETE)

• Endpoint: /api/v1/namespaces/{namespace}/pods/{pod_id}
• Operation: DELETE
• Parameters: URL parameters (namespace, pod_id)
• Description: Delete the specified Pod.
• Status Code: 200 OK, 202 Accepted if the deletion is asynchronous, or 204 No Content if the operation is successfully but there is no content in the response.
• Response: Confirmation of deletion.

Authentication and Authorization in APIs (Kubernetes as a Case Study)

In any API, authentication and authorization are critical to ensure that only legitimate users can perform allowed operations on the system’s resources. Conceptually, these principles apply to all APIs, not just Kubernetes.

Authentication and Authorization

1. Authentication

• Authentication verifies the identity of a user or client.
• Typically, a user presents a credential, such as a client ID and secret, a JWT token, or a certificate.
• The application validates the credential and extracts the user identity.

Principle: If the identity cannot be verified, the API must return HTTP 401 Unauthorized.

2. Authorization

• Authorization determines what the authenticated user can do.
• Once a user is identified, the system maps the user to roles, and roles define permissions on resources.
• Each role can be scoped to a part of the system (i.e. in Kubernetes Namespaced resources or Cluster-scoped resources).

Principle: If the user is authenticated but lacks permission for an operation, the API must return HTTP 403 Forbidden.

3. Role Mapping and Bindings (Case Study: Kubernetes)

When you design your application you need to:

• Define a set of roles;
• Define a scope for each role;
• Define the operations the role can do in that scope.

Kubernetes illustrates these concepts:

• A Role defines operations allowed on namespaced resources (e.g., Pods, Services).
• A ClusterRole defines operations allowed on cluster-scoped resources (e.g., Nodes, PersistentVolumes).
• Users are associated to roles via:
  • RoleBinding → links user to a Role in a namespace
  • ClusterRoleBinding → links user to a ClusterRole cluster-wide

This design allows a many-to-many mapping: users can have multiple roles, and roles can apply to multiple users.

4. Context of Operation

When a user interacts with the API, the system maintains a context that encapsulates all relevant information:

• User identity — who is making the request
• Roles — the permissions assigned to this user
• Scope — the resource context (namespaced or cluster-scoped) where operations are allowed

This context is used by the application to enforce authorization rules consistently for each request.

5. Key Takeaways for API Design

• Authentication and authorization should be explicit and predictable: unauthorized requests → 401, forbidden operations → 403.
• Roles should be mapped clearly to resources, respecting their scope.
• Maintaining a user context makes it easier to check permissions and enforce rules consistently.

Kubernetes demonstrates this clean separation of identity (authentication), capability (authorization), and scope (context), providing a model that can be applied to any API design.

Error Handling in APIs

One of the key differences between a professional API and one created by inexperienced developers is how errors are handled. Proper error handling ensures that clients can reliably understand what went wrong and respond appropriately.

While frameworks like FastAPI (Python) or Spring Boot (Java) provide built-in mechanisms for error handling, the principles are framework-agnostic.

API Error Handling

1. Resource Not Found (404)

• When performing operations like GET, PUT, PATCH, DELETE on a resource identified by an ID, if the resource does not exist, the API must return 404 Not Found.
• Returning 200 OK with a message like “resource not found” is incorrect, because clients rely on the HTTP status code to handle errors programmatically.

Example:

GET /api/v1/namespaces/default/pods/nonexistent
Response: 404 Not Found

2. Validation Errors (400, 422)

If the client sends invalid input, such as:

• Missing required fields
• Incorrect data types
• Malformed JSON

the API should return 400 Bad Request or 422 Unprocessable Entity.

Frameworks like FastAPI + Pydantic or Spring Boot with validation annotations make it easy to automatically validate input and return the appropriate status code.

3. Authentication and Authorization Errors (401, 403)

• 401 Unauthorized: The client’s identity could not be verified (e.g., missing or invalid token).
• 403 Forbidden: The client is authenticated but does not have permission to perform the requested operation.

These errors were covered in the Authentication and Authorization section, but they are a crucial part of robust error handling.

4. Method Not Allowed (405)

405 Method Not Allowed is returned when the client attempts to use an HTTP method that is not supported for a given resource.

Example:

POST /api/v1/customers/{id}
Response: 405 Method Not Allowed

This error is typically implemented automatically by the framework (e.g., FastAPI or Spring Boot), based on how routes and HTTP methods are defined. The application logic should not manually handle this case.

5. Rate Limiting (429)

429 Too Many Requests indicates that the client has exceeded the allowed request rate.

This error is:

• Typically enforced at the API gateway or infrastructure level
• Not implemented inside the application code

Common use cases include:

• Abuse prevention
• Traffic control
• Fair usage policies

The response often includes headers such as Retry-After to indicate when the client can retry.

6. Server Errors (500, 503)

• 500 Internal Server Error: Indicates unexpected failures, such as programming errors or unhandled exceptions.
• 503 Service Unavailable: Indicates that a dependency is temporarily unavailable, for example when the API cannot reach the database or an external service is down.

These codes allow clients to distinguish between a system failure and a temporary unavailability.

API Documentation

APIs should be self-describing, meaning that each endpoint should be clear and understandable without requiring external documentation. However, tools like Swagger have greatly simplified the publication and management of API documentation. Frameworks such as FastAPI allow you to automatically generate Swagger documentation for every API, making it interactive and easily accessible.

API Documentation with Swagger

When using Swagger, it’s a best practice to group APIs by the entities they belong to. In FastAPI, this can be achieved using tags. For example, all APIs related to a specific entity, such as Customer, can be grouped under the Customer tag. This approach helps to keep the documentation clear and well-structured, with each group of APIs representing the “rules” for that particular entity.

For every API, it’s essential to properly document the following details in the code:

• Summary: A brief description of what the API does, allowing users to quickly understand its purpose.
• Description: A more detailed explanation of the API, providing additional context on its behavior.
• Parameters: A list of all the parameters the API accepts, along with information about their type and whether they are required.
• Error Codes: A list of possible error codes the API may return, along with brief explanations of what each code means.

These details will automatically be included in the Swagger documentation, making it comprehensive and easy to navigate.

Moreover, Swagger also acts as a client, allowing developers to directly test the APIs from the documentation interface, which simplifies testing and debugging during development.

Conclusion

In this article we covered:

• Entity modeling from requirements: How Clean Architecture principles and user stories identify domain entities and their boundaries prior to writing any API code.
• Resource mappings and relationships: Structuring RESTful endpoints with plural naming conventions and shallow nesting hierarchies (maximum 2 entities).
• HTTP semantics and operations: Mapping CRUD and action-based workflows to standard HTTP methods (GET, POST, PUT, PATCH, DELETE) with predictable status codes.
• Authentication, authorization, and context: Enforcing access controls via roles, scopes, and execution contexts, using Kubernetes RBAC (Role, ClusterRole, RoleBinding) as an architectural blueprint.
• Standardized error handling and Swagger documentation: Returning machine-readable status codes and using tags and OpenAPI metadata to generate self-documenting APIs.

The next article will put these architectural principles into practice: building a production-ready asynchronous REST API using FastAPI, with thread pools for blocking operations and dependency injection patterns.

If you enjoyed this article, don’t forget to give it a clap 👏, share it with your friends 🔗, and follow me for more tips and tutorials on software development 📘. Your support helps me create more content like this — thank you! 🙌

Posted on 13 Sep 2026

Introduction

The previous article covered the theory: the GIL, thread pools, process pools, and why the event loop beats them all for I/O-bound concurrency at scale. Theory is useful until you sit down to write something real.

This article builds a concrete project from scratch — an async news aggregator that fetches RSS feeds concurrently and formats a digest ready to send via WhatsApp. The code is the same used in the open-source Alfred project: it starts minimal and grows with one new pattern per section: gather, TaskGroup, asyncio.timeout, Semaphore, and format_news. By the end you have a working, production-grade async application and a clear mental model of when and why to reach for each tool.

You should read this article if:

• You understand the theory of asyncio but have not yet built something real with it.
• You want to see TaskGroup, Semaphore, and asyncio.timeout used together in a single project rather than in isolation.
• You are comfortable with Python but new to aiohttp and concurrent I/O patterns.

This article is part of the Modern Python Application series.

The Project: An Async News Aggregator

The aggregator in this article is based on the news.py module from the open-source Alfred project — a personal assistant that fetches daily news and delivers it via WhatsApp. The article walks through the same patterns the real module uses, introducing them one step at a time. The final section points out where the production code goes further.

The aggregator fetches headlines from five RSS sources — the same default feeds configured in Alfred:

The final output is a formatted text digest ready to be sent via WhatsApp, with each item showing title and a short summary stripped of HTML.

Install the dependencies first:

pip install aiohttp feedparser

• aiohttp — async HTTP client; see the aiohttp documentation for the full API reference
• feedparser — synchronous RSS/Atom/RDF parser; we feed it raw XML already in memory; see the feedparser documentation for the full entry field reference

Step 1 — One Feed, One Coroutine

Start with the simplest possible version: fetch a single RSS feed and parse it. The real Alfred module also strips HTML from each entry’s summary — let’s include that from the start, since it is a two-liner and avoids HTML noise in the output.

# news.py
import asyncio
import re

import aiohttp
import feedparser

TIMEOUT = aiohttp.ClientTimeout(total=10)
HEADERS = {"User-Agent": "Mozilla/5.0 (compatible; AlfredBot/1.0)"}
MAX_SUMMARY = 150


def _strip_html(text: str) -> str:
    """Remove HTML tags, collapse whitespace, and truncate to MAX_SUMMARY chars."""
    clean = re.sub(r"\s+", " ", re.sub(r"<[^>]+>", "", text)).strip()
    return clean[:MAX_SUMMARY] + "…" if len(clean) > MAX_SUMMARY else clean


async def fetch(session: aiohttp.ClientSession, url: str, limit: int = 3) -> list[dict]:
    async with session.get(url) as resp:
        resp.raise_for_status()
        text = await resp.text()
    feed = feedparser.parse(text)
    return [
        {
            "title": e.get("title", "").strip(),
            "link": e.get("link", ""),
            "summary": _strip_html(e.get("summary", "")),
        }
        for e in feed.entries[:limit]
    ]

async def main() -> None:
    async with aiohttp.ClientSession(timeout=TIMEOUT, headers=HEADERS) as session:
        items = await fetch(session, "https://feeds.bbci.co.uk/news/world/rss.xml")
    for item in items:
        print(item["title"])
        print(item["summary"])

asyncio.run(main())

A few things worth noting:

• HEADERS sets a User-Agent string. Many RSS servers silently return 403 for requests that look like bots without a recognisable user agent — this one-liner prevents that class of failure entirely.
• aiohttp.ClientSession is created once and reused for all requests. Creating a session per request is the most common aiohttp mistake — it bypasses connection pooling entirely. One session per application is the rule.
• ClientTimeout(total=10) sets a 10-second cap on the full operation. Without a timeout, a hanging feed hangs your entire program indefinitely.
• feedparser.parse(text) is synchronous but runs in microseconds once the XML is already in memory — no need to offload it to a thread.
• resp.raise_for_status() raises aiohttp.ClientResponseError for any 4xx or 5xx response immediately, rather than letting malformed data propagate silently.
• _strip_html() removes HTML tags from the summary using two regex passes: first strips tags, then collapses whitespace. The truncation at MAX_SUMMARY chars keeps WhatsApp messages readable.

A coroutine object does nothing until it is awaited. fetch(session, url) returns a coroutine — the HTTP request only fires when the event loop reaches await.

Step 2 — Multiple Feeds with asyncio.gather

Fetching feeds one by one defeats the purpose of async. asyncio.gather runs multiple coroutines concurrently — all five requests are in-flight simultaneously, and the total wall time equals the slowest single response, not the sum.

DEFAULT_FEEDS = {
    "BBC News":      "https://feeds.bbci.co.uk/news/world/rss.xml",
    "Repubblica":    "https://www.repubblica.it/rss/homepage/rss2.0.xml",
    "Wired":         "https://www.wired.com/feed/rss",
    "Sky Sport":     "https://sport.sky.it/rss/sport.xml",
    "Science Daily": "https://www.sciencedaily.com/rss/top/science.xml",
}

async def main() -> None:
    async with aiohttp.ClientSession(timeout=TIMEOUT, headers=HEADERS) as session:
        results = await asyncio.gather(
            *[fetch(session, url) for url in DEFAULT_FEEDS.values()],
            return_exceptions=True,
        )
    for source, result in zip(DEFAULT_FEEDS.keys(), results):
        if isinstance(result, Exception):
            print(f"{source}: error — {result}")
        else:
            for item in result:
                print(f"[{source}] {item['title']}")

return_exceptions=True is the key choice here. Without it, the first failed feed raises immediately and you lose all results. With it, exceptions are returned as values alongside successful results — you check isinstance(result, Exception) and continue. This is the right default for an aggregator where partial results are genuinely useful.

The limitation: if one feed fails, the others keep running regardless. There is no automatic cancellation. That is sometimes what you want (independent sources) and sometimes not (a batch of dependent requests where partial data is useless). The next section shows the alternative.

Step 3 — Structured Concurrency with TaskGroup

asyncio.TaskGroup (Python 3.11+, see the official docs) enforces a stricter contract: if any task raises an unhandled exception, all siblings are cancelled and an ExceptionGroup is raised at the async with exit. Think of it as gather with a safety net (here, that safety net never triggers — fetch_safe absorbs every exception internally before TaskGroup can see it).

For the aggregator, feeds are independent sources — a failure in one should not kill the others. The right pattern here is to absorb errors inside fetch_safe and return them as a third tuple element, then let TaskGroup manage the concurrency cleanly:

async def fetch_safe(
    session: aiohttp.ClientSession,
    source: str,
    url: str,
    limit: int = 3,
) -> tuple[str, list[dict], Exception | None]:
    try:
        items = await fetch(session, url, limit=limit)
        return source, items, None
    except Exception as exc:
        return source, [], exc

async def main() -> None:
    results: dict[str, list[dict]] = {}
    failed: dict[str, Exception] = {}

    async with aiohttp.ClientSession(timeout=TIMEOUT, headers=HEADERS) as session:
        async with asyncio.TaskGroup() as tg:
            tasks = {
                source: tg.create_task(fetch_safe(session, source, url))
                for source, url in DEFAULT_FEEDS.items()
            }

    for source, task in tasks.items():
        _, items, error = task.result()
        if error is not None:
            failed[source] = error
        else:
            results[source] = items

tg.create_task() schedules each coroutine as a Task immediately. The async with block does not exit until all tasks are complete — no manual await for each one. The tasks dictionary gives you named access to each result after the group closes.

Notice that fetch_safe now returns a 3-tuple (source, items, error | None) instead of mixing exceptions and data in a single list. The error travels alongside the result as a first-class field — the caller unpacks it cleanly with _, items, error = task.result() and routes failures to a separate failed dict without any isinstance check.

Use TaskGroup when you want structured lifetimes for a set of concurrent tasks. Use gather(return_exceptions=True) when you need to collect all results including errors as data.

As Hynek Schlawack notes, TaskGroup is now the recommended default for scheduling a nested group of tasks — it has fewer sharp edges than gather and its cancellation semantics are unambiguous.

Step 4 — Per-Request Timeouts with asyncio.timeout

The global ClientTimeout covers the session default, but individual feeds can be slower or faster. asyncio.timeout (Python 3.11+) wraps any block of async code with a deadline — if the block takes longer than n seconds, it raises TimeoutError and cancels the inner task cleanly.

async def fetch_safe(
    session: aiohttp.ClientSession,
    source: str,
    url: str,
    timeout: float = 8.0,
    limit: int = 3,
) -> tuple[str, list[dict], Exception | None]:
    try:
        async with asyncio.timeout(timeout):
            items = await fetch(session, url, limit=limit)
        return source, items, None
    except TimeoutError as exc:
        return source, [], exc
    except Exception as exc:
        return source, [], exc

asyncio.timeout is more composable than asyncio.wait_for — it is an async context manager that wraps a block rather than a single coroutine call. This means you can cover multiple await expressions with a single deadline, which is useful when a feed requires a follow-up request to resolve redirects.

One important rule: TimeoutError must be caught outside the async with asyncio.timeout(...) block, not inside it. The context manager converts the internal CancelledError into TimeoutError at its exit — catching it inside would swallow it before the conversion.

Step 5 — Rate Limiting with asyncio.Semaphore

Five feeds is manageable, but extend this to 50 sources and you risk triggering rate limits or overwhelming slow servers with a burst of simultaneous connections. asyncio.Semaphore is a counter that limits how many coroutines can be inside a critical section at the same time.

async def fetch_safe(
    session: aiohttp.ClientSession,
    source: str,
    url: str,
    semaphore: asyncio.Semaphore,
    timeout: float = 8.0,
    limit: int = 3,
) -> tuple[str, list[dict], Exception | None]:
    async with semaphore:  # acquire slot; blocks if all slots are taken
        try:
            async with asyncio.timeout(timeout):
                items = await fetch(session, url, limit=limit)
            return source, items, None
        except TimeoutError as exc:
            return source, [], exc
        except Exception as exc:
            return source, [], exc


async def _fetch_all(
    feeds: dict[str, str], limit: int = 3
) -> tuple[dict[str, list[dict]], dict[str, Exception]]:
    semaphore = asyncio.Semaphore(3)
    results: dict[str, list[dict]] = {}
    failed: dict[str, Exception] = {}

    async with (
        aiohttp.ClientSession(timeout=TIMEOUT, headers=HEADERS) as session,
        asyncio.TaskGroup() as tg,
    ):
        tasks = {
            source: tg.create_task(fetch_safe(session, source, url, semaphore, limit=limit))
            for source, url in feeds.items()
        }

    for source, task in tasks.items():
        _, items, error = task.result()
        if error is not None:
            failed[source] = error
        else:
            results[source] = items

    return results, failed

The semaphore wraps the entire request block including the timeout. When three tasks are already inside, the fourth async with semaphore: suspends at acquire() — the event loop runs other work while it waits. As soon as one of the active tasks exits the block, the semaphore releases a slot and the waiting task resumes.

The _fetch_all helper wraps everything: it creates the semaphore, opens the session and the TaskGroup together using a parenthesised async with block (Python 3.10+), spawns one task per feed, then collects results into results and failed dicts. Returning both separately is cleaner than mixing exceptions and data in a single list — the caller sees clean items and can act on errors independently.

asyncio.Semaphore is not thread-safe. Use it only inside a single event loop — never across threads.

Step 6 — Formatting the Digest with format_news

All the data is now in results. The last step before delivery is turning it into a human-readable string — one that WhatsApp can render with bold headers and bullet points using its own Markdown-like syntax.

def format_news(results: dict[str, list[dict]]) -> str:
    blocks: list[str] = []
    for source, items in results.items():
        if not items:
            continue
        lines: list[str] = [f"*{source}*"]
        for item in items:
            lines.append(f"• {item['title']}")
            if item.get("summary"):
                lines.append(f"  {item['summary']}")
        blocks.append("\n".join(lines))
    if not blocks:
        return ""
    return "📰 *Ultime Notizie*\n\n" + "\n\n".join(blocks)

format_news is a plain synchronous function — no async, no await. By the time it runs, all the I/O is done and results is already in memory. This is the right pattern: keep async code to the network and disk boundary, and let everything else be normal Python.

• *source* produces bold text in WhatsApp.
• • item['title'] is a bullet item.
• The summary line is indented with two spaces and shown only when non-empty.
• Sources with zero items are skipped entirely.

The Complete Aggregator

Putting all the pieces together — this is the full news.py module from Alfred, with DEFAULT_FEEDS hardcoded for readability:

# news.py
import asyncio
import re

import aiohttp
import feedparser

TIMEOUT = aiohttp.ClientTimeout(total=10)
HEADERS = {"User-Agent": "Mozilla/5.0 (compatible; AlfredBot/1.0)"}
MAX_SUMMARY = 150

DEFAULT_FEEDS = {
    "BBC News":      "https://feeds.bbci.co.uk/news/world/rss.xml",
    "Repubblica":    "https://www.repubblica.it/rss/homepage/rss2.0.xml",
    "Wired":         "https://www.wired.com/feed/rss",
    "Sky Sport":     "https://sport.sky.it/rss/sport.xml",
    "Science Daily": "https://www.sciencedaily.com/rss/top/science.xml",
}


def _strip_html(text: str) -> str:
    clean = re.sub(r"\s+", " ", re.sub(r"<[^>]+>", "", text)).strip()
    return clean[:MAX_SUMMARY] + "…" if len(clean) > MAX_SUMMARY else clean


async def fetch(session: aiohttp.ClientSession, url: str, limit: int = 3) -> list[dict]:
    async with session.get(url) as resp:
        resp.raise_for_status()
        text = await resp.text()
    feed = feedparser.parse(text)
    return [
        {
            "title": e.get("title", "").strip(),
            "link": e.get("link", ""),
            "summary": _strip_html(e.get("summary", "")),
        }
        for e in feed.entries[:limit]
    ]


async def fetch_safe(
    session: aiohttp.ClientSession,
    source: str,
    url: str,
    semaphore: asyncio.Semaphore,
    timeout: float = 8.0,
    limit: int = 3,
) -> tuple[str, list[dict], Exception | None]:
    async with semaphore:
        try:
            async with asyncio.timeout(timeout):
                items = await fetch(session, url, limit=limit)
            return source, items, None
        except TimeoutError as exc:
            return source, [], exc
        except Exception as exc:
            return source, [], exc


async def _fetch_all(
    feeds: dict[str, str], limit: int = 3
) -> tuple[dict[str, list[dict]], dict[str, Exception]]:
    semaphore = asyncio.Semaphore(3)
    results: dict[str, list[dict]] = {}
    failed: dict[str, Exception] = {}

    async with (
        aiohttp.ClientSession(timeout=TIMEOUT, headers=HEADERS) as session,
        asyncio.TaskGroup() as tg,
    ):
        tasks = {
            source: tg.create_task(fetch_safe(session, source, url, semaphore, limit=limit))
            for source, url in feeds.items()
        }

    for source, task in tasks.items():
        _, items, error = task.result()
        if error is not None:
            failed[source] = error
        else:
            results[source] = items

    return results, failed


def format_news(results: dict[str, list[dict]]) -> str:
    blocks: list[str] = []
    for source, items in results.items():
        if not items:
            continue
        lines: list[str] = [f"*{source}*"]
        for item in items:
            lines.append(f"• {item['title']}")
            if item.get("summary"):
                lines.append(f"  {item['summary']}")
        blocks.append("\n".join(lines))
    if not blocks:
        return ""
    return "📰 *Ultime Notizie*\n\n" + "\n\n".join(blocks)


async def main() -> None:
    results, failed = await _fetch_all(DEFAULT_FEEDS)

    print(format_news(results))

    if failed:
        print("--- Failed feeds ---")
        for source, error in failed.items():
            print(f"{source}: {error}")


if __name__ == "__main__":
    asyncio.run(main())

Run it:

python news.py

The expected output looks like:

📰 *Ultime Notizie*

*BBC News*
• Article title one
  Short summary of the article…
• Article title two
  Short summary of the article…

*Repubblica*
• ...

How the Event Loop Sees This

Every await in the code above is a yield point — the moment a coroutine suspends and hands control back to the event loop. The diagram below shows what happens with five concurrent fetch(url) calls: the event loop runs all five as tasks of the same coroutine function, each suspended at its own await, each holding a different URL.

None of the feeds block each other. While fetch(url) for BBC News is waiting for bytes from the BBC server, the event loop resumes fetch(url) for Repubblica, and so on. The total wall time collapses to the latency of the single slowest feed — not the sum of all five.

The event loop does not make individual requests faster. It makes the waiting invisible by running other work while one request is in flight.

Patterns Compared

Conclusion

In this article we covered:

• Starting with a single async def fetch() coroutine, a shared aiohttp.ClientSession, and a _strip_html() helper to produce clean summaries from the first line.
• Adding a User-Agent header to avoid silent 403 rejections from RSS servers that block unrecognised clients.
• Fetching multiple feeds concurrently with asyncio.gather(return_exceptions=True) to collect partial results when some sources fail.
• Replacing gather with asyncio.TaskGroup for structured concurrency with clean task lifetimes, and switching fetch_safe to a 3-tuple return so errors travel as data rather than exceptions.
• Wrapping individual requests with asyncio.timeout to enforce per-source deadlines without blocking the event loop.
• Using asyncio.Semaphore(3) and a _fetch_all helper with a parenthesised async with block to cap concurrent connections and keep the call site clean.
• Converting the results dict to a WhatsApp-ready string with a plain synchronous format_news function — keeping async code at the I/O boundary and normal Python everywhere else.

The next article starts a three-part mini-series on Python CLI development — building the same Task Manager CLI first with argparse and the Command Pattern, then with Click, then with Typer.

If you enjoyed this article, don’t forget to give it a clap 👏, share it with your friends 🔗, and follow me for more tips and tutorials on software development 📘. Your support helps me create more content like this — thank you! 🙌

Posted on 06 Sep 2026

Introduction

Your Python application works fine until the moment it has to do more than one thing at a time. A web server blocking on every database query, a scraper waiting for each HTTP response before firing the next — the code is correct, it just does not scale. Python offers three concurrency models to fix this: threads, processes, and the event loop. Each solves a different problem, and choosing the wrong one costs you performance or correctness.

This article is part of the Modern Python Application series.

You should read this article if:

• You have hit a performance wall with sequential Python code and want to understand your options.
• You are confused about when threads help and when the GIL gets in the way.
• You want to understand why asyncio became the standard for high-concurrency services.

The Restaurant Metaphor

Think of your Python program as a restaurant.

Sequential model — one waiter takes an order, walks to the kitchen, and stands there until the dish is ready before returning. The cook (I/O, a DB query, a network call) does all the work while the waiter stands idle.

Thread pool model — a dispatcher assigns each order to a pool waiter who walks to the kitchen and waits. Multiple orders are in-flight simultaneously. The catch: every waiter occupies memory even while idle, and coordinating handoffs costs time — the equivalent of context switching between threads. The more waiters, the higher the overhead. This model worked well for enterprise web apps when concurrency stayed in the hundreds.

Event loop model — one efficient waiter drops an order at the kitchen pass and immediately moves to the next table. She never waits. Periodically she checks which dishes are ready and delivers them. If the restaurant can afford multiple waiters of this kind — one per CPU core — throughput scales proportionally, each running her own independent event loop. Node.js popularised this model in 2010; Python’s asyncio followed in 2014 and today underpins FastAPI, aiohttp, and every high-throughput Python service.

The event loop does not make individual operations faster — it makes the waiting disappear.

Parallelism vs Concurrency

Two terms that are often used interchangeably but mean different things.

Parallelism — multiple tasks execute at the same instant on multiple CPU cores. Wall-clock time is divided by the number of cores.

Concurrency — multiple tasks are in progress at the same time, but not necessarily executing simultaneously. A single core interleaves them: work on A, pause, switch to B, pause, switch to C. No two tasks run at the exact same instant, but all make progress.

Concurrency is about dealing with many things at once. Parallelism is about doing many things at once.

The trade-off: concurrency hides waiting (I/O), parallelism reduces computation time. This maps directly onto Python’s three models:

• asyncio — concurrency on a single thread. Hides I/O waiting. Does not reduce computation time.
• ThreadPoolExecutor — concurrency across threads. Hides I/O waiting. Does not reduce computation time (GIL).
• ProcessPoolExecutor — true parallelism. Reduces computation time for CPU-bound work.

Threads vs Processes

A thread and a process are both units of execution, but they differ in one fundamental way: what they share.

A thread lives inside a process. All threads in the same process share the same memory — the same heap, the same global variables, the same open file handles. Communication between threads is fast and free: one thread writes to a variable, another reads it. The downside is that this sharing requires coordination, and in CPython it means sharing the GIL.

A process is a fully isolated execution environment. It has its own memory space, its own Python interpreter, and its own GIL. Processes do not share memory — to pass data between them, objects must be serialised (pickled) and sent through a pipe or queue. The upside: with no shared GIL, multiple processes run Python bytecode in true parallel on separate cores.

The diagram shows why this matters for Python concurrency. Threads inside Process A share one GIL — only one runs at a time. Process B and Process C each have their own GIL and run simultaneously on separate cores. This is exactly why ProcessPoolExecutor bypasses the GIL while ThreadPoolExecutor does not.

The GIL: Why Threads Are Not What You Think

Python’s threading module exists and works, but there is a catch: the Global Interpreter Lock (GIL). The GIL is a mutex inside CPython that allows only one thread to execute Python bytecode at a time, regardless of how many CPU cores your machine has.

This is fundamentally different from Java or C#, where the JVM and CLR manage memory with a garbage collector that does not need reference counting — so there is no GIL. In those languages, four threads on a four-core machine execute bytecode in true parallel. If you have fewer cores than threads, you get concurrency (threads take turns); if you have as many cores as threads, you get genuine parallelism.

In CPython the picture is different.

In Java, four threads run simultaneously on four cores — wall time equals t. In CPython, the same four threads run one at a time on a single core — wall time is 4t, with cores 2–4 sitting idle waiting for the GIL.

This has two practical consequences.

When the GIL Is Released

The GIL is not held permanently. CPython releases it in two situations: during I/O waits (network, disk, database) and inside C extensions that explicitly drop it (NumPy, PyCryptodome, lxml). In both cases, other threads can run while one is waiting or computing in C.

• I/O-bound tasks: threads work well. When a thread waits for a network response, a file read, or a database query, CPython releases the GIL for the duration of that wait. Another thread picks it up and runs. You get real concurrency — just not on the CPU calculation itself, but on the waiting.
• CPU-bound tasks: threads give you nothing. All threads fight over the single GIL. A CPU-heavy calculation running in four threads on a four-core machine runs no faster than on one core — and slightly slower due to context switching overhead. In contrast, a Java developer using a thread pool for CPU work gets genuine parallelism; the same pattern in CPython gives worse results than a single thread.

import threading
import time
import urllib.request

def fetch(url):
    with urllib.request.urlopen(url) as response:
        return response.read()

urls = [
    "https://httpbin.org/delay/1",
    "https://httpbin.org/delay/1",
    "https://httpbin.org/delay/1",
]

start = time.time()
threads = [threading.Thread(target=fetch, args=(url,)) for url in urls]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(f"Elapsed: {time.time() - start:.1f}s")  # ~1s, not 3s

Three one-second HTTP requests finish in roughly one second because all three threads wait concurrently while the GIL sits idle during the actual network I/O.

This is exactly the I/O-bound case: the GIL is released during the network wait, so the threads are genuinely concurrent. If instead of making an HTTP request each thread ran a CPU-heavy calculation, the result would be ~3 seconds — or worse, due to the extra context switching overhead on top of serial execution.

The following example replaces the HTTP call with a CPU-bound task — summing a large range of numbers, which keeps the GIL held the entire time:

import threading
import time

def cpu_task():
    # time.monotonic() returns a monotonically increasing clock (never goes
    # backwards, unaffected by system clock adjustments) — reliable for
    # measuring durations. We busy-loop for exactly 1s on any machine.
    deadline = time.monotonic() + 1.0
    while time.monotonic() < deadline:
        pass  # pure Python busy loop — GIL never released

start = time.time()
threads = [threading.Thread(target=cpu_task) for _ in range(3)]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(f"Elapsed: {time.time() - start:.1f}s")  # ~3s, not 1s

All three threads fight over the GIL. Only one runs at a time — the others wait. Total time is roughly three times the time of a single task, with no benefit over running them sequentially.

Free-Threaded Python (3.13+)

PEP 703, accepted in October 2023, introduced an experimental --disable-gil build flag in CPython 3.13, made more stable in 3.14. With the nogil build, threads can execute Python bytecode in parallel on multiple cores — real CPU-bound parallelism without multiprocessing.

import sys
print(sys._is_gil_enabled())  # False in a nogil build

The single-threaded performance penalty is approximately 5–10% (PEP 703). As of 2026, the nogil build is production-ready for many workloads and represents the most significant change to CPython’s concurrency model in the language’s history.

Threads: The Right Tool for I/O-Bound Work

For I/O-bound concurrency where you need dozens — not thousands — of concurrent operations, ThreadPoolExecutor from concurrent.futures is the idiomatic choice. The threading module gives you lower-level control, but ThreadPoolExecutor handles pool management, exception propagation, and cleanup automatically.

from concurrent.futures import ThreadPoolExecutor, as_completed
import urllib.request

def fetch(url):
    with urllib.request.urlopen(url, timeout=10) as r:
        return url, len(r.read())

urls = [
    "https://httpbin.org/bytes/1024",
    "https://httpbin.org/bytes/2048",
    "https://httpbin.org/bytes/512",
]

with ThreadPoolExecutor(max_workers=10) as executor:
    futures = {executor.submit(fetch, url): url for url in urls}
    for future in as_completed(futures):
        url, size = future.result()
        print(f"{url} → {size} bytes")

Breaking down the key choices:

• max_workers=10 — caps the pool size. The default in Python 3.13+ is min(32, cpu_count + 4), which preserves at least 5 workers for I/O overlap.
• as_completed() — yields futures as they finish, not in submission order. Use this when tasks have different durations and you want to process results as they arrive.
• Context manager (with) — guarantees the pool shuts down cleanly and all threads complete before the block exits.

Use ThreadPoolExecutor for I/O-bound work that needs straightforward parallel execution without handling thousands of concurrent connections.

Processes: The Right Tool for CPU-Bound Work

When you need real parallelism for CPU-heavy work — image processing, numerical computation, data transformation — use ProcessPoolExecutor. Each worker runs in a separate Python interpreter with its own GIL, so all cores are genuinely available.

from concurrent.futures import ProcessPoolExecutor
import math

def is_prime(n):
    if n < 2:
        return False
    for i in range(2, int(math.sqrt(n)) + 1):
        if n % i == 0:
            return False
    return True

if __name__ == "__main__":
    candidates = range(10_000_000, 10_000_100)
    with ProcessPoolExecutor() as executor:
        results = list(executor.map(is_prime, candidates))
    primes = [n for n, p in zip(candidates, results) if p]
    print(f"Found {len(primes)} primes")

Two rules that catch beginners:

• if __name__ == "__main__": is mandatory on macOS and Windows. The spawn start method (default since Python 3.14 for ProcessPoolExecutor) imports the module in each worker. Without this guard, the import triggers new worker spawns, causing an infinite loop.
• Only picklable objects can be passed to and returned from workers. Lambda functions, file handles, and database connections are not picklable.

The default worker count is os.process_cpu_count() — one worker per physical core. Spawning processes is expensive; use the context manager to amortize startup cost across many tasks. For project setup and dependency management best practices, see How to Set Up Your Next Python Project.

InterpreterPoolExecutor (Python 3.14+)

Python 3.14 introduced InterpreterPoolExecutor, a new executor that gives each thread its own sub-interpreter with its own GIL. It achieves true multi-core parallelism without the process spawn cost.

from concurrent.futures import InterpreterPoolExecutor

def cpu_task(n):
    return sum(i * i for i in range(n))

with InterpreterPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(cpu_task, [1_000_000] * 4))

print(results)
# [333332833333500000, 333332833333500000,
#  333332833333500000, 333332833333500000]
# Four tasks completed in parallel — wall time ≈ 1× single task duration

Each interpreter is fully isolated — no shared mutable state. Data must still be picklable. This is the direct answer to the GIL problem for many CPU-bound workloads that previously required multiprocessing.

The Three Models Compared

The Event Loop: asyncio for High-Concurrency I/O

When concurrent connections number in the thousands — API gateways, WebSocket servers, real-time data feeds — thread pools hit a wall. Each thread consumes a non-trivial amount of memory for its stack, and context switching between hundreds of threads adds measurable overhead.

The event loop solves this with cooperative concurrency: a single thread runs many coroutines, each of which voluntarily suspends at await points to let others run. No threads, no context switching overhead, no GIL juggling. This is the same model that powers FastAPI and that we will use in the next article when building a REST API.

import asyncio
import aiohttp

async def fetch(session, url):
    async with session.get(url) as response:
        return await response.text()

async def main():
    urls = ["https://httpbin.org/delay/1" for _ in range(10)]
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, url) for url in urls]
        results = await asyncio.gather(*tasks)
    print(f"Fetched {len(results)} responses")

asyncio.run(main())

Ten requests that each take one second complete in roughly one second — all ten coroutines are suspended at await response.text() simultaneously, and the event loop resumes each one as its response arrives.

TaskGroup: Structured Concurrency (Python 3.11+)

asyncio.gather() has a subtle flaw: if one task raises an exception, the remaining tasks are not cancelled — they continue running as orphans. Python 3.11 introduced TaskGroup, which cancels all sibling tasks when any one fails.

import asyncio
import aiohttp

async def fetch(session, url):
    async with session.get(url) as response:
        response.raise_for_status()
        return await response.text()

async def main():
    urls = [
        "https://httpbin.org/get",
        "https://httpbin.org/status/500",  # this will fail
        "https://httpbin.org/get",
    ]
    async with aiohttp.ClientSession() as session:
        async with asyncio.TaskGroup() as tg:
            tasks = [tg.create_task(fetch(session, url)) for url in urls]
    # If any task raises, all others are cancelled automatically

Use TaskGroup when a failure in one task should cancel all the others — e.g. a batch of dependent requests where partial results are useless. Use gather() when tasks are independent and you want all results even if some fail, passing return_exceptions=True to collect errors instead of raising them.

Mixing Sync and Async: asyncio.to_thread()

Most existing Python libraries — requests, psycopg2, boto3 — are synchronous. Calling them directly inside an async def blocks the entire event loop. asyncio.to_thread() (Python 3.9+) runs a blocking function in a thread pool without blocking the loop. This is the same pattern we used in the cron jobs article when mixing sync schedulers with async application code.

import asyncio
import requests  # synchronous library

def blocking_fetch(url):
    return requests.get(url).text

async def main():
    # Runs blocking_fetch in a thread; event loop continues while it waits
    result = await asyncio.to_thread(blocking_fetch, "https://httpbin.org/get")
    print(result[:100])

asyncio.run(main())

This replaces the older loop.run_in_executor(None, func) pattern and is the idiomatic bridge between sync libraries and async application code.

Which Model Should I Use?

Four questions get you to the right answer:

1. Is the task I/O-bound? If yes, go to question 2. If no (CPU-bound), use ProcessPoolExecutor — or InterpreterPoolExecutor on Python 3.14+ to avoid process spawn cost.
2. Do you need thousands of concurrent connections? If yes, use asyncio. If no, continue.
3. Do you need to call synchronous libraries inside async code? Use asyncio.to_thread() as the bridge.
4. Moderate I/O concurrency with straightforward code? Use ThreadPoolExecutor.

Scaling to Production: Gunicorn with Uvicorn Workers

The event loop model scales further when combined with multiple worker processes. Gunicorn with Uvicorn workers runs one event loop per CPU core — back to the restaurant metaphor: multiple efficient waiters, one per kitchen station.

gunicorn app:app -w 4 -k uvicorn.workers.UvicornWorker

Breaking down the options:

• -w 4 — four worker processes, one per CPU core. Each runs its own independent event loop.
• -k uvicorn.workers.UvicornWorker — each worker uses Uvicorn’s ASGI event loop instead of a synchronous WSGI worker.

This is the standard production deployment for FastAPI and Starlette applications.

Conclusion

In this article we covered:

• Why the GIL limits thread-based CPU parallelism in standard CPython, and how free-threaded Python 3.13+ changes that.
• ThreadPoolExecutor for I/O-bound work with moderate concurrency.
• ProcessPoolExecutor for CPU-bound workloads requiring real multi-core parallelism.
• InterpreterPoolExecutor (Python 3.14+) as a process-free alternative for CPU-bound tasks.
• The event loop model and why it outperforms thread pools at high concurrency.
• asyncio.TaskGroup for structured concurrency and safer error propagation.
• asyncio.to_thread() as the bridge between synchronous libraries and async applications.
• Gunicorn + Uvicorn workers for scaling the event loop across all CPU cores in production.

The next article puts the event loop theory into practice — building a real async news aggregator with aiohttp, TaskGroup, Semaphore, and asyncio.timeout in a single working module.

If you enjoyed this article, don’t forget to give it a clap 👏, share it with your friends 🔗, and follow me for more tips and tutorials on software development 📘. Your support helps me create more content like this — thank you! 🙌

Posted on 24 Aug 2026

Introduction

Most Python tutorials show you how to write a SELECT statement. They rarely show you what to do when you have ten services, each opening connections and sprinkling session.commit() everywhere — until one forgotten rollback corrupts your data at 2 AM.

This article builds the database layer of a layered Python application. It covers the DatabaseSessionManager, which centralises session lifecycle and transaction management, the DAO models that map your tables to Python objects, the DatabaseInitializer service that creates your schema at startup, and the DAO vs DTO boundary that keeps your architecture honest.

All the code shown here is extracted from runalyze, a personal project I built to analyse my Garmin running workouts. You can browse the full source layout on GitHub.

This post is Part 5 of the How to Write Modern Python Applications series. The previous article covered How to Create Cron Jobs in Python for Your Applications.

You should read this article if:

• you want a reusable, transactional database layer for any Python application (batch job, API, web app)
• you want SQLAlchemy’s ORM to handle SQL for you, without writing raw queries
• you want session lifecycle — commit, rollback, close — managed in one place so services never have to think about it

Project Layout

Think of the database layer as a set of concentric rings. At the centre is the raw connection (database.py). Wrapping it are the table mappings (models/). Around those are the services that do the actual work. Controllers and routes sit on the outermost ring — they call services, and nothing more.

src/runanalyze/
│
├── core/
│   ├── config.py              # Pydantic Settings (Part 3 of this series)
│   └── database/
│       └── database.py        # DatabaseSessionManager
│
├── models/
│   ├── base.py                # SQLAlchemy declarative Base
│   ├── activity.py            # ActivityDAO
│   └── activity_sample.py     # ActivitySampleDAO
│
├── services/
│   ├── db_initializer.py      # Creates tables at startup
│   └── garmin_sync.py         # Business logic (Garmin data fetch & sync)
│
├── controllers/               # Call services, never DAOs directly
└── routes/                    # Flask/FastAPI routes

The models/ folder holds Data Access Objects (DAOs) — pure table-to-class mappings. Business logic lives exclusively in services/. Controllers and routes never touch a DAO directly.

The Declarative Base

Before you can define any table, SQLAlchemy needs a registry to collect all model metadata. One line creates it, in models/base.py:

from sqlalchemy.orm import declarative_base

Base = declarative_base()

Every DAO you write inherits from this Base. When you later call Base.metadata.create_all(engine), SQLAlchemy uses this registry to create every table it knows about in a single pass.

Configuration

The session manager reads its database URL from the Settings object. The full explanation of how Pydantic Settings works is in the previous article. Here is the relevant section of config.yaml:

database:
  sqlite:
    path: "data/garmin.db"
    echo: false
    pool_pre_ping: true

log:
  level: "INFO"
  console: true
  file: "logs/runanalyze.log"
  rotation: "10 MB"
  retention: "7 days"
  compression: "zip"

The database.sqlite section maps to a SQLiteSettings Pydantic model, which exposes computed properties like absolute_path and database_url used by the session manager. For a PostgreSQL backend, replace the sqlite block with host, port, user, and dbname, and inject the password via an environment variable.

DAO vs DTO

Before diving into the code, one conceptual boundary deserves its own section — because getting it wrong quietly corrupts the architecture.

A DAO (Data Access Object) is a Python class that mirrors a database table. It is a SQLAlchemy ORM model. Its job is to represent a row in memory and participate in transactions. It lives in the models/ layer.

A DTO (Data Transfer Object) is a plain data class — a dataclass, a Pydantic model, a TypedDict — that carries data between layers. No SQLAlchemy, no session, no database awareness whatsoever.

The critical rule is this:

A DAO must never cross a service boundary. Controllers, routes, and any caller outside the service layer must only ever see DTOs.

Why does this matter? If a controller receives an ActivityDAO instance directly, it is holding a live SQLAlchemy object bound to a session. The session may be closed. Accessing a lazy-loaded relationship raises a DetachedInstanceError. Serialising it to JSON is not straightforward. Testing the controller requires a real database.

The clean version:

1. The service queries the database, gets back ActivityDAO objects.
2. The service maps them to ActivityDTO (a plain Pydantic model or dataclass).
3. The service returns the DTO.
4. The controller receives the DTO and does whatever it wants with it — serialize it, pass it further, log it.

In the runalyze project, the batch sync service does not have a controller above it, so no DTOs are used today. But the boundary is still respected: the DAOs never leave the service. The moment a controller or API route is added, DTOs become mandatory.

The DatabaseSessionManager

The DatabaseSessionManager is the single object responsible for creating the engine, building sessions, and managing transactions. Services never call session.commit() or session.rollback() directly — the context manager does it for them.

# core/database/database.py
from collections.abc import Generator
from contextlib import contextmanager
from pathlib import Path

from sqlalchemy import create_engine, event
from sqlalchemy.exc import SQLAlchemyError
from sqlalchemy.orm import Session, sessionmaker

from runanalyze.core.config import SQLiteSettings, config


class DatabaseSessionManager:
    def __init__(self) -> None:
        sqlite_settings: SQLiteSettings = config.database.sqlite

        # Ensure the target directory exists
        db_file_path: Path = sqlite_settings.absolute_path
        db_file_path.parent.mkdir(parents=True, exist_ok=True)

        self.database_url = sqlite_settings.database_url

        self.engine = create_engine(
            self.database_url,
            echo=sqlite_settings.echo,
            pool_pre_ping=sqlite_settings.pool_pre_ping,
            connect_args={"check_same_thread": False}
            if self.database_url.startswith("sqlite")
            else {},
        )

        # SQLite only: enforce foreign-key constraints
        @event.listens_for(self.engine, "connect")
        def set_sqlite_pragma(dbapi_connection, connection_record):
            cursor = dbapi_connection.cursor()
            cursor.execute("PRAGMA foreign_keys=ON")
            cursor.close()

        self.SessionFactory = sessionmaker(
            bind=self.engine, autocommit=False, autoflush=False
        )

    @contextmanager
    def get_session(self) -> Generator[Session, None, None]:
        """Yields a transactional session. Commits on success, rolls back on error."""
        session = self.SessionFactory()
        try:
            yield session
            session.commit()
        except SQLAlchemyError as e:
            session.rollback()
            raise e
        except Exception as e:
            session.rollback()
            raise e
        finally:
            session.close()


# Global instance — import this in every service
db_manager: DatabaseSessionManager = DatabaseSessionManager()

A few design decisions are worth understanding:

• yield inside @contextmanager — control returns to the caller (with block) at the yield line. When the block exits, execution resumes in the try/except/finally. The caller never manages the transaction.
• pool_pre_ping=True — before lending a connection from the pool, SQLAlchemy sends a lightweight SELECT 1. Stale connections dropped by the server after idle timeout are discarded and replaced transparently.
• check_same_thread=False — SQLite rejects connections used from a thread other than the one that created them. SQLAlchemy’s session isolation makes multi-thread use safe, so this restriction is lifted explicitly.
• PRAGMA foreign_keys=ON — SQLite does not enforce foreign-key constraints by default. This listener fires on every new connection and switches enforcement on, so ON DELETE CASCADE works as expected.

If a service method raises after yield, the except block executes rollback. If it exits cleanly, commit fires. The service never decides which one to call.

DAO Models

A DAO is a Python class that mirrors a database table. SQLAlchemy translates between instances of that class and rows in the table.

ActivityDAO

The ActivityDAO maps the activities table, which holds one row per Garmin workout:

# models/activity.py
from sqlalchemy import BigInteger, Column, Float, String
from sqlalchemy.orm import relationship

from .base import Base


class ActivityDAO(Base):
    __tablename__ = "activities"

    id = Column(BigInteger, primary_key=True)  # Garmin Connect activity ID
    name = Column(String(length=120), nullable=False)
    activity_type = Column(String(length=50), nullable=True)
    start_time = Column(String(length=50), nullable=False)
    duration_secs = Column(Float, nullable=False)
    distance_meters = Column(Float, nullable=False)
    avg_hr = Column(Float, nullable=True)
    max_hr = Column(Float, nullable=True)
    calories = Column(Float, nullable=False)
    avg_speed_m_s = Column(Float, nullable=False)

    # Training metrics computed from raw samples
    tss = Column(Float, nullable=True)
    vo2max = Column(Float, nullable=True)
    aerobic_decoupling = Column(Float, nullable=True)
    aerobic_decoupling_pure = Column(Float, nullable=True)

    # One-to-many: second-by-second heart rate and speed samples
    samples = relationship(
        "ActivitySampleDAO",
        back_populates="activity",
        cascade="all, delete-orphan",
        passive_deletes=True,
    )

    # One-to-one: weather conditions at the time of the activity
    weather = relationship(
        "ActivityWeatherDAO",
        back_populates="activity",
        cascade="all, delete-orphan",
        passive_deletes=True,
        uselist=False,
    )

The relationship fields do not create columns. They tell SQLAlchemy how to follow foreign keys to load related rows. Two options work together on the samples relationship:

• cascade="all, delete-orphan" — when an ActivityDAO is deleted, all its child ActivitySampleDAO rows are deleted too, at the ORM level
• passive_deletes=True — also lets the database’s own ON DELETE CASCADE handle the deletion, which is far more efficient for large child sets than issuing individual DELETE statements

The weather relationship adds uselist=False because there is exactly one weather record per activity — a one-to-one association.

ActivitySampleDAO

The ActivitySampleDAO maps the activity_samples table, which stores per-second heart rate and speed for every workout:

# models/activity_sample.py
from sqlalchemy import BigInteger, Column, Float, ForeignKey, Integer
from sqlalchemy.orm import relationship

from .base import Base


class ActivitySampleDAO(Base):
    __tablename__ = "activity_samples"

    activity_id = Column(
        BigInteger,
        ForeignKey("activities.id", ondelete="CASCADE"),
        primary_key=True,
    )
    timestamp_secs = Column(
        Integer, primary_key=True
    )  # seconds elapsed since activity start
    heart_rate = Column(Integer, nullable=True)
    speed_m_s = Column(Float, nullable=True)

    activity = relationship("ActivityDAO", back_populates="samples")

The composite primary key (activity_id, timestamp_secs) enforces uniqueness at the database level: the same activity cannot have two samples at the same second.

The DatabaseInitializer

The DatabaseInitializer is a startup service with one job: create every table that does not yet exist. It lives in services/ — not in core/ — because schema initialisation is a business-level decision, not an infrastructure primitive.

# services/db_initializer.py
from runanalyze.core.database import db_manager
from runanalyze.models.activity import ActivityDAO  # noqa: F401
from runanalyze.models.activity_sample import ActivitySampleDAO  # noqa: F401
from runanalyze.models.activity_weather import ActivityWeatherDAO  # noqa: F401
from runanalyze.models.base import Base
from runanalyze.models.daily_metrics import DailyMetricsDAO  # noqa: F401


class DatabaseInitializer:
    def __init__(self) -> None:
        self._engine = db_manager.engine

    def initialize_tables(self) -> None:
        """Create tables if they do not already exist."""
        try:
            Base.metadata.create_all(bind=self._engine)
        except Exception as e:
            raise e

The DAO imports look like dead code, but they are essential. SQLAlchemy’s Base.metadata only knows about tables whose classes have been imported at least once. Without those imports, create_all would find an empty registry and create nothing.

create_all is idempotent — it checks whether each table exists before trying to create it. Running it on every startup is safe and removes the need for any startup condition check.

Using Sessions in a Service

With the infrastructure in place, a service uses db_manager.get_session() as a plain context manager. Here is a simplified version of the Garmin sync service:

# services/garmin_sync.py
from runanalyze.core.database import db_manager
from runanalyze.models.activity import ActivityDAO
from sqlalchemy import exists


class GarminSyncService:
    def save_if_new(self, activity_data: dict) -> None:
        activity_id = activity_data["activityId"]

        with db_manager.get_session() as session:
            already_exists = session.query(
                exists().where(ActivityDAO.id == activity_id)
            ).scalar()

            if already_exists:
                return

            new_activity = ActivityDAO(
                id=activity_id,
                name=activity_data.get("activityName", "Unknown"),
                start_time=activity_data.get("startTimeLocal"),
                duration_secs=activity_data.get("duration", 0.0),
                distance_meters=activity_data.get("distance", 0.0),
                avg_hr=activity_data.get("averageHR"),
                max_hr=activity_data.get("maxHR"),
                calories=activity_data.get("calories", 0.0),
                avg_speed_m_s=activity_data.get("averageSpeed", 0.0),
            )
            session.add(new_activity)
        # commit fires automatically when the `with` block exits cleanly

No session.commit(). No session.rollback(). No session.close(). All of that is owned by the DatabaseSessionManager. And note that new_activity — an ActivityDAO — never leaves this method. The caller gets nothing back, or it would get a DTO in a real API scenario.

Simple Queries vs. the Repository Pattern

For most CRUD operations, SQLAlchemy’s ORM is sufficient and requires no extra abstraction:

# Fetch all activities
activities = session.query(ActivityDAO).all()

# Fetch by primary key
activity = session.get(ActivityDAO, activity_id)

# Filter by date
recent = session.query(ActivityDAO).filter(ActivityDAO.start_time >= "2024-01-01").all()

These reads are clean, readable, and need no additional layer.

Complex queries are a different story. Imagine you need the top-10 activities by average heart rate, joined with aggregated sample counts, filtered by activity type and date range. That logic does not belong directly in a service method — it mixes business intent with query mechanics and becomes hard to test.

The Repository Pattern solves this by giving each aggregate root its own class that owns all query logic for that table:

class ActivityRepository:
    def __init__(self, session: Session):
        self._session = session

    def top_by_avg_hr(self, limit: int = 10) -> list[ActivityDAO]:
        return (
            self._session.query(ActivityDAO)
            .filter(ActivityDAO.avg_hr.is_not(None))
            .order_by(ActivityDAO.avg_hr.desc())
            .limit(limit)
            .all()
        )

The service stays clean — it calls repo.top_by_avg_hr() and never sees the query internals. The repository returns DAOs, which the service immediately maps to DTOs before returning to the caller.

Use the Repository Pattern when queries grow beyond two or three filter conditions, or when the same complex query appears in more than one service.

Schema Migrations with Alembic

create_all is fine for development and greenfield deployments. The moment you need to add a column, rename a table, or drop an index on a database that already holds production data, you need migrations.

Alembic is the standard migration tool for SQLAlchemy. It tracks schema changes in versioned scripts, applies them in order, and supports rollback. A typical migration looks like this:

# alembic/versions/0001_add_elevation_to_activities.py
def upgrade():
    op.add_column(
        "activities", sa.Column("elevation_meters", sa.Float(), nullable=True)
    )


def downgrade():
    op.drop_column("activities", "elevation_meters")

Setting up Alembic is beyond the scope of this article. The short version — run these three commands in your project root:

alembic init alembic
alembic revision --autogenerate -m "initial schema"
alembic upgrade head

Breaking down the revision command:

• --autogenerate — compares your current SQLAlchemy models against the live database schema and generates the migration script automatically
• -m "initial schema" — labels the migration version with a human-readable message

After that, every schema change follows the same cycle: update your models, run alembic revision --autogenerate -m "description", review the generated script, then alembic upgrade head to apply it.

Admin vs. Application User

In production databases — PostgreSQL in particular — the application should never connect as an admin user. A common setup uses two credentials:

• admin user — used once at deployment time by DatabaseInitializer to create tables and grant privileges
• application user — used at runtime by DatabaseSessionManager for all reads and writes

Both credentials come from environment variables and map to two separate engine instances: one for schema setup, one for everything else.

This limits the blast radius of a compromised credential: even if an attacker obtains the application user’s password, they cannot drop tables or alter the schema.

Conclusion

In this article we covered:

• how the DatabaseSessionManager centralises session lifecycle — commit, rollback, close — so services never manage transactions directly
• how declarative_base() creates the metadata registry that create_all relies on
• the DAO vs DTO boundary: DAOs stay inside services, DTOs cross service boundaries — controllers and routes must never handle a raw DAO
• how to define DAO models with relationships, cascades, and composite primary keys, using real examples from the runalyze project
• how the DatabaseInitializer service creates the schema at startup in an idempotent, safe way
• when simple ORM queries are enough and when the Repository Pattern earns its place
• why Alembic is the right tool for schema migrations in production

If you enjoyed this article, don’t forget to give it a clap 👏, share it with your friends 🔗, and follow me for more tips and tutorials on software development 📘. Your support helps me create more content like this — thank you! 🙌

Posted on 11 Aug 2026

Introduction

You have already built the same Task Manager CLI twice. In Part 1 you used Python’s built-in argparse module and the Command Pattern to get a clean, extensible structure. In Part 2 you replaced argparse with Click — the code got shorter, type validation became automatic, and the Command Pattern survived without a shared execute(**kwargs) interface.

Now we do it a third time with Typer. Typer turns Python’s own type annotations into a complete CLI — no option decorator stacks, no explicit type parameters, just function signatures. The result is the most readable version of the three, and it is the one I use in my own projects.

You should read this article if:

• You have read Parts 1 and 2 and want to see the evolution complete.
• You want a CLI library that is natively type-safe and plays well with mypy and IDE autocompletion.
• You want a real-world reference showing Typer in a real project before committing to it.

argparse vs Click vs Typer at a Glance

Before diving into code, here is the complete picture:

Typer is to CLIs what FastAPI is to web APIs: the same idea of driving behaviour from type annotations rather than from imperative configuration.

Why Typer?

Click removed the argparse boilerplate. Typer removes the option decorator stack.

To be precise: Typer still uses a decorator to register a command (@app.command()), just like Click. What disappears are all the @click.option and @click.argument decorators that pile up above every function in Click. With one option those are manageable; with four or five options the function itself disappears under a tower of decorators.

With Click, every option requires an explicit @click.option("--name", type=str, required=True) decorator. The type is declared twice — once in the decorator, once in the function signature — and they can silently drift apart. Typer merges the two: the function signature is the definition, and the single @app.command() decorator is the only one you need.

Two more practical gains:

1. Docstrings become help text. Write """Deploy a new agent sandbox.""" on a function and Typer uses it for --help. You document the code and the CLI at the same time.
2. Optional and Annotated work as you expect. name: str | None = None is an optional option. Annotated[str, typer.Option(help="Task name")] adds metadata without a separate decorator.

Typer is built on top of Click, so every Click feature is still available if you need it. You are not giving anything up.

Installing Typer

uv add "typer[all]"

Breaking down the options:

• typer — the core library
• [all] — installs rich (coloured output, formatted help pages) and shellingham (shell auto-completion detection); highly recommended for end-user CLIs

Running uv add records the dependency in pyproject.toml and updates uv.lock — no manual editing needed.

The Task Manager CLI — Code Walkthrough

The CLI syntax stays the same as in Parts 1 and 2:

python3 cli.py task add --name "Buy milk"
python3 cli.py task list
python3 cli.py task delete --id 1
python3 cli.py stats summary
python3 cli.py stats export --output report.csv

No external repository this time — all code lives inline in this article.

Project Structure

task-cli/
├── cli.py
└── commands/
    ├── __init__.py
    ├── base.py
    ├── add_task_command.py
    ├── list_task_command.py
    ├── delete_task_command.py
    └── stats_command.py

The structure is identical to Parts 1 and 2. Only cli.py changes significantly.

The Base Command Class

Same marker pattern introduced in the Click version — no shared execute() method, just a semantic boundary:

# commands/base.py
from abc import ABC


class Command(ABC):
    """Marker base class for CLI commands."""

The Concrete Command Classes

The command classes are unchanged from the Click version. Each method carries an explicit, typed signature:

# commands/add_task_command.py
from .base import Command


class AddTaskCommand(Command):
    def execute(self, name: str) -> None:
        with open("tasks.txt", "a") as f:
            f.write(name + "\n")
        print(f"Task added: {name}")

# commands/list_task_command.py
from .base import Command


class ListTaskCommand(Command):
    def execute(self) -> None:
        try:
            with open("tasks.txt") as f:
                tasks = [line.strip() for line in f if line.strip()]
        except FileNotFoundError:
            tasks = []
        if not tasks:
            print("No tasks found.")
            return
        print("Tasks:")
        for i, t in enumerate(tasks, 1):
            print(f"{i}. {t}")

# commands/delete_task_command.py
from .base import Command


class DeleteTaskCommand(Command):
    def execute(self, task_id: int) -> None:
        try:
            with open("tasks.txt") as f:
                tasks = [line.strip() for line in f if line.strip()]
        except FileNotFoundError:
            print("No tasks found.")
            return
        if task_id < 1 or task_id > len(tasks):
            print(f"Invalid task ID: {task_id}")
            return
        removed = tasks.pop(task_id - 1)
        with open("tasks.txt", "w") as f:
            f.write("\n".join(tasks))
        print(f"Task deleted: {removed}")

# commands/stats_command.py
import csv
import os
from .base import Command


class StatsCommand(Command):
    TASKS_FILE = "tasks.txt"

    def summary(self) -> None:
        tasks = self._load_tasks()
        print(f"You have {len(tasks)} task(s).")
        for i, t in enumerate(tasks, 1):
            print(f"{i}. {t}")

    def export(self, output: str) -> None:
        tasks = self._load_tasks()
        if not tasks:
            print("No tasks to export.")
            return
        with open(output, "w", newline="") as csvfile:
            writer = csv.writer(csvfile)
            writer.writerow(["ID", "Task"])
            for i, task in enumerate(tasks, 1):
                writer.writerow([i, task])
        print(f"Tasks exported to {output}")

    def _load_tasks(self) -> list[str]:
        if not os.path.exists(self.TASKS_FILE):
            return []
        with open(self.TASKS_FILE) as f:
            return [line.strip() for line in f if line.strip()]

The CLI Entry Point

This is where Typer replaces Click. Compare the structure to the Click version and notice what disappears: no @click.option decorator stacks, no explicit type= parameters.

# cli.py
from typing import Annotated
import typer
from commands.add_task_command import AddTaskCommand
from commands.list_task_command import ListTaskCommand
from commands.delete_task_command import DeleteTaskCommand
from commands.stats_command import StatsCommand

app = typer.Typer(help="Task Manager CLI", no_args_is_help=True)
task_app = typer.Typer(help="Manage tasks.", no_args_is_help=True)
stats_app = typer.Typer(help="Show or export task statistics.", no_args_is_help=True)

app.add_typer(task_app, name="task")
app.add_typer(stats_app, name="stats")


@task_app.command("add")
def task_add(
    name: Annotated[str, typer.Option("--name", "-n", help="Task name.")],
) -> None:
    """Add a new task."""
    AddTaskCommand().execute(name=name)


@task_app.command("list")
def task_list() -> None:
    """List all tasks."""
    ListTaskCommand().execute()


@task_app.command("delete")
def task_delete(
    task_id: Annotated[int, typer.Option("--id", "-i", help="Task ID to delete.")],
) -> None:
    """Delete a task by ID."""
    DeleteTaskCommand().execute(task_id=task_id)


@stats_app.command("summary")
def stats_summary() -> None:
    """Show a summary of all tasks."""
    StatsCommand().summary()


@stats_app.command("export")
def stats_export(
    output: Annotated[
        str, typer.Option("--output", "-o", help="Output CSV file name.")
    ] = "tasks.csv",
) -> None:
    """Export tasks to a CSV file."""
    StatsCommand().export(output=output)


if __name__ == "__main__":
    app()

A few things worth noting:

• app.add_typer(task_app, name="task") registers a sub-app — the equivalent of @click.group. Each sub-app is an independent typer.Typer() instance.
• Annotated[str, typer.Option(...)] is the idiomatic Typer way to attach help text and flags to a parameter while keeping the type annotation separate and readable by mypy.
• The function docstring ("""Add a new task.""") becomes the command description in --help automatically.
• no_args_is_help=True on each app means running python3 cli.py or python3 cli.py task without a subcommand prints the help page instead of doing nothing.

Running the CLI

python3 cli.py task add --name "Write Part 3"
python3 cli.py task add --name "Review PR"
python3 cli.py task list
python3 cli.py task delete --id 1
python3 cli.py stats summary
python3 cli.py stats export --output report.csv

The --help output for the top-level app:

python3 cli.py --help

Typer renders a rich, coloured help page listing all registered sub-apps and commands — at no extra cost.

Side-by-Side: the Same Command in Three Libraries

To make the progression concrete, here is task add implemented in all three:

argparse (Part 1)

add_parser = subparsers.add_parser("add", help="Add a new task")
add_parser.add_argument("--name", required=True, help="Task name")
add_parser.set_defaults(func=self.commands["add"].execute)

Click (Part 2)

@cli.command(help="Add a new task")
@click.option("--name", "-n", required=True, help="Task name")
def add(name):
    AddTaskCommand().execute(name=name)

Typer (this article)

@task_app.command("add")
def task_add(
    name: Annotated[str, typer.Option("--name", "-n", help="Task name.")],
) -> None:
    """Add a new task."""
    AddTaskCommand().execute(name=name)

A Real-World Reference: golem-cli

The pattern above is not just a toy example. golem-cli is the CLI client for a real Agentic AI Platform-as-a-Service project I am working on, built with exactly this structure.

The cli.py entry point is pure Typer wiring — no business logic:

# src/golem_cli/cli.py (excerpt)
app = typer.Typer(
    name="golem", help="golem — Golem Agent-as-a-Service CLI", no_args_is_help=True
)

agent_app = typer.Typer(help="Manage agent sandboxes.", no_args_is_help=True)
app.add_typer(agent_app, name="agent")


@agent_app.command("create")
def agent_create(
    config: Annotated[
        Path, typer.Option("--config", "-c", help="Path to the runner config YAML.")
    ],
    ttl: Annotated[
        int, typer.Option("--ttl", "-t", help="Sandbox time-to-live in seconds.")
    ] = 3600,
) -> None:
    """Deploy a new agent sandbox."""
    AgentCommand().create(config=config, ttl_seconds=ttl)

The business logic lives entirely in AgentCommand, ChatCommand, and CpCommand — the Command Pattern is preserved cleanly. The CLI file’s only job is to map flags to method calls.

The project structure mirrors what you built in this series:

golem-cli/
└── src/
    └── golem_cli/
        ├── cli.py              ← Typer wiring only
        └── commands/
            ├── base.py         ← Marker ABC
            ├── agent_command.py
            ├── chat_command.py
            └── cp_command.py

When Typer is your chosen library, this is what a real project looks like.

When to Keep Using Click

Typer is the better default for new projects, but Click still wins in specific cases:

• You are extending an existing Click-based codebase. Typer can wrap Click objects, but mixing them adds complexity — stay pure.
• You need Click plugins or third-party Click extensions. The ecosystem is built around Click’s decorator API.
• You need fine-grained control over context objects and pass-through behaviour. Click’s @click.pass_context is more explicit for advanced orchestration.

For everything else — new projects, in-house tools, developer utilities — Typer’s type-hint–driven approach is cleaner and safer.

Conclusion

In this article we covered:

• Why Typer eliminates the decorator boilerplate that Click still requires, by inferring command structure from type annotations
• How to build a complete Task Manager CLI with Typer: sub-apps, typed options, docstring-driven help text
• How the same Command Pattern from Parts 1 and 2 carries over unchanged — business logic in command classes, wiring in cli.py
• A side-by-side comparison of argparse, Click, and Typer on the same command, with a concrete table of differences
• How golem-cli applies this exact pattern in a real project

This is the final article in the Modern Python Application series. The three CLI posts together form a complete toolkit: argparse for zero-dependency scripts, Click for mature ecosystems, Typer for new projects where type safety and readability matter. For a deeper dive into the Command Pattern used throughout, the Refactoring.Guru reference is still the best starting point.

If you enjoyed this article, don’t forget to give it a clap 👏, share it with your friends 🔗, and follow me for more tips and tutorials on software development 📘. Your support helps me create more content like this — thank you! 🙌

Posted on 01 Aug 2026

Introduction

This is the eighth and final article of the Getting Started with Docker series. In the previous article we hardened a container against attacks. Back in Part 3 we wrote our first Dockerfile and learned the basic instructions and the layer cache.

Now it is time to put everything together. A Dockerfile that just “works” is not enough — a well-written image should be:

• Functional — it starts correctly, exposes the right ports, and tells Docker how to check its own health
• Lightweight — it ships only what the application needs to run, nothing else
• Debuggable — when something goes wrong at 2 AM, you can find out why quickly
• Secure — it does not run as root and does not carry unnecessary attack surface

This article collects the practices that matter most for each of these four dimensions, using the Nginx example from the series as a running example.

1. Functional: Make the Image Behave Predictably

Use the exec form for CMD and ENTRYPOINT

# Good — exec form, PID 1, signals delivered directly
CMD ["nginx", "-g", "daemon off;"]

# Avoid — shell form wraps the process, SIGTERM may not reach it
CMD nginx -g daemon off;

The exec form (JSON array) runs your process as PID 1 directly, so docker stop can deliver SIGTERM to it and your application can shut down gracefully instead of being killed after the timeout.

Combine ENTRYPOINT and CMD

Use ENTRYPOINT for the fixed executable and CMD for default arguments the user can override:

ENTRYPOINT ["nginx"]
CMD ["-g", "daemon off;"]

Now docker run my-nginx -v runs nginx -v instead of the default arguments, without anyone needing to know the entrypoint.

Declare a HEALTHCHECK

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD curl -f http://localhost/ || exit 1

Without a HEALTHCHECK, Docker only knows whether the main process is alive — not whether your application is actually serving requests. With it, docker ps shows healthy / unhealthy, and orchestrators (Compose, Swarm, Kubernetes) can restart or stop routing traffic to a broken container automatically.

Use ENV for configuration, not for secrets

ENV NGINX_PORT=80

Environment variables are the standard way to make an image configurable at runtime. Give sensible defaults in the Dockerfile and let users override them with docker run -e or Compose’s environment: — but keep this to configuration, never credentials (see the security article for why).

Add a .dockerignore

.git
node_modules
*.md
Dockerfile
.env

A .dockerignore file works exactly like .gitignore: it keeps unnecessary or sensitive files out of the build context, which makes builds faster and prevents accidentally COPY-ing things like .git history or local .env files into the image.

2. Lightweight: Ship Only What You Need

Start from a minimal base image

FROM nginx:alpine

alpine-based images are a fraction of the size of their Debian/Ubuntu-based equivalents because they use musl libc and busybox instead of a full userland. For runtimes that support it, distroless images go even further — no shell, no package manager, just the runtime and your application.

Use multi-stage builds

This is the single most effective technique for keeping images small. Build tools, source code, and dependencies stay in an intermediate builder stage that never ships — only the final artifact is copied into the runtime image.

# Stage 1: build the static assets
FROM node:20-alpine AS builder
WORKDIR /build
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: serve them from a minimal image
FROM nginx:alpine
COPY --from=builder /build/dist /usr/share/nginx/html
USER nginx
EXPOSE 80

The node_modules folder, the TypeScript compiler, and every build dependency exist only in the builder stage. The final image contains just nginx and the compiled static files — often a 20x reduction in size, and a much smaller attack surface as a bonus.

When to reach for it: any time your build step needs tools your runtime does not — a compiler or SDK (Go, Rust, Java/Maven, C/C++), a bundler/transpiler for a front-end app (webpack, vite, tsc), or a test suite you want to run during the build but never ship. As a rule, if docker build needs to install something with apt/apk/npm/pip just to produce an artifact, and the running application never calls that tool again, it belongs in a builder stage. It is also useful to keep build-time secrets (private registry tokens, npm auth) out of the final image’s layer history, since the builder stage is simply discarded.

When it is not worth it: if your image never compiles or bundles anything — say, a Python script that only needs its runtime dependencies installed once, or a set of static files that already exist in the repository — a single stage is simpler and there is no build-only tooling to strip out. Multi-stage solves a specific problem (build tools leaking into the runtime image); do not add a stage just because it looks more sophisticated.

Chain RUN instructions and clean up in the same layer

# Good — one layer, no leftover cache
RUN apk add --no-cache curl \
  && curl -O https://example.com/tool.tar.gz \
  && tar -xzf tool.tar.gz \
  && rm tool.tar.gz

# Avoid — the download persists in an earlier layer even if a later RUN removes it
RUN apk add curl
RUN curl -O https://example.com/tool.tar.gz
RUN tar -xzf tool.tar.gz
RUN rm tool.tar.gz

Each RUN is a layer, and layers are additive — deleting a file in a later layer does not shrink the image, it only hides the file. Install, use, and clean up temporary files within the same RUN instruction. The --no-cache flag on apk add (or rm -rf /var/lib/apt/lists/* on Debian-based images) avoids leaving the package index cached in a layer.

Beware of chown -R in its own layer

A subtler version of the same problem: changing ownership of a large tree after it has already been copied.

# Avoid — chown touches every file, duplicating them into a new layer
COPY app /app
RUN chown -R appuser:appgroup /app

# Good — ownership is set as part of the copy, no extra layer
COPY --chown=appuser:appgroup app /app

chown looks like it only changes metadata, but the overlay filesystem Docker uses has to copy each affected file into the new layer to record that change. Run RUN chown -R over a directory that a previous layer already populated, and you can end up shipping that directory’s contents twice — once in the COPY layer, once again in the RUN chown layer — roughly doubling its footprint in the image.

You can catch this with docker history, which lists the size of every layer in build order:

docker history my-nginx:1.2.0

If a RUN chown -R ... layer is anywhere near the size of the COPY layer above it, that is the tell — a metadata-only change should be close to 0 B. For a more visual, layer-by-layer breakdown (including a wasted-space estimate), the dive tool is built exactly for this:

dive my-nginx:1.2.0

The fix is almost always the same: set ownership with --chown on the COPY (or ADD) instruction that creates the files, instead of a separate RUN chown -R afterwards.

Order instructions from least to most frequently changing

As covered in Part 3, put dependency installation before COPY-ing application code. Dependencies change rarely, code changes often — this ordering keeps the layer cache useful across builds.

3. Debuggable: Make Failures Easy to Diagnose

A lightweight, locked-down image is worth little if nobody can figure out why it crashed. A few practices keep images small and secure without sacrificing debuggability.

Log to stdout/stderr, not to files

# Nginx already does this — logs are symlinked to stdout/stderr
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
  && ln -sf /dev/stderr /var/log/nginx/error.log

Docker captures anything written to stdout/stderr and makes it available through docker logs, which works with every logging driver and log aggregator. Logs written to a file inside the container are invisible unless you exec in and go looking for them.

docker logs -f my-app

Keep a shell available, or know how to add one temporarily

Distroless and scratch images are excellent for production but have no shell, so docker exec -it my-app sh will not work. If you need that debugging path, alpine is usually the right trade-off — it is still tiny but keeps sh, curl, and basic tools available:

docker exec -it my-app sh

If you are committed to a distroless final stage, keep a debug-friendly variant of the same image (many distroless images ship a :debug tag with Busybox included) for troubleshooting.

Add LABEL metadata

LABEL org.opencontainers.image.source="https://github.com/you/my-nginx" \
      org.opencontainers.image.version="1.0" \
      org.opencontainers.image.description="Custom Nginx image with static site"

When an incident happens at 2 AM, docker inspect on a labeled image immediately tells you which commit and which repository it was built from — no guessing which tag maps to which source.

Always use explicit, meaningful tags

Reusing latest makes it impossible to know which build is actually running. Tag every image with a version (and ideally the commit SHA), as shown in Part 3:

docker build -t my-nginx:1.2.0 -t my-nginx:$(git rev-parse --short HEAD) .

4. Secure: Don’t Undo Your Own Work

We covered security in depth in the previous article; here is the short version to keep in mind while writing every Dockerfile:

• Never run as root — add a USER instruction, or run with --user
• Prefer a read-only filesystem at runtime, with tmpfs for the few paths that need writing
• Drop Linux capabilities with --cap-drop=ALL and add back only what is required
• Never bake secrets into ENV or image layers — use secrets managers or untracked .env files
• Scan images for known CVEs with Docker Scout or Trivy before shipping
• Pin base images to a digest and keep them updated

A lightweight image built with multi-stage builds is already a security win: fewer packages means fewer CVEs to track.

Putting It All Together

Here is the Nginx example from the series, now applying all four dimensions at once:

# ── Stage 1: build ──────────────────────────────────────────────
FROM node:20-alpine AS builder
WORKDIR /build
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# ── Stage 2: runtime ─────────────────────────────────────────────
FROM nginx:alpine

LABEL org.opencontainers.image.source="https://github.com/you/my-nginx" \
      org.opencontainers.image.version="1.2.0"

# Functional: only the compiled static assets, owned by the runtime user
COPY --from=builder --chown=nginx:nginx /build/dist /usr/share/nginx/html

# Debuggable: send Nginx logs to stdout/stderr
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
  && ln -sf /dev/stderr /var/log/nginx/error.log

# Secure: drop to the non-root user the base image already provides
USER nginx

EXPOSE 80

# Functional: let Docker (and orchestrators) know when we're actually serving traffic
HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD curl -f http://localhost/ || exit 1

ENTRYPOINT ["nginx"]
CMD ["-g", "daemon off;"]

Run it the same secure way we did in the previous article:

docker build -t my-nginx:1.2.0 .

docker run -d -p 8080:80 \
  --read-only \
  --tmpfs /tmp --tmpfs /var/cache/nginx --tmpfs /var/run \
  --cap-drop=ALL --cap-add=NET_BIND_SERVICE \
  --name my-app \
  my-nginx:1.2.0

Checklist

A quick reference to keep next to your Dockerfile:

• Exec form for CMD/ENTRYPOINT
• HEALTHCHECK defined
• .dockerignore excludes build artifacts, VCS metadata, and secrets
• Minimal base image (alpine, distroless, or scratch)
• Multi-stage build separates build tools from the runtime image
• Related RUN commands chained and cleaned up in the same layer
• Ownership set with COPY --chown rather than a separate RUN chown -R
• Instructions ordered from least to most frequently changing
• Application logs to stdout/stderr
• Image carries LABEL metadata and an explicit version tag
• USER set to a non-root user
• Runs with a read-only filesystem and dropped capabilities
• No secrets in ENV or image layers
• Scanned for CVEs before shipping

Conclusion

In this article we brought together everything the series covered, organized around four goals for every image you write:

• Functional: exec-form commands, HEALTHCHECK, sensible ENV defaults, and a .dockerignore
• Lightweight: minimal base images and multi-stage builds to keep build tools out of the final image
• Debuggable: logs to stdout/stderr, LABEL metadata, explicit tags, and a shell available when you need one
• Secure: non-root users, read-only filesystems, dropped capabilities, and vulnerability scanning

This concludes the Getting Started with Docker series. You now have a solid foundation to build, network, persist, orchestrate, secure, and package containerized applications the right way. The natural next step is Docker Swarm for multi-host deployments, or Kubernetes for large-scale container orchestration.

If you enjoyed this article, don’t forget to give it a clap 👏, share it with your friends 🔗, and follow me for more tips and tutorials on software development 📘. Your support helps me create more content like this — thank you! 🙌

Posted on 31 Jul 2026

Introduction

This is the seventh article of the Getting Started with Docker series. In the previous article we orchestrated a multi-container application with Docker Compose. Now we turn our attention to security.

Containers are isolated — but isolation is not the same as security. A container running as root, with a writable filesystem and full Linux capabilities, is a significant risk if it is ever compromised. The good news is that Docker provides several mechanisms to reduce the attack surface, and most of them require only a few lines of configuration.

1. Never Run as Root

By default, processes inside a container run as root (UID 0). This means that if an attacker exploits a vulnerability in your application, they have root access inside the container — and depending on how the container is configured, they may be able to escape to the host.

The fix is simple: add a USER instruction to your Dockerfile.

FROM nginx:alpine

COPY html /usr/share/nginx/html

# Switch to the non-root user that the nginx image already provides
USER nginx

EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

The official nginx:alpine image already includes a non-root nginx user. For your own images, create a dedicated user:

FROM alpine:3.20

RUN addgroup -S appgroup && adduser -S appuser -G appgroup

COPY --chown=appuser:appgroup app /app

USER appuser

CMD ["/app/start.sh"]