Home / Cluster rendering

Cluster rendering

The cluster module (srcs/core/cluster/*, interfaces srcs/common/cluster/*) lets you spread the rendering of an image, tile by tile, across several machines on the network. One machine acts as the server: it owns the scene and hands out the tiles to render. The other machines are workers (clients): they compute the tiles they receive and send the pixels back to the server, which assembles the final image.

Overview

The goal of the cluster is to speed up a render by making several computers work in parallel. The image is split into rectangular tiles (default size 32×32), each identified by a tile_id. The server keeps a queue of tiles to render and sends them to connected workers; each worker renders its tile locally and sends back the corresponding array of pixels. The server accumulates these pixels into the image buffer.

Rendering is progressive, sample by sample (sample): each tile is tied to a sample number, and a fresh full pass can be relaunched to refine the result. Tile coordination is handled by ClusterRenderCoordinator, and accumulation by ClusterRenderer (which implements both ISceneRenderer and IClusterTileSink).

Server and workers

The module distinguishes two roles, exposed through ClusterMode (CLIENT, SERVER, NONE):

On the server side, each worker is represented by a Connection that tracks a ConnectionState (PENDING, CONNECTED, DISCONNECTED, REFUSED). On the worker side, the client tracks its own ClientState (throughout the session: fetching data, receiving, rendering, sending, idling) and the server’s render state (ServerRenderState: IDLING or RENDERING).

API — IClusterModule

The entry point is the IClusterModule interface (srcs/common/cluster/IClusterModule.hpp), implemented by ClusterModule. It holds either a server or a client (never both at once) and refuses to start if a mode is already active.

namespace rc
{
    enum class ClusterMode { CLIENT, SERVER, NONE };

    class IClusterModule
    {
        public:
            virtual ClusterMode   getClusterMode()   const = 0;
            virtual IClusterServer *getClusterServer() const = 0;
            virtual IClusterClient *getClusterClient() const = 0;

            // Start a server that owns `scene`; port 0 = automatic port.
            virtual void startServer(IScene *scene, size_t port = 0) = 0;

            // Join a server as a worker.
            virtual void joinCluster(const std::string &address, size_t port) = 0;

            // Leave the cluster (only valid in CLIENT mode).
            virtual void leaveCluster() = 0;
    };
}

Session walkthrough

Communication goes over TCP sockets. Each message is framed by a 6-byte header (2-byte packet id + 4-byte payload size, in network order), followed by the serialized payload. The nominal session flow is:

1. Connect — the worker opens a TCP connection to the server and sends ClientJoinRequest.
2. Accept — the server marks the connection CONNECTED and replies with ServerJoinResponse.
3. Request the scene — the worker sends ClientFetchSceneData.
4. Send the scene — the server serializes the scene to JSON (SceneRegister) and returns ServerSceneData.
5. Prepare — the worker parses the JSON (SceneParser), builds the BVH and becomes idle (IDLING).
6. Distribute — during rendering, the server assigns tiles with ServerRenderRequest (id, sample, coordinates).
7. Return pixels — the worker renders the tile and sends back ClientTileData (the tile’s pixels).
8. Assemble — the server validates the tile (markComplete) and accumulates the pixels into the image (applyTileSample).
9. State & cancellation — the server broadcasts its state (ServerRenderState) and can cancel in-flight tiles (ServerCancelRender).

Worker (client)                         Server (owns the scene)
     |                                            |
     |  1. TCP connect + ClientJoinRequest        |
     | -----------------------------------------> |
     |                                            |  → CONNECTED
     |        2. ServerJoinResponse               |
     | <----------------------------------------- |
     |                                            |
     |     3. ClientFetchSceneData                |
     | -----------------------------------------> |
     |                                            |
     |     4. ServerSceneData (scene as JSON)     |
     | <----------------------------------------- |
     |  (parseScene + buildBvh, state IDLING)     |
     |                                            |
     |     5. ServerRenderRequest (tile)          |
     | <----------------------------------------- |
     |  (render_tile_sample over the tile)        |
     |     6. ClientTileData (pixels)             |
     | -----------------------------------------> |
     |                                            |  markComplete + applyTileSample
     |            ...  more tiles  ...             |
     |   ServerRenderState / ServerCancelRender   |
     | <----------------------------------------- |

A tile queue with a guard timeout (requeueTimedOut, ~5 s) reassigns to another worker any tile that has not come back in time, which makes the render tolerant of slow or lost workers.

Packet types

Packets are typed by PacketID and rebuilt on receipt by PacketFactory::createPacket(id, payload). Each packet implements IPacket (serialize / deserialize / handle / getId) and processes itself through an IPacketHandler (the connection on the server, the client on the worker). The table below lists the packets actually registered in the factory:

On-the-wire format (identical in both directions):

┌───────────────┬────────────────────┬───────────────────────┐
│ id (2 bytes)  │ size (4 bytes)     │ payload (N bytes)      │
└───────────────┴────────────────────┴───────────────────────┘
      uint16            uint32              IPacket serialization
  (network order)   (network order)

Starting a cluster

Options are parsed by srcs/config/Options.hpp. The scene file is an optional positional argument; without it, a default scene is loaded.

On the server machine (it owns the scene):

# Start the UI and a cluster server on port 8080
./raytracer --server 8080 scenes/demo.cfg

# Automatic port (chosen by the system) if the number is omitted
./raytracer --server scenes/demo.cfg

On each worker (it joins the server):

# Join the server at the given IP and port
./raytracer --connect 192.168.1.10 8080

Joining from the interface

Besides the command line, you can join a cluster from the graphical interface through the JoinClusterWindow (srcs/plugins/user_interface/windows/JoinClusterWindow.hpp). It is a small “Join a cluster” pop-up (400×180) offering:

• an IP field (the server address);
• a Port field, restricted to unsigned integers;
• a JOIN button.

Validating a field or clicking JOIN fires a windowCallback(ip, port) callback that ultimately calls joinCluster(ip, port) on the module. Validating the IP field moves focus to the Port field, and the port must be filled in before a connection is attempted.

Limits & behavior

Based on the current code:

• IPv4 only — sockets use AF_INET / inet_pton(AF_INET, …); IPv6 addresses are not supported.
• Plaintext transport — raw TCP, with no authentication or encryption. Meant for a trusted local network.
• Scene sent as JSON — the server re-serializes the scene through SceneRegister and the worker rebuilds it with SceneParser; each worker builds its own BVH on receipt.
• Slow-worker tolerance — a tile that has not returned before the guard timeout (~5 s) is reassigned; fine-grained handling of disconnects during a render is still minimal (handleClientDisconnect is currently an almost-empty extension point).
• One role at a time — the module refuses to start a server or join a cluster if it is already in SERVER or CLIENT mode.
• Server-driven rendering — workers only render the tiles the server assigns them; they stay idle until a request arrives.