Developer Docs

Media Intelligence & Processing

The Ngwenya platform handles media with extreme care, ensuring high performance, zero redundancy, and native support for rich media across all Malets. The Media Service acts as the core ingest point, but delegates CPU-heavy workloads offline.

This document details the three core pillars of our "Media Intelligence" infrastructure: EXIF Extraction, SHA-256 Deduplication, and Asynchronous Video Transcoding.

1. SHA-256 Deduplication & Ref-Counted Storage

To prevent storage bloat and ensure fast uploads for Malet Owners, every incoming media file is hashed using SHA-256 before it touches Cloudflare R2 / S3.

The Upload Flow

When a file is uploaded, the Media Service:

  1. Calculates the SHA-256 hash of the raw buffer.
  2. Queries the database for an existing MediaFile where:
    • hash matches.
    • maletId matches (deduplication is intentionally isolated to a single Malet for privacy).
    • status is COMPLETE.

Fast-Path Cloning

If an identical file exists, the service skips all processing and S3 uploads. Instead, it generates a new database record that shares the exact same key, url, and thumbnailUrl as the original, returning instantly.

Safe Deletion (Reference Counting)

Because multiple database records may point to the same physical S3 object, deletion must be careful. When a MediaFile is deleted, the service uses countDocuments({ key: file.key }):

  • If count > 1: Only the database record is removed. The S3 object remains.
  • If count === 1: The database record AND the physical S3 object are permanently deleted.

2. EXIF Metadata Extraction

Rich media often contains valuable metadata. For images, we use the exifr library to extract data before image optimization occurs (our optimization pipeline intentionally strips EXIF data to protect the privacy of Visitors / Buyers and Malet Owners).

// Example EXIF Output stored in MediaFile.metadata.exif
{
  "Make": "Apple",
  "Model": "iPhone 15 Pro",
  "DateTimeOriginal": "2026-04-05T12:00:00Z",
  "latitude": 37.7749,
  "longitude": -122.4194
}

This extracted metadata is attached to the MediaFile graph, allowing future features (like map-based discovery in the Lobby) to utilize the data without compromising the raw image file's privacy footprint.

3. Asynchronous Video Transcoding

Video processing (generating HLS streams from MP4/MOV) is incredibly CPU-intensive. To protect the core GraphQL Gateway and Media Service from latency spikes, video processing is fully decoupled.

Architecture

sequenceDiagram
    participant Client as Frontend
    participant Media as Media Service (Port 30012)
    participant TCP as TCP Event Bus
    participant Video as Video Microservice (Port 30021)
    participant S3 as R2 / S3 Storage

    Client->>Media: Upload raw video via REST
    Media->>Media: Hash buffer, save to DB (status: PROCESSING)
    Media->>S3: Upload raw chunk to R2
    Media--)TCP: emit('transcode_video_hls', { id, key })
    Media->>Client: Return 201 (Client shows spinner)

    Note over Video: Asynchronous FFmpeg Worker
    TCP-->>Video: Receive transcode job
    Video->>S3: Download raw video
    Video->>Video: FFmpeg -> HLS (.m3u8, .ts), Poster (.jpg)
    Video->>S3: Upload HLS playlist & Poster to R2
    Video->>Media: Update DB (status: COMPLETE, urls added)
    Note over Client: Next poll / WebSocket shows Video ready

The `apps/video` Microservice

  • Transport: NestJS TCP (Port 30021) — no HTTP overhead.
  • Engine: Wraps fluent-ffmpeg via Alpine Linux ffmpeg package.
  • Output: Generates Apple HTTP Live Streaming (HLS) standard formats (VOD playlist, 6-second .ts segments) for adaptive streaming, plus a 1-second mark .jpg poster frame.

Media Client Adapter Strategies

The Media service interacts with the Video service via a Strategy pattern (apps/media/src/video-transcode/):

  • RemoteStrategy: Emits the TCP event to apps/video.
  • NoOpStrategy: Used for local development if the video service is disabled. Skips transcoding entirely to allow local media work to continue uninterrupted.

Future Scale: The Rust Rewrite Path

While the NestJS worker processes videos flawlessly, Node.js memory overhead limits vertical scaling capacity. Because the system is heavily decoupled via a language-agnostic TCP contract, the apps/video service is documented and primed for a future drop-in replacement using a lightweight Rust binary (ffmpeg-sys + tokio), guaranteeing massive density improvements as the platform scales.

  • Media Infrastructure — Multi-size thumbnails, blur placeholders, Cloudflare R2 storage, and presigned upload flow
  • Media Analytics API — Upload volume, storage usage, type distribution, and processing health analytics