Universal Search Index — Developer Guide

Overview

The Universal Search Index extends Mallnline's Lobby discovery engine beyond commerce items (Products and Services) to include Blog Posts, Malets themselves, and Platform Guides as first-class searchable entities. This transforms the search experience from a product catalog into a comprehensive platform-wide discovery system.

Architecture

graph TD
    subgraph Sources
        P["Products Subgraph"]
        S["Services Subgraph"]
        B["Blogs Subgraph"]
        M["Malets Subgraph"]
        G["Guide Module (Search)"]
    end

    subgraph Search Subgraph
        SC["SearchController (TCP)"]
        GS["GuideService"]
        MS["MeilisearchService"]
        BI["BulkImportService"]
        RC["ReconciliationService"]
    end

    subgraph Index
        MI["Meilisearch Index"]
    end

    P -- "item_upserted / item_deleted" --> SC
    S -- "item_upserted / item_deleted" --> SC
    B -- "blog_post_upserted / blog_post_deleted" --> SC
    M -- "malet_upserted / malet_tags_updated" --> SC

    SC --> MS --> MI
    GS --> MS
    BI --> MS
    RC --> MS

    style MI fill:#f59e0b,color:#fff
    style SC fill:#3b82f6,color:#fff
    style GS fill:#10b981,color:#fff

Data Flow

1. Source subgraphs emit TCP events when content changes (create, update, status transition, delete)
2. SearchController receives events, enriches with Malet metadata (name, verification, section tags), and indexes via MeilisearchService
3. GuideService manages platform guides internally within the search subgraph — no external TCP events needed
4. BulkImportService and ReconciliationService cover all five entity types for administrative integrity

ItemType Enum

The ItemType enum determines how each indexed document is categorized and filtered in the Lobby:

enum ItemType {
	PRODUCT = 'PRODUCT', // Physical or digital goods from a Malet
	SERVICE = 'SERVICE', // Bookable services from a Malet
	BLOG_POST = 'BLOG_POST', // Published blog posts from a Malet
	MALET = 'MALET', // The Malet itself (Mall Outlet)
	GUIDE = 'GUIDE' // Platform-managed user guides and help articles
}

Filtering by Type

Visitors can filter search results by type using the types array:

query {
	search(query: "photography", filters: { types: [MALET, BLOG_POST] }) {
		results {
			id
			name
			type
			slug
			contentSource
		}
		facets {
			types {
				type
				count
			}
		}
		totalCount
	}
}

The types array uses OR logic — results matching any of the specified types are returned. If omitted, all types are searched.

ContentSource Discriminator

The ContentSource enum separates user-generated marketplace content from platform-managed documentation:

enum ContentSource {
	USER_GENERATED = 'USER_GENERATED', // Products, Services, Blog Posts, Malets
	PLATFORM = 'PLATFORM' // Guides, help articles, manuals
}

Filtering by Content Source

query {
	search(query: "getting started", filters: { contentSource: PLATFORM }) {
		results {
			id
			name
			type
			slug
		}
		totalCount
	}
}

This is useful for building separate search experiences — e.g., a help center that only searches platform guides, or a Lobby that only shows marketplace content.

Entity Indexing Details

Products & Services (USER_GENERATED)

Commerce items are indexed with full pricing, category, Malet enrichment, and availability:

Blog Posts (USER_GENERATED)

Blog posts are indexed only when their status is PUBLISHED. Status transitions to DRAFT, ARCHIVED, or SCHEDULED trigger automatic removal from the index.

TCP Events:

// Emitted from BlogQueryService on create/update
'blog_post_upserted'  → { id: string }  // Index/re-index if PUBLISHED
'blog_post_deleted'   → { id: string }  // Remove from index

Malets (USER_GENERATED)

Malets are indexed as standalone entities so Visitors can discover Malets directly by name, description, or handle:

TCP Event:

'malet_upserted' → { id: string }  // Index/re-index Malet

Guides (PLATFORM)

Platform guides are managed entirely within the search subgraph via the GuideService. No external TCP events are needed — indexing happens automatically on CRUD operations.

Guide Module

The Guide Module is a self-contained CRUD system inside the search subgraph for managing platform documentation — user guides, help articles, and manuals.

GuideCategory Enum

enum GuideCategory {
	GETTING_STARTED = 'GETTING_STARTED',
	SELLER_GUIDE = 'SELLER_GUIDE',
	BUYER_GUIDE = 'BUYER_GUIDE',
	API_REFERENCE = 'API_REFERENCE',
	TROUBLESHOOTING = 'TROUBLESHOOTING',
	BEST_PRACTICES = 'BEST_PRACTICES',
	PLATFORM_UPDATES = 'PLATFORM_UPDATES'
}

GraphQL API

Queries

# Public: published guides sorted by category + order
query {
	guides {
		id
		title
		slug
		category
		isPublished
		sortOrder
	}
}

# Admin: all guides including unpublished
query {
	allGuides {
		id
		title
		slug
		category
		isPublished
	}
}

# Get by slug
query {
	guideBySlug(slug: "getting-started-with-malets") {
		id
		title
		body
		excerpt
		category
		tags
	}
}

Mutations

# Create
mutation {
	createGuide(
		input: {
			title: "Getting Started with Malets"
			body: "# Welcome\n\nThis guide covers..."
			category: GETTING_STARTED
			slug: "getting-started-with-malets"
			tags: ["onboarding", "setup"]
		}
	) {
		id
		slug
	}
}

# Update
mutation {
	updateGuide(
		id: "guide-id"
		input: {
			title: "Updated Title"
			isPublished: false # Removes from search index
		}
	) {
		id
		title
		isPublished
	}
}

# Delete
mutation {
	deleteGuide(id: "guide-id")
}

Cross-Subgraph Event Sync

Blogs → Search

The blogs subgraph emits TCP events to the SEARCH_SERVICE client whenever blog post lifecycle changes occur:

sequenceDiagram
    participant BO as Malet Owner
    participant BQ as BlogQueryService
    participant SC as SearchController
    participant MS as MeilisearchService

    BO->>BQ: createOne(blog, status: PUBLISHED)
    BQ->>SC: emit('blog_post_upserted', { id })
    SC->>SC: Fetch blog + Malet metadata
    SC->>MS: indexItem(enriched document)

    BO->>BQ: updateOne(blog, status: DRAFT)
    BQ->>SC: emit('blog_post_deleted', { id })
    SC->>MS: deleteItem(id)

Key behaviors:

• Only PUBLISHED blog posts are indexed
• Transitioning away from PUBLISHED (→ DRAFT, ARCHIVED, SCHEDULED) triggers deletion from the index
• The SEARCH_SERVICE TCP client is registered inside the NestjsQueryGraphQLModule.forFeature() child context, matching the DI scope of BlogQueryService

Malets → Search

Malet events flow from the malets subgraph:

• malet_upserted — Indexes/re-indexes the Malet as a standalone entity
• malet_tags_updated — Batch-updates sectionTags on all indexed items belonging to the Malet
• malet_verification_updated — Updates maletVerificationStatus across all Malet items

SearchFiltersInput

The full SearchFiltersInput type supports filtering across all entity types:

input SearchFiltersInput {
	type: ItemType # Single type filter
	types: [ItemType!] # Multi-type filter (OR logic, overrides type)
	contentSource: ContentSource # PLATFORM or USER_GENERATED
	maletId: String # Filter by owning Malet
	organizationId: String # Filter by Organization
	categories: [String!] # Filter by categories (OR logic)
	sectionTags: [String!] # Filter by section tags (OR logic)
	priceRange: PriceRangeInput # Min/max price
	dateRange: DateRangeInput # After/before dates
	isAvailable: Boolean # Availability status
	maletVerificationStatus: String # Malet verification filter
	location: GeoFilterInput # Geo-radius search
}

Bulk Import & Reconciliation

Both administrative services have been updated to cover all five entity types:

Bulk Import

# Import all entity types
mutation {
	bulkImportAll {
		jobId
		status
	}
}

# Import by specific type
mutation {
	bulkImportByType(input: { type: BLOG_POST }) {
		jobId
		status
	}
}

Supported types: PRODUCT, SERVICE, BLOG_POST, MALET, GUIDE

Reconciliation

The daily 3 AM cron job detects drift across all entity types:

• Missing items → re-indexed from source database
• Orphaned items → removed from Meilisearch
• Stale items → re-indexed with fresh data

For guides, the GuideService.reindexAll() method is available for manual re-indexing.

Module Structure

apps/search/src/
├── index/
│   ├── item-index.model.ts         # ItemIndex, ItemType, ContentSource
│   └── stubs.ts                    # ProductStub, ServiceStub, MaletStub,
│                                   # ReviewStub, BlogPostStub
├── guides/
│   ├── guide.entity.ts             # Guide entity, GuideCategory, DTOs
│   ├── guide.service.ts            # CRUD + Meilisearch indexing
│   ├── guide.service.spec.ts       # 11 unit tests
│   ├── guide.resolver.ts           # GraphQL queries + mutations
│   └── guide.module.ts             # NestJS module
├── meilisearch/
│   └── meilisearch.service.ts      # filterableAttributes, primaryKey: 'id'
├── bulk-import/
│   └── bulk-import.service.ts      # Covers all 5 entity types
├── reconciliation/
│   └── reconciliation.service.ts   # Drift detection for all 5 types
└── search.controller.ts            # TCP event handlers

apps/blogs/src/blog/
├── blog.query-service.ts           # Emits blog_post_upserted/deleted
└── blog.module.ts                  # SEARCH_SERVICE client registration

Environment Variables

The blogs subgraph requires these variables for search event emission:

# Search service TCP connection (blogs subgraph)
SEARCH_SERVICE_HOST=127.0.0.1
SEARCH_SERVICE_PORT_TCP=3021

These are already configured in the platform .env file and used by other subgraphs (e.g., malets, products).

Testing

Search Subgraph

# All search tests (133 tests across 14 suites)
npm run test -- apps/search --no-coverage

Key coverage:

• SearchController: Blog post, Malet, and guide event handlers
• GuideService: CRUD, Meilisearch indexing, reindexAll
• BulkImportService: All 5 entity types with contentSource tagging
• ReconciliationService: Drift detection across all types

Blogs Subgraph

# All blogs tests (63 tests across 9 suites)
npm run test -- apps/blogs --no-coverage

Key coverage:

• BlogQueryService: TCP event emission on create/update, status-aware indexing

Gotchas & Troubleshooting

Primary Key Auto-Detection Failure

Meilisearch auto-detects the primary key from document fields ending in id. Because ItemIndex contains both id and maletId, auto-detection fails with error index_primary_key_multiple_candidates_found, silently rejecting every batch.

Prevention: MeilisearchService.onModuleInit() explicitly calls createIndex('items', { primaryKey: 'id' }) before any indexing, and all addDocuments() calls pass { primaryKey: 'id' } as a defensive measure.

Symptoms: bulkImportAll returns success: true, indexed: 119 but GET /indexes/items/stats shows numberOfDocuments: 0. Check GET /tasks?limit=5 for status: "failed" entries with the error code above.

Recovery:

# Delete and recreate the index with the correct primaryKey
curl -X DELETE -H "Authorization: Bearer $MEILI_KEY" $MEILI_HOST/indexes/items
curl -X POST -H "Authorization: Bearer $MEILI_KEY" -H "Content-Type: application/json" \
  $MEILI_HOST/indexes -d '{"uid":"items","primaryKey":"id"}'
# Wait a few seconds, then re-run bulk import

ValidationPipe Stripping Filter Fields

The search service uses ValidationPipe({ whitelist: true, transform: true }) in main.ts. The whitelist: true option strips any properties from input DTOs that lack class-validator decorators. If you add a new field to SearchFiltersInput without @IsOptional(), it will be silently dropped and the filter will not apply.

Symptoms: Search results ignore the filter entirely — totalCount returns the same value whether the filter is set or not.

Fix: Always add @IsOptional() (from class-validator) above the @Field() decorator on every nullable field in SearchFiltersInput.

Meilisearch Tasks Are Asynchronous

All document write operations (addDocuments, deleteDocuments, updateSettings) are enqueued by Meilisearch and processed asynchronously. The Node.js SDK resolves immediately when the task is enqueued, not when it completes. This means a bulkImportAll mutation can return success: true while the actual indexing task fails minutes later.

Debugging: Use GET /tasks?limit=10 against the Meilisearch REST API to inspect task statuses, error messages, and processing duration.

Related

• Search Engine Administration — Synonyms and backend admin logic.
• Mallnline Algorithm — How trending scores and personalization influence search ranking.
• Tag Registry — Platform-managed curated tags that flow into the search index as filterable facets
• Blogs Connectivity & SEO — Canonical URLs, RSS feeds, and sitemaps for blog posts
• Universal Search Guide — Frontend integration with the search API
• Org Context & Tier Access — How searchStore auto-injects organizationId, OrgScopeIndicator, and org-scoped content suggestions
• Handle System & Sigil Taxonomy — Sigil-aware search uses parseSigil for prefix matching and entity-specific filtering in the search bar