Combined Product Search + Enrich Endpoint (consumer-context-service)

1. Problem Statement

consumer-agent needs a single endpoint that searches for products by natural language query AND returns enriched results (prices, images, availability, offers, points) in one call. Currently, product search and enrichment are separate: POST /v1/products/search returns bare FIDOs, then POST /v1/products/enrich adds retailer + offer data. This requires two round-trips and increases latency.

2. Background & Context

Existing Endpoints

Endpoint	Returns	Missing
POST /v1/products/search	FIDO IDs + basic metadata (name, brand, category)	No prices, images, offers
GET /v1/products/{fido_id}/enrich	Full enrichment (price, image, offers, PPD)	Requires known FIDO ID
POST /v1/products/enrich	Batch enrichment	Requires known FIDO IDs

The new endpoint combines search + enrich in a single call.

Existing Pipeline Components (All Reusable)

• FidoSearchClient (internal/clients/fido_search.go): Searches pam-ml-search by description
• FPS (internal/clients/fps.go): Product metadata (name, brand, category)
• FIDORA (internal/clients/fidora.go): Retailer-specific data (price, image, URL, availability)
• Button (internal/clients/button.go): PPD, merchant logo, stickers
• FidoIndex + Offer Guardian: FIDO → offer mapping + offer details
• NELI: User offer eligibility filtering
• EnrichWithRetailerContext(): Existing orchestration method

3. Requirements

Functional

• Accept natural language product query + optional limit + optional user_id
• Search via FIDO Search (pam-ml-search) to find matching products
• Enrich each match with: FPS metadata, FIDORA retailer context, Button PPD, offers
• When user_id provided: filter offers by NELI eligibility
• Return enriched results sorted by search relevance

Non-Functional

• Latency: <2s p95 for typical queries (5-10 results)
• Concurrent enrichment: use goroutine pool for parallel per-product enrichment

Acceptance Criteria

• POST /v1/products/search/enriched returns enriched products
• MCP tool search_products_enriched wraps the endpoint
• Results include: name, brand, price, image, URL, retailer, availability, offers, PPD
• Latency <2s p95 in staging
• Existing endpoints unaffected

4. Solution Design

4a. REST Endpoint

POST /v1/products/search/enriched

Request:
{
  "query": "Tide vs Gain laundry detergent",
  "limit": 10,
  "user_id": "abc123"
}

Response:
{
  "query": "Tide vs Gain laundry detergent",
  "results": [
    {
      "fido_id": "fido_xxx",
      "name": "Tide Original Laundry Detergent 64oz",
      "brand": "Tide",
      "category": "Laundry",
      "price_cents": 1299,
      "image_url": "https://cdn.fetchrewards.com/...",
      "product_url": "https://www.target.com/...",
      "retailer_name": "Target",
      "retailer_id": "ret_xxx",
      "venue_type": "GROCERY",
      "available": true,
      "ereceipt": true,
      "ppd_points": 15,
      "ppd_type": "PPD",
      "merchant_logo": "https://...",
      "offers": [
        {
          "offer_id": "off_xxx",
          "points": 200,
          "description": "Buy Tide, earn 200 points",
          "expires": "2026-05-15"
        }
      ],
      "search_score": 0.92
    }
  ],
  "result_count": 5
}

4b. Domain Service Method

internal/products/service.go

func (s *Service) SearchAndEnrich(
    ctx context.Context,
    query string,
    limit int,
    userID string,
) ([]EnrichedProductResult, error) {
    // 1. Search for products
    searchResults, err := s.fidoSearch.SearchProducts(ctx, []string{query})
    if err != nil {
        return nil, fmt.Errorf("search failed: %w", err)
    }

    // 2. Collect unique FIDO IDs (up to limit)
    fidoIDs := extractFidoIDs(searchResults, limit)
    if len(fidoIDs) == 0 {
        return []EnrichedProductResult{}, nil
    }

    // 3. Batch get product context from FPS
    productContexts, err := s.BatchGetProductContext(ctx, fidoIDs)
    if err != nil {
        return nil, fmt.Errorf("fps batch failed: %w", err)
    }

    // 4. Concurrent per-product enrichment
    results := make([]EnrichedProductResult, 0, len(fidoIDs))
    var wg sync.WaitGroup
    var mu sync.Mutex
    sem := make(chan struct{}, 10) // limit concurrency

    for _, fidoID := range fidoIDs {
        pc, ok := productContexts[fidoID]
        if !ok {
            continue
        }

        wg.Add(1)
        go func(fid string, pc ProductContext) {
            defer wg.Done()
            sem <- struct{}{}
            defer func() { <-sem }()

            // Retailer enrichment (FIDORA + Button + Retailer + eReceipt)
            s.EnrichWithRetailerContext(ctx, &pc)

            // Offer enrichment
            var offers []OfferContext
            if s.offerService != nil {
                offers, _ = s.offerService.GetOffersForProduct(ctx, fid, userID)
            }

            mu.Lock()
            results = append(results, EnrichedProductResult{
                ProductContext: pc,
                Offers:         offers,
                SearchScore:    getSearchScore(searchResults, fid),
            })
            mu.Unlock()
        }(fidoID, pc)
    }
    wg.Wait()

    // Sort by search score
    sort.Slice(results, func(i, j int) bool {
        return results[i].SearchScore > results[j].SearchScore
    })

    return results, nil
}

4c. MCP Tool

internal/mcp/tools.go

// Register: "search_products_enriched"
// Parameters: query (string, required), limit (int, optional), user_id (string, optional)
// Calls: products.SearchAndEnrich(ctx, query, limit, userID)

4d. Types

internal/products/types.go

type EnrichedProductResult struct {
    ProductContext
    Offers      []offers.OfferContext `json:"offers,omitempty"`
    SearchScore float64              `json:"search_score"`
}

5. Dependencies

• All upstream services already integrated (FPS, FIDORA, Button, OG, NELI, FidoSearch)
• No new external dependencies

6. Risks & Open Questions

Risk	Impact	Mitigation
pam-ml-search quality for NL queries	Poor results	Test with real product queries; consider FIDORA BM25 as alternative
Enrichment adds latency per product	>2s for 10 results	Cap concurrency, cache FPS/FIDORA results
Offer matching per product is N lookups	Adds latency	FidoIndex is in-memory, OG is cached — should be fast

Open questions:

1. Use pam-ml-search, FIDORA BM25, or both for the search step?
2. Should results include the “best” offer only or all offers per product?
3. Maximum result limit?

7. Testing Strategy

• Unit tests: SearchAndEnrich with mocked clients
• Integration tests: Against staging backend services
• Latency benchmark: 50 representative queries, measure p50/p95/p99

8. Rollout

New endpoint — no changes to existing endpoints. Deploy and consumer-agent starts calling it.