Phase 4 — Search pipeline, ranking, dedupe, validation

Status: Implemented
Date: 2026-10-12
OpenAI model: gpt-6-luna (cheap, accurate; $0.10 / $0.50 per 1M tokens)

Pipeline

NL query
  → OpenAI interpretation (gpt-6-luna, reasoning_effort=none)
  → merge with user filter edits
  → Indeed fetch (paginated, max 4 pages)
  → local JobListingFilter
  → dedupe by indeedJobKey
  → availability validation (source signals)
  → optional OpenAI / heuristic ranking
  → in-memory cache (15m TTL)
  → UI results

What shipped

Component	Path
Orchestrator	`Services/Search/JobSearchPipeline.swift`
Local filter	`Services/Search/JobListingFilter.swift`
Dedupe	`Services/Search/JobListingDeduper.swift`
Criteria merge	`Services/Search/SearchCriteriaMerge.swift`
Ranking	`Services/OpenAI/OpenAIRankingService.swift`
Validation	`Services/Validation/ListingValidationServices.swift`
Cache	`Services/Indeed/InMemoryIndeedListingCache.swift`
Model default	`Services/OpenAI/OpenAIModels.swift` → `gpt-6-luna`

Indeed source

Live DI uses MockJobDataSource (via JobDataSourceIndeedAdapter) with openable Indeed browse/search URLs until the licensed Option C vendor is ready. Fixtures remain for mapper/contract tests.

Why links used to look “expired”: interim keys like fixture-ios-… / ios001 are not real Indeed jk values. Opening /viewjob?jk=… with those keys shows Indeed’s expired/not-found page. The app now opens /jobs?q=…&l=… (live Indeed search for that role) when the key is not a real 16-char hex jk.

Open on Indeed

JobListingOpener + OpenIndeedLinkButton are available on AI Job Search rows (and Saved Jobs). Xcode console logs (JobOpen / SearchPipeline) explain fallbacks and unavailable listings.

Cache / TTL

Policy	Value	Use
Listing metadata cache	15 minutes	Search result pages in `InMemoryIndeedListingCache`
Saved job revalidation	24 hours	Reserved for Phase 5 persistence

Feature flags

FeatureFlags.isAIInterpretationEnabled  // NL → criteria
FeatureFlags.isAIRankingEnabled         // optional batch rank (degrades to heuristic)
FeatureFlags.isListingValidationEnabled // source availability pass

HTTP HEAD validation exists (HTTPJobValidationService) but is not wired in live DI yet — fixtures/source signals only, to avoid unsolicited Indeed probing before vendor expired-signal policy.

Acceptance checklist

Duplicate listings collapsed by indeedJobKey
Shared local filter for Indeed-oriented fields (incl. freshness / exclude / exact phrase best-effort)
Open Indeed URLs remain regional Indeed hosts from fixtures/mapper
OpenAI uses gpt-6-luna for interpretation + ranking
Pipeline wired into JobSearchViewModel
Unit tests for filter, dedupe, merge, ranking order, pipeline, cache

Search_Pipeline.md 3.0 KB History Raw