# Indeed access spike notes

**Phase:** 2  
**Scope:** Technical + policy constraints only — **not** authorization to ship scraping  
**Date:** 2026-10-12

---

## 1. ToS / automation constraints (planning)

- Indeed’s terms generally prohibit unauthorized automated access / scraping of site content.
- App Store review and user trust require a **permitted** data story.
- A controlled spike that measures HTML fragility (CAPTCHA, rate limits, layout churn) may inform a **don’t-build** decision; it must **not** become a production path without permission.

**Spike posture for this repo:** no live scrape client. Fixtures simulate permitted JSON payloads.

---

## 2. Field coverage needed by the app

Mapped to `JobListing`:

| Domain field | Required | Notes |
|--------------|----------|-------|
| `indeedJobKey` | Yes | Dedupe + save identity |
| `title` | Yes | |
| `company` | Yes | |
| `location` | Yes | Free-text OK |
| `remoteType` | Preferred | Map remote/hybrid/onsite/unknown |
| `salary` | Optional | min/max/currency/period |
| `postedAt` | Preferred | Relative date UI |
| `snippet` | Preferred | Detail / row preview |
| `indeedURL` | Yes | Must be Indeed TLD |
| `availability` | Preferred | open / unknown / unavailable |
| employment / experience | Optional | Filter support |

---

## 3. Regional TLDs

Indeed hosts vary by country (examples): `indeed.com`, `indeed.co.uk`, `pk.indeed.com`, `ca.indeed.com`, etc.

**Canonicalization rules** (implemented in `IndeedURLCanonicalizer`):

- Accept only hosts matching `indeed.` / `*.indeed.` patterns
- Prefer HTTPS
- Extract job key from query `jk=` or known path segments when present
- Preserve regional host when opening in browser so the user lands on the correct locale

Country setting (Settings → Search defaults / criteria.country) should map to the vendor’s locale parameter **and** preferred TLD when the provider supports it.

---

## 4. Pagination, caching, rate limits (contract defaults)

| Concern | Default (until vendor docs override) |
|---------|--------------------------------------|
| Page size | 25 |
| Max pages per user search (UI) | 4 (100 results) before “load more” |
| Cache TTL (listing metadata) | 15 minutes in-memory (Phase 4+) |
| Saved-job revalidation TTL | 24 hours (Phase 5) |
| Rate limit handling | Surface `IndeedDataError.rateLimited(retryAfter:)` — backoff UI |
| Empty page | Treat as end of results |

---

## 5. Error taxonomy

| Error | User-facing intent |
|-------|--------------------|
| `unauthorized` | Source credentials missing/invalid |
| `forbidden` | License / geo not allowed |
| `rateLimited` | Retry later |
| `invalidResponse` | Decode / schema failure |
| `unsupportedFilter` | Best-effort note (hide or label in UI later) |
| `network` | Offline / transport |
| `cancelled` | Task cancelled |
| `nonIndeedURL` | Mapper rejected non-Indeed link |

---

## 6. `indeedJobKey` extraction

Priority:

1. Explicit `jobKey` / `jk` field from provider DTO  
2. Query item `jk` on Indeed URL  
3. Stable path token when vendor uses `/viewjob?jk=`  
4. Fallback: hash of canonical Indeed URL (last resort; prefer real key)

---

## 7. Technical viability checklist (when a vendor is chosen)

Measure against sandbox credentials (still no scrape):

- [ ] Latency p50 / p95 for page of 25
- [ ] Geo coverage for target countries (e.g. Pakistan, US, UK)
- [ ] Remote/hybrid signals accuracy
- [ ] Salary presence rate
- [ ] Pagination cursor vs offset
- [ ] Expired listing signals

Record results in this doc when a vendor sandbox is available.

---

## 8. Outcome for Phase 2

- DADR recommends **licensed / partner Indeed-only** channel  
- In-repo: protocol + mapper + fixtures + unit tests  
- **No production network adapter** until DADR approval + contract