Phone Numbe field Improvements, and API documentations

AGENTS.md

@@ -48,7 +48,7 @@
 - All multi-step financial operations use db.transaction()
 - JWT stored in httpOnly cookies (not localStorage)
 - CSRF protection on all mutation endpoints
-- Rate limiting on all endpoints (login: 10/min, API: 100/min)
+- Rate limiting on all endpoints (login: 10/min, API: 500/min, integration: 100/min, connect: 30/min)
 - All list endpoints filter by location/business context
 - All tables have indexes on locationId, businessId, userId, deletedAt, status
 - Error boundaries wrap every route in App.tsx
@@ -57,11 +57,17 @@
 - Audit logging for sensitive operations
 
 ## Integrations
-- FinaFlow exposes integration endpoints under `api/integration/*` mounted in `api/boot.ts`.
-- Incoming webhooks are handled by `api/lib/webhook-handlers.ts` and verified in `api/boot.ts` using `X-Fina-Signature`.
+- External REST API lives under `/api/v1/` (versioned). All routes use API key auth (`Authorization: Bearer fna_...`).
+- Unified response envelope: `{ data, meta: { requestId } }` for success, `{ error: { code, message }, meta: { requestId } }` for errors.
+- API key scopes are defined in `api/lib/api-scopes.ts`. Shared business logic lives in `api/lib/integration-service.ts`.
+- v1 routes are mounted in `api/routes/v1/index.ts` and wired in `api/boot.ts` via `app.route("/api/v1", v1)`.
+- Incoming webhooks are handled by `api/routes/v1/webhooks.ts` (FinaBill) and `api/lib/webhook-handlers.ts` (providers), verified using `X-Fina-Signature`.
 - Outgoing webhooks are dispatched by `api/lib/webhook-dispatcher.ts` and recorded in `webhookDeliveries`.
 - API-key auth for integration endpoints is implemented in `api/lib/api-key-auth.ts` and `api/lib/api-key-middleware.ts`.
-- To add a new incoming webhook provider: extend `handleProviderWebhook` in `api/lib/webhook-handlers.ts` and mount the route before the catch-all in `api/boot.ts`.
+- Legacy tRPC integration endpoints (`integrationFinabill.*`) are thin wrappers over `integration-service.ts` — kept for backward compat.
+- Old paths `/api/integration/daily-sales` and `/api/webhooks/finabill` redirect to v1 with deprecation headers.
+- Fina Connect pairing endpoints (`/api/connect/*`) remain in `boot.ts` — they're M2M protocol, not CRUD.
+- To add a new incoming webhook provider: extend `handleProviderWebhook` in `api/lib/webhook-handlers.ts` and mount the route in `api/routes/v1/webhooks.ts`.
 - Run integration tests: `npx vitest run api/__tests__/webhook-dispatcher.test.ts api/__tests__/integration-finabill.test.ts`
 - Dev server: `npm run dev` (Portless) or `npm run dev:app` (no Portless).
 

CHANGELOG.md

@@ -2,6 +2,52 @@
 
 ## [Unreleased]
 
+### Fixed
+- **CI integration test isolation** — Vitest 4 no longer honors `singleFork`, so API suites were running files in parallel against one Postgres DB. Test bootstrap also truncated `locations` and dropped budget plan tables on every setup pass, which raced with seeded data and produced `budget_plan_buckets does not exist`, `Location not found for business or inactive`, and missing location counters after business reset. Forced `fileParallelism: false` + `maxWorkers: 1`, made bootstrap once-only and non-destructive, and verified migration 0014 tables exist after apply (`vitest.config.ts`, `api/test/setup.ts`, `api/__tests__/budgets-router.test.ts`).
+
+### Added
+- **Unified REST API v1** (`/api/v1/`) — all external integration endpoints are now available as standard REST routes with a consistent `{ data, meta }` / `{ error, meta }` response envelope. No more tRPC wire format required for external consumers.
+  - `GET /api/v1/verify` — verify API key
+  - `GET /api/v1/accounts` — list accounts (scope: `accounts:read`)
+  - `GET /api/v1/suppliers` — list suppliers (scope: `suppliers:read`)
+  - `POST /api/v1/suppliers` — upsert supplier (scope: `suppliers:write`)
+  - `GET /api/v1/categories` — list expense categories (scope: `categories:read`)
+  - `GET /api/v1/business/profile` — get business profile (scope: `business:read`)
+  - `GET /api/v1/locations` — list locations (scope: `locations:read`)
+  - `GET /api/v1/users` — list users (scope: `users:read`)
+  - `POST /api/v1/users` — upsert user (scope: `users:write`)
+  - `GET /api/v1/roles` — list role templates (scope: `users:read`)
+  - `POST /api/v1/daily-sales` — ingest daily sales (scope: `sales:write`)
+  - `POST /api/v1/webhooks/finabill` — incoming FinaBill webhook with HMAC verification
+  - `POST /api/v1/wallet-webhooks/:provider` — incoming mobile wallet webhook (mpesa, airtel_money, sasapay)
+- **API documentation at `/docs`** — Scalar-powered interactive API reference served from the Hono app, reading from `docs/api-reference/openapi.yaml`.
+- **OpenAPI 3.1 spec** (`docs/api-reference/openapi.yaml`) — covers all 12 v1 endpoints, 6 outgoing webhook events, authentication, scopes, pagination, and error contracts.
+- **Documentation pages** — `docs/authentication.md` (API keys, scopes, rate limits, error codes) and `docs/webhooks.md` (incoming/outgoing webhook contract, signature verification, retry policy, event catalog).
+- **Request ID tracing** — every v1 response includes `X-Request-Id` header and `meta.requestId` in the JSON body.
+- **Centralized scope registry** (`api/lib/api-scopes.ts`) — granular `resource:action` scopes with legacy alias resolution so existing `read`/`write` keys keep working.
+- **Shared zod schemas** (`api/schemas/index.ts`) — single source of truth for request validation, webhook payload schemas, and pagination. Includes all 6 outgoing webhook event payload schemas.
+- **Pagination on all v1 list endpoints** — `?offset=0&limit=20` query params with `{ page, limit, total, totalPages }` in response meta.
+- **Service layer** (`api/lib/integration-service.ts`) — shared business logic extracted from the tRPC router, used by both REST and tRPC paths.
+- **Rate limiting on connect endpoints** — `/api/connect/*` endpoints now have a 30 req/min limiter to prevent brute-force pairing attacks.
+- **Rate limiting on integration endpoints** — all `/api/v1/` routes have a 100 req/min limiter.
+- **tRPC `.meta()` descriptions** on all `integrationFinabill` router procedures for future doc generation.
+
+### Changed
+- **Granular API scopes** — scopes now follow `resource:action` naming: `accounts:read`, `suppliers:read`, `suppliers:write`, `categories:read`, `business:read`, `locations:read`, `users:read`, `users:write`, `sales:write`, `journal:write`, `webhooks`. Legacy `read`/`write` scopes still work via alias resolution.
+- **`DEFAULT_CONNECT_SCOPES` tightened** — removed `admin` (overly broad), `coa:read` and `supplier:read` (redundant). Now uses the shared registry from `api-scopes.ts`.
+- **`integrationFinabillRouter` refactored** — tRPC router is now a thin wrapper over `integration-service.ts`. All business logic lives in the service layer.
+- **`boot.ts` cleaned up** — inline webhook handler, `constantTimeCompare`, and daily-sales handler removed. All moved to proper modules.
+- **`/debug-sentry` gated** — only available when `NODE_ENV !== "production"`.
+- **`constantTimeCompare` moved** to `api/lib/crypto.ts` alongside other crypto utilities.
+- **Old paths deprecated** — `/api/integration/daily-sales` and `/api/webhooks/finabill` now return `307` redirects to their v1 equivalents with `Deprecation: true` header.
+- **AGENTS.md Integrations section** updated to reflect the new v1 API surface, scope model, and architecture.
+- **Removed unused `Sentry` import** from `api/middleware.ts`.
+
+### Fixed
+- **Webhook catch-all parses body** — `/api/webhooks/:provider` now actually reads the request body instead of passing an empty `{}`.
+- **CSRF exemption for v1** — `/api/v1` routes are properly exempted from CSRF (machine-to-machine via API key).
+- **AGENTS.md rate limits corrected** — updated from stale 100/min to actual 500/min, with integration (100/min) and connect (30/min) limits documented.
+
 ## [1.1.1] - 2026-07-11
 
 ### Changed

api/__tests__/budgets-router.test.ts

@@ -39,16 +39,27 @@ describe("Budgets Router", () => {
 
   const slugPrefix = "BGT_";
 
+  async function cleanupBudgetPlansForBusiness(businessId: number) {
+    // Budget plan tables come from migration 0014. If bootstrap is incomplete,
+    // skip plan cleanup rather than failing the whole suite setup/teardown.
+    try {
+      await db.delete(bbl).where(sql`${bbl.bucketId} IN (SELECT id FROM ${bpb} WHERE ${bpb.planId} IN (SELECT id FROM ${bp} WHERE ${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${businessId})))`);
+      await db.delete(bpb).where(sql`${bpb.planId} IN (SELECT id FROM ${bp} WHERE ${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${businessId}))`);
+      await db.delete(bp).where(sql`${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${businessId})`);
+    } catch (error: unknown) {
+      const msg = String((error as { message?: string })?.message ?? error);
+      if (!/relation .* does not exist/i.test(msg)) throw error;
+    }
+  }
+
   beforeAll(async () => {
     db = getDb();
 
     // Clean up any leftover data from previous runs
     const existingBiz = await db.select({ id: businesses.id }).from(businesses)
       .where(sql`${businesses.slug} LIKE ${slugPrefix + "%"}`);
     for (const b of existingBiz) {
-      await db.delete(bbl).where(sql`${bbl.bucketId} IN (SELECT id FROM ${bpb} WHERE ${bpb.planId} IN (SELECT id FROM ${bp} WHERE ${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${b.id})))`);
-      await db.delete(bpb).where(sql`${bpb.planId} IN (SELECT id FROM ${bp} WHERE ${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${b.id}))`);
-      await db.delete(bp).where(sql`${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${b.id})`);
+      await cleanupBudgetPlansForBusiness(b.id);
       await db.delete(expenseCategories).where(eq(expenseCategories.businessId, b.id));
       const locs = await db.select({ id: locations.id }).from(locations).where(eq(locations.businessId, b.id));
       for (const l of locs) {
@@ -129,12 +140,10 @@ describe("Budgets Router", () => {
 
   afterAll(async () => {
     if (!biz) return;
-    await db.delete(bbl).where(sql`${bbl.bucketId} IN (SELECT id FROM ${bpb} WHERE ${bpb.planId} IN (SELECT id FROM ${bp} WHERE ${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${biz.id})))`);
-    await db.delete(bpb).where(sql`${bpb.planId} IN (SELECT id FROM ${bp} WHERE ${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${biz.id}))`);
-    await db.delete(bp).where(sql`${bp.locationId} IN (SELECT id FROM ${locations} WHERE ${locations.businessId} = ${biz.id})`);
+    await cleanupBudgetPlansForBusiness(biz.id);
     await db.delete(expenseCategories).where(eq(expenseCategories.businessId, biz.id));
-    await db.delete(accounts).where(eq(accounts.locationId, loc.id));
-    await db.delete(locations).where(eq(locations.id, loc.id));
+    if (loc) await db.delete(accounts).where(eq(accounts.locationId, loc.id));
+    if (loc) await db.delete(locations).where(eq(locations.id, loc.id));
     await db.delete(userBusinesses).where(eq(userBusinesses.businessId, biz.id));
     await db.delete(businesses).where(eq(businesses.id, biz.id));
     if (user) await db.delete(users).where(eq(users.id, user.id));

api/__tests__/docs-pages.test.ts

@@ -0,0 +1,179 @@
+import { describe, it, expect, beforeAll } from "vitest";
+
+let app: Awaited<ReturnType<typeof import("../boot")>["default"]>;
+
+beforeAll(async () => {
+  process.env.NODE_ENV = "development";
+  const mod = await import("../boot");
+  app = mod.default;
+}, 120_000);
+
+describe("documentation pages", () => {
+  describe("GET /docs", () => {
+    it("returns HTML with the navigation header", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs"));
+      expect(res.status).toBe(200);
+      expect(res.headers.get("content-type")).toContain("text/html");
+
+      const html = await res.text();
+      expect(html).toContain("FinaFlow Documentation");
+      expect(html).toContain('href="/"');
+      expect(html).toContain('href="/docs"');
+      expect(html).toContain('href="/docs/api"');
+      expect(html).toContain('href="/dashboard"');
+    });
+
+    it("links to the API reference, authentication, and webhooks guides", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs"));
+      const html = await res.text();
+      expect(html).toContain('href="/docs/api"');
+      expect(html).toContain('href="/docs/authentication"');
+      expect(html).toContain('href="/docs/webhooks"');
+    });
+
+    it("has a documentation card structure", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs"));
+      const html = await res.text();
+      expect(html).toContain("API Reference");
+      expect(html).toContain("Authentication");
+      expect(html).toContain("Webhooks");
+      expect(html).toContain("Interactive");
+    });
+  });
+
+  describe("GET /docs/api", () => {
+    it("returns HTML with Scalar API reference", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/api"));
+      expect(res.status).toBe(200);
+      expect(res.headers.get("content-type")).toContain("text/html");
+
+      const html = await res.text();
+      expect(html).toContain("FinaFlow API Reference");
+      expect(html).toContain("api-reference");
+      expect(html).toContain("cdn.jsdelivr.net/npm/@scalar/api-reference");
+    });
+
+    it("includes the navigation header with correct links", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/api"));
+      const html = await res.text();
+      expect(html).toContain('href="/"');
+      expect(html).toContain('href="/docs"');
+      expect(html).toContain('href="/docs/api"');
+      expect(html).toContain('href="/dashboard"');
+    });
+
+    it("references the openapi.yaml spec", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/api"));
+      const html = await res.text();
+      expect(html).toContain("/openapi.yaml");
+    });
+
+    it("has the API link highlighted as active", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/api"));
+      const html = await res.text();
+      expect(html).toContain("color:#C73E1D");
+    });
+  });
+
+  describe("GET /docs/authentication", () => {
+    it("returns HTML with authentication content", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/authentication"));
+      expect(res.status).toBe(200);
+      expect(res.headers.get("content-type")).toContain("text/html");
+
+      const html = await res.text();
+      expect(html).toContain("Authentication");
+      expect(html).toContain("API key");
+      expect(html).toContain("Scopes");
+    });
+
+    it("includes the navigation header", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/authentication"));
+      const html = await res.text();
+      expect(html).toContain('href="/"');
+      expect(html).toContain('href="/docs"');
+      expect(html).toContain('href="/docs/api"');
+      expect(html).toContain('href="/dashboard"');
+    });
+
+    it("has a breadcrumb back to documentation", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/authentication"));
+      const html = await res.text();
+      expect(html).toContain('href="/docs"');
+      expect(html).toContain("Documentation");
+    });
+  });
+
+  describe("GET /docs/webhooks", () => {
+    it("returns HTML with webhooks content", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/webhooks"));
+      expect(res.status).toBe(200);
+      expect(res.headers.get("content-type")).toContain("text/html");
+
+      const html = await res.text();
+      expect(html).toContain("Webhooks");
+      expect(html).toContain("HMAC");
+      expect(html).toContain("X-Fina-Signature");
+    });
+
+    it("includes the navigation header", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/webhooks"));
+      const html = await res.text();
+      expect(html).toContain('href="/"');
+      expect(html).toContain('href="/docs"');
+      expect(html).toContain('href="/docs/api"');
+      expect(html).toContain('href="/dashboard"');
+    });
+
+    it("documents all webhook events", async () => {
+      const res = await app.fetch(new Request("http://localhost/docs/webhooks"));
+      const html = await res.text();
+      expect(html).toContain("sale.recorded");
+      expect(html).toContain("expense.created");
+      expect(html).toContain("bill.paid");
+      expect(html).toContain("coa.updated");
+      expect(html).toContain("supplier.updated");
+      expect(html).toContain("journal.created");
+    });
+  });
+
+  describe("GET /openapi.yaml", () => {
+    it("returns the OpenAPI spec", async () => {
+      const res = await app.fetch(new Request("http://localhost/openapi.yaml"));
+      expect(res.status).toBe(200);
+
+      const text = await res.text();
+      expect(text).toContain("openapi: 3.1.0");
+      expect(text).toContain("FinaFlow Integration API");
+      expect(text).toContain("/verify");
+      expect(text).toContain("/accounts");
+      expect(text).toContain("/suppliers");
+      expect(text).toContain("/daily-sales");
+      expect(text).toContain("/webhooks/finabill");
+    });
+
+    it("documents the webhook event catalog", async () => {
+      const res = await app.fetch(new Request("http://localhost/openapi.yaml"));
+      const text = await res.text();
+      expect(text).toContain("saleRecorded");
+      expect(text).toContain("expenseCreated");
+      expect(text).toContain("billPaid");
+    });
+  });
+
+  describe("navigation consistency", () => {
+    it("all doc pages share the same nav structure", async () => {
+      const paths = ["/docs", "/docs/api", "/docs/authentication", "/docs/webhooks"];
+      const navChecks = ['href="/"', 'href="/docs"', 'href="/docs/api"', 'href="/dashboard"'];
+
+      for (const path of paths) {
+        const res = await app.fetch(new Request(`http://localhost${path}`));
+        expect(res.status).toBe(200);
+        const html = await res.text();
+        for (const check of navChecks) {
+          expect(html, `${path} missing nav link: ${check}`).toContain(check);
+        }
+      }
+    });
+  });
+});

api/__tests__/integration-finabill.test.ts

@@ -300,7 +300,7 @@ describe("integration API key authentication", () => {
       caller.integrationFinabill.upsertSupplier({
         name: "Test Supplier",
       })
-    ).rejects.toThrow(/API key missing required scope: write/);
+    ).rejects.toThrow(/API key missing required scope: suppliers:write/);
   });
 
   it("rejects expired API keys", async () => {