Building a Lead Intelligence Engine on the Edge

At ShapeShifters, the promise we make to clients is that their website will know who is ready to buy before the sales team picks up the phone. That promise runs on a package called @shape-shifters-dev/intel.

This is the project log for that package. It covers how the tracking layer works, how we score behavioral data, the architecture decisions behind rule-based vs ML scoring, and where the system is headed next.

What the Package Does

@ss/intel is a unified intelligence layer installed on every client website we build. It handles four things:

1. Tracking — capturing visitor behavior from the first pageview, anonymously, before any form is filled
2. Scoring — turning that behavioral history into a single number that tells the sales team who to call
3. SEO — analyzing Sanity CMS documents and generating meta, alt text, schema markup, and keyword clusters
4. Gateway — a status and export API that our DigitalTwin master system reads from to monitor all client sites from one place

The package ships as a set of subpath exports: @ss/intel/tracking, @ss/intel/scoring, @ss/intel/seo, @ss/intel/handlers, @ss/intel/plugin, @ss/intel/gateway, @ss/intel/d1. Each one is independent. A client might install only tracking and scoring. Another gets the full stack.

The runtime target is Cloudflare Workers. Everything runs on the edge. No Node.js servers, no cold starts, no region-latency surprises for clients in Dubai, London, or anywhere else their buyers are coming from.

The Data Model

Three tables. That is the whole schema.

events (id, lead_id, type, page, referrer, data, created_at)
leads  (id, email, name, score, first_seen, last_seen, metadata)
sessions (id, lead_id, started_at, ended_at, pages)

Stored in Cloudflare D1, which is SQLite running on the edge. No external database to manage, no connection pooling to worry about, no latency hop to a separate region. The D1 binding lives in the Worker and queries run in the same datacenter as the request.

Every row in events belongs to a lead_id. That ID starts as an anonymous string generated in the browser. It becomes a real identity the moment the visitor identifies themselves, which i will get to in the merge flow section.

The Tracking Layer

The tracking layer is a React context provider called LeadProvider. It wraps the Next.js app and exposes a useTracking() hook that any component can call.

The events we capture:

Each event fires a POST to /api/intel/events, which writes a row to D1 and triggers a score recalculation for that lead.

Session management works off a 30-minute inactivity timeout. Mouse movement and keyboard events reset the timer. When a session times out and the visitor becomes active again, a new session ID is generated. Pages visited get tracked per session, so we can see the journey, not just the individual actions.

The Consent Gate

Before anything hits D1, we check consent. The consent state lives in localStorage under intel_consent. If analytics consent has not been granted, events are buffered in localStorage instead of being sent to the server. Once the visitor accepts, the buffer flushes.

This is also the first place the anonymous ID appears. A visitor who has not consented still gets an anon_ prefixed ID stored in localStorage. Their events queue up locally. If they accept consent, everything flushes. If they convert without ever accepting, the merge endpoint still works because the anonymous ID travels with the identification request and all queued events are sent along with it.

Anonymous to Identified: The Merge Flow

This is the part that makes lead scoring actually useful, because most of the valuable behavioral data happens before a visitor fills out a form.

The flow:

1. Visitor lands on the site. LeadProvider generates an anonymous ID: anon_abc123
2. Every event that fires gets stored: either in D1 if consent is granted, or in localStorage if not
3. Visitor submits a contact form. The form component calls identify("real_lead_id", { email, name })
4. identify() fires a POST to /api/intel/identify with the anonymous ID, the real lead ID, and any queued localStorage events
5. The server creates the identified lead in D1, inserts any queued events under the real ID, then runs UPDATE events SET lead_id = ? WHERE lead_id = ? and UPDATE sessions SET lead_id = ? to move all existing D1 records over
6. The anonymous lead record is deleted

From that point on, the lead profile is complete. The sales team sees the full behavioral history, including everything the visitor did before they ever identified themselves.

The Scoring Engine

The score is not a sum. It is a weighted sum with exponential time decay.

Where:

• $points_i$ is the base point value for event type $i$ (from the table above)
• $days_i$ is the number of days since that event occurred
• $\lambda$ is the decay constant, defaulting to 0.05

With $\lambda = 0.05$, an event from 14 days ago retains about 50% of its original point value. An event from 90 days ago retains about 1%. Events older than 90 days are excluded entirely.

The practical effect: a lead who visited the pricing page yesterday scores higher than a lead who visited it three weeks ago, even if their total event history looks similar. Recency is signal. Staleness is noise.

Scores are recalculated on every new event. The score column on the leads table is always current. The hot threshold is score > 50, which triggers the red highlight in the Sanity Studio plugin and the real-time alert to the sales team.

The point map is configurable per client. A real estate developer might weight pricing_view at 20 instead of 10 because in their market, anyone looking at pricing is already serious. A medical clinic might weight video_complete higher because patients who watch the full procedure explainer are more qualified than those who skim it. This is the rule-based layer, and it's where the domain knowledge lives.

Rule-Based vs ML: The Architecture Decision

Every system we ship starts rule-based. The point map above is the rule set. It encodes what we know about the industry: which actions predict conversion, and how much weight each one carries.

This is the right starting point for two reasons.

First, a new client has no conversion history we can learn from. An ML model trained on fifty conversions is not a model, it's noise with a label. The rule-based point map, informed by our experience across the vertical, is more accurate at the start than any model trained on thin data would be.

Second, the sales team needs to trust the system. When they see a score of 85, they should be able to understand why. With rules, you can explain it: this person viewed pricing twice, submitted a form, and came back the next day. The rule set is auditable. A model's decision boundary is not.

The graduation point comes when two conditions are met: the client has enough real conversion data to learn from, and the rule set has grown complex enough that maintaining it is becoming its own job. At that point, we start building a model that learns the conversion patterns directly from the data, rather than from our assumptions about what the data should say.

The thing that changed this calculus recently is LLMs. A young system that does not yet have enough real conversions to train a model can now generate synthetic behavioral histories that match the business context. Not a replacement for real conversion data, but a way to give a model something to start from before the pipeline fills in with real signal.

The architecture supports this graduation. The ScoringConfig interface accepts a custom pointMap that overrides the defaults. The same scoring endpoint that runs rule-based weights today can run model-derived weights tomorrow without any structural change.

The Sanity Studio Plugin

Every client site we build uses Sanity as the CMS. The @ss/intel/plugin export adds two tools directly into the Sanity Studio UI: a Lead Intelligence dashboard and an SEO Analyzer.

The Lead Intelligence tool shows:

• Total leads and hot lead count (score > 50)
• Total event count across all leads
• A full leads table with score, last seen, and event count
• A lead detail view with the complete event timeline for any selected lead

The SEO Analyzer runs document-level audits across six dimensions: title and meta, content quality, link health, structured data, readability, and mobile signals. Each dimension produces a score and a set of actionable suggestions that editors can act on without leaving the CMS.

Both tools talk to the same /api/intel/* endpoints that power the frontend tracking. One API, two consumers.

The DigitalTwin Gateway

Our internal system, DigitalTwin, is a master application that reads status and exports data from every client site we manage. The @ss/intel/gateway subpath exposes three endpoints that DigitalTwin polls:

• GET /api/intel/status — returns lead counts, hot lead count, new leads this week, SEO score averages, and content freshness
• GET /api/intel/export — paginated export of leads, events, or SEO analysis
• POST /api/intel/update — allows DigitalTwin to push configuration updates back to the client site

The gateway endpoints are authenticated with a shared secret. DigitalTwin holds the key. Client sites verify it on every request.

This architecture means we can see the health of every client system from one place, without logging into each one individually. When a client's hot lead count drops to zero for three days in a row, DigitalTwin flags it. That is a signal we need to look at their tracking setup, or their traffic, before the client notices it themselves.

What is Next

The current scoring engine is rule-based with configurable weights. The next layer is a model that learns the weights from conversion data rather than having them set by hand. The infrastructure for this is already in place: the event history is clean, the conversion events are labelled, and the scoring engine accepts external weight maps.

The other piece in progress is the scanner. @ss/intel/scanner reads // @ss tags from component files and generates a blueprint of which sections of a site are instrumented for lead tracking, SEO, and chat. This makes it possible to audit a site's instrumentation coverage from the outside, and to auto-generate Sanity schemas and GROQ queries for new sections without writing them by hand.

The goal across all of it is the same: the moment your sales team picks up the phone, they should already know who they are calling and why.