Two systems, described honestly.

The problem

A retail analytics platform needed sales data from merchants’ e-commerce and POS systems — Shopify, Clover, Qu, Poynt — landing in one warehouse schema its reporting and ML layer could query. Each platform models orders, refunds, payments and inventory differently. Every merchant needed their own set of tables. And the whole thing had to run unattended.

The constraint

The hard requirement wasn’t throughput, it was correctness under repetition. Syncs get interrupted. Cron overlaps itself. A merchant reinstalls the app. Someone re-runs yesterday’s job by hand. Every one of those has to end with the same rows in the warehouse — not duplicates, not gaps, not silent drift that surfaces three weeks later in a report nobody trusts any more.

What I built

An embedded Shopify app handling OAuth and the install/uninstall lifecycle, feeding a multi-tenant ETL service in PHP 8 and PostgreSQL.

• Per-merchant provisioning — a registry entry plus DDL for 24 tables, including fact tables of up to 130 columns, created from nothing on first install
• A sync engine built around idempotency: per-vertical transactions, commit before marker, watermarks with a min-rule, lease locks so two runs can’t overlap, tombstones for deletes, and delete-then-insert rather than blind appends
• Data pulled in-memory over the Shopify GraphQL Admin API and mapped straight through, with no intermediate raw database to drift out of date
• A cron scheduler that returns honest exit codes, and a read-only ops endpoint exposing per-action health

The outcome

The pipeline is verified by a drill, not by hope: install, sync, uninstall, reinstall, deliberately drop the tables, re-provision, refill — and the resulting facts come out byte-identical to the snapshot taken before. A merchant onboarded from a completely clean slate has all 24 tables created and filled in about a minute. 46 automated tests cover the mapping and sync contracts.

The problem

A no-code predictive analytics platform lets business users connect their data, build a model and get a prediction — without writing code or hiring a data scientist. That means the hard parts can’t be exposed to the user: connecting messy data sources, running models that take minutes, and returning something a non-technical person can act on.

The constraint

Two languages, two runtimes, one product. The application layer was PHP; the modelling had to be Python. Neither could block the other, model runs were far too slow for a request/response cycle, and the whole thing was multi-tenant with subscriptions and per-role access on top.

What I built and owned

Both codebases, across five years and roughly 1,150 commits.

PHP side — the API behind the entire product: datasets, models, predictions and results, on a custom framework rather than Laravel or Symfony. Tenants, roles and privileges, subscription state, token authentication.

• A public REST API letting customers create datasets and run predictions programmatically, never opening the UI at all
• Data source connectors for CSV, MySQL, PostgreSQL and BigQuery
• Real-time job status pushed over WebSockets, so a model run that takes minutes reports progress instead of leaving the user staring at a spinner

Python side — the model execution service: classification, regression, forecasting, cohort and ABC analysis, RFM and CLV segmentation, built on PyCaret, scikit-learn and pandas. Containerised with Docker so runs are reproducible, with model artifacts stored on S3.

Then the part users actually noticed: an OpenAI integration that turns model output into plain-language narratives. A segmentation result stops being a table of cluster IDs and becomes a paragraph explaining what the clusters are and what to do about them.

The outcome

Around 1,000 platform users, with integrations delivered for several partners and clients on top of the product itself.