Data engineering for an ecommerce brand is the work of moving data out of Shopify, the ad platforms, the email platform and the support desk into one place, cleaning it, and modeling it so that revenue, margin and acquisition cost come out the same in every report. It is the layer under reporting, attribution and any AI tool that uses customer or order data. This guide covers what data engineering is, the data an ecommerce brand works with, the tools involved, what it costs and where to start. Brand Media Works does this work for ecommerce brands.
What is data engineering?
Data engineering is the work of building and running the systems that collect data from where it is created, move it to one place, clean and reshape it, and make it reliable enough to report on and build with. What is data engineering? explains the parts of the work, the difference between ETL and ELT, and how data engineering differs from data analytics and data science.
What data does an ecommerce brand need to bring together?
Most ecommerce brands run on four or five systems that each hold part of the picture. Each one has its own rules for how much history it keeps, how often its numbers change after the fact, and how much data it will hand over in a day. The table sets out the main ones, from the connector documentation of the data integration vendors that move this data.
| Source | What it holds | What a pipeline has to handle |
|---|---|---|
| Shopify | Orders, customers, products, discounts, shipping settings | Shopify deprecated its REST Admin API on 1 October 2024. A move of shipping settings into Markets began rolling out on 1 October 2026 and is planned to cover all shops by 1 July 2027, after which the old shipping data is a frozen snapshot. Shopify's bulk export truncates nested discount codes at about 100 per discount. |
| Meta Ads | Spend, reach, clicks and conversions by attribution window | Meta removed the 7-day and 28-day view-through attribution windows from its Ads Insights API on 12 January 2026. Insights data goes back 37 months at most. Conversions are revised after the fact, so pipelines re-read recent days: the last 9 days by default in one vendor's connector, and up to 28 days in another's. |
| Google Ads | Spend, clicks, conversions by campaign | Granular reporting data is limited to 37 months under Google's data retention policy, and click-level data to the last 90 days. Money values arrive in micros, so $180.00 is stored as 180,000,000. Google stopped issuing new developer tokens on 9 September 2026. |
| Klaviyo | Profiles, events, campaign and flow results | Klaviyo keeps revising conversion attribution after a send, for up to 5 days by default and up to 90 days if the window has been raised. Its reporting API allows 225 requests a day. |
Two things follow for any brand. First, a pipeline is not a one-time copy. It has to re-read recent days to pick up revised conversions, and keep up as each platform changes its API. Second, history has to be stored. If Meta and Google Ads only hold 37 months of detailed data, a brand that wants to compare this year with three or four years ago needs its own copy in a warehouse before the platforms drop it.
What does an ecommerce data stack look like?
A modern data stack has four layers. Each has tools that specialize in it, and each charges in its own way.
| Layer | What it does | Common tools and how they charge |
|---|---|---|
| Ingestion | Copies data from Shopify, ad platforms and email into the warehouse on a schedule | Fivetran charges by monthly active rows, with a Free plan up to 500,000 rows. Airbyte Cloud starts at $20 a month with 5 credits included. |
| Warehouse | Stores every source side by side, with full history | Google BigQuery gives the first 10 GiB of storage and 1 TiB of queries free each month. Snowflake bills compute by the second, with an extra-small warehouse using 1 credit an hour. Amazon Redshift Serverless starts at $1.50 an hour. |
| Modeling | Turns raw tables into agreed definitions, such as net revenue, new customers and blended acquisition cost | dbt has a free one-seat Developer plan, and its Starter plan is $100 per user a month. |
| Use | Dashboards, finance reports, audience segments sent back to ad and email platforms, and AI tools | Reporting tools, the ad and email platforms themselves, and private or public AI models |
Not every brand needs every layer on day one. A brand with one or two sources can start with a pipeline into a reporting tool. A warehouse earns its place once several sources need joining, history needs keeping, or the same numbers have to feed finance, marketing and AI tools.
What does a data engineer do for an ecommerce brand?
A data engineer builds the pipelines, models the data into clean tables, and keeps both working as the sources change. Much of the job is maintenance: fixing a sync when a platform changes its API, re-reading revised conversions, and checking that today's numbers still match finance. What does a data engineer do? covers the day-to-day work, how the role differs from a data analyst, and when a brand needs one in-house.
What does data engineering cost?
Published prices run from about $1,000 for a first dashboard on one source and about $4,000 for a pipeline from one or two sources, to about $10,000 for a build across around three sources, and $30,000 and more for a full data warehouse. The tools add a monthly bill on top. How much marketing data engineering costs sets out the published prices in full, with the monthly cost of each tool.
Where should an ecommerce brand start?
- Pick one question. Blended customer acquisition cost by channel, contribution margin by order, or repeat rate by first product are all good first questions, because finance and marketing both need them.
- Connect only the sources that answer it. For blended acquisition cost, that is Shopify plus the ad platforms. Klaviyo and the support desk can follow.
- Keep the history. Load as much history as the platforms still hold, and keep loading it, so the 37-month limits stop mattering.
- Agree the definitions once. Decide how revenue treats discounts, refunds, shipping and tax, and build that into the modeling layer so every report uses the same rule.
- Then build on it. Dashboards, audience syncs, workflow automation and AI tools all work better on data that already agrees.
Marketing data engineering sets out how Brand Media Works scopes this work. If the end goal is an AI tool that uses customer or order data, private AI for business covers how to keep that data inside the brand's own account.
How do you build the warehouse itself?
How to build a marketing data warehouse walks through connecting Shopify, Klaviyo, Meta Ads and Google Ads, comparing the free BigQuery transfers with connector tools such as Fivetran at published prices. It sets out the joins that make the numbers agree, the monthly running cost on each route, and what to check before trusting the reports.
Which ingestion tool should you use?
Fivetran vs Airbyte vs custom pipelines compares the two main connector tools on how they charge, their published prices and where BigQuery's free transfers fit. It also sets out when custom code is worth the upkeep, given how often Shopify, Meta and Klaviyo change their APIs, and how to test the options on a brand's own data.
Frequently asked questions
Is Shopify's own reporting enough?
For store sales, often yes. It stops being enough when the brand needs to join store data with ad spend, email results or costs from other systems, or wants one definition of revenue across every report.
Do we need a full-time data engineer?
Not to start. A first pipeline is a bounded project of a few weeks. Ongoing maintenance, such as fixing syncs when a platform changes its API, can be handled by a part-time partner until the data work grows enough to justify a hire.
How long does it take to get one set of numbers?
Published estimates put a pipeline from one or two sources at about two weeks and a build across around three sources at about four weeks. A full warehouse with every channel takes longer and is best done once the first project has shown which numbers the team uses.