After years of building and rescuing data platforms, one pattern keeps repeating: organisations spend months choosing tools, then wonder why their brand-new platform becomes an expensive white elephant.
The scenario is familiar: executives conclude that the organisation needs a new data platform, or that an existing one must be migrated to modern technologies. Sometimes the goal is better insight into existing data; increasingly, it is to provide a foundation for AI initiatives. So far, so good. The problems tend to begin with what happens next.
Attention turns quickly to technology choices. Snowflake, Databricks, Microsoft Fabric, BigQuery, usually a shortlist of familiar names, each with its strengths and weaknesses. Discussions expand to ingestion tools, transformation frameworks, BI platforms, and discovery layers. Critical foundations are skipped or postponed. The result is depressingly common: a shiny new platform that reproduces all the problems of the old one, often at greater cost. Inaccurate metrics, slow processing, spiralling spend, data silos, and low trust across the business.
This is preventable, but not through better tools.
Success hinges on starting with a clear, business-led data strategy, not a tool shopping list. That strategy should be accessible to everyone, from the CFO to data engineers, and clearly explain why the platform exists and how it creates value. It must be anchored in business outcomes, better decisions, greater efficiency, reduced risk, or new revenue by explicitly tying technical work to measurable impact and delivering a balanced pipeline of use cases that combines quick wins with longer-term goals.
Addressing governance, security, and compliance from day one. Define ownership, data quality expectations, access controls, and release processes early. Do not build first and figure this out later. Early decisions here prevent costly rework and can often be handled with built-in platform capabilities.
Investing in data modelling. Well-designed models aligned to business concepts are essential for scalability and reuse. Avoid building reports directly from source data for short-term gains; these shortcuts almost always create long-term problems.
Designing for change, not perfection. Assume technologies, requirements, and data volumes will evolve, and favour modular, adaptable architectures over over-engineering.
Planning for operations and maintenance. Monitoring, incident response, documentation, and ongoing improvement are critical to long-term reliability and trust.
Using tools for what they are good at. Select technologies based on strengths and intended use, integrate where appropriate, and avoid forcing tools to behave in ways they were not designed for.
Finally, getting the team right. Successful platforms are built by teams that combine technical excellence with deep business understanding. Hiring for tool-specific experience alone is a mistake, but so is assuming domain expertise replaces engineering skill. Balance here underpins everything else.
In the end, tools are just components. The success of a data platform depends on having the right strategy, structure, and discipline to use them effectively.
