The Holy Digital Trinity

Cloud Architecture

Data Engineering

Data Science

These three represent the "man behind the curtain" hidden from the view of even technologically advanced people in the modern era.  Between these three scopes of work, this is how mobile applications and SaaS vendors are able to execute and scale all the variety of things we do in our digital lives. 

Cloud architecture is, predictably, the knowledge of 'services' provided by the cloud vendors and how to configure them.  This covers data center hardware, virtualization, applications, as well as the services themselves.  This typically serves as the metaphorical foundation slab upon which the rest of the data house is built.

Data engineering is the movement and transformation of data.  Now when I say transformation, most of the time we're talking about switching data types (string to date, integer to float, nvarchar(max) to anything that is not nvarchar max) and not business specific logic or aggregation acting upon that data.  This runs the gamut of structured (SQL and Oracle) and unstructured (basically everything else, but now mostly either json files from API calls and flat files stored in a data lake/object storage).  Data engineering is building the walls, doors, windows, and roof and running the utility lines in the metaphor of the data house.  Note that you generally can't build walls where there is no foundation.

I use this metaphor repeatedly because it serves as a pretty broadly understood analogy as well as appropriately divides the functions.  A house, at its most fundamental level, is to provide shelter and distribute utilities (plumbing, electrical, heat) necessary for survival.  Between cloud architecture (in an on prem environment, this is replaced by traditional infrastructure server and networking teams) and data engineering, this provides everything you need from a data perspective.

Data science now carries the immediate mental connotation with machine learning and artificial intelligence; but it also covers data cleansing and domain-specific understanding and logical transformation of the data to make the data applicable to the business.  So in our metaphor, data science is responsible for the selection of appliances, paint colors of the walls, flooring choice, etc.  All the things that we don't need for technical survival but are a fundamental part of our daily lives.  Note that we often in industry split things like data cleansing and transformation out into analyst or tech roles.  Generally this approach works with simple reporting but begins to fall apart when we get into the world of…."predictive" or "prescriptive" analytics.

Leave a Reply

Your email address will not be published. Required fields are marked *