In the early 2010s, we had huge increases in data volumes, velocity, and variety leading to the term big data to describe the data itself, and data-driven tech companies like Facebook and Airbnb started using the phrase data engineer. Large IT firms started creating data engineering, a type of software engineering focused on data. This change in approach was particularly focused on cloud computing.

How did it used to be? Around the 1970s/1980s the term information engineering methodology (IEM) was created to describe database design and the use of software for data analysis and processing. These techniques were intended to be used by database administrators (DBAs) and by systems analysts based upon an understanding of the operational processing needs of organizations for the 1980s. DBA’s use software to store and organize data. The DBA sets up and configures the databases to be used, whereas the data engineer works with the data scientist to help shape the data in the rights ways to prepare it for analyses.

A systems analyst, also known as business technology analyst, is an information technology (IT) professional who specializes in analyzing, designing and implementing information systems. A systems analyst uses analysis and design techniques to solve business problems using information technology.

In the early 2000s, the data and data tooling was generally held by the information technology (IT) teams in most companies.[8] Other teams then used data for their work (e.g. reporting), and there was usually little overlap in data skillset between these parts of the business.