In datateam, we have so many roles: Data scientis, Data Analyst, Machine Learning Engineer, Analytics Engineer and of course, Data Engineer (Some company have a role name Data Architect, but less common with others role). I think so much articles talk about the different between these role, so I don’t going to explain again. In this post, I just talk about Data Engineer, and how I think you could do to make good effect with your team, your company and your customers.
To be clear, just define: What is Data Engineering? Who is Data Engineer? What do these role do in daily basic? … Sad but true: It’s depend. I don’t have specific definition because these role still elvoving. By referrence in “Fundamemtals of Data Engineering - Joe Reis, Matt Housley”, we got several definitions, but in short, I just got last definition of the authors:
Data engineering is the development, implementation, and maintenance of systems and processes that take in raw data and produce high-quality, consistent information that supports downstream use cases, such as analysis and machine learning. Data engineering is the intersection of security, data management, DataOps, data architecture, orchestration, and software engineering. A data engineer manages the data engineering lifecycle, beginning with getting data from source systems and ending with serving data for use cases, such as analysis or machine learning.