Qualifications for Data Engineer
Experience manipulating and feature extraction from large disconnected datasets containing a mixture of structured and unstructured data using R and Python.
Experience in supporting and working with cross-functional teams in particular with Data Scientists and Client practitioners.
Experience building processes supporting data transformation, metadata catalogs, landing zones and data lakes.
Advanced DevOps experience building and optimizing large scale data pipelines, data science architectures and curating data sets.
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Experience with the following platforms –
Experience with data science tools: R (tidyverse) and Python (pandas, numpy etc), Juypyter/RStudio.
Experience with Client frameworks: Keras, TensorFlow.
Experience with big data tools: Hadoop, Spark, Kafka and others.
Experience with relational SQL and NoSQL databases and working knowledge of SQL.
Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, Oozie or similar.
Experience with object-oriented/object function scripting languages: Python, Java, C#, Go, Scala, etc.
Source: Job Diva – Job Listing