About the job
As part of the Bapco Digital Factory, you will apply multiple programming languages and advanced knowledge of database design, analyze raw data, develop, and maintain datasets, improve data quality and efficiency, find trends in data sets, and develop algorithms to make data more useful to the enterprise.
You have strong numerical and analytical skills, advanced technical expertise with data models, data mining, and segmentation techniques, programming languages, and hands-on experience with the database design. You understand different approaches to data architecture and applications used to handle both structured and unstructured data types, and use a variety of big data technologies, such as open source data ingestion and processing frameworks.
The Data Engineering Specialist reports to the Data & Analytics Team Lead.
- Design and evaluate open source and vendor tools for data lineage and work closely with all business units and engineering teams to develop strategies for long term data platform architecture and Big Data solutions.
- Access various cloud and on-premise sources to organize and analyze raw data, create data integrations in databases, data warehouses and data lakes, to provide data analysts, data scientists and business users with simplified access to prepared data sets.
- Identify opportunities for data acquisition, from internal and external sources, develop and maintain scalable data pipelines from different sources and build out new Application Programming Interface (API) integrations to support continuing increases in data volume and complexity.
- Develop, maintain, and test big data solutions, design and implement Big Data tools and frameworks, implement ETL processes, build cloud data platforms and maintain the production system.
- Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility, and fostering data-driven decision making across the organization.
- Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes as and when needed
- Perform data analysis required to troubleshoot data related issues and assist in the resolution of data issues.
- Accurately document work, write unit and integration tests in line with standard procedures and contribute substantially to create a data engineering knowledgebase; update timely and accurately relevant documentation and diagrams as and when needed, to ensure traceability and auditability of changes.
- Design data integrations and data quality framework and define company data assets (data models) and automated jobs to populate data models.
- Follow all approved IT standard operating procedures, information security zero-trust security principles and policies, apply strict information security hygiene in the design and operation of data platforms, data pipelines and big data solutions, on-prem and on Cloud.
- BSc/MSc Degree in Computer Science or equivalent
- Minimum 10 years of experience, with at least 5 years of data engineering experience in a complex enterprise environment (schema design and dimensional data modeling; C#, Python or Java; SQL)
- Experience designing, building, and maintaining data processing systems
- Knowledge of best practices and IT operations in a high availability environment
- Work experience with relational database management systems, ETL tools and REST-oriented APIs for creating and managing data integration jobs, Hadoop, NoSQL databases, Spark
- Experience with or knowledge of Agile Software Development methodologies
- Excellent analytical, problem solving and troubleshooting skills. Process oriented with great documentation skills
- Team-oriented, comfortable switching between technologies, with ability to effectively prioritize and execute tasks under pressure
- Excellent oral and written communication skills with a keen sense of customer service
- Fluent English