Data Engineer

*Daily rate up to £400 outside IR35
*Leeds based client, weekly office visit required
*6 month outside of IR35 contract
*First day will be onsite in Leeds to go through induction and equipment pickup

Urgent requirement to start ASAP

What you will be doing;

In this role, you will focus on building out and enhancing the end-to-end (E2E) Quality Engineering (QE) framework for both Azure Data Factory (ADF) and Databricks. This includes developing a robust E2E logging and alerting process to ensure timely monitoring and response to potential issues. You will also work on parameterizing ADF and Databricks processes, incorporating functions to streamline and automate repetitive tasks, improving efficiency and scalability. Additionally, you will implement dependency management within the job orchestration pipeline to ensure that tasks are executed in the correct sequence, minimizing errors and optimizing performance. Lastly, you will establish a permissions and security model for Unity Catalog to ensure proper data governance and access control across the system.

We need you to have;
*ADF (metadata driven pipeline development)
*DataBricks (medallion layer engineering, automated testing)
*DevOps
*SQL Server/Azure SQL (lite-DBA experience)
*IaC
*Logging, monitoring and altering (build/implementation across technologies)
Roll non-negotiables;
*Must currently live within commutable distance of Leeds
*Be a UK Citizen
*Start ASAP
To be considered…..

Please either apply by clicking online or emailing me directly to . For further information please call me on / . I can make myself available outside of normal working hours to suit from 7am until 10pm. If unavailable please leave a message and either myself or one of my colleagues will respond. By applying for this role you give express consent for us to process & submit (subject to required skills) your application to our client in conjunction with this vacancy only.