Senior Data Engineer

<p>Equal Experts is an innovative consultancy specialising in the delivery of custom software solutions for&nbsp;blue-chip enterprise and public sector clients across a range of industry sectors. We work on important&nbsp;projects for our clients, deliver market-leading propositions across the digital, online and mobile&nbsp;channels, and are recognised for our leadership in the application of agile and lean delivery methods to&nbsp;assure delivery. We have offices in Australia, India, South Africa, the UK, the EU and the USA.</p> <p><br>Our Data Engineers are highly skilled in creating data pipelines for ingest, transformation and storage of&nbsp;data, usually at scale, and are familiar with standing-up analytics clusters.</p> <p><br>This role involves using modern agile technical practices - including Continuous Integration and&nbsp;Deployment, TDD, and XP - to deliver timely and pragmatic solutions, as well as helping others to do&nbsp;their jobs in a more effective way.</p> <p>&nbsp;</p> <p>Responsibilities</p> <ul> <li>Working with the client and the team to determine data processing and access needs and how&nbsp;best to deliver them.</li> <li>Creation and support of highly available data pipelines and storage solutions (e.g. data&nbsp;warehouses).</li> <li>Delivery of software using agile (pair programming, TDD, CI/CD, etc) practices.</li> <li>Automation of data infrastructure and deployments.&nbsp;</li> <li>Advocating agile practices to the client organisation.</li> <li>Mentoring client team members.</li> <li>Helping improve the team and clients’ data capabilities.</li> </ul> <p>&nbsp;</p> <p>Required Skills</p> <ul> <li>Significant experience working on data pipelines, platforms and projects.</li> <li>Experience in developing solutions in one or more of the main Cloud providers (AWS, GCP,&nbsp;Azure).</li> <li>Strong background in at least one programming language, Python or Scala preferable.</li> <li>Working with a variety of databases, data warehouses and distributed file systems in production&nbsp;environments.</li> <li>Strong experience in SQL and data modelling.</li> <li>Exposure with data processing at scale.</li> <li>Knowledge about GDPR and data security.</li> <li>Working with source control and infrastructure as code (Git/Github, Terraform, ansible etc.) and&nbsp;CI/CD tools.</li> <li>Understanding and being able to support data scientists or business intelligence teams - in data&nbsp;prep, mentoring and support functions.</li> <li>Strong collaboration and communication skills, including the ability to communicate effectively&nbsp;while working remotely with globally distributed</li> </ul> <p><em>*This role is only open to candidates that are based in South Africa&nbsp;</em></p>