Description:
An opportunity exists within a Big 4 Banking client for a Data Streaming Engineer with skills in Pyspark, Kafka and Dynamo (AWS preferable.)
Key Skills Required:
Mandatory Requirements:
- Minimum of 10years experience as a solution designer, developer, tester or support role
- Minimum of 3years of hands-on development, testing and administration experience with Big Data/Data Lake services in cloud and on-premise
- Experience in applying industry best practices, design patterns and first-of-a-kind technical solutions on data and non-data specific platforms and applications
- Experience in implementing near real-time data flows/pipelines and distributed streaming applications using technologies such as Pyspark, Kafka & Dynamo (with AWS as preferable)
- Experience in implementing traditional ETL, data warehousing and BI solutions
- Hands-on development experience and working skill level in Python, Scalar, SQL, shell scripting and other programming/scripting languages
- Hands-on implementation of services sensitive to performance SLAs with high availability, fault-tolerance, automatic fail-over and geographical redundancy
- Working knowledge and hands-on experience with Cloud Platform such as Azure, AWS, GCP and IBM and other modern data platforms
- Solid understanding of containerisation, virtualisation, infrastructure and networking
- Diverse experience in software development methodologies and project delivery frameworks such as Agile Sprints, Kanban, Waterfall etc
- Experience in presenting to and influencing stakeholders and senior managers
- Team leadership and people manager experience
Key Responsibilities:
- Develop, test and support future-ready data solutions for customers across industry verticals
- Develop, test and support end-to-end batch and near real-time data flows/pipelines
- Demonstrate understanding in data architectures, modern data platforms, big data, ML/AI, analytics, cloud platforms, data governance and information management and associated technologies
- Develop and demonstrate Proof of Concepts and Working Demos
- Lead or collaborate with other internal/external consultants in consulting, workshops and delivery engagements
- Mentor junior consultants in the practice and delivery engagements