Lead Data Engineer - Spark, Scala
Societe Generale
Software Engineering, Data Science
Bengaluru, Karnataka, India
Posted on Nov 1, 2025
Responsibilities
Responsibilities
- Design, development and testing of components / modules involving Spark, Scala, Hive and related big-data technologies in a datalake architecture
- Implement design, development and deployment of new Data pipelines and maintain the existing ones.
- Lead migrations such as move to public cloud ( Azure ), version upgrades.
- Oversee team of Data Engineers located in both offshore and onshore.
- Building views that are plugged by Data Visualization tools such as Power BI Release & Deployment – Deploy using CD/CI practices and tools in various environments – development, UAT and production and follow production processes.
- Ensure Craftsmanship practices are followed.
- Follow Agile@Scale process in terms of participation in PI Planning and follow-up, Sprint planning, Back-log maintenance in Jira.
- Develop/propose tactical solutions for Project specific needs