Senior Lead Software Engineer - Big Data

JPMorganChase
JPMorganChase

Software Engineering · Full-time

Bengaluru, Karnataka, India

Posted on Nov 20, 2026

Be an integral part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products.

As a Senior Lead Software Engineer at JPMorganChase within the Commercial and Investment Bank and C360 Technology team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. You will set the technical direction, build reusable frameworks and guide a team of engineers. You also have a solid understanding of the broader software and application landscape that our data solutions integrate with.

Job responsibilities

  • Take full responsibility for the data platforms, including architecture, roadmap, reliability, cost, operational health and production support.
  • Mentor and guide engineers. Set technical direction, clear blockers quickly, delegate effectively and answer for the team's deliverables and code quality.
  • Architect and build data warehouse and data lake solutions on Databricks. Apply sound data modeling, history tracking and data organization practices so the data is well structured and ready for analytics.
  • Create reusable, configurable ETL and processing frameworks covering ingestion, transformation, data quality, reconciliation and auditability, and drive their adoption across teams.
  • Define and govern standard approaches for bringing data in from databases, files, APIs and event sources, covering both full and incremental loads and ship high-volume batch and streaming pipelines with Spark/PySpark and Kafka/Kinesis. Make sure they're reliable, resilient to failure and meet agreed service levels.
  • Tune performance and cost across Spark, Databricks, SQL engines and NoSQL stores. Design secure, resilient solutions on AWS using services such as S3, Glue, EMR, Lambda, Redshift and Athena.
  • Govern the models, aggregates and data marts behind reporting so that business and regulatory reports are accurate, reconciled, timely and fast.
  • Drives adoption and governance of approved AI-assisted engineering practices across teams to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test acceleration, release readiness, incident/root-cause analysis), while establishing measurable validation standards (secure coding, peer review, automated testing) and promoting reuse of proven patterns and automation within the SDLC/TLM toolchain.
  • Applies knowledge of tools within the Software Development Life Cycle toolchain, including approved AI-assisted development and automation capabilities, to improve the value realized by automation at scale.
  • Enforce modular design, code reviews, CI/CD, automated testing, monitoring and Git-based workflows. Work with application, API and UI teams, using your knowledge of the full software stack to shape integration decisions.
  • Promote reuse-first practices that use enterprise-authorized AI capabilities for design analysis, test generation, control validation and documentation. Validate all outputs, respect data sensitivity, and keep work traceable and auditable.

    Required qualifications, capabilities, and skills

  • Formal training or certification on software engineering concepts and 5+ years applied experience
  • Experience in data engineering with a track record of leading teams, mentoring engineers, removing obstacles and being accountable for delivery, platform stability and incident resolution.
  • Expertise in designing and developing data warehouses and data lakes on Databricks, with strong data modeling skills. Familiarity with Snowflake and comparable platforms such as Redshift, BigQuery or Azure Synapse.
  • Exceptional SQL skills, including complex queries, optimization and analytical problem solving on large datasets with solid grounding in reporting: business-friendly data layers, BI tools (e.g., Tableau, Power BI, QuickSight), dashboard performance, and accuracy controls.
  • Outstanding production-grade Python and PySpark, with clean, efficient, testable and maintainable code.
  • Mastery of data ingestion and processing designs, with a history of building enterprise ETL frameworks and running batch and streaming workloads.
  • Broad knowledge of relational databases (e.g., PostgreSQL, Oracle), NoSQL stores (e.g., DynamoDB, MongoDB, Cassandra) and AWS data services. You can tune performance across these technologies and choose the right one for each use case.
  • Demonstrated experience leading effective use of enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security
  • Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching senior engineers/leads on compliant usage patterns and controls.
  • A well-rounded understanding of software engineering and application development, including the SDLC, design principles, microservices, APIs (e.g., Spring Boot) and front-end concepts. You also troubleshoot well and know CI/CD and GitHub/Bitbucket workflows.
  • Experience using enterprise-authorized AI capabilities in engineering work. You review and validate AI-assisted outputs before using them, escalate when unsure, and follow data handling requirements.

    Preferred qualifications, capabilities, and skills

  • Infrastructure provisioning in AWS using Infrastructure as Code (e.g., Terraform, AWS CloudFormation).
  • Databricks and/or AWS certifications.
  • Hands-on experience with Snowflake, or with migrating data between platforms.
  • Experience with orchestration tools (e.g., Airflow) and with data governance and lineage tools.
  • Experience with containerization (Docker/Kubernetes) and in financial services or other regulated environments.

Lead the engineering of our enterprise data warehouse and data lake on Databricks and be accountable for it end to end