Data Engineer 2 - Partner Engineering Platform (PEP) Reporting
Microsoft
Partner Engineering Platform (PEP) Reporting team within WECE powers high scale telemetry insights that help Microsoft’s OEM and silicon partners improve device reliability and enhance the Windows customer experience.
As a Data Engineer 2, you will design, build, and optimize cloud scale data systems that process high-volume Windows OEM telemetry, enabling engineering teams to make fast, accurate, data driven decisions.
You will work across distributed data platforms, near real-time ingestion flows, and large analytical datasets, contributing to next generation reporting and partner experiences across the Windows ecosystem.
This role is ideal for engineers who enjoy solving complex data problems, delivering reliable and efficient pipelines, and collaborating with Program Management, Technical Program Managers, Engineering, and Telemetry teams within a supportive, learning-oriented environment.
Responsibilities
- Build and maintain scalable data pipelines using Azure Data Lake, Lakehouse patterns, Azure Datastores, and Parquet-based datasets.
- Develop Kusto (ADX) queries, tables, and ingestion processes; optimize KQL for performance and cost efficiency.
- Design and tune Cosmos DB collections, partitioning, indexing, and RU consumption.
- Implement streaming and batch workflows using COSMOS Streams and large distributed data systems.
- Improve pipeline reliability, latency, and availability through monitoring, diagnostics, and performance engineering.
- Ensure secure and compliant data access using Azure RBAC, ACLs, managed identities, and fine-grained access patterns.
- Collaborate with PM/TPM, platform, reliability, OEM engagement, and telemetry teams to deliver high value insights.
Qualifications
Required Qualifications:
- Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 1+ year(s) experience in business analytics, data science, software development, data modeling, or data engineering
- OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 2+ years experience in business analytics, data science, software development, data modeling, or data engineering
- OR equivalent experience.
Other Requirements: Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Preferred Qualifications:
• 3+ years in data engineering, software engineering, or largescale analytics.
• Strong experience with KQL and Azure Data Explorer (Kusto).
• Hands-on experience with Azure Cosmos DB, Azure Data Lake/Lakehouse, and Parquet datasets.
• Strong SQL skills and proficiency in one programming language (Python, C#, or similar).
• Demonstrated ability to optimize data pipelines for performance, scalability, and availability.
• Familiarity with Azure Synapse, Fabric Semantic Models, PySpark, Scala, and distributed compute engines.
• Experience building high-volume data pipelines (batch and streaming) using Cosmos Streams or equivalent.
• Experience with device telemetry, reliability metrics, or OEM ecosystem data flows.
• Knowledge of Azure access control models (RBAC/ABAC) and secure data governance.
• Experience building observability dashboards and pipeline health metrics.
What Success Looks Like
- Reliable, fast, cost optimized pipelines that power OEM and partner analytics.
- Clear data models and Kusto assets that improve downstream engineering and telemetry insights.
- Strong cross team collaboration and technical clarity supporting PEP’s reliability mission.
#W+DJOBS
Data Engineering IC3 - The typical base pay range for this role across the U.S. is USD $100,600 - $199,000 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $131,400 - $215,400 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.