Data Engineer, Global Talent Intelligence
Microsoft
We are hiring a Data Engineer to support the Global Talent Intelligence team. The Global Talent Intelligence (GTI) team sits at the intersection of talent, technology, and strategy—partnering with senior leaders across Microsoft to anticipate market shifts and inform high‑impact business decisions. At the core of this work is a purpose‑built talent data engine that transforms vast, complex external signals into clear, actionable intelligence.
As a Data Engineer on GTI, you will drive the next iteration of design and scale for the infrastructure that powers how Microsoft relies on talent movement as a leading indicator of market movement. This role goes beyond traditional data engineering: you will build curated pipelines, models, and power interfaces that enable GTI to move from manual analysis to proactive and predictive insights—unlocking earlier visibility into where skills are emerging, where competition is intensifying, and where strategic intervention matters most.
Our GTI data engineer works across an unusually rich and diverse signal ecosystem, synthesizing supply and demand dynamics, competitor and investment signals, innovation indicators, emerging talent and skills, and geo‑specific workforce risk factors. These signals power tools and intelligence products that directly influence executive decision‑making—from workforce planning and site strategy to critical skill investments and long‑term growth bets.This is a highly collaborative role, partnering closely with talent intelligence consultants, analysts, and business stakeholders to ensure data is not only technically sound, but strategically meaningful—moving insights into action.
Success in this role requires:
- Strong data engineering fundamentals, paired with modern AI adoption and automation practices
- A product mindset for designing systems and interfaces that scale, evolve, and drive action
- Curiosity about how talent dynamics shape markets, strategy, and competitive advantage
- The ability to create clarity from complexity and translate insight into direction
- Dedication to building secure, compliant, and validated data systems
- Creativity and discipline in designing for optimization, reliability, and scale
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
- Anticipates the need for data governance and designs data modeling and data handling procedures, with direct support and partnership with Corporate, External, and Legal Affairs (CELA), to ensure compliance with applicable laws and policies across all aspects of the development process.
- Tags data based on categorization (e.g., personally identifiable information [PII], pseudo-anonymized, financial).
- Documents data type, build data dictionary, classifications, and lineage to ensure traceability.
- Governs accessibility of data within assigned data pipelines.
- Provides guidance on contributions to the data glossary to document the origin, usage, and format of data for each program.
- Independently implements data governance and privilege of least access practices leveraging security tools.
- Builds responsible AI-compliant data products and/or applications.
- Plans and creates efficient techniques and operations (e.g., inserting, aggregating, joining) to transform raw data into a form (e.g., dimensional data model) that is compatible with downstream data consumers, databases, and formats that support applications, analytics and reporting.
- Independently uses software, query languages, and computing tools (e.g., cloud-based) to transform raw data across end-to-end pipelines.
- Evaluates data to ensure data quality and completeness using queries, data wrangling, and statistical techniques.
- Merges data into distributed systems, products, or tools for further processing.
- Identifies opportunities to leverage and contribute to the development of data tools that are used to transform, manage, and access data, scaling with efficiency and reduced time to new data insights.
- Writes, implements, and validates code to test storage and availability of data platforms and drives the implementation of sustainable design patterns to make data platforms more usable and robust to failure and change.
- Analyzes relevant data sources that allow others to develop insights into data architecture designs or solution fixes.
- Identifies data sources and builds code to extract raw data from identified upstream sources using query languages, tools, or application programming interfaces (APIs) while ensuring quality, scale, and reliability of the data across several pipeline components.
- Contributes to the code review process by providing feedback and suggestions for implementation. Uses knowledge of the end-to-end business case to implement orchestration techniques that automate data extraction logic across multiple sources of data.
- Leverages data protocols, reduction techniques, and aggregation approaches to validate the quality of extracted data across a data pipeline, consistent with the service level agreement (SLA).
- Offers feedback on methods and tools used to track and maintain data source control and versioning.
- Applies deep knowledge of data to validate that the correct data is ingested and that the data is applied accurately across the pipeline.
- Creates a data design document.
- Leads the design of a data model that is appropriate for the project and prepares design specification documents to model the flow and storage of data for a data pipeline.
- Designs assigned components of the data model for a functional area of a project, while accounting for scale, and industry standards and best practices (e.g., data mesh).
- Partners with stakeholders (e.g., business partners, Data Science Specialists) to understand the business domain so as to make iterative improvements to design specifications, data models, or data schemas, so that data is easy to connect, ingest, has a clear lineage, and the data model is responsive to work with.
- Considers tradeoffs between analytical requirements with compute/storage consumption for data and anticipates cost that could be influenced by the cadence of data extraction, transformation, and loading into moderately complex data products or datasets in cloud and local environments.
- Demonstrates an advanced understanding of security, performance, and costs associated with data that are used to assess the total cost of ownership (TOC).
- Collaborates with appropriate stakeholders across teams and escalates concerns around data requirements by assessing and conducting feature estimation.
- Assesses data costs, access, usage, use cases, dependencies across products, and availability for business or customer scenarios related to one or more product features.
- Informs clients on feasibility of data needs and suggests transformations or strategies to acquire data if requirements cannot be met.
- Negotiates agreements with partners and system owners to align on project delivery, data ownership between both parties, and the shape and cadence of data extraction for one or more features.
- Proposes new data metrics or measures to assess data across varied service lines.
- Defines data source contracts (e.g., nature of data, data schemas, data latency, data availability, data privacy, ethical use of data).
- Performs root cause analysis in response to detected problems/anomalies to identify the reason for alerts and implement solutions that minimize points of failure.
- Implements and monitors self-healing processes across multiple product features to prevent issues from recurring in the future and retain data quality and optimal performance (e.g., latency, cost) throughout the data lifecycle.
- Uses cost analysis to drive product/program level solutions that reduce budgetary risks.
- Documents the problem and associated solutions through postmortem reports and shares insights with team and the customer.
- Provides data-based insights into the health of data products owned by the team according to SLAs across multiple features.
- Implements and practices both agile and data operations (DataOps) practices.
- Maintains involvement with, and awareness of current and upcoming data engineering practices (e.g., tools, technology) through Microsoft's internal data community with the purpose of connecting into a data mesh.
- Writes code to implement performance monitoring protocols across data pipelines.
- Builds visualizations and smart aggregations (e.g., advanced statistics) to monitor issues with patterns in data quality and pipeline health that could threaten pipeline performance.
- Develops and updates troubleshooting guides (TSGs) and operating procedures for reviewing, addressing, and/or fixing advanced problems/anomalies flagged by automated testing.
- Supports and monitors platforms, analyzing telemetry data to understand the health of the systems and takes proactive steps for live site improvement.
Qualifications
Required/minimum qualifications
- Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering
- OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering
- OR equivalent experience.
- Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years experience in business analytics, data science, software development, data modeling, or data engineering
- OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 8+ years experience in business analytics, data science, software development, data modeling, or data engineering
- OR equivalent experience.
- 2+ years experience with data governance, data compliance and/or data security.
Data Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.