AI Data Engineer & Analyst

Overview We are seeking a highly skilled AI Data Engineer & Analyst to lead the design and implementation of an AI-powered strategic account intelligence platform. This is a greenfield build within the Microsoft ecosystem (Microsoft Fabric, OneLake, Azure AI, Power BI, Purview, Entra ID), delivering an end-to-end solution from data ingestion and semantic modeling through to AI-driven recommendations and executive-ready dashboards. This role is ideal for a hands-on technical professional who thrives at the intersection of data engineering, machine learning, and business intelligence — someone who can architect a robust data platform and bring it to life with actionable AI-powered insights. You will work in close partnership with an external Microsoft-certified consultancy engaged to support the full project rollout. The consultancy will provide expert guidance and hands-on support across all project phases, from strategy and planning through infrastructure evaluation and technical execution, ensuring the platform is built to enterprise standards and aligned with Microsoft best practices. Key Responsibilities Data Architecture & Engineering (40%) • Design and implement the canonical object model in Microsoft Fabric / OneLake (Lakehouse / Data Warehouse) • Build and maintain automated ingestion pipelines using Data Factory in Fabric / Azure Data Factory • Implement identity resolution across disparate source systems (deterministic matching on shared keys + probabilistic/ML-based entity resolution) • Create golden records with source-confidence scoring per attribute and conflict resolution rules • Ensure schema validation, data quality checks, and attribute-level completeness thresholds across all pipelines • Design for scalability to hundreds of accounts with near-real-time query performance AI / Machine Learning (30%) • Develop the opportunity scoring model using Azure AI Services / Azure Machine Learning / Fabric Data Science • Build NLP-based news signal classification and entity extraction from external feeds • Implement the NBA recommendation engine with explainable evidence chains linking scores to underlying data points, signals, and model factors • Design and implement decision-lineage capture and the continuous learning feedback loop • Produce ranked recommendations with confidence intervals • Plan for quarterly model review cycles assessing recommendation quality, bias, and drift Business Intelligence & Visualization (20%) • Develop interactive Power BI dashboards in Direct Lake mode over OneLake • Build account-level opportunity score dashboards with drill-through to evidence chains • Create tracking and reporting dashboards for recommendation outcomes, acceptance rates, pipeline impact, and model performance • Generate auto-formatted sales-ready outputs (one-page account briefs, exec summaries, talking points) Governance, Security & Collaboration (10%) • Implement Role-Based Access Control (RBAC) via Microsoft Entra ID with object-level permissions • Configure data lineage tracking through Microsoft Purview (Data Map, Unified Catalog) • Ensure full audit logging of data access, recommendation generation, and user actions • Collaborate with business stakeholders (Account Directors, Sales Leaders, Strategic Planning teams) to validate data models, refine scoring logic, and iterate on outputs • Partner with an external Microsoft-certified consultancy providing end-to-end project support, from strategy and infrastructure planning through to technical execution, leveraging their specialist Microsoft expertise to accelerate delivery, validate architectural decisions, and ensure best-practice implementation across the full Microsoft stack • Document architecture decisions, data flows, and operational runbooks Required Qualifications Bachelor's degree in Computer Science, Data Science, Information Systems, Statistics, or a related technical field Experience: • 5+ years of professional experience in data engineering, data analytics, or business intelligence • 3+ years working within the Microsoft data ecosystem (Azure / Fabric / Power BI) • Demonstrated experience building end-to-end data platforms from ingestion through modeling to dashboards • Proven track record with AI/ML model development and deployment in a production or near-production environment • Experience with entity resolution / identity matching across multiple data sources Technical Skills (Required): Data Platform Microsoft Fabric (OneLake, Lakehouse, Data Warehouse, Data Factory, Fabric Data Science) AI / ML Azure AI Services, Azure Machine Learning, model training & scoring pipelines BI & Dashboards Power BI (Direct Lake mode, DAX, advanced data modeling, report design) Data Governance Microsoft Purview (Data Map, Unified Catalog, Lineage), source-confidence scoring Identity & Access Microsoft Entra ID (RBAC, object-level permissions) Programming Python (pandas, scikit-learn, or equivalent ML frameworks), SQL (T-SQL / Spark SQL) Data Formats Delta Lake, Parquet, JSON Data Modeling Canonical / semantic data modeling, star schema, entity-relationship design