Apache Spark-Data Engineer

Design, build, and manage scalable data pipelines using Apache Spark for large-scale data processing and transformation. Develop, optimize, and maintain SQL queries for complex data manipulation, retrieval, and reporting requirements. Implement scalable cloud-based data storage solutions using AWS services, including Amazon S3. Manage metadata, data governance, and data quality processes using Collibra to ensure compliance with organizational standards. Execute interactive data queries and analysis using Presto to enable seamless access across multiple data sources. Orchestrate, schedule, and automate data workflows using Apache Airflow for efficient pipeline execution. Work in Agile delivery teams to support iterative improvements, stakeholder requirements, and project objectives.