Data Engineer

Job Description: • Design, develop, optimize, and maintain data architecture and pipelines that adhere to ETL principles and business goals. • Solve complex data problems to deliver insights that help the organization achieve its goals. • Code in Python with tools like Apache Spark to build a multi-cluster data warehouse. • Interact with other technology teams to define, prioritize, and ensure smooth deployments for other operational components. • Advise, consult, mentor, and coach other data and analytics professionals on data standards and practices. • Foster a culture of sharing, reuse, design for scale stability, and operational efficiency of data and analytical solutions. • Codify best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases to facilitate data capturing and management. • Strong experience in mapping attributes, data profiling, data cleansing, and technical data quality etc. • Strong experience in Ansi SQL and in-depth knowledge with structured, semi-structured & unstructured data. • Must have good knowledge of data lake and working experience of migration projects in cloud with providers like AWS or Microsoft AZURE or GCP. • Good to have experience in working with No SQL, Spark SQL using AWS Glue, EMR, and columnar data store. • Good to understand data security features like data masking, data encryption, role based & fine grain access control mechanisms etc. Qualifications: • 3+ years of relevant experience in data engineering/analytics space. • Expertise in SQL and data analysis and strong hands-on expertise with at least one programming language: Python. • Strong knowledge in one or more of the following big data tools: Hive, Hadoop Impala, Spark, Kafka. • Strong expertise in ETL, reporting tools, data governance, data warehousing, and hands-on experience. • Experience developing solutions for cloud computing services and infrastructure. • Experience developing and maintaining data warehouses in big data solutions. • Up to date on industry trends within the analytics space from a data acquisition processing, engineering, and management perspective. • Experience in agile development. • Strong people skills, specifically in collaboration and teamwork. • High level of curiosity, creativity, and problem-solving capabilities.