Big Data Cloud Engineer (Banking Sector, HDFS, YARN, Kafka, Linux, AWS, Shell scripting)

Responsibilities: • Administer, maintain, and support large-scale Hadoop and Cloudera (CDH/CDP) environments across enterprise distributed infrastructures. • Design and manage Hadoop cluster architecture, deployment, configuration, and lifecycle management for highly available production environments. • Manage and optimize core Big Data ecosystem components including HDFS, YARN, MapReduce, Spark, PySpark, Kafka, Hive, Impala, HBase, Oozie, and Zookeeper. • Perform administration, monitoring, troubleshooting, and performance tuning of Hadoop clusters to ensure high availability, scalability, and operational stability. • Support and maintain real-time and batch data processing platforms using Spark, Kafka Streams, Hive, and HBase technologies. • Configure and support Hadoop ecosystem tools including Hue, Arcadia, Airflow, NiFi, Sqoop, Datameer, Splunk, and AEN. • Implement and manage enterprise-grade security frameworks using Kerberos, Ranger, Sentry, SSL, IAM policies, and access control mechanisms. • Manage and support AWS cloud infrastructure, including EC2, S3, EMR, IAM, and VPC services for Big Data workloads. • Perform capacity planning, cluster expansion, metadata management, partition optimization, and resource tuning to improve platform performance. • Develop and maintain automation utilities using Shell scripting, Python, PySpark, and Linux scripting for operational efficiency and proactive monitoring. • Execute OS patching, cluster upgrades, maintenance activities, backup strategies, and infrastructure management with minimal business disruption. • Monitor production environments using enterprise monitoring and observability tools including NewRelic and other operational monitoring platforms. • Lead incident management, root cause analysis (RCA), problem resolution, and production support activities for mission-critical Big Data platforms. • Collaborate with DevOps, infrastructure, database, and application teams to support enterprise analytics and data engineering initiatives. • Participate in Agile delivery processes, operational reviews, and continuous improvement initiatives. • Mentor junior team members and contribute to technical knowledge sharing and operational best practices. • Support enterprise platforms and applications within Financial/Banking domain environments with strict compliance and availability requirements. • Provide support for PeopleSoft Administration and related enterprise integration environments where required. Requirements: • 10+ years of experience in Hadoop Administration and Cloudera Administration (CDH/CDP). • Experience in Financial/Banking Sector • Strong experience in Hadoop Cluster Architecture, Deployment, Administration, and Performance Tuning. • Hands-on experience in HDFS, YARN, Kafka, Spark, Hive, Impala, MapReduce, HBase, Oozie, and Zookeeper. • Strong experience in Linux Administration and enterprise infrastructure troubleshooting. • Hands-on experience in AWS cloud services, including EC2, S3, EMR, IAM, and VPC. • Experience in Shell scripting, Python, and PySpark for automation and operational support. • Experience in PeopleSoft Administration and Oracle GoldenGate. • Strong understanding of data security, authentication, authorization, and compliance frameworks including Kerberos, Ranger, and Sentry. • Experience in monitoring, incident management, production support, and performance analysis using tools such as NewRelic. • Experience in supporting large-scale, high-availability, multi-region production environments. • Experience working with CI/CD pipelines, DevOps practices, and GitHub/Jenkins-based automation frameworks. • Experience in Hadoop ecosystem tools including Hue, Arcadia, Airflow, NiFi, Sqoop, Datameer, Splunk, and AEN. • Experience in PeopleSoft Administration and enterprise application support environments. • Experience in Financial/Banking sector projects and mission-critical enterprise systems is highly preferred. • Strong analytical, troubleshooting, stakeholder management, and communication skills.