Principal Duties And Responsibilities:-
• Administer, maintain, and support Data Warehouse and Big Data platforms including Teradata, Informatica, Hadoop, and Apache NiFi environments.
• Design, develop, deploy, and optimize data integration workflows, ETL processes, and report solutions to support business and operational requirements.
• Develop, analyze, and optimize SQL queries, reports, and data extraction processes to ensure efficient data access and reporting performance.
• Monitor, maintain, and troubleshoot scheduled data refreshes, batch jobs, and automated data processing workflows.
• Analyze business requirements and develop ad hoc queries, datasets, and analytical solutions to support decision-making and operational needs.
• Monitor end-to-end data pipelines, ensuring timely data ingestion, processing, validation, and delivery across all platforms.
• Ensure all data feeds, integrations, and BI data exports are delivered accurately and within agreed service levels and business requirements.
• Administer Apache NiFi clusters, including flow development, deployment, monitoring, troubleshooting, performance tuning, and cluster maintenance.
• Manage NiFi security, user access controls, SSL certificates, LDAP/Active Directory integration, and platform governance policies.
• Monitor and troubleshoot NiFi processors, queues, repositories, cluster communication, and data flow performance issues.
• Administer Hadoop clusters, including deployment, configuration, expansion, upgrades, node commissioning/decommissioning, and high-availability configurations.
• Install, configure, maintain, and monitor Hadoop ecosystem components including HDFS, YARN, Hive, Spark, Kafka, Impala, Oozie, ZooKeeper, HBase, and related services.
• Perform cluster monitoring, capacity planning, performance tuning, and troubleshooting to ensure optimal platform availability and efficiency.
• Implement and maintain Hadoop security controls including Kerberos authentication, LDAP/Active Directory integration, authorization, and data protection mechanisms.
• Create and manage backup, snapshot, disaster recovery, and business continuity procedures for Big Data platforms.
• Support recovery activities from node failures, service interruptions, and infrastructure incidents while minimizing operational impact.
• Execute platform upgrades, migrations, and lifecycle management activities for Hadoop environments, including private and public cloud deployments.
• Collaborate with business, development, infrastructure, and vendor teams to resolve technical issues and implement platform enhancements.
• Develop and maintain operational procedures, technical documentation, monitoring standards, and best practices for supported platforms.
• Utilize Linux administration and shell scripting skills to automate operational tasks, monitoring activities, and system maintenance processes.
Skills
• Teradata Administration and Development
• Informatica PowerCenter Administration and Support
• Apache NiFi Administration and Data Flow Development
• Hadoop Ecosystem Administration (HDFS, YARN, Hive, Spark, Kafka, HBase, Impala, Oozie, ZooKeeper, Ranger)
• Ambari Administration
• Kerberos, LDAP, and Active Directory Integration
• SQL Performance Tuning and Query Optimization
• Linux System Administration
• Shell Scripting and Automation
• Data Warehouse and Big Data Operations Support
