Position title

Databricks Administrator

Job Location

Bangalore, Bengaluru , Karnataka, India

Description

Hadoop/Databricks Operations is part of the Hadoop Support team. This team is responsible for managing the Hadoop/Databricks plant across the system. Working within an Agile delivery (Scrum/Kanban/Scrumban)/ DevOps methodology. Support the application development teams and
help them in debugging and fixing an issue.

Responsibilities

Build, Upgrade and maintain Hadoop clusters with several nodes.
 Monitoring and troubleshooting of services running on Hadoop clusters.
 Performance tuning.
 Setup of backup and recovery.
 Troubleshooting User issues which include User on boarding, job failures.
 Installing and integrating of a new services onto the Hadoop cluster.
 Working with Vendor to discuss/apply issue bugs, patches and issues.
 Deploy and automate the implementations/fixes using Ansible scripts.
 Hands-on experience on the administration side of DataBricks preferably on Azure.
 Prior experience in a support role on admin side of Databricks on one of Azure/AWS/GCP cloud
 Excellent knowledge of git and Jenkins. Understanding of distributed systems and databases,
cloud computing environments – Azure/Linux(mandatory skills)
 General familiarity with Docker and Kubernetes concepts
 Hands-on experience in Azure stack (Azure Data Lake, Azure Data Factory, Azure Databricks)
 Good understanding of other Azure services like Azure Data Lake Analytics & U-SQL, Azure SQL
DW
 Demonstrated analytical and problem-solving skills, particularly those that apply to a big data
environment.
 Working within an Agile delivery (Scrum/Kanban/Scrumban)/ DevOps methodology.

Skills

Experience in Cloudera Hadoop distribution CDH 6.x and CDP 7 is preferred.
 Experience with Cluster maintenance tasks like adding and removing nodes, enabling High
availability, installing services, applying patches.
 Unix/Linux knowledge including the ability to understand hardware, Operating system and
network settings.
 Experience with Hadoop Ecosystem components which include HDFS, YARN, Hive, Impala, Spark,
Sqoop, Kafka, Flume and Solr.
 Unix Shell, perl or python scripting.
 Kerberos and LDAP integration experience.
 TLS/SSL certificates knowledge to enable encryption cross Hadoop services.
 Some development experience in Databricks on Azure/AWS/GCP
 Some developer skills in Python
 Some experience with Terraform for IaaC

Hiring organization