Azure Databricks Admin

Create and manage Azure Databricks workspaces, Unity Catalog setup allocating resources and configuring settings based on specific needs for all environments (Dev/Test/Prod).

• Design and implement access control mechanisms, user permissions, and security policies to safeguard sensitive data. : 6 to 10 yesr expereince

• Configure and launch Databricks clusters, optimizing performance and cost through effective instance type selection, auto-scaling, and termination policies.

• Monitor cluster health and performance, troubleshoot issues proactively, and adjust configurations as necessary.

• Install and configure libraries, dependencies, and environment variables essential for data engineering and analytics tasks.

• Implement robust security practices, including network security settings, integration with Azure Active Directory, and data encryption.

• Stay informed about security best practices and updates, applying them proactively to maintain a secure environment.

• Monitor resource utilization and costs associated with Databricks usage, identifying opportunities for optimization and cost reduction.

• Analyze cluster performance metrics and implement tuning strategies to ensure optimal query execution times and resource utilization.

• Track system logs and identify potential issues before they impact users or operations.

• Collaborate with data engineers, data scientists, and other stakeholders to understand their analytics needs and translate them into effective configurations.

• Provide technical support to users, troubleshooting issues and resolving technical inquiries.

• Document setup processes, configurations, and troubleshooting steps for future reference and knowledge sharing