Cloudera Platform Engineer

Education - masters or bachelor's degree in computer science, information systems management or related field.

Preferred Experience: More than 6+ years of experience in information technology, Minimum of 3-5 years of experience in managing and administering Hadoop/Cloudera environments.

Proven experience with Cloudera CDP (Cloudera Data Platform), Cloudera Manager, and related tools.

Strong knowledge of Hadoop ecosystem components (HDFS, YARN, Hive, HBase, Spark, Impala, etc.).

Proficiency in Linux system administration with experience with scripting languages (Python, Bash, etc.) and configuration management tools (Ansible, Puppet, etc.).

Familiarity with data security practices and tools (Kerberos, Ranger, Sentry).

Excellent problem-solving and analytical with strong communication and interpersonal skills.

Ability to work independently and in a team environment.

Cloudera Certified Administrator for Apache Hadoop (CCAH) or similar certification. Experience with DevOps practices and tools (Docker, Kubernetes, Jenkins).

Knowledge & Skills Technical Managerial

Cluster Management: Deploy Cloudera clusters from scratch, including hardware and software configuration.

Configure clusters to meet specific business requirements and workloads.

Plan and execute cluster upgrades to ensure systems are up to date with the latest features and security patches.

Perform necessary tests to ensure system stability post-upgrade.

Continuously monitor the health and performance of the clusters using Cloudera Manager and other monitoring tools.

Implement alerting mechanisms to proactively address issues before they impact performance. System Administration:

Manage Hadoop components such as HDFS, YARN, Hive, HBase, Spark, Impala, Kafka, and others. Configure and manage data storage policies, replication, and fault tolerance.

Implement and manage security policies, including Kerberos for authentication and Ranger or Sentry for authorization.

Conduct regular security audits and compliance checks.

Develop and maintain scripts to automate routine administrative tasks (e.g., backups, log rotation, user management).

Use configuration management tools (Ansible etc.) to ensure consistent and repeatable cluster configurations. Performance Optimization:

Optimize cluster performance by tuning Hadoop configuration parameters and adjusting resource allocations.

Conduct regular performance testing and benchmarking.

Identify and resolve performance bottlenecks in the Hadoop ecosystem components.

Work with application developers to optimize data processing jobs for efficiency. Best Practices Implementation:

Establish and enforce best practices for data storage, retrieval, and processing.

Provide guidance on the optimal use of Hadoop ecosystem tools. Collaboration and Support:

Work closely with data engineers, data scientists, and analysts to understand data requirements and provide solutions.

Assist in the design and implementation of data models and storage strategies.

Provide Level 2 and Level 3 support for Cloudera platform-related issues.

Troubleshoot and resolve technical issues related to cluster operations promptly.

Maintain comprehensive documentation of the platform architecture, configurations, and standard operating procedures.

Document troubleshooting steps and resolutions for future reference.

Participate in an on-call rotation to provide after hours support as needed.

Ensure rapid response to critical issues affecting cluster performance or availability.

Communicate effectively with stakeholders regarding system status, issues, and planned enhancements.

Provide regular updates and reports on cluster performance and health.

Collaborate with cross-functional teams to understand data requirements and provide solutions. Participate in planning and design sessions for new data initiatives.

Provide training and support to junior team members and other stakeholders.

Mentor team members on best practices and new technologies within the Cloudera ecosystem.

Behavioral Competencies Thinking Related People Related Self Related

Business Acumens Organization leadership skills Coaching and mentoring Excellent analytical skills Demonstrate critical and systems of thinking ability. Ability to negotiate and influence Disciplined, organized Flexibility / Adaptability Visionary Performance driven