Description

Greymatter Innovationz helps you stay digitally relevant across domains, technologies, and skillsets, every day.

We are looking for:

Lead AWS Cloud Engineer – Databricks Platform

Location: Princeton, NJ (Hybrid – 3 days onsite)

Duration: Contract Or FTE

Position Overview:

We are seeking a highly skilled and experienced AWS Cloud Engineer with a strong background in Databricks to join our dynamic cloud engineering team. The ideal candidate will have extensive experience in designing, deploying, and managing cloud infrastructure on AWS, along with in-depth expertise in the Databricks platform for data engineering, analytics, and machine learning workloads. The candidate should also have prior leadership experience, as they will be responsible for guiding a team of engineers in cloud-related initiatives, best practices, and ensuring the successful delivery of data-driven solutions.

Key Responsibilities:

  • Cloud Infrastructure Design & Implementation:
  • Lead the design, implementation, and optimization of scalable and secure AWS cloud solutions.
  • Utilize AWS services such as EC2, S3, Lambda, RDS, VPC, IAM, and others for building robust cloud-based architectures.
  • Work closely with the architecture team to define infrastructure requirements and recommend suitable AWS technologies.
  • Databricks Platform Expertise:
  • Leverage Databricks for building and managing data pipelines, running Spark-based analytics workloads, and orchestrating machine learning models.
  • Design, optimize, and maintain Databricks clusters and notebooks for data processing, transformation, and analysis.
  • Integrate Databricks with other AWS services (e.g., S3, Redshift, Glue) to build comprehensive end-to-end data solutions.
  • Provide leadership in best practices for working with Databricks to ensure reliability, scalability, and efficiency of data workflows.
  • Leadership & Mentoring:
  • Lead a team of cloud engineers, guiding them in implementing solutions that align with organizational goals and industry best practices.
  • Mentor junior team members and provide technical guidance on cloud-related projects, Databricks integration, and infrastructure optimization.
  • Facilitate cross-functional collaboration between engineering, data science, and business teams.
  • Automation & Optimization:
  • Automate cloud infrastructure provisioning and management using Infrastructure-as-Code tools (e.g., Terraform, CloudFormation).
  • Continuously monitor and optimize AWS resources to improve performance and reduce costs.
  • Troubleshooting & Support:
  • Provide expert-level troubleshooting and support for cloud infrastructure, Databricks platform issues, and production systems.
  • Identify and resolve performance bottlenecks, security risks, and system inefficiencies across AWS and Databricks environments.
  • Documentation & Reporting:
  • Document cloud infrastructure, Databricks configurations, and solutions for internal knowledge sharing.
  • Prepare regular reports on system performance, cost management, and the status of cloud initiatives.

Key Requirements:

  • Experience:
  • 5+ years of experience in AWS cloud engineering, with a proven track record of delivering cloud-based solutions.
  • 3+ years of hands-on experience with Databricks platform, including building and managing data pipelines, notebooks, and Spark-based workloads.
  • 2+ years of experience leading or mentoring a team of engineers, with strong leadership and team-building skills.
  • Technical Skills:
  • Proficient in AWS cloud services (EC2, S3, Lambda, RDS, VPC, IAM, CloudWatch, etc.).
  • Strong knowledge of Databricks, Apache Spark, and data engineering best practices.
  • Experience with containerization and orchestration tools (Docker, Kubernetes).
  • Familiar with Infrastructure-as-Code tools (Terraform, CloudFormation).
  • Experience with data processing frameworks, data lakes, and analytics pipelines.
  • Additional Skills:
  • Strong understanding of security best practices in cloud environments.
  • Ability to manage complex cloud projects with multiple stakeholders.
  • Excellent communication skills, with the ability to explain technical concepts to non-technical stakeholders.
  • Experience with Agile methodologies is a plus.

Education & Certifications:

  • Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field (or equivalent experience).
  • AWS Certified Solutions Architect – Professional (preferred).
  • Databricks Certified Associate Developer (preferred, but not required).

At Greymatter Innovationz,

We offer:

Motivating Work Environment.

Excellent Work Culture.

Help you to upgrade yourself to the next level.

And More!!!