Description

Senior DevOps with Infrastructure

Role: Senior DevOps with Infrastructure

Type: Remote – working EST Hours

Security Clearance: Must be eligible for up to a Top-Secret Security Clearance

Job Overview

The Senior DevOps Engineer is a critical hands-on role responsible for the architecture, deployment, and performance optimization of a cloud-native, real-time data processing platform.

Key Responsibilities

  • Design, deploy, and manage the underlying data platform using Kubernetes to ensure high-scalability and resiliency for distributed data workload.
  • Build and maintain high-throughput ETL/ELT pipelines using Python and Apache Spark.
  • Implement and tune the distributed SQL query engine Trino against the Iceberg table format to optimize query performance and data reliability.
  • Develop core services, APIs, and automation tooling for the data platform primarily using the C# programming language, ensuring speed and reliability.
  • Architect and implement robust authentication and authorization integrations across the platform, including seamless connectivity via LDAP/SSO protocols.
  • Utilize Infrastructure-as-Code (e.g., Terraform, Helm) and GitOps principles to manage the platform lifecycle and enforce immutable infrastructure practices.
  • Work closely with security teams to implement and audit IAM policies and ensure container security best practices are followed.

Must-Have:

  • 7+ years of experience in DevOps in Infrastructure with hands-on experience with Kubernetes administration, tuning, and deployment in production environments.
  • C# for building platform services and integration points.
  • Deploying, configuring, and optimizing Trino for large, federated data sets.
  • Implementing Iceberg table format for data lake transactions and schema evolution.
  • Python and mandatory practical experience with Apache Spark for data processing.
  • Integrating enterprise authentication and authorization protocols (LDAP/SSO) into microservices.