Description
Sr. ETL Developer (Azure Databricks and PySpark) – Remote
for
Next Pathway – The Automated Cloud Migration Company
Location: Remote
Duration: 5-Month Contract (Strong Potential for Extension)
Company: Next Pathway
Next Pathway is seeking an experienced Sr. ETL Developer to design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and PySpark
Responsibilities
- Analyze and understand legacy ETL workflows and business logic to accurately migrate and modernize data pipelines into Azure Databricks while preserving existing functionality.
- Build data ingestion and transformation workflows for structured and semi-structured data.
- Integrate Databricks with Azure Data Factory, ADLS, and Azure SQL for end-to-end data processing.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Implement data validation, error handling, and monitoring to ensure reliable data pipelines.
- Collaborate with business , architects, and developers to deliver data engineering solutions.
- Support code reviews, testing, deployment, and production maintenance following best practices
