Job Description
Summary
The Azure Data Engineer is responsible for designing, building, and supporting scalable data platforms and pipelines within a cloud-based environment. This role focuses on delivering secure, high-performance data solutions that enable advanced analytics and data-driven decision making.
Key Responsibilities
- Design and implement scalable and secure data processing pipelines using Azure Data Factory, Azure Databricks, and related Azure services.
- Build and manage ELT and ETL workflows supporting batch and streaming ingestion use cases.
- Develop and optimize complex SQL queries and data models to support enterprise reporting and analytics.
- Implement API-based and streaming ingestion methods for low-latency data processing.
- Manage and optimize data storage solutions including Azure Data Lake, Azure SQL, Azure Synapse Analytics, and Azure Cosmos DB.
- Monitor, troubleshoot, and optimize data pipelines to ensure reliability, performance, and availability.
- Implement data security controls including encryption, access management, auditing, and compliance requirements.
- Automate data workflows and deployments using DevOps practices and CI/CD pipelines.
- Document data architectures, pipelines, and operational procedures to ensure clarity and regulatory alignment.
- Support data governance initiatives including metadata management, data lineage, and data cataloging.
Required Qualifications
- Strong expertise in data software engineering with hands-on experience in Azure Databricks and Python.
- Expert-level experience writing and optimizing complex SQL in large-scale data environments.
- Proven experience designing data models and ETL or ELT solutions in enterprise settings.
- Hands-on experience with Apache Spark and Azure Data Factory.
- Strong understanding of Azure cloud services including Azure Data Lake, Azure SQL, Azure Synapse Analytics, and Azure App Service.
- Experience building data pipelines for streaming and real-time processing.
- Knowledge of DevOps processes including CI/CD and infrastructure as code.
- Working knowledge of Apache Kafka and NoSQL solutions such as Azure Cosmos DB preferred.
- Familiarity with both Azure and AWS cloud infrastructure offerings.
Tech Stack
Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure SQL, Azure Data Lake, Apache Spark, Apache Kafka, Python, SQL, Azure Cosmos DB, AWS
Compensation & Benefits
Competitive compensation aligned with market standards.
Health benefits, PTO, remote flexibility (varies by client).
Work Schedule
Monday–Friday, flexible hours, occasional on-call (if applicable).
About the Client
Our client is a technology-driven organization focused on building modern, secure, and scalable data platforms to support enterprise analytics and business intelligence.
Why Join the Team?
- Work on complex data engineering challenges that drive meaningful business outcomes.
- Leverage modern cloud and data technologies at enterprise scale.
- Collaborate with high performing teams delivering impactful data initiatives.
How to Apply
Submit your resume to people@ignitetalentpartners.com. Shortlisted candidates will be contacted.