Databricks - Engineer

• Develop and maintain scalable data pipelines and data processing solutions using PySpark.
• Design and implement data ingestion, transformation, and processing workflows to support analytical and operational data requirements.
• Build and maintain data solutions using Snowflake or Delta Lake on Databricks to support enterprise data platforms.
• Develop batch and near real-time data processing pipelines leveraging modern data engineering practices.
• Implement event-driven data integration solutions using Apache Kafka or Amazon Kinesis.
• Develop data streaming workflows to support reliable and scalable data movement across systems.
• Configure and maintain workflow orchestration solutions using Apache Airflow or Databricks Workflows.
• Monitor and support scheduled data processing jobs to ensure successful execution and operational stability.
• Perform data validation, reconciliation, and quality checks to ensure data accuracy and consistency.
• Utilize AI-assisted engineering tools, code generation capabilities, and developer productivity solutions to improve data engineering efficiency while adhering to organizational engineering standards.
• Support development of reliable data products by contributing to data quality, metadata, documentation, and business-ready datasets that enable trusted data consumption.
• Troubleshoot data pipeline failures, workflow issues, and processing bottlenecks through root cause analysis.
• Support testing, deployment, and production support activities for data engineering solutions.
• Maintain technical documentation, workflow definitions, and operational procedures.
• Collaborate with various teams and stakeholders to support successful data platforms and analytics initiatives.

Behavioral Competencies

• Demonstrates ownership and accountability while delivering high-quality data engineering solutions.
• Collaborate effectively with various teams and business stakeholders to ensure smooth delivery.
• Maintains a quality-first mindset while developing scalable and reliable data processing solutions.
• Apply strong analytical and problem-solving skills to resolve data engineering and platform challenges.
• Shows adaptability and willingness to learn evolving data engineering technologies and practices.
• Communicates clearly by sharing progress, risks, and dependencies proactively.