What you’ll bring:

• Big Data Technologies: Proficiency in working with big data technologies, particularly in the context of Azure Databricks, which may include Apache Spark for distributed data processing.
• Azure Databricks: In-depth knowledge of Azure Databricks for data engineering tasks, including data transformations, ETL processes, and job scheduling.
• SQL and Query Optimization: Strong SQL skills for data manipulation and retrieval, along with the ability to optimize queries for performance in Snowflake.
• ETL (Extract, Transform, Load): Expertise in designing and implementing ETL processes to move and transform data between systems, utilizing tools and frameworks available in Azure Databricks.
• Data Integration: Experience with integrating diverse data sources into a cohesive and usable format, ensuring data quality and integrity.
• Python/PySpark: Knowledge of programming languages like Python and PySpark for scripting and extending the functionality of Azure Databricks notebooks.
• Version Control: Familiarity with version control systems, such as Git, for managing code and configurations in a collaborative environment.
• Monitoring and Optimization: Ability to monitor data pipelines, identify bottlenecks, and optimize performance for both Azure Data Factory
• Security and Compliance: Understanding of security best practices and compliance considerations when working with sensitive data in Azure and Snowflake environments.