Experienced IT professional with over 7 years of industry experience, specializing in Azure Data Engineering for the past 3+ years. Skilled in leveraging Azure Databricks, Data Factory, and Data Lake Storage to design and implement scalable ETL pipelines. Proficient in PySpark for custom business logic, data validation, and performance optimization. Experienced in Agile methodologies, collaborating across teams, and ensuring data quality control throughout the pipeline process.
Client: Banco Santander S.A., USA.
Client: Athena Health.
Programming: Python, PySpark, SQL
Big Data Frameworks: Apache Spark, Delta Lake
Azure Services: Azure Databricks, Azure Data Factory (ADF), ADLS Gen2, Azure SQL, Azure Key Vault
Databases: Azure SQL
Concepts: ETL/ELT, Schema Evolution, Broadcast Joins, Partitioning, Window Functions
Tools: Git, Azure DevOps, Visual Studio Code
OS: Windows, Linux
Employee of the quarter Q4 2021