TruGreen - Microsoft Azure Data Engineer/Architect ,
Sep, 2021 - Jul, 20242 years 10 months
Co-Leader of the design and implementation of a medallion data lakehouse architecture that leveraged low-cost storage solutions to optimize data warehousing and analytics capabilities using Databricks and delta tables. This innovative approach consolidated disparate data systems into a unified platform, streamlining data access and accelerating data-driven initiatives. Teams across the organization were empowered to access and analyze comprehensive, up-to-date datasets, facilitating faster insights and driving more informed decision-making.
Spearheaded the migration of a legacy IBM AS/400 system to Azure Synapse, reducing reporting processing time from 6 hours to under 1 hour, enabling real-time, company-wide distribution of critical metrics and boosting decision-making speed by 85%.
Designed and implemented scalable data pipelines using Databricks and the Medallion Architecture, transforming raw data into organized Bronze, Silver, and Gold layers, resulting in a 30% increase in data processing efficiency and a 40% reduction in storage costs.
Developed highly optimized Python scripts for ETL/ELT workflows, streamlining data cleansing, transformation, and loading processes, achieving an 80% reduction in processing time and improving data accuracy by 25%.
Implemented Azure Purview for data governance, improving compliance with GDPR and HIPAA regulations and enhancing audit readiness by 30%.
Integrated Azure Machine Learning with Synapse pipelines to enable predictive analytics for customer segmentation, achieving a 25% improvement in churn prediction accuracy.
Optimized costs by deploying automated scaling for Databricks clusters and Azure Synapse, reducing monthly cloud expenses by 20%.
Leveraged Azure Data Share to enable secure, cross-functional data collaboration, reducing data silos and enhancing enterprise-wide insights.
Designed and implemented targeted customer campaigns by leveraging Azure Synapse Analytics to analyze and segment large-scale customer datasets, resulting in a 15% increase in customer engagement.
Developed automated workflows using Azure Data Factory to ingest, transform, and load customer data into Azure SQL Database, enabling near real-time campaign personalization and execution.
Enhanced campaign effectiveness by implementing data matching and cleansing routines within Azure Databricks, ensuring the quality and reliability of customer datasets.