Pavan Kumar Gade

• Provisioned and automated AWS services (EC2, RDS, EMR, DMS, CloudWatch) using CloudFormation templates.
• Administered EMR clusters for enterprise DataLake projects, enabling migration from on-prem Hadoop to AWS Cloud.
• Configured auto-scaling and monitoring for EMR clusters, troubleshooting performance issues and optimizing cluster utilization.
• Automated Talend job deployments on Linux EC2 using CI/CD pipelines in Jenkins, reducing manual deployment efforts.
• Developed Python-based Lambda functions for DMS task automation, Glue job provisioning, and S3 optimization reporting.
• Collaborated with AWS Support to resolve cluster slowdowns and infrastructure bottlenecks.

Key Achievements:

• Reduced deployment time by 40% through automation of Talend jobs.
• Improved cluster stability by implementing advanced monitoring and troubleshooting processes.
• Enabled seamless migration of data pipelines, ensuring business continuity during modernization projects.

• Built and managed Elasticsearch, Logstash, and Kibana (ELK) clusters to centralize log monitoring and analysis.
• Designed Grok filters for Logstash to process Ezproxy logs and configured Ezpaarse for ingestion into Elasticsearch.
• Automated daily log ingestion and system operations with custom Shell scripts.
• Created Kibana dashboards and visualizations for real-time monitoring and analytics.
• Collaborated with development teams to debug software/system issues and streamline deployment processes.

Key Achievements:

• Improved system observability by implementing centralized log management using ELK.
• Reduced manual intervention by automating log ingestion and system scripts.
• Supported smoother releases by actively participating in risk analysis and deployment planning.