Pasumarthi Pradeep Shiva

Role 2: Team lead/Manager:

• Managing 12 member team with different working experience people.
• Managed to deliver 2 new features of KAFKA w.r.t critical applications and non critical applications
• Provided technical guidance and mentorship to junior team members, fostering a collaborative learning environment within the organization.
• Collaborated with system architects, design analysts and others to understand business and industry requirements.

Role 1: Streaming Data Engineer

• Company Overview: Lloyds technology Centre (LTC), Hyderabad
• It’s a streaming platform which will be used by multiple applications across the bank.
• As part of this project, I am responsible to create real time streaming data pipelines for requested applications and deploying required kafka components w.r.t different usecases.
• This confluent kafka platform is deployed in GKE container and its available in two zones.
• I am responsible to onboard different applications and enable different businesses to get realtime streaming advantages.
• Some of the applications: Credit cards, Debit cards, Loans, Business commercial banking, faster payments, customer experience.
• I am leading this entire team very scaled platform with nearly 75+ applications.
• Lloyds technology Centre (LTC), Hyderabad

• It's a data migration project which is having multiple sources that holds the data.
• We will process the data to GCP Big Query in multiple modes like batch processing and real time processing.
• We have created multiple pipelines for batch and real time processing’s.
• Nearly 15 sources we have migrated to GCP within a year.
• Pipelines involved, GCS buckets, bigquery, data flow, Pub/Sub and Kafka.
• For batch processing we will create a pipeline with Juniper which will connect to the source database or file system along with target environment in GCP and schedule this pipeline in hourly, daily, monthly basis.
• For real-time processing we will capture the data from the source in a real-time fashion by using Attunity tool to connect and capture the source data and process the data to Kafka region.
• Once the data landed in Kafka it will go to PubSub topics with the help of Juniper integration tool and then we use Dataflow to load the data into the destination BigQuery from PubSub topics.
• Making sure every process involved is adhere to Data Governance.
• Managed multiple projects simultaneously while maintaining strict deadlines and high-quality standards.
• Streamlined development workflows to increase team efficiency and reduce time spent on repetitive tasks.
• This entire mechanism performed in real time fashion.
• Technologies used: GCP, BigQuery, Juniper, Jira, Kafka, Attinuty, Dataflow

• Client: Macys Inc. USA.
• Project Description: Macys Inc is an online retail and as well as physical stores available through out the USA.
• Doing maintaining Macys data across all modules like sales, customer info, products, pricing etc need to provide production support for already built datastage jobs and some enhancements as well.
• Technologies used: IBM DataStage, Control M, Service Now, Unix, DB2, Oracle, Informatica, DAC, Confluence

• Client: Macys Inc. USA.
• Project Description: This is a migration from tradition ETL to GCP along with Airflows.
• Need to migrate existing code logics and pipelines to GCP by using Airflows, Dataflow pipelines.
• And need to call airflow jobs from control M scheduler.
• As part of this project GCP components like Google Bigquery, Google Pub/SUB, Data flows, Air flows, Alert ops, Cloud storage, and different Data management tools.
• Technologies used: GCP components, IBM DataStage, Control M, Airflow, Adobe Analytics etc.

• Title: Batch Integration HUB
• Role: Datastage Developer
• Client: ANZ (Australia and New Zeeland Bank)
• Project Description: As a banking project, there were so many data flows across the organization and need to develop ETL jobs for those dataflows.
• Batch Integration Hub is a intermediate place for source and Target syatems.
• As per business requirements need to build Datastage jobs with business logic inside.
• And need to schedule jobs daily, weekly, monthly yearly across all locations of bank exists.
• Uses Tools: IBM Datastage, Control M, Unix, Oracle, Bit bucket, Confluence, JIRA.