Skip to content

vishnoiprem/data.code.cloud

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

98 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Prem Vishnoi

Lead Data Engineer | Expert in Big Data and AI

+65-92716405 | [email protected] | LinkedIn | Blog

PROFESSIONAL SUMMARY

Data Engineering Leader with 14+ years of experience transforming complex data ecosystems across e-commerce, retail, and financial services. Expert in building scalable data platforms, leading cross-functional teams, and delivering innovative cloud solutions.

TECHNICAL EXPERTISE

  • Core: Python, SQL, Java, Shell Scripting
  • Big Data: Apache Spark, Hadoop, Kafka, Flink, HBase
  • Cloud: AWS (Solutions Architect), Azure, Aliyun
  • Data Platform: Databricks, Airflow, dbt, Teradata, MySQL, Postgres
  • ML/AI: PyTorch, Scikit-learn, NLP, LLMs
  • Visualization: Looker, Power BI, Quickview

PROFESSIONAL EXPERIENCE

Lead Data Engineer | CPaxtra (2024 - Present)

  • Led 16-person engineering team building next-gen data platform
  • Built end-to-end data pipelines serving 50+ stakeholders across retail operations
  • Developed real-time data products for last-mile delivery reducing delivery time by 30%
  • Architected Lakehouse solution reducing operational costs by 50%
  • Implemented data validation frameworks improving reliability by 20%
  • Tech: Azure, AWS, Databricks, Spark, Kafka, Python

Lead Data Engineer | Xendit.co, Singapore (2024)

  • Led 4-person team developing real-time and batch data pipelines
  • Built automated FX data pipeline reducing manual processing by 80%
  • Optimized AWS/Databricks costs by 40% through resource optimization
  • Reduced DBT job execution time by 60% through code refactoring
  • Tech: Databricks, AWS, Airflow, Python, DBT, Looker

VP Data Engineer | Lazada Group, Singapore (2018 - 2024)

  • Built Data Lakehouse on Alibaba Cloud using ODS, CDM, ADS layers supporting $10B+ GMV
  • Built real-time last-mile delivery platform tracking 5M+ parcels daily
  • Developed real-time hub analytics reducing backlog by 35% with capacity alerts
  • Created campaign data platform supporting 11.11, 12.12 events handling 10M+ orders/hour
  • Implemented data governance and lineage for 1000+ tables across 6 countries
  • Tech: Alicloud, Kafka, Flink, Hologres, Python, SpringBoot

Sr. Big Data Consultant | SCB Bank, Singapore (2016 - 2018)

  • Developed batch data pipelines for Anti-Money Laundering (AML) operations
  • Managed data workflows across 15+ international markets
  • Improved data ingestion efficiency by 15%
  • Tech: Hadoop, Hive, Kafka, Spark, SQL, Teradata, Scala

Data Engineer | DBS Bank, Singapore (2015 - 2016)

  • Migrated to Hadoop-based big data platform
  • Developed scalable data ingestion frameworks
  • Tech: Hadoop, Hive, Kafka, Spark, SQL, Teradata

ETL Data Engineer | PayPal Inc., India (2014 - 2015)

  • Designed ETL pipelines handling 500M daily transactions
  • Reduced processing times by 50%
  • Tech: ETL, Python, Hadoop, Spark, SQL

Data Warehouse Engineer | Exilant Tech, India (2011 - 2014)

  • Built data warehouse for Apple's customer care platform
  • Improved data accuracy by 10%
  • Tech: SQL, Informatica, Teradata, Tableau, Python

EDUCATION

  • PGP in AI & Machine Learning | University of Texas at Austin
  • Bachelor of Engineering | University of Rajasthan

CERTIFICATIONS & ACHIEVEMENTS

  • AWS Solutions Architect Associate
  • LLM Model Learning Certificate
  • Internal Hackathon Winner at PayPal
  • Promoted twice in 4 years for exceptional performance

Head of data | Expert in Big Data and AI

About

coding interview for big 5

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages