I am a dedicated Middle Data Engineer with over 5 years of experience, focused on designing and implementing data pipelines to support business intelligence across e-commerce, healthcare, logistics, and financial services sectors. Skilled in Apache Airflow, Spark, and AWS, I build scalable data solutions that enhance data accessibility and reliability. I thrive in collaborative environments, working with cross-functional teams to optimize data workflows and deliver actionable data infrastructure.

Programming Languages: Python, SQL

Skills:

• Data Engineering: Apache Airflow, Apache Spark, ETL Development

• Databases: MySQL, PostgreSQL, MongoDB

• Data Processing: Pandas, PySpark

• Cloud Platforms: AWS (Redshift, S3, Glue), Google Cloud Platform

• Others: Data Pipeline Optimization, Cloud Integration, Schema Design

Tools: Docker, Kubernetes, Git, Jenkins, Visual Studio Code.

E-commerce Data Pipeline Project - Japan  

Data Engineer, 07/2023 - Present  

Project description 

• Developed and maintained data pipelines for an e-commerce platform to process real-time sales and inventory data.

Responsibilities 

• Designed and implemented ETL pipelines using Apache Airflow to ingest data from MySQL into AWS Redshift.

• Optimized data processing workflows with Apache Spark, reducing latency by 30% for daily reports.

• Collaborated with data analysts to ensure data quality and schema consistency across pipelines.

• Integrated AWS Glue for automated data cataloging and transformation.

• Monitored pipeline performance using CloudWatch, resolving issues to maintain 99.9% uptime.

• Documented pipeline architecture to support team onboarding and future scaling.

Technologies 

• Apache Airflow, Apache Spark, Python, SQL, MySQL, AWS Redshift, AWS Glue, PySpark

‍

Healthcare Data Integration Project - United States

Data Engineer, 05/2022 - 06/2023

Project description 

• Built data integration solutions to consolidate patient records and operational data for a U.S. healthcare provider.  

Responsibilities 

• Constructed ETL processes using Python and Apache Spark to transform data from PostgreSQL to AWS S3.

• Worked with healthcare teams to design data models for patient outcome analysis.

• Implemented data validation checks to ensure 100% data integrity during migration.

• Deployed pipelines on AWS Glue, improving data availability for reporting by 25%.

• Assisted in troubleshooting pipeline failures, reducing downtime by 15%.

Technologies 

• Apache Spark, Python, SQL, PostgreSQL, AWS S3, AWS Glue, PySpark

‍

Financial Services Data Workflow Project - Singapore

Data Engineer, 04/2021 - 04/2022

Project description 

• Supported a financial services firm with data workflows for transaction processing and fraud detection.  

Responsibilities

• Built ETL pipelines using Python to process transaction data from MongoDB.

• Optimized data ingestion for real-time fraud detection systems.

• Collaborated with analysts to ensure data accuracy for reporting.

• Documented workflows for future reference.

Technologies

• Python, SQL, MongoDB, AWS S3

‍

Logistics Data Processing Project - Australia

Data Engineer 05/2020 - 03/2021

Project description 

• Supported a logistics firm with data processing for shipment tracking and route optimization.  

Responsibilities

• Built batch processing scripts using Python to handle shipment data.

• Integrated data into MySQL databases for real-time tracking.

• Collaborated with developers to optimize query performance.

Technologies

• Python, SQL, MySQL

AWS Certified Data Engineer - Associate, 2023

English: Advanced  

• IELTS 6.5

‍