Welcome to my GitHub profile! I am Muhammed Musharaf, a passionate Big Data Engineer, specializing in building scalable data platforms. With expertise in real-time data processing, batch ETL, and modern data architecture, I love turning data into actionable insights using cutting-edge technologies.
My GitHub repositories showcase my contributions to the world of data engineering. Feel free to explore my projects and open-source contributions.
Explore some of my key projects below:
- Ecomm Analytics Pipeline - Real-time analytics for e-commerce using Kafka, Spark Streaming, and Hudi.
- Lakehouse from Scratch - A full-stack lakehouse setup with Delta Lake, Hive, and Trino.
- ML Feature Pipeline - A pipeline for ML feature engineering and consistency with Feast.
- COVID Batch ETL - Ingesting public health data with Airflow, Spark, and Delta Lake.
You can find more of my projects in my DataMush Portfolio.
Connect with me on various platforms:
I'm constantly learning new skills and obtaining certifications:
Some of the technologies I work with include:
- Data Engineering: Apache Kafka, Apache Spark, Delta Lake, Hudi, Trino, Airflow
- Databases: PostgreSQL, MySQL, MongoDB
- Machine Learning: MLlib, TensorFlow, Scikit-learn, Feast
- DevOps: Docker, Kubernetes, CI/CD
- Data Visualization: Superset, Grafana, Tableau
Feel free to reach out for collaborations, mentorship, or just a casual chat about tech.