PySpark-Tutorial provides basic algorithms using PySpark

Implementation of Spark code in Jupyter notebook. Topics include: RDDs and DataFrame, exploratory data analysis (EDA), handling multiple DataFrames, visualization, Machine Learning

A big data project to develop a real-time data pipeline for analyzing the popularity and sentiments of trending topics on Twitter.

Getting started with PySpark for Big data analysis

Efficiently tackle large datasets and perform big data analysis with Spark and Python

MapReduce Job Development, RDDs Programming, Medical Data Management, Sales Analysis, And Efficient Data Integration For Big Data Analysis. Spark: Big Data Processing, SQOOP Integration, And Spark Structured Streaming For Real-Time Data.

This repository is part of my journey to learn **PySpark**, the Python API for Apache Spark. I explored the fundamentals of distributed data processing using Spark and practiced with real-world data transformation and querying use cases.

📚 Master PySpark in 18 days with structured lessons, hands-on tasks, and an end-to-end project, covering essential concepts and ML model training.

Here I play with the services offered by Apache Spark and try to learn them in more depth.

Data Mining using Spark Rdds

Spark, RDDs and Map Reduce applications related to the BigData @polito course (2019-2020). A set of personal notes are already provided.

Project on MapReduce for the Μ111 - Big Data Management course, NKUA, Spring 2023.

Analysis of Clinical Trial Dataset using PySpark RDD implementation.

This project illustrates Apache Spark RDD operations, from creation and transformation to actions and results, enhancing users' understanding of distributed data processing.

📈📊 Big Data Notebooks . ▫️ Análisis masivos de datos con pyspark ▫️ Ingesta de datos. ▫️ Algoritmos de machine learning con datos masivos. ▫️ Procesamiento de mensajes en tiempo real con Kafka.