Big Data Analytics

SOEN 471/6111
Closed
Concordia University
Montreal, Quebec, Canada
tristan glatard
Associate Professor
(1)
2
Timeline
  • January 18, 2023
    Experience start
  • April 15, 2023
    Experience end
Experience
5/5 project matches
Dates set by experience
Preferred companies
Montreal, Quebec, Canada
Any
Any industries
Categories
Machine learning Artificial intelligence Data analysis Data modelling Data science
Skills
python data analytics research
Learner goals and capabilities

A team of 3-5 students will implement a data-science project using Big Data technologies Apache Spark, Dask or scikit-learn.

Learners
Graduate
Any level
150 learners
Project
90 hours per learner
Learners self-assign
Teams of 4
Expected outcomes and deliverables
  • Project summary: The project summary will be a 400-word abstract available as a Markdown (.md) document in a public or private GitHub repository. The summary will report on project definition and model design. It will describe the dataset used in the project and its main characteristics (number and type of features), the research questions to be addressed in the project, the class of models to be applied to the dataset, and the algorithms that will be used. At least two algorithms must be used and compared.
  • Project data model: The project data model will be delivered as a Jupyter notebook containing code and explanations to implement data preparation, model training and preliminary model evaluation.
  • Final project presentation: The final project presentation will go through the final Jupyter notebook implemented for the project, putting special emphasis on model evaluation and summarizing the other project milestones.
Project timeline
  • January 18, 2023
    Experience start
  • April 15, 2023
    Experience end
Project Examples

In this assignment, students will work on a dataset to answer specific exploratory questions by applying one or more techniques seen in class: supervised learning, recommender systems, unsupervised clustering, frequent itemset mining, data stream analytics, graph analysis, and similarity search. Students will implement the project in Python, using Jupyter notebooks and a data analytics library among Apache Spark, Dask or scikit-learn.

As a participating organization, you’ll be asked to provide a particular dataset and a first set of related questions to be answered by the team using the dataset.

The expected project milestones are as follows:

  1. Project definition: students will summarize the project, including: (1) the dataset of interest, (2) the set of exploratory questions to be answered with the dataset, using techniques studied in class.
  2. Model design: students will choose a class of models in {supervised learning, recommender systems, unsupervised clustering, frequent itemset mining, data stream analytics, graph analysis, similarity search}. They will outline how the data model could be applied to the dataset to answer the exploratory question(s). They will research algorithms and techniques to implement this class of model.
  3. Data preparation: students will inspect the dataset, identify missing data, outliers, data types (categorical data in particular), and write Apache Spark or Dask programs to correct for potential issues.
  4. Model implementation: students will implement the model with Apache Spark, Dask or scikit-learn.
  5. Model evaluation: students will identify evaluation metrics for the model, implement, and discuss them.
Companies must answer the following questions to submit a match request to this experience:

The proposed project includes a dataset that the students will be able to access and analyze.