https://github.com/andy-joy-25/cs-552-modern-nlp
Repository for EPFL's CS-552: Modern NLP class
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (4.7%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
Repository for EPFL's CS-552: Modern NLP class
Basic Info
- Host: GitHub
- Owner: andy-joy-25
- Language: Jupyter Notebook
- Default Branch: main
- Homepage: https://nlp.epfl.ch/cs-552-modern-nlp/
- Size: 187 MB
Statistics
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
- Releases: 0
Fork of epfl-nlp/cs-552-modern-nlp
Created over 2 years ago
· Last pushed almost 2 years ago
https://github.com/andy-joy-25/cs-552-modern-nlp/blob/main/
# CS-552: Modern Natural Language Processing ### Course Description Natural language processing is ubiquitous in modern intelligent technologies, serving as a foundation for language translators, virtual assistants, search engines, and many more. In this course, we cover the foundations of modern methods for natural language processing, such as word embeddings, recurrent neural networks, transformers, and pretraining, and how they can be applied to important tasks in the field, such as machine translation and text classification. We also cover issues with these state-of-the-art approaches (such as robustness, interpretability, sensitivity), identify their failure modes in different NLP applications, and discuss analysis and mitigation techniques for these issues. #### Quick access links: - [Platforms](#class) - [Lecture Schedule](#lectures) - [Exercise Schedule](#exercises) - [Grading](#evaluation) - [Contact](#contact) ## Class | Platform | Where & when | |:------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------| | Lectures | **Wednesdays: 11:15-13:00** [[STCC - Cloud C](https://plan.epfl.ch/?room=%3DSTCC%20-%20Cloud%20C&dim_floor=0&lang=en&dim_lang=en&tree_groups=centres_nevralgiques%2Cmobilite_acces_grp%2Censeignement%2Ccommerces_et_services&tree_group_layers_centres_nevralgiques=information_epfl%2Cguichet_etudiants&tree_group_layers_mobilite_acces_grp=metro&tree_group_layers_enseignement=&tree_group_layers_commerces_et_services=&baselayer_ref=grp_backgrounds&map_x=2532938&map_y=1152803&map_zoom=11)] & **Thursdays: 13:15-14:00** [[CE16](https://plan.epfl.ch/?room=%3DCE%201%206&dim_floor=1&lang=en&dim_lang=en&tree_groups=centres_nevralgiques%2Cmobilite_acces_grp%2Censeignement%2Ccommerces_et_services&tree_group_layers_centres_nevralgiques=information_epfl%2Cguichet_etudiants&tree_group_layers_mobilite_acces_grp=metro&tree_group_layers_enseignement=&tree_group_layers_commerces_et_services=&baselayer_ref=grp_backgrounds&map_x=2533400&map_y=1152502&map_zoom=13)] | | Exercises Session | **Thursdays: 14:15-16:00** [[CE11](https://plan.epfl.ch/?room=%3DCE%201%201&dim_floor=1&lang=en&dim_lang=en&tree_groups=centres_nevralgiques%2Cmobilite_acces_grp%2Censeignement%2Ccommerces_et_services&tree_group_layers_centres_nevralgiques=information_epfl%2Cguichet_etudiants&tree_group_layers_mobilite_acces_grp=metro&tree_group_layers_enseignement=&tree_group_layers_commerces_et_services=&baselayer_ref=grp_backgrounds&map_x=2533297&map_y=1152521&map_zoom=13)] | | Project Assistance## Exercise Schedule | Week | Date | Topic | Instructor | |:------------|:--------|:--------------------------------------------------------------------------------------|:-------------------------------------------------------------------:| | **Week 1** | 22 Feb | Setup + Word embeddings [[code][1e]] | Mete Ismayilzada | | | | | | | **Week 2** | 29 Feb | Word embeddings review
(not every week) | **Wednesdays: 13:15-14:00** [[STCC - Cloud C](https://plan.epfl.ch/?room=%3DSTCC%20-%20Cloud%20C&dim_floor=0&lang=en&dim_lang=en&tree_groups=centres_nevralgiques%2Cmobilite_acces_grp%2Censeignement%2Ccommerces_et_services&tree_group_layers_centres_nevralgiques=information_epfl%2Cguichet_etudiants&tree_group_layers_mobilite_acces_grp=metro&tree_group_layers_enseignement=&tree_group_layers_commerces_et_services=&baselayer_ref=grp_backgrounds&map_x=2532938&map_y=1152803&map_zoom=11)] | | QA Forum & Annoucements | Ed Forum [[link](https://edstem.org/eu/courses/1159/discussion/)] | | Grades | Moodle [[link](https://moodle.epfl.ch/course/view.php?id=17143)] | All lectures will be given in person and live streamed on Zoom. The link to the Zoom is available on the Ed Forum (pinned post). Beware that, in the event of a technical failure during the lecture, continuing to accompany the lecture live via zoom might not be possible. Recording of the lectures will be made available on SwitchTube. We will reuse some of last year's recordings and we may record a few new lectures in case of different lecture contents. ## Lecture Schedule
Language and Sequence-to-sequence models [[code][2e]] | Mete Ismayilzada
Badr AlKhamissi | | | | | | | **Week 3** | 6 Mar | Assignment 1 Q&A | Mete Ismayilzada | | **Week 3** | 7 Mar | Language and Sequence-to-sequence models review
Attention + Transformers [[code][3e]] | Badr AlKhamissi | | | | | | | **Week 4** | 13 Mar | **\[Online only\]** Pretraining S2S: BART, T5 [[slides][4s]] | Antoine Bosselut | | **Week 4** | 14 Mar | Attention + Transformers review
Pretraining and Transfer Learning Pt. 1 [[code][4e]] | Badr AlKhamissi
Simin Fan | | | | | | | **Week 5** | 20 Mar | No lecture | - | | **Week 5** | 21 Mar | Pretraining and Transfer Learning Pt. 1 review
Transfer Learning Pt. 2 [[code][5e]] | Simin Fan | | | | | | | **Week 6** | 27 Mar | Assignment 2 Q&A | Simin Fan, Silin Gao | | **Week 6** | 28 Mar | Transfer Learning Pt. 2 review
Text Generation & Assignment 2 Q&A [[code][6e]] | Simin Fan
Deniz Bayazit, Silin Gao | | | | | | | | | ***EASTER BREAK*** | | | | | | | | **Week 7** | 10 Apr | Assignment 3 Q&A | Badr AlKhamissi
Deniz Bayazit | | **Week 7** | 11 Apr | Text Generation review
In-context Learning [[code][7e]] | Deniz Bayazit
Mete Ismayilzada | | | | | | | **Week 8** | 17 Apr | No lecture | - | | **Week 8** | 18 Apr | Assignment 3 Q&A
A1 Grade Review Session | Badr AlKhamissi
Deniz Bayazit
Mete Ismayilzada | | **Week 9** | 24 & 25 Apr | Project | TA meetings on-demand | | | | | | | **Week 10** | 1 & 2 May | Project | TA meetings on-demand | | | | | | | **Week 11** | 8 & 9 May | Project
Milestone 1 Feedback | TA meetings on-demand | | | | | | | **Week 12** | 15 & 16 May | Project | TA meetings on-demand | | | | | | | **Week 13** | 22 May | Project | Badr AlKhamissi
Deniz Bayazit | | | | | | | **Week 13** | 23 May | A3 Grade Review Session | TA meetings on-demand | | | | | | | **Week 14** | 30 May | Project
Milestone 2 Feedback | TA meetings on-demand | ### Exercises Session format: - TAs will provide a small discussion over the **last week's exercises**, answering any questions and explaining the solutions. _(10-15mins)_ - TAs will present **this week's exercise**. _(5mins)_ - Students will be solving this week's exercises and TAs will provide answers and clarification if needed. _**Note**: Please make sure you have already done the setup prerequisites to run the coding parts of the exercises. You can find the instructions [here][0e]._ ## Grading: Your grade in the course will be computed according to the following guidelines. ### Submission Format Assignment and project release annoucements will be on Ed. Your work will be submitted as a repository created by [GitHub classroom](https://classroom.github.com/). Clicking the assignment link (announced on its release date) will automatically create a repository under your username (ensure it matches the one on the CS-552 GitHub registration form). Your last push to the repository will be considered as your final submission, with its timestamp determining any late days (see below for the policy). All large files such as model checkpoints need to be pushed to the repository with [Git LFS](https://git-lfs.com/). Large files can take time to upload, therefore please avoid last-minute uploads that can create potential submission delays. We also propose to use [Colab](https://colab.research.google.com/) as a free GPU resource. You can find tutorials on all of these resources [here][0t]. ### Late Days Policy All assignments and milestones are due at 23:59 on their due date. As we understand that circumstances can make it challenging to abide by these due dates, you will receive 7 late days over the course of the semester to be allocated to the assignments and project milestones as you see fit. No further extensions will be granted. The only exception to this rule is for the final report, code, and data. No extensions will be granted beyond June 14th. We will automatically calculate the late days according to your last commit; hence you don't have to inform us. For group projects, when everyone has some late days, we will deduct individually from everyone. In the scenario where one person has no more late days, that student will lose points for the late submission. The other students in the team will continue to use their late days (i.e. no points will be deducted from them). After you have used all your allotted late days, the penalty policy is a 25% deduction of the grade per day. ### Assignments (40%): There will be three assignments throughout the course. They will be released and due according to the following schedule: #### Assignment 1 (10%) - Released: 28 February 2024 - Due: 17 March 2024 - Grade released: 14 April 2024 - Grade review sessions: 18 and 25 April 2024 #### Assignment 2 (15%) - Released: 20 March 2024 - Due: 7 April 2024 - Grade released: 5 May 2024 - Grade review sessions: 9 and 16 May 2024 #### Assignment 3 (15%) - Released: 3 April 2024 - Due: 21 April 2024 - Grade released: 19 May 2024 - Grade review sessions: 29 and 30 May 2024 ### Project (60%): The project will be divided into 2 milestones and a final submission. Each milestone will be worth 15% of the final grade with the remaining 30% being allocated to the final report. Each team will be supervised by one of the course TAs or AEs. More details on the content of the project and the deliverables of each milestone will be released at a later date. #### Milestone 1: - Due: 5 May 2024 #### Milestone 2: - Due: 28 May 2024 #### Final Deliverable: - The final report, code, and date will be due on June 14th. Students are welcome to turn in their materials ahead of time, as soon as the semester ends. - Due: 14 June 2024 ## Contacts Please email us at **nlp-cs552-spring2024-ta-team [at] groupes [dot] epfl [dot] ch** for any administrative questions, rather than emailing TAs individually. All course content questions need to be asked via [Ed](https://edstem.org/eu/courses/1159/discussion/). **Lecturer**: [Antoine Bosselut](https://people.epfl.ch/antoine.bosselut) **Teaching assistants**: [Negar Foroutan Eghlidi](https://people.epfl.ch/negar.foroutan), [Badr AlKhamissi](https://people.epfl.ch/badr.alkhamissi), [Deniz Bayazit](https://people.epfl.ch/deniz.bayazit?lang=en), [Beatriz Borges](https://people.epfl.ch/beatriz.borges), [Zeming (Eric) Chen](https://people.epfl.ch/zeming.chen?lang=en), [Simin Fan](https://people.epfl.ch/simin.fan?lang=en), [Silin Gao](https://people.epfl.ch/silin.gao?lang=en), [Mete Ismayilzada](https://people.epfl.ch/mahammad.ismayilzada) [0t]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/tutorials.md [0e]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/Setup [1e]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/Week%201%20-%20Word%20Embeddings [2e]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/Week%202%20-%20N-gram%20%26%20Neural%20Language%20Models [3e]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/Week%203%20-%20RNNs [4e]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/Week%204%20-%20Pretraining%20%26%20Finetuning [5e]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/Week%205%20-%20Biases%20%26%20Prompting [6e]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/Week%206%20-%20Text%20Generation [7e]:https://github.com/epfl-nlp/cs-552-modern-nlp/tree/main/Exercises/Week%207%20-%20In-context%20Learning
Owner
- Name: Anand Choudhary
- Login: andy-joy-25
- Kind: user
- Repositories: 2
- Profile: https://github.com/andy-joy-25