CSCI 544 — Applied Natural Language Processing

Spring 2017

There will be no office hours on January 16 (Martin Luther King’s Birthday), February 20 (Presidents’ Day), or March 12–19 (Spring Recess). There will also be no office hours on April 10 because code demonstrations are being held that day.

Synopsis

This course covers both fundamental and cutting-edge topics in Natural Language Processing (NLP) and provides students with hands-on experience in NLP applications.

This graduate course is intended for:

• students who want to understand current NLP technologies
• students interested in building NLP applications
• students interested in applications of NLP like sentiment analysis, dialogue systems, question answering systems, among others

Recommended preparation: Proficiency in programming, algorithms and data structures, basic knowledge of linear algebra and machine learning.

Related Courses

This course is part of USC’s curriculum in natural language processing. There is a sister course, CSCI 662 Advanced Natural Language Processing, offered in the Fall semester, which covers complementary (and advanced) material and is intended for PhD students (or students who want to continue to a PhD program).

Coursework and Grading

• Homework assignments (8 × 5%): A mix of programming assignments and written exercises.
• Reading quizzes (10 × 1%): Short, web-based quizzes relating to reading assignments.
• Midterm exam (10%): In-class exam with questions similar to those from the homework assignments and quizzes.
• Final exam (10%): In-class exam with questions similar to those from the homework assignments and quizzes.
• Research project (30%): Group project, graded on the project’s relevance to the course, degree of difficulty, amount of work, correctness, and written report.

Grading scale

The following scale is used for determining final grades (note that A is the highest grade given by USC).

• A: 92%, A–: 90%, B+: 87%, B: 82%, B–: 80%, C+: 77%, C: 72%, C–: 70%, D+: 67%, D: 62%, D–: 60%

Grade challenges

• Grades on an assignment will only be changed if there is an error in grading; a student who wishes to challenge a grade must identify the grading error before asking for a grade change.
• Students are welcome to discuss any aspect of the homework assignments, but there will be no negotiation on grades, and no changes other than the correction of grading errors.

Late Policy

• There will be a penalty for turning in homework late; the penalty will vary by assignment. In no instance will credit be given to a homework assignment submitted after the solution has been discussed in class.
• Reading quizzes are intended to make sure that students are prepared for class, so no late quizzes will be accepted. (One exception: students who registered late will take the first quiz after they have registered.)
• Homework assignments will not be accepted by email. If there are technical or other issues with the submission system you should write to us and we will work to fix these issues, but do not send homework by email just because you weren’t able to submit it through the system.
• Homework is usually due at the end of the day. We may not be available to solve issues close to the deadline, so you should plan on submitting your homework early, even if only as a draft. Multiple submissions are generally allowed, and the last submission will be graded. Quizzes may only be submitted once.

Communication

Please use the class discussion boards on Piazza for questions and issues regarding homework assignments and the course in general. This way, the entire class can participate and see the questions and answers. Email should be reserved for communication of a personal nature. If we receive questions by email where the response could be helpful for the class, we may ask you to repost the question on the discussion boards.

Resources

• Blackboard (for reading quizzes and class announcements)
• Piazza (for class discussion boards)
• Vocareum (for coding assignments)
• Crowdmark (for written assignments)

Schedule

Note: The weeks of January 16 and February 20 are instructional weeks. Class will not be held on these days because they are university holidays, but work will be assigned for the week and is due at the appropriate time.

Topics listed in the schedule are tentative and subject to change.

• January 9: Introduction and basic concepts; Naive Bayes.
  • The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
  • UTF-8, a transformation format of ISO 10646 (You only need to refer to section 3, “UTF-8 definition”).
• January 16: No class (MLK Birthday)
  • The first homework assignment is due January 20.
    • All students who were in the Blackboard system by January 11 should have received a personalized link by email in the evening of January 11; if you didn’t, please check your email junk/spam settings, and if you still can’t find the email, please write to Ramesh.
    • Students who registered on or after January 12 will receive the email link after they are added to Blackboard (with some delay: we have to manually get the new registrations and send links).
• January 23: Text classification: generative and discriminative models.
  • Manning, Raghavan and Schutze, Introduction to information retrieval, Chapter 13: Text classification and Naive Bayes.
• January 30: Perceptron and linear classifiers; project discussion.
  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Chapter 7: Logistic Regression.
  • Hal Daumé III, A Course in Machine Learning (v. 0.99 draft), Chapter 4: The Perceptron.
  • The second homework assignment is due February 1.
• February 6: Part-of-speech tagging and sequence labeling.
  • Brill (1992): A Simple Rule-Based Part of Speech Tagger
  • Ratnaparkhi (1996): A Maximum Entropy Model for Part-Of-Speech Tagging
• February 13: Discriminative sequence labeling (conditional random fields).
  • Lafferty, McCallum and Pereira (2001): Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
  • Morency (2016): Modeling Human Communication Dynamics (video; concentrate on 0:30–0:46)
  • The third homework assignment is due February 15. All students should have received a personalized link by email; if you didn’t, please check your email junk/spam settings, and if you still can’t find the email, please write to Ramesh.
• February 20: No class (Presidents’ day)
  • Research project proposals due February 22.
  • The fourth homework assignment is due February 26. All students should have received a personalized link by email; if you didn’t, please check your email junk/spam settings, and if you still can’t find the email, please write to Ramesh.
• February 27: Language modeling and speech recognition.
  • Young (1996): A review of large-vocabulary continuous-speech recognition. This link should allow free download when accessed from the USC network.
• March 6: Midterm; Annotation and evaluation.
  • The fifth homework assignment is due March 11.
• March 13: Spring Recess
• March 20: Syntax (parsing).
  • Charniak (1997): Statistical Techniques for Natural Language Parsing
  • Jurafsky and Martin, Speech and Language Processing (3rd edition draft), Chapter 14: Dependency Parsing.
• March 27: Semantics, discourse, coreference.
  • Mikolov, Yih and Zweig (2013): Linguistic Regularities in Continuous Space Word Representations
  • Levy, Goldberg and Dagan (2015): Improving Distributional Similarity with Lessons Learned from Word Embeddings.
  • The sixth homework assignment is due April 1. All students should have received a personalized link by email; if you didn’t, please check your email junk/spam settings, and if you still can’t find the email, please write to Ramesh.
• April 3: Reinforcement learning (Georgila), Educational applications (Core)
  • Levin, Pieraccini and Eckert (2000): A Stochastic Model of Human-Machine Interaction for Learning Dialog Strategies
• April 10: Dialogue.
  • Lee and Stent (2016): Task Lineages: Dialog State Tracking for Flexible Interaction (Sigdial 2016 best paper; a video of the talk is also available)
  • Research project code demonstrations will take place on April 10 in PHE 516.
  • The seventh homework assignment is due April 16. All students should have received a personalized link by email; if you didn’t, please check your email junk/spam settings, and if you still can’t find the email, please write to Ramesh.
• April 17: Machine translation.
  • Sutskever, Vinyals and Le (2014): Sequence to Sequence Learning with Neural Networks
• April 24: Langauge generation.
  • The eighth homework assignment is due April 27.
  • Research project reports due April 28.
• May 8 at 4:30pm: Final Exam

Statement on Academic Conduct and Support Systems

Academic Conduct
Plagiarism – presenting someone else’s ideas as your own, either verbatim or recast in your own words – is a serious academic offense with serious consequences. Please familiarize yourself with the discussion of plagiarism in SCampus in Part B, Section 11, “Behavior Violating University Standards” https://policy.usc.edu/student/scampus/part-b. Other forms of academic dishonesty are equally unacceptable. See additional information in SCampus and university policies on scientific misconduct, http://policy.usc.edu/scientific-misconduct.

Discrimination, sexual assault, intimate partner violence, stalking, and harassment are prohibited by the university. You are encouraged to report all incidents to the Office of Equity and Diversity/Title IX Office http://equity.usc.edu and/or to the Department of Public Safety http://dps.usc.edu. This is important for the health and safety of the whole USC community. Faculty and staff must report any information regarding an incident to the Title IX Coordinator who will provide outreach and information to the affected party. The sexual assault resource center webpage http://sarc.usc.edu fully describes reporting options. Relationship and Sexual Violence Services https://engemannshc.usc.edu/rsvp provides 24/7 confidential support.

Support Systems
A number of USC’s schools provide support for students who need help with scholarly writing. Check with your advisor or program staff to find out more. Students whose primary language is not English should check with the American Language Institute http://ali.usc.edu, which sponsors courses and workshops specifically for international graduate students. The Office of Disability Services and Programs http://dsp.usc.edu provides certification for students with disabilities and helps arrange the relevant accommodations. If an officially declared emergency makes travel to campus infeasible, USC Emergency Information http://emergency.usc.edu will provide safety and other updates, including ways in which instruction will be continued by means of Blackboard, teleconferencing, and other technology.

USC Viterbi Honor Code

The Code was developed by Viterbi students, and its text is as follows:

Engineering enables and empowers our ambitions and is integral to our identities. In the Viterbi community, accountability is reflected in all our endeavors.

• Engineering+ Integrity.
• Engineering+ Responsibility.
• Engineering+ Community.
• Think good. Do better. Be great.

These are the pillars we stand upon as we address the challenges of society and enrich lives.

Academic Integrity Violations

All coding and writing must be done individually (unless instructed otherwise), and not copied from other students. Copying or plagiarism is grounds for failure of an assignment, or in serious cases failure of the course.

Use of the internet or other outside resources to find solutions to homework problems is considered cheating.

Please read my Personal note on academic integrity.