Labs

• at least 50% homework to get the credit
• we will get the feedback approximately a week after the deadline (or later)
• we should create the merge request before the deadline
  • (it is okay to make a tiny change after the deadline if we found a bug)
• we should run the automated tests before we create the merge request
  • the tests should not crash, they may not pass
• first assignment
  • we can switch the domain later
  • but we should imagine two domains
  • at least one of them should be our preferred
  • we want a task-oriented system
    • anything that operates on top of a database or API
  • we don't need to focus on parsing errors (or misunderstanding) in the flowchart
  • we can use anything as the backend – even a local file
• second assignment
  • look at the data in data/hw2 (only the train data?)
  • write a script that separates the user and the system turns
  • some search calls (after silence) will be parts of the text
    • we ignore only the lines without tabs
  • don't forget to comment the results
  • we should run the tests locally
• third assignment
  • NLU, just understanding, no reply
  • we will install dialmonkey using pip (see readme)
    • it's an empty shell
  • there is a config file
    • there are some components
  • jupyter notebook with demo snippets
  • pick one domain
  • implement ruse-based NLU
    • there are restaurant examples we can look at
    • we need to update our repo from the upstream
  • we should create a config file – we can start with the sample config and replace the dummy NLU with our own implementation
  • we should test it (write 15 test utterances)
    • input on the left, dialogue act on the right
    • we should separate it by a real tab character
    • there can be multiple intents on the right (with & separating them)
• fourth assignment
  • we won't work with our domain, we will use DSTC2 restaurant data
  • for each sentence, there is a DA annotation (sentence-level, not token-level)
  • our goal is to do sentence-level classification (no need for sequence analysis)
  • idea: for each intent-slot pair we should train a classifier (~ 50 classifiers total)
    • the goal is to make it work, we don't have to follow that strictly
  • we should put the classification results back together into DAs
  • there's an evaluation script
  • then we should set it up so that we can use it to chat
• fifth assignment
  • belief tracker
  • so far we've been filling dial.nlu
  • now, we need to fill dial.state dictionary … key = slot, value = dict (value → probability)
    • dial.state = {'price': {'cheap': 0.2, 'moderate': 0.5, 'expensive': 0.1, None: 0.2}, 'area': {'north': 0.5, 'east': 0.1, None: 0.4}}
    • after each turn we update the values with probabilities from dial.nlu
    • (initially: dial.state = {'price': {None: 1.0}, 'area': {None: 1.0}})
  • the probabilities should sum up to one! (it's a job of NLU to assure that)
  • in hw4 we don't need to set the probabilities (but we can do that)
  • in hw5 the NLU needs to set the probabilities (in sklearn, predict_proba)
• assignments 6 & 7
  • implement the rule-based policy
  • we should take the instructions about the policy with a grain of salt
  • we can merge hw3, we should not delete the branch
  • the database can be CSV table, SQLite database, …
• hw 2
  • how to count the bigrams?
• assignment 8
  • we don't need to optimize the number of used templates (in the third requirement) – if we do not find the match, we can go one by one
  • load the upstream changes into the repository, there is a reference implementation we can use