# Labs

- at least 50% homework to get the credit
- we will get the feedback approximately a week after the deadline (or later)
- we should create the merge request before the deadline
	- (it is okay to make a tiny change after the deadline if we found a bug)
- we should run the automated tests before we create the merge request
	- the tests should not crash, they may not pass
- first assignment
	- we can switch the domain later
	- but we should imagine two domains
	- at least one of them should be our preferred
	- we want a **task-oriented** system
		- anything that operates on top of a database or API
	- we don't need to focus on parsing errors (or misunderstanding) in the flowchart
	- we can use anything as the backend – even a local file
- second assignment
	- look at the data in `data/hw2` (only the train data?)
	- write a script that separates the user and the system turns
	- some search calls (after silence) will be parts of the text
		- we ignore only the lines without tabs
	- don't forget to comment the results
	- we should run the tests locally
- third assignment
	- NLU, just understanding, no reply
	- we will install dialmonkey using pip (see readme)
		- it's an empty shell
	- there is a config file
		- there are some components
	- jupyter notebook with demo snippets
	- pick one domain
	- implement ruse-based NLU
		- there are restaurant examples we can look at
		- we need to update our repo from the upstream
	- we should create a config file – we can start with the sample config and replace the dummy NLU with our own implementation
	- we should test it (write 15 test utterances)
		- input on the left, dialogue act on the right
		- we should separate it by a real `tab` character
		- there can be multiple intents on the right (with `&` separating them)
- fourth assignment
	- we won't work with our domain, we will use DSTC2 restaurant data
	- for each sentence, there is a DA annotation (sentence-level, not token-level)
	- our goal is to do sentence-level classification (no need for sequence analysis)
	- idea: for each intent-slot pair we should train a classifier (~ 50 classifiers total)
		- the goal is to make it work, we don't have to follow that strictly
	- we should put the classification results back together into DAs
	- there's an evaluation script
	- then we should set it up so that we can use it to chat
- fifth assignment
	- belief tracker
	- so far we've been filling dial.nlu
	- now, we need to fill dial.state dictionary … key = slot, value = dict (value → probability)
		- `dial.state = {'price': {'cheap': 0.2, 'moderate': 0.5, 'expensive': 0.1, None: 0.2}, 'area': {'north': 0.5, 'east': 0.1, None: 0.4}}`
		- after each turn we update the values with probabilities from dial.nlu
		- (initially: `dial.state = {'price': {None: 1.0}, 'area': {None: 1.0}}`)
	- the probabilities should sum up to one! (it's a job of NLU to assure that)
	- in hw4 we don't need to set the probabilities (but we can do that)
	- in hw5 the NLU needs to set the probabilities (in sklearn, predict_proba)
- assignments 6 & 7
	- implement the rule-based policy
	- we should take the instructions about the policy with a grain of salt
	- we can merge hw3, we should not delete the branch
	- the database can be CSV table, SQLite database, …
- hw 2
	- how to count the bigrams?
- assignment 8
	- we don't need to optimize the number of used templates (in the third requirement) – if we do not find the match, we can go one by one
	- load the upstream changes into the repository, there is a reference implementation we can use