From f2255db77661c312fa29107a4292eb5b31b08a16 Mon Sep 17 00:00:00 2001 From: Pavan Mandava Date: Tue, 15 Nov 2022 10:55:57 +0100 Subject: [PATCH] Fixed README formatting issues --- README.md | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 51a774f..3a13d33 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ Create an environment for baseline training with a specific python version (Pyth ```shell conda create -n python=3.6 ``` + ### Create conda environment (for prompt learning) Create an environment for prompt-based methods (Python 3.10 is **required**) ```shell @@ -48,6 +49,7 @@ To deactivate the conda environment, run: (Only after running all the experiment ```shell conda deactivate ``` + #### Download and extract SOLOIST pre-trained model Download and unzip the pretrained model, this is used for fine-tuning the baseline and prompt-based methods. For more details about the pre-trained SOLOIST model, refer to the GitHub [repo](https://github.com/pengbaolin/soloist). @@ -66,10 +68,12 @@ Clone the repository for source code ```shell git clone https://git.pavanmandava.com/pavan/master-thesis.git ``` + Pull the changes from remote (if local is behind the remote) ```shell git pull ``` + Change directory ```shell cd master-thesis @@ -120,7 +124,8 @@ pip install requirements.txt Train a separate model for each data split. Edit the [train_baseline.sh](baseline/train_baseline.sh) file to modify the hyperparameters while training (learning rate, epochs). Use `CUDA_VISIBLE_DEVICES` to specify a CUDA device (GPU) for training the model. ```shell sh train_baseline.sh -d -``` +``` + Pass the data split name to `-d` flag. Possible values are: `5-dpd`, `10-dpd`, `50-dpd`, `100-dpd`, `125-dpd`, `250-dpd` Example training command: `sh train_baseline.sh -d 50-dpd` @@ -131,13 +136,15 @@ Choose a checkpoint of the saved baseline model to generate belief state predict Set the `MODEL_CHECKPOINT` environment variable with the path to the chosen model checkpoint. It should only contain the path from the "experiment-{datetime}" folder. ```shell export MODEL_CHECKPOINT=// -``` +``` + Example: `export MODEL_CHECKPOINT=experiment-20220831/100-dpd/checkpoint-90000` Generate belief states by running decode script ```shell sh decode_baseline.sh ``` + The generated predictions are saved under `OUTPUTS_DIR_BASELINE` folder. Some of the generated belief state predictions are uploaded to this repository and can be found under [outputs](outputs) folder. ### Baseline Evaluation @@ -147,7 +154,8 @@ The standard Joint Goal Accuracy (JGA) is used to evaluate the belief prediction Edit the [evaluate.py](baseline/evaluate.py) to set the predictions output file before running the evaluation ```shell python evaluate.py -``` +``` + ### Results from baseline experiments |data-split| JGA | |--|:--:| @@ -171,21 +179,27 @@ Change directory to `prompt-learning` and install the requirements. Make sure th ```shell cd prompt-learning pip install requirements.txt -``` +``` + ### Train the prompt model Train a separate model for each data split. Edit the [train_prompting.sh](prompt-learning/train_prompting.sh) file to modify the default hyperparameters for training (learning rate, epochs). ```shell sh train_prompting.sh -d -``` +``` + Pass the data split name to `-d` flag. Possible values are: `5-dpd`, `10-dpd`, `50-dpd`, `100-dpd`, `125-dpd`, `250-dpd` Example training command: `sh train_baseline.sh -d 50-dpd` **Some `train_prompting.sh` flags**: + `--num_epochs 10` - Number of epochs + `--learning_rate 5e-5` - Initial learning rate for Optimizer + `--with_inverse_prompt` - Use Inverse Prompt while training + `--inverse_prompt_weight 0.1` - Weight of the inverse prompt for loss function **Note:** The defaults in `train_prompting.sh` are the best performing values. @@ -197,6 +211,7 @@ Generate belief states by running the below script: ```shell sh test_prompting.sh -m ``` + The argument `-m` takes the relative path of saved model from `SAVED_MODELS_PROMPT` env variable. It takes the following structure `-m //` Example: `sh test_prompting.sh -m 50-dpd/experiment-20221003T172424/epoch-09` @@ -210,6 +225,7 @@ Edit the [evaluate.py](prompt-learning/evaluate.py) to set the predictions outpu ```shell python evaluate.py ``` + ### Results from prompt-based belief state generations |data-split| JGA* | |--|:--:| @@ -241,4 +257,5 @@ Generate belief states by running the below script: ```shell sh test_prompting.sh -m ``` + // TODO :: Add results