Fixed README formatting issues

main
Pavan Mandava 3 years ago
parent 287ec9164d
commit f2255db776

@ -31,6 +31,7 @@ Create an environment for baseline training with a specific python version (Pyth
```shell ```shell
conda create -n <baseline-env-name> python=3.6 conda create -n <baseline-env-name> python=3.6
``` ```
### Create conda environment (for prompt learning) ### Create conda environment (for prompt learning)
Create an environment for prompt-based methods (Python 3.10 is **required**) Create an environment for prompt-based methods (Python 3.10 is **required**)
```shell ```shell
@ -48,6 +49,7 @@ To deactivate the conda environment, run: (Only after running all the experiment
```shell ```shell
conda deactivate conda deactivate
``` ```
#### Download and extract SOLOIST pre-trained model #### Download and extract SOLOIST pre-trained model
Download and unzip the pretrained model, this is used for fine-tuning the baseline and prompt-based methods. For more details about the pre-trained SOLOIST model, refer to the GitHub [repo](https://github.com/pengbaolin/soloist). Download and unzip the pretrained model, this is used for fine-tuning the baseline and prompt-based methods. For more details about the pre-trained SOLOIST model, refer to the GitHub [repo](https://github.com/pengbaolin/soloist).
@ -66,10 +68,12 @@ Clone the repository for source code
```shell ```shell
git clone https://git.pavanmandava.com/pavan/master-thesis.git git clone https://git.pavanmandava.com/pavan/master-thesis.git
``` ```
Pull the changes from remote (if local is behind the remote) Pull the changes from remote (if local is behind the remote)
```shell ```shell
git pull git pull
``` ```
Change directory Change directory
```shell ```shell
cd master-thesis cd master-thesis
@ -120,7 +124,8 @@ pip install requirements.txt
Train a separate model for each data split. Edit the [train_baseline.sh](baseline/train_baseline.sh) file to modify the hyperparameters while training (learning rate, epochs). Use `CUDA_VISIBLE_DEVICES` to specify a CUDA device (GPU) for training the model. Train a separate model for each data split. Edit the [train_baseline.sh](baseline/train_baseline.sh) file to modify the hyperparameters while training (learning rate, epochs). Use `CUDA_VISIBLE_DEVICES` to specify a CUDA device (GPU) for training the model.
```shell ```shell
sh train_baseline.sh -d <data-split-name> sh train_baseline.sh -d <data-split-name>
``` ```
Pass the data split name to `-d` flag. Possible values are: `5-dpd`, `10-dpd`, `50-dpd`, `100-dpd`, `125-dpd`, `250-dpd` Pass the data split name to `-d` flag. Possible values are: `5-dpd`, `10-dpd`, `50-dpd`, `100-dpd`, `125-dpd`, `250-dpd`
Example training command: `sh train_baseline.sh -d 50-dpd` Example training command: `sh train_baseline.sh -d 50-dpd`
@ -131,13 +136,15 @@ Choose a checkpoint of the saved baseline model to generate belief state predict
Set the `MODEL_CHECKPOINT` environment variable with the path to the chosen model checkpoint. It should only contain the path from the "experiment-{datetime}" folder. Set the `MODEL_CHECKPOINT` environment variable with the path to the chosen model checkpoint. It should only contain the path from the "experiment-{datetime}" folder.
```shell ```shell
export MODEL_CHECKPOINT=<experiment-folder>/<data-split-name>/<checkpoint-folder> export MODEL_CHECKPOINT=<experiment-folder>/<data-split-name>/<checkpoint-folder>
``` ```
Example: `export MODEL_CHECKPOINT=experiment-20220831/100-dpd/checkpoint-90000` Example: `export MODEL_CHECKPOINT=experiment-20220831/100-dpd/checkpoint-90000`
Generate belief states by running decode script Generate belief states by running decode script
```shell ```shell
sh decode_baseline.sh sh decode_baseline.sh
``` ```
The generated predictions are saved under `OUTPUTS_DIR_BASELINE` folder. Some of the generated belief state predictions are uploaded to this repository and can be found under [outputs](outputs) folder. The generated predictions are saved under `OUTPUTS_DIR_BASELINE` folder. Some of the generated belief state predictions are uploaded to this repository and can be found under [outputs](outputs) folder.
### Baseline Evaluation ### Baseline Evaluation
@ -147,7 +154,8 @@ The standard Joint Goal Accuracy (JGA) is used to evaluate the belief prediction
Edit the [evaluate.py](baseline/evaluate.py) to set the predictions output file before running the evaluation Edit the [evaluate.py](baseline/evaluate.py) to set the predictions output file before running the evaluation
```shell ```shell
python evaluate.py python evaluate.py
``` ```
### Results from baseline experiments ### Results from baseline experiments
|data-split| JGA | |data-split| JGA |
|--|:--:| |--|:--:|
@ -171,21 +179,27 @@ Change directory to `prompt-learning` and install the requirements. Make sure th
```shell ```shell
cd prompt-learning cd prompt-learning
pip install requirements.txt pip install requirements.txt
``` ```
### Train the prompt model ### Train the prompt model
Train a separate model for each data split. Edit the [train_prompting.sh](prompt-learning/train_prompting.sh) file to modify the default hyperparameters for training (learning rate, epochs). Train a separate model for each data split. Edit the [train_prompting.sh](prompt-learning/train_prompting.sh) file to modify the default hyperparameters for training (learning rate, epochs).
```shell ```shell
sh train_prompting.sh -d <data-split-name> sh train_prompting.sh -d <data-split-name>
``` ```
Pass the data split name to `-d` flag. Pass the data split name to `-d` flag.
Possible values are: `5-dpd`, `10-dpd`, `50-dpd`, `100-dpd`, `125-dpd`, `250-dpd` Possible values are: `5-dpd`, `10-dpd`, `50-dpd`, `100-dpd`, `125-dpd`, `250-dpd`
Example training command: `sh train_baseline.sh -d 50-dpd` Example training command: `sh train_baseline.sh -d 50-dpd`
**Some `train_prompting.sh` flags**: **Some `train_prompting.sh` flags**:
`--num_epochs 10` - Number of epochs `--num_epochs 10` - Number of epochs
`--learning_rate 5e-5` - Initial learning rate for Optimizer `--learning_rate 5e-5` - Initial learning rate for Optimizer
`--with_inverse_prompt` - Use Inverse Prompt while training `--with_inverse_prompt` - Use Inverse Prompt while training
`--inverse_prompt_weight 0.1` - Weight of the inverse prompt for loss function `--inverse_prompt_weight 0.1` - Weight of the inverse prompt for loss function
**Note:** The defaults in `train_prompting.sh` are the best performing values. **Note:** The defaults in `train_prompting.sh` are the best performing values.
@ -197,6 +211,7 @@ Generate belief states by running the below script:
```shell ```shell
sh test_prompting.sh -m <tuned-prompt-model-path> sh test_prompting.sh -m <tuned-prompt-model-path>
``` ```
The argument `-m` takes the relative path of saved model from `SAVED_MODELS_PROMPT` env variable. It takes the following structure `-m <data-split-name>/<experiment-folder>/<epoch-folder>` The argument `-m` takes the relative path of saved model from `SAVED_MODELS_PROMPT` env variable. It takes the following structure `-m <data-split-name>/<experiment-folder>/<epoch-folder>`
Example: `sh test_prompting.sh -m 50-dpd/experiment-20221003T172424/epoch-09` Example: `sh test_prompting.sh -m 50-dpd/experiment-20221003T172424/epoch-09`
@ -210,6 +225,7 @@ Edit the [evaluate.py](prompt-learning/evaluate.py) to set the predictions outpu
```shell ```shell
python evaluate.py python evaluate.py
``` ```
### Results from prompt-based belief state generations ### Results from prompt-based belief state generations
|data-split| JGA* | |data-split| JGA* |
|--|:--:| |--|:--:|
@ -241,4 +257,5 @@ Generate belief states by running the below script:
```shell ```shell
sh test_prompting.sh -m <tuned-prompt-model-path> sh test_prompting.sh -m <tuned-prompt-model-path>
``` ```
// TODO :: Add results // TODO :: Add results

Loading…
Cancel
Save