@ -182,8 +182,6 @@ python evaluate.py
| 125-dpd | 35.79 |
| 125-dpd | 35.79 |
| 250-dpd | **40.38** |
| 250-dpd | **40.38** |
< img src = "images/baseline_results.png" alt = "Baseline results" width = "540" height = "415" >
## Prompt Learning Experiments
## Prompt Learning Experiments
### Install the requirements
### Install the requirements
@ -266,9 +264,6 @@ python evaluate.py -o path/to/outputs/file
< table > < tr > < th > < / th > < th colspan = "2" > w = 0.1< / th > < th colspan = "2" > w = 0.3< / th > < th colspan = "2" > w = 0.5< / th > < th colspan = "2" > w = 0.7< / th > < / tr > < tr > < th > Dataset< / th > < th > JGA< / th > < th > JGA*< / th > < th > JGA< / th > < th > JGA*< / th > < th > JGA< / th > < th > JGA*< / th > < th > JGA< / th > < th > JGA*< / th > < / tr > < tr > < td > 5-dpd< / td > < td > 30.66< / td > < td > 71.04< / td > < td > 31.67< / td > < td > 73.19< / td > < td > 30.77< / td > < td > 72.85< / td > < td > 29.98< / td > < td > 70.93< / td > < / tr > < tr > < td > 10-dpd< / td > < td > 42.65< / td > < td > 86.43< / td > < td > 41.18< / td > < td > 83.48< / td > < td > 40.05< / td > < td > 80.77< / td > < td > 40.38< / td > < td > 85.18< / td > < / tr > < tr > < td > 50-dpd< / td > < td > 47.06< / td > < td > 91.63< / td > < td > 46.49< / td > < td > 91.18< / td > < td > 47.04< / td > < td > 91.18< / td > < td > 46.27< / td > < td > 90.05< / td > < / tr > < tr > < td > 100-dpd< / td > < td > 47.74< / td > < td > 92.31< / td > < td > 48.42< / td > < td > 92.42< / td > < td > 48.19< / td > < td > 92.65< / td > < td > 48.3< / td > < td > 92.65< / td > < / tr > < tr > < td > 125-dpd< / td > < td > 46.49< / td > < td > 91.86< / td > < td > 46.15< / td > < td > 91.18< / td > < td > 46.83< / td > < td > 91.74< / td > < td > 46.15< / td > < td > 90.95< / td > < / tr > < tr > < td > 250-dpd< / td > < td > 47.06< / td > < td > 92.08< / td > < td > 47.62< / td > < td > 92.65< / td > < td > 47.4< / td > < td > 92.31< / td > < td > 47.17< / td > < td > 92.09< / td > < / tr > < / table >
< table > < tr > < th > < / th > < th colspan = "2" > w = 0.1< / th > < th colspan = "2" > w = 0.3< / th > < th colspan = "2" > w = 0.5< / th > < th colspan = "2" > w = 0.7< / th > < / tr > < tr > < th > Dataset< / th > < th > JGA< / th > < th > JGA*< / th > < th > JGA< / th > < th > JGA*< / th > < th > JGA< / th > < th > JGA*< / th > < th > JGA< / th > < th > JGA*< / th > < / tr > < tr > < td > 5-dpd< / td > < td > 30.66< / td > < td > 71.04< / td > < td > 31.67< / td > < td > 73.19< / td > < td > 30.77< / td > < td > 72.85< / td > < td > 29.98< / td > < td > 70.93< / td > < / tr > < tr > < td > 10-dpd< / td > < td > 42.65< / td > < td > 86.43< / td > < td > 41.18< / td > < td > 83.48< / td > < td > 40.05< / td > < td > 80.77< / td > < td > 40.38< / td > < td > 85.18< / td > < / tr > < tr > < td > 50-dpd< / td > < td > 47.06< / td > < td > 91.63< / td > < td > 46.49< / td > < td > 91.18< / td > < td > 47.04< / td > < td > 91.18< / td > < td > 46.27< / td > < td > 90.05< / td > < / tr > < tr > < td > 100-dpd< / td > < td > 47.74< / td > < td > 92.31< / td > < td > 48.42< / td > < td > 92.42< / td > < td > 48.19< / td > < td > 92.65< / td > < td > 48.3< / td > < td > 92.65< / td > < / tr > < tr > < td > 125-dpd< / td > < td > 46.49< / td > < td > 91.86< / td > < td > 46.15< / td > < td > 91.18< / td > < td > 46.83< / td > < td > 91.74< / td > < td > 46.15< / td > < td > 90.95< / td > < / tr > < tr > < td > 250-dpd< / td > < td > 47.06< / td > < td > 92.08< / td > < td > 47.62< / td > < td > 92.65< / td > < td > 47.4< / td > < td > 92.31< / td > < td > 47.17< / td > < td > 92.09< / td > < / tr > < / table >
< img src = "images/prompt_results.png" alt = "Prompt-based methods results" width = "600" height = "540" >
## Multi-prompt Learning Experiments
## Multi-prompt Learning Experiments
### Prompt Ensemble
### Prompt Ensemble
@ -316,8 +311,6 @@ sh test_prompting.sh -m <saved-model-path>
| 250-dpd | 48.30 | 93.44 |
| 250-dpd | 48.30 | 93.44 |
< img src = "images/ensemble_results.png" alt = "Prompt Ensembling results" width = "560" height = "417" >
### Prompt Augmentation
### Prompt Augmentation
Prompt Augmentation, also called *demonstration learning* , provides a few additional *answered prompts* that can demonstrate to the PLM, how the actual prompt slot can be answered. Sample selection of answered prompts are hand-crafted and hand-picked manually. Experiments are performed on different sets of *answered prompts* .
Prompt Augmentation, also called *demonstration learning* , provides a few additional *answered prompts* that can demonstrate to the PLM, how the actual prompt slot can be answered. Sample selection of answered prompts are hand-crafted and hand-picked manually. Experiments are performed on different sets of *answered prompts* .
@ -330,14 +323,7 @@ sh test_prompting.sh -m <tuned-prompt-model-path>
#### Results from Prompt Augmentation
#### Results from Prompt Augmentation
< table > < tr > < th > < / th > < th colspan = "2" > Sample 1< / th > < th colspan = "2" > Sample 2< / th > < / tr >
< table > < tr > < th > < / th > < th colspan = "2" > Sample 1< / th > < th colspan = "2" > Sample 2< / th > < / tr >
< tr > < th > Data< / th > < th > JGA< / th > < th > JGA*< / th > < th > JGA< / th > < th > JGA*< / th > < / tr > < tr > < td > 5-dpd< / td > < td > 26.02< / td > < td > 58.6< / td > < td > 27.6< / td > < td > 59.39< / td > < / tr > < tr > < td > 10-dpd< / td > < td > 33.26< / td > < td > 70.14< / td > < td > 34.95< / td > < td > 77.94< / td > < / tr > < tr > < td > 50-dpd< / td > < td > 38.8< / td > < td > 71.38< / td > < td > 39.77< / td > < td > 74.55< / td > < / tr > < tr > < td > 100-dpd< / td > < td > 35.97< / td > < td > 70.89< / td > < td > 38.46< / td > < td > 74.89< / td > < / tr > < tr > < td > 125-dpd< / td > < td > 36.09< / td > < td > 73.08< / td > < td > 36.18< / td > < td > 76.47< / td > < / tr > < tr > < td > 250-dpd< / td > < td > 35.63< / td > < td > 72.9< / td > < td > 38.91< / td > < td > 76.7< / td > < / tr > < / table >
< tr > < th > Data< / th > < th > JGA< / th > < th > JGA*< / th > < th > JGA< / th > < th > JGA*< / th > < / tr > < tr > < td > 5-dpd< / td > < td > 26.02< / td > < td > 58.6< / td > < td > 27.6< / td > < td > 59.39< / td > < / tr > < tr > < td > 10-dpd< / td > < td > 33.26< / td > < td > 70.14< / td > < td > 34.95< / td > < td > 77.94< / td > < / tr > < tr > < td > 50-dpd< / td > < td > 38.8< / td > < td > 71.38< / td > < td > 39.77< / td > < td > 74.55< / td > < / tr > < tr > < td > 100-dpd< / td > < td > 35.97< / td > < td > 70.89< / td > < td > 38.46< / td > < td > 74.89< / td > < / tr > < tr > < td > 125-dpd< / td > < td > 36.09< / td > < td > 73.08< / td > < td > 36.18< / td > < td > 76.47< / td > < / tr > < tr > < td > 250-dpd< / td > < td > 35.63< / td > < td > 72.9< / td > < td > 38.91< / td > < td > 76.7< / td > < / tr > < / table >
< img src = "images/demonstration_results.png" alt = "Prompt Augmentation results" width = "560" height = "425" >
### Comparison of all the results
< img src = "images/comparison_results.png" alt = "Comparison of results" width = "600" height = "515" >
## Analysis
## Analysis