Added images to README.md

main
Pavan Mandava 3 years ago
parent 7b80bec2dc
commit f49890b1e4

@ -182,6 +182,8 @@ python evaluate.py
| 125-dpd | 35.79 |
| 250-dpd | **40.38** |
![Baseline results](images/baseline_results.png)
## Prompt Learning Experiments
### Install the requirements
@ -263,7 +265,8 @@ python evaluate.py -o path/to/outputs/file
<table> <tr> <th> </th> <th colspan="2">w = 0.1</th> <th colspan="2">w = 0.3</th> <th colspan="2">w = 0.5</th> <th colspan="2">w = 0.7</th> </tr> <tr> <th>Dataset</th> <th>JGA</th> <th>JGA*</th> <th>JGA</th> <th>JGA*</th> <th>JGA</th> <th>JGA*</th> <th>JGA</th> <th>JGA*</th> </tr> <tr> <td>5-dpd</td> <td>30.66</td> <td>71.04</td> <td>31.67</td> <td>73.19</td> <td>30.77</td> <td>72.85</td> <td>29.98</td> <td>70.93</td> </tr> <tr> <td>10-dpd</td> <td>42.65</td> <td>86.43</td> <td>41.18</td> <td>83.48</td> <td>40.05</td> <td>80.77</td> <td>40.38</td> <td>85.18</td> </tr> <tr> <td>50-dpd</td> <td>47.06</td> <td>91.63</td> <td>46.49</td> <td>91.18</td> <td>47.04</td> <td>91.18</td> <td>46.27</td> <td>90.05</td> </tr> <tr> <td>100-dpd</td> <td>47.74</td> <td>92.31</td> <td>48.42</td> <td>92.42</td> <td>48.19</td> <td>92.65</td> <td>48.3</td> <td>92.65</td> </tr> <tr> <td>125-dpd</td> <td>46.49</td> <td>91.86</td> <td>46.15</td> <td>91.18</td> <td>46.83</td> <td>91.74</td> <td>46.15</td> <td>90.95</td> </tr> <tr> <td>250-dpd</td> <td>47.06</td> <td>92.08</td> <td>47.62</td> <td>92.65</td> <td>47.4</td> <td>92.31</td> <td>47.17</td> <td>92.09</td> </tr> </table>
> **Note:** All the generated output files for the above reported results are available in this repository. Check [outputs/prompt-learning](outputs/prompt-learning) directory to see the output JSON files for each data-split.
![Prompt-based methods results](images/prompt_results.png)
## Multi-prompt Learning Experiments
@ -312,6 +315,8 @@ sh test_prompting.sh -m <saved-model-path>
| 250-dpd | 48.30 | 93.44 |
![Prompt Ensembling results](images/ensemble_results.png)
### Prompt Augmentation
Prompt Augmentation, also called *demonstration learning*, provides a few additional *answered prompts* that can demonstrate to the PLM, how the actual prompt slot can be answered. Sample selection of answered prompts are hand-crafted and hand-picked manually. Experiments are performed on different sets of *answered prompts*.
@ -325,8 +330,12 @@ sh test_prompting.sh -m <tuned-prompt-model-path>
<table> <tr> <th></th> <th colspan="2">Sample 1</th> <th colspan="2">Sample 2</th> </tr>
<tr> <th>Data</th> <th>JGA</th> <th>JGA*</th> <th>JGA</th> <th>JGA*</th> </tr> <tr> <td>5-dpd</td> <td>26.02</td> <td>58.6</td> <td>27.6</td> <td>59.39</td> </tr> <tr> <td>10-dpd</td> <td>33.26</td> <td>70.14</td> <td>34.95</td> <td>77.94</td> </tr> <tr> <td>50-dpd</td> <td>38.8</td> <td>71.38</td> <td>39.77</td> <td>74.55</td> </tr> <tr> <td>100-dpd</td> <td>35.97</td> <td>70.89</td> <td>38.46</td> <td>74.89</td> </tr> <tr> <td>125-dpd</td> <td>36.09</td> <td>73.08</td> <td>36.18</td> <td>76.47</td> </tr> <tr> <td>250-dpd</td> <td>35.63</td> <td>72.9</td> <td>38.91</td> <td>76.7</td> </tr> </table>
![Prompt Augmentation results](images/demonstration_results.png)
### Comparison of all the results
> **Note:** All the generated output files for the above reported results are available in this repository. Check [outputs/multi-prompt](outputs/multi-prompt) directory to see the output JSON files for each data-split.
![Comparison of results](images/comparison_results.png)
## Analysis

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 264 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 130 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 199 KiB

Loading…
Cancel
Save