Update LaTex code for thesis draft

writing/latex/sections/05_results.tex

@@ -111,17 +111,17 @@ Table \ref{table:11} shows the results of prompt augmentation under few-shot set
 \hline
 \textbf{\makecell{Data split (\# dialogs)}} & \textbf{JGA}  & \textbf{JGA*}\\
 \hline
-\textsl{5-dpd} (25) & --- & --- \\
+\textsl{5-dpd} (25) & 27.8 & 68.1 \\
 \hline
-\textsl{10-dpd} (50) & --- & ---  \\
+\textsl{10-dpd} (50) & 38.91 & 74.43  \\
 \hline
-\textsl{50-dpd} (250) & --- & --- \\
+\textsl{50-dpd} (250) & 39.52 & 82.81 \\
 \hline
-\textsl{100-dpd} (500) & --- & --- \\
+\textsl{100-dpd} (500) & \textbf{42.42} & \textbf{83.71} \\
 \hline
-\textsl{125-dpd} (625) & --- & --- \\
+\textsl{125-dpd} (625) & 40.16 & 82.92 \\
 \hline
-\textsl{250-dpd} (1125) & --- & --- \\
+\textsl{250-dpd} (1125) & 41.52 & 85.07 \\
 \hline
 \end{tabular}
 \endgroup
@@ -129,4 +129,6 @@ Table \ref{table:11} shows the results of prompt augmentation under few-shot set
 \label{table:11}
 \end{table}
 
-\newpage
+Prompt augmentation (also called \textit{demonstration learning}) provides additional context to the language models in the form of \textquote{\textsl{answered prompts}} at inference time. The hand-crafted answered prompts are supposed to help the language model understand the DST task better and generate accurate responses. Table \ref{table:11} shows the experimental results from the prompt augmentation method. Results show that the demonstration learning struggled to generate the belief states accurately. The performance is inadequate across all data splits when compared to other prompt-based methods. Only a limited number of answered prompts can be provided to the GPT-2 LM due to the max input sequence length of 1024, which led to bias during the slot generation process.
+
+\paragraph{} Overall, the multi-prompt methods (prompt ensembling and prompt augmentation) struggled to improve the performance of the DST task. However, the prompt ensembling approach with multiple value-based prompts showed minor improvements over a single value-based prompt. 

writing/latex/sections/06_analysis.tex

@@ -122,7 +122,7 @@ In the prompt-based methods, the value-based prompt takes the candidate values a
 \label{table:15}
 \end{table}
 
-\noindent The data instance listed in table \ref{table:15} contains multiple (slot, value) pairs. For the belief slots \textsl{stars}, \textsl{stay}, and \textsl{people}, the value is the same. The value-based prompt can only generate one slot with the repeated value \textit{3}. This is a main drawback of the value-based prompt under the existing belief state annotation system.
+\noindent The data instance listed in table \ref{table:15} contains multiple (slot, value) pairs. For the belief slots \textsl{stars}, \textsl{stay}, and \textsl{people}, the value is the same. The value-based prompt can only generate one slot with the repeated value 3. This is a main drawback of the value-based prompt under the existing belief state annotation system.
 
 
 \subsubsection{Error Analysis of Value Extraction} \label{subsec:value_errors}
@@ -156,7 +156,7 @@ In the prompt-based methods, the value-based prompt takes the candidate values a
 \label{table:16}
 \end{table}
 
-At inference time, the value-based prompt requires the belief state values in order to generate slots. The value extraction methods apply a set of rules on POS tags and named entities to extract value candidates directly from utterances. The rule-based extraction has an accuracy of \textit{79\%} on the test split. Table \ref{table:16} highlights instances where the values cannot be extracted using rule-based methods. In the first example, the value \textquote{\textit{dont care}} does not appear in the utterances and cannot be extracted from POS tags. When the user requirement is \textit{free} wifi or \textit{free} parking, the existing annotation system for belief states considers it as the value \textquote{\textit{yes}}. The rule-based methods adopted for value extraction can only extract the value \textquote{\textit{free}} from the utterances. The values \textquote{\textit{dont care}} and \textquote{\textit{yes}} also occur twice in the examples shown in table \ref{table:16}, as described in the previous section (sec \ref{subsec:value_errors}) the value-based prompt cannot handle repeated values for slot generation. 
+At inference time, the value-based prompt requires the belief state values in order to generate slots. The value extraction methods apply a set of rules on POS tags and named entities to extract value candidates directly from utterances. The rule-based extraction has an accuracy of \textit{79\%} over all the values and a turn-level accuracy of \textit{49\%} on the test split. Table \ref{table:16} highlights instances where the values cannot be extracted using rule-based methods. In the first example, the value \textquote{\textit{dont care}} does not appear in the utterances and cannot be extracted from POS tags. When the user requirement is \textit{free} wifi or \textit{free} parking, the existing annotation system for belief states considers it as the value \textquote{\textit{yes}}. The rule-based methods adopted for value extraction can only extract the value \textquote{\textit{free}} from the utterances. The values \textquote{\textit{dont care}} and \textquote{\textit{yes}} also occur twice in the examples shown in table \ref{table:16}, as described in the previous section (sec \ref{subsec:value_errors}) the value-based prompt cannot handle repeated values for slot generation. 
 
 \vspace{0.5cm}
 \begin{table}[h!]

writing/latex/sections/07_conclusion.tex

@@ -1,3 +1,3 @@
 \section{Conclusion}\label{sec:conclusion}
 
-This work explored the use of prompt-based methods for dialog state tracking (DST) in task-oriented dialogue systems. The prompt-based methods, which include value-based prompt and inverse prompt, learned the DST task efficiently under low-resource few-shot settings without relying on the pre-defined set of slots and values. Experiments show that the prompt-based methods significantly outperformed the baseline Soloist model under low-resource settings. Analysis of generated belief states shows the prompt-based approach has some limitations. Additionally, multi-prompt methods such as prompt ensembling and prompt augmentation are applied to the DST task. Results show that the prompt ensemble model achieved minor improvements, and the performance of prompt augmentation is limited due to the bias in answered prompts. Error analysis of value extraction highlights the limitations of the rule-based methods. Further research is necessary to overcome the limitations of value extraction methods.
+This work explored the use of prompt-based methods for dialog state tracking (DST) in task-oriented dialogue systems. The prompt-based methods, which include value-based prompt and inverse prompt, learned the DST task efficiently under low-resource few-shot settings without relying on the pre-defined set of slots and values. Experiments show that the prompt-based methods significantly outperformed the baseline Soloist model under low-resource settings. Analysis of generated belief states shows the prompt-based approach has some limitations. Additionally, multi-prompt methods such as prompt ensembling and prompt augmentation are applied to the DST task. Results show that the prompt ensemble model achieved minor improvements, and the performance of prompt augmentation is limited due to the bias in answered prompts. Error analysis of value extraction highlights the limitations of the rule-based methods. Further research is necessary to overcome the limitations of prompt-based methods and value extraction methods.

writing/latex/thesis.tex

@@ -56,10 +56,6 @@
 %%
 %%
 
-%% TODO command
-%% Use this to highlight text in yellow color
-\newcommand\todo[1]{\colorbox{yellow}{#1}}
-
 
 % Start document
 \begin{document}