|
|
|
@ -86,7 +86,6 @@
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
\item Ontology of domains
|
|
|
|
\item Ontology of domains
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
\item Represents knowledge \& information required for specific tasks
|
|
|
|
|
|
|
|
\item Contains pre-defined set of slots and all possible values for each slot
|
|
|
|
\item Contains pre-defined set of slots and all possible values for each slot
|
|
|
|
\item Some Neural-based models solve the DST as classification task
|
|
|
|
\item Some Neural-based models solve the DST as classification task
|
|
|
|
\end{itemize}
|
|
|
|
\end{itemize}
|
|
|
|
@ -113,7 +112,6 @@
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
\item New way of efficiently using the generation capabilities of PLMs to solve different language tasks
|
|
|
|
\item New way of efficiently using the generation capabilities of PLMs to solve different language tasks
|
|
|
|
\item Downstream task is converted to a textual prompt and given as input, the PLM directly generates the outputs from prompts
|
|
|
|
\item Downstream task is converted to a textual prompt and given as input, the PLM directly generates the outputs from prompts
|
|
|
|
\item Prompting methods can be effectively used under \textsl{zero-shot} and \textsl{few-shot} settings when there's not enough training data
|
|
|
|
|
|
|
|
\item GPT-3 \parencite{brown2020gpt3}, Few-shot Bot \parencite{madotto2021fsb}, \textsc{PET} \parencite{schick2021pet} explored prompt-based methods for several tasks
|
|
|
|
\item GPT-3 \parencite{brown2020gpt3}, Few-shot Bot \parencite{madotto2021fsb}, \textsc{PET} \parencite{schick2021pet} explored prompt-based methods for several tasks
|
|
|
|
\end{itemize}
|
|
|
|
\end{itemize}
|
|
|
|
\end{itemize}
|
|
|
|
\end{itemize}
|
|
|
|
@ -128,7 +126,6 @@
|
|
|
|
\end{figure}
|
|
|
|
\end{figure}
|
|
|
|
\vspace{-4pt}
|
|
|
|
\vspace{-4pt}
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
\item Prompt Types: \textsl{prefix} \& \textsl{cloze} prompts
|
|
|
|
|
|
|
|
\item Prompt selection: manual, discrete, \& continuous prompts
|
|
|
|
\item Prompt selection: manual, discrete, \& continuous prompts
|
|
|
|
\item Training strategy: Fixed-prompt LM Fine Tuning
|
|
|
|
\item Training strategy: Fixed-prompt LM Fine Tuning
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
@ -159,8 +156,7 @@
|
|
|
|
\end{itemize}
|
|
|
|
\end{itemize}
|
|
|
|
\item Research Objectives
|
|
|
|
\item Research Objectives
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
\item Can the dialog states be extracted from the PLM using prompts?
|
|
|
|
\item Can the prompt-based methods learn the DST task efficiently under low-resource settings without depending on the ontology?
|
|
|
|
\item Can the prompt-based methods learn the DST task under low-resource settings without depending on the ontology of domains?
|
|
|
|
|
|
|
|
\item Compare prompt-based approach with the baseline model
|
|
|
|
\item Compare prompt-based approach with the baseline model
|
|
|
|
\item Identify the drawbacks \& limitations of prompt-based approach
|
|
|
|
\item Identify the drawbacks \& limitations of prompt-based approach
|
|
|
|
\item Can different multi-prompt techniques help improve the performance of DST task?
|
|
|
|
\item Can different multi-prompt techniques help improve the performance of DST task?
|
|
|
|
@ -184,7 +180,6 @@
|
|
|
|
\item Six data splits are created to perform few-shot experiments
|
|
|
|
\item Six data splits are created to perform few-shot experiments
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
\item Different proportions of dialogues in each split
|
|
|
|
\item Different proportions of dialogues in each split
|
|
|
|
\item All the five domains are evenly distributed in each split
|
|
|
|
|
|
|
|
\end{itemize}
|
|
|
|
\end{itemize}
|
|
|
|
\end{itemize}
|
|
|
|
\end{itemize}
|
|
|
|
|
|
|
|
|
|
|
|
@ -253,9 +248,7 @@
|
|
|
|
\end{table}
|
|
|
|
\end{table}
|
|
|
|
|
|
|
|
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
|
|
|
|
|
|
|
|
\item The pre-trained Soloist is used to fine-tune the prompting methods
|
|
|
|
\item The pre-trained Soloist is used to fine-tune the prompting methods
|
|
|
|
\item All MultiWOZ data splits are used in the fine-tuning phase
|
|
|
|
|
|
|
|
\item Loss function for value-based prompt
|
|
|
|
\item Loss function for value-based prompt
|
|
|
|
$$\mathcal{L}=-\sum_{t}^{|D|} \log P\left(s_{t} \mid c_{t}, f\left(v_{t}\right)\right)$$
|
|
|
|
$$\mathcal{L}=-\sum_{t}^{|D|} \log P\left(s_{t} \mid c_{t}, f\left(v_{t}\right)\right)$$
|
|
|
|
\item Loss function for inverse prompt
|
|
|
|
\item Loss function for inverse prompt
|
|
|
|
@ -435,7 +428,7 @@
|
|
|
|
\end{block}
|
|
|
|
\end{block}
|
|
|
|
|
|
|
|
|
|
|
|
\begin{itemize}
|
|
|
|
\begin{itemize}
|
|
|
|
\item Open-ended generation
|
|
|
|
%% \item Open-ended generation
|
|
|
|
\item Susceptible to generating random slot-value pairs
|
|
|
|
\item Susceptible to generating random slot-value pairs
|
|
|
|
\item Repeated slot-value generations
|
|
|
|
\item Repeated slot-value generations
|
|
|
|
\item From the above example:
|
|
|
|
\item From the above example:
|
|
|
|
|
