Conceptual talks
Intro
This project is part of my PhD studies.
Establishing mutual understanding in conversations requires effective grounding and negotiation of conceptual meaning, in order to mitigate the differences in the understanding among the interacting parties, especially for concepts related to real-world entities, qualities, and experiences.
In this project, I aim to build a taxonomy that describes how people ground and negotiate meaning for concepts in a dialogue.
Behind the scenes
I'm making use of the human dialogue corpus I collected here. Here's a small taste, translated from Chinese:
| A | This white thing made of silica gel, what is this for? I'm not sure. Looks like something for cooking, but also not really. |
| B | I think that silica gel releases toxic matter under high temperature. So normally you wouldn't use it for cooking. [...] |
| A | It seems that I need to change my cooking wares. |
| B | [Inaudible] |
| A | Because I bought the ones made of silica gel. |
| B | Oh! It's okay, it's okay. Most of them have gone through safety checks, so it's not a problem. |
| A | Hmm. It seems that we don't have other disagreements, if you say it like that. |
At first, this may seem quite different from typical human-LLM dialogue. After all, this is two people talking to each other. However, if we imagine it as A talking to an MLLM, or a speech-enabled robot, this becomes quite natural at once.
It also shows how fluid the reasons for applying a concept can be. If we imagine the speakers as conversational agents and assume everything they said was "properly backed up (i.e. grounded)", then this dialogue has already invoked many different things to ground and how to ground them:
- To visual features (with uncertainty): A thinks that the spatula "looks like something for cooking" but is uncertain.
- To factual knowledge (the most typical kind of LLM grounding): B claims silica gel "normally" is not used in cooking because they are toxic when heated.
- To personal experience (more relevant for robots and long-term interaction): A talks about having silica cookware at home.
- To future implications (important for planning and collaboration): A ponders having to replace all their cookware after this discussion.
- Retractions (nothing is "golden", practicality reigns): B contradicts themselves later (presumably because the earlier claim, if true, would give A a lot of work.)
Not to mention that proper symbol grounding is only one part of successful communication. Hopefully by building this taxonomy, we can understand where to go from the current state of affairs.