Compare several conversations exported from Multi-Pip Chat (⤓ Export) and see how alike or different they turned out. Nothing leaves this device.
How to use it
- Drop two or more export files on the shelf, or tap to pick them.
- Read the comparison.
- ⤓ Report downloads it. Clear empties the shelf.
What it measures
- Distinct-1 / 2 / 3: how varied the wording is. It's the share of single words, word pairs and word triples that are unique. Higher means less repetitive.
- Pairwise overlap: how much two conversations share, phrase for phrase.
- Replayed openings: runs that started word for word the same (the same seed and settings do that). It also notes when the replay length is the context window + 1: the chats stay identical until the window first has to drop a line.
- Recurring phrases: four-word phrases that keep coming back, after any replayed opening.
- Invented names: names the models made up that recur across runs.
- Flags: each run's filter flags (speaker slips, garble, repeats, heavy themes and so on). These only point things out; nothing is changed.
Good for checking whether a batch of synthetic dialogue is genuinely varied, or for comparing settings (stateful vs. not, different seeds or temperatures).