The HaLLMark Effect: Supporting Provenance and Transparent Use of Large Language Models in Writing with Interactive Visualization

The use of Large Language Models (LLMs) for writing has sparked controversyboth among readers and writers. On one hand, writers are concerned that LLMswill deprive them of agency and ownership, and readers are concerned aboutspending their time on text generated by soulless machines. On the other hand,AI-assistance can improve writing as long as writers can conform to publisherpolicies, and as long as readers can be assured that a text has been verifiedby a human. We argue that a system that captures the provenance of interactionwith an LLM can help writers retain their agency, conform to policies, andcommunicate their use of AI to publishers and readers transparently. Thus wepropose HaLLMark, a tool for visualizing the writer's interaction with the LLM.We evaluated HaLLMark with 13 creative writers, and found that it helped themretain a sense of control and ownership of the text.