---
variant: detail
kind: inputs
slug: interaction-augmented-instruction
url: /inputs/interaction-augmented-instruction
title: >-
  Interaction-Augmented Instruction: Modeling the Synergy of Prompts and
  Interactions in Human-GenAI Collaboration
source_path: content/inputs/interaction-augmented-instruction.md
frontmatter:
  title: >-
    Interaction-Augmented Instruction: Modeling the Synergy of Prompts and
    Interactions in Human-GenAI Collaboration
  url: 'https://arxiv.org/html/2510.26069v2'
  source: arXiv
  consumed: '2026-07-01T00:00:00.000Z'
  note: >-
    The paper argues that text prompts struggle with fine-grained and
    referential intent, and models how GUI interactions such as clicking and
    brushing can combine with prompts in human-GenAI systems.
  tags:
    - ai
    - human-ai-interaction
    - interfaces
    - prompting
agent_metadata:
  source_path: content/inputs/interaction-augmented-instruction.md
  html_url: /inputs/interaction-augmented-instruction
  markdown_url: /inputs/interaction-augmented-instruction.md
  source_url: >-
    https://github.com/flaming-codes/thinkinglabs/blob/main/content/inputs/interaction-augmented-instruction.md
  summary: >-
    This source supports the claim that visual and artifact-oriented prompting
    should not be reduced to chat. Pointing, selecting, brushing, editing, and
    manipulating controls can carry intent that is expensive or impossible to
    express clean...
  word_count: 135
  approx_token_count: 277
  token_estimate: chars/4
---
This source supports the claim that visual and artifact-oriented prompting should not be reduced to chat. Pointing, selecting, brushing, editing, and manipulating controls can carry intent that is expensive or impossible to express cleanly in words.