|information about the structure of the tree (e.g., characteristics of the nodes labels, the relationships between parent and child nodes, and the level at which the goal node was located).|
Thus, at each navigation point, the subject had to decide whether to rely on the system's assistance or rather choose his or her own path through the tree.
As independent variables we defined the Representation of the Agent (RoA) (with four levels) and Quality of the Agent (QoA) (with two levels). As dependent variables we chose the ratio of the number of clicks on recommended nodes and number of expansions, and subjective ratings of trustworthiness on a questionnaire. RoA was manipulated within subjects, whereas QoA was manipulated between subjects. Thus, the experiment had a 4 (RoA) x 2 (QoA) mixed design.
Representation of Agents. The representation of the agents could take on four values: Text, Audio, Video-Agent or Persona. In case of Text, recommendations were given in text, accompanied by a blinking of the intended nodes. In the Audio condition, the recommendations were given auditorily, again accompanied by blinking. In the case of Persona, the recommendations were given by the Persona cartoon who, besides speaking the recommendation, pointed with a pointer stick at the intended nodes. The Video-Agent condition differed from Persona only in that the agent was represented by a real-looking person.
Quality of Agents. The quality of the recommendations could take on two values: high or low. In the case of high-quality assistance, the probability of system's recommendations being correct was .75, whereas with low-quality assistance, this probability was .25. Subjects were allocated to the QoA condition randomly. The order of RoA conditions was counterbalanced. Furthermore, so as to control for the particular voice, we used both a normal pitch voice and a low-pitch voice.
The experiment started with a test session, consisting of a sequence of four test trees. The aim of this first test session was to have the subjects grow acquainted with the environment and get a feeling of how the nodes in the trees were organized. Here, the system would not give any recommendations at all. After this first test session, the subjects would proceed with a second test session. Here, the system would give recommendations by simply encircling the nodes it recommended. The aim of this session was to have the subjects experience the quality of the recommendations. After the test sessions, the sequence of experimental sessions with either Text, Audio, Persona, or Video-Agent