Amy's Lab Notebook
- Set up a meeting with Talie [Done]
- Set up a meeting with Jeff [on-going]
- Figure out metrics [on-going]
- Replicate chat [on-going]
###Task Updates
Questions:
- Should this actually be a single-session experiment (so not over time?), as they exist in social media -it is mostly used in short spontaneous interactions. However, it is difficult to tell if people use them internally in messaging. This may also be helpful for gaining more information on TI's relationship to personal closeness, message perception and immediate conversation feedback.
- How do you do a qualitative analysis with one main researcher (Not sure what the restrictions on collaboration for thesis)
- How do you systemically evaluate uses in the wild?
LIWC Use: Transcript analysis for use of first-person vs other pronouns, positive or negative affect terms Citation: Boyd, R. L., Ashokkumar, A., Seraj, S., & Pennebaker, J. W. (2022). The development and psychometric properties of LIWC-22. Austin, TX: University of Texas at Austin. https://www.liwc.app/ Background/Why: Linguistic Inquiry and Word Count program is a widely used tool that contains analysis software and a reference of words associated with different linguistic dimensions, including pronouns, affect terms, cognition terms, and social and communicative processes with high reliability. This was used in Hancock et al., 2007 to evaluate text transcripts for means of emotional express in text.
Self-Assessment Mankin (Bradley and Lang) Use: Collect information on the intended and perceived emotional arousal and valence in text responses Citation: Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The Self-Assessment Manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9 Background/Why: The SAM is a pictorial questionnaire that measures emotional reactions to a stimulus. In particular, since it is designed to be quick any easy -making it good for repeated self-assessments. In particular, it was used in Aoki et. al., 2022 to assess the impacts of chat balloon shape on emotional arousal and valence.
**Meeting with Talie on Friday to touch base on metrics of self-disclosure
####TI design ideas Figma
- Figure out IRB things
- Compare differences in AI/Human transcript
- Continue exploration of social networks to discover common use cases of tone indicators
- I chatted with the IRB personnel who would be in charge of CLPS studies and they mentioned that I would need to get my study IRB approved since it would deal with vulnerable populations. Additionally, I would have to use the expedited form (can't do the undergrad one) and would need someone else to be the PI for the application submission. Turnaround time would be 2-3 weeks
- How feasible would it be/should I just not use neurodivergence as a specific aspect of the study (would prefer to though)
- To-do: ask Zainab about this
- Set up IRB discussion and identify main questions
- Find a CBT chat and replicate the discussion in the chats
- Continue exploration of social networks to discover common use cases of tone indicators
Original Conversation Transcript Replicated Transcript
General rules I followed while recreating the conversation:
- Re-used user prompts as much as possible while responding directly to the AI responses, preserving timing and sequences
- When some areas of concern were not in direct response to the AI (which was also seen with the human helper), I included that too
- Tried to hit on all areas of discussion the user specifically brought up in the AI version
Observations:
- Lacking turn-taking and back and forth, much more solution-oriented > ended every response with a question
- Missed a lot of disclosure of details since they lacked back-and-forth
- Much less space for the user to jump in and talk (every message gets 2 paragraphs of text)
- SO LONG, very clear imbalance of time between helper and thinker
- Hit points that the helper didn’t (especially re: what are you going to do now)
- Very structured responses from AI, made it noticeable inhuman
- Thoughts, feelings etc insights were a lot longer (not necessarily more complex than human)
Observations (looked mostly on Reddit rn):
- Most common tone indicators I saw were /srs or /s (serious), and /j (joking), /neg and /pos (denoting pos/negative tone)
- Primarily used as a means to indicate sarcasm or false statements in joke form (i.e "X is the best character in the series/j")
- Used heavily in back and forth discussion (also could just be the structure)
- Concern about abbreviations
- "Just /s and /j. Let's not overcomplicate it, or it'll stop being useful."
- "Seems like another annoying thing I'd have to learn. I struggle with acronyms don't need another thing to try and decipher. People would change and add their own and expect everyone to just understand it, and somehow, they just do. Rather stick to plain ol' English, thanks."
- "I think in some cases they can be super useful, like if I was talking to someone I didn't know well yet and they insulted me as a joke, "/j" would take all anxiety from that and I could play along. Obviously "/s" is also useful in many contexts. I do think some of them don't really work though? I don't think anybody genuinely trying to threaten someone would think to put "/th", nor with clickbait. I also don't really get how one would use the positive/negative connotation ones."
- Start to ideate on how lines of questioning can be approached on the basis of a system
- Continue to do research with a focus on observing the methodology
- Try to identify the different programs you would need to use?
- Do self-disclosure patterns change across different conversation sizes and formats (small groups, teams, forums)? *
- Is there a long-term divergence in the way people treat CA, does that then inform the way they perceive and interact with humans? *
- How does computer-mediated self-presentation work across groups of varying self-consciousness?
- What makes online communication rewarding for participants?
- Tone indicators *
- How accurate is semantic analysis on small texts?
- How do you measure self-disclosure and self-presentation in images? (think B-Real vs Instagram)
- Effects of Implicit versus explicit cues?
- How do people learn new nonverbal cues?
- Groups of: 1, 2, 5<, 10+ (can they carry over between conversations?) Chatbot roles:
- Human reminders ie "responding to text late my worry -person-"
- Lead conversations
- Perform engagement Possible tasks:
- Discuss a recent problem
- Discuss an ethical dilemma
- Just get to know each other* Potential Metrics
- Perception of a conversational partner
- Level of self-disclosure
- Amount of texts
- Outspokeness/even participation
- Assertive/proactiveness
- Anxiety?
Tone indicator system
- Social media analysis
- Manual vs Automatic
- One-on-one vs group
- In the moment vs reflection
- Notification feature?
Metrics
- Perception of a conversational partner
- Perception of self-disclosure
- Actual comparative sensitive disclosure
Questions
- What if tone tags were applied on the other end?
- Do you use it for everyone?
- Would they interfere with the natural flow of conversation?
- Does the practice of having to tone tag messages affect the way conversations are had?
- Hover feature?
Moderator Chatbot for Deliberative Discussion: Effects of Discussion Structure and Discussant Facilitation*** this was super helpful
-
I’m unsure how this would be received outside a study?
-
““The effectiveness of dyadic chatbots has mainly been assessed by manipulating message-level variables such as conversational style [42], empathic responses [33], typeface [8], and self-disclosure [48].”
-
Chatbots can increase the participation of lurkers
-
How do chatbots fit into unstructured conversation?
- How do they relate to social loafing
-
Seems like the deliberations they can address are highly limited, it may be good for an educational setting.
* Logic structure for how their chatbot operated
* Tracked several different sources: behavioral data from the chat log or user response, quantitative data from the surveys, and qualitative data from open-ended surveys and analyzed for (1) consensus reaching, (2) opinion expression, (3) discussion quality, and (4) perceived satisfaction (task cohesion, communication effectiveness, communication fairness, communication efficiency)
[WIP -still need to add in some more notes]
- read lit review (sent in slack) >> how do you measure things like closeness or empathy
- review live typing set-up
- Look into semantic search similarity
- Effects of Self-Consciousness and Social Anxiety on Self-Disclosure Among Unacquainted Individuals
"Public self-consciousness does not seem to be related to the level at which subjects disclose but rather appears to influence subjects' beliefs about how their partners view them." "It appears that the different subscales of the self-consciousness scale may influence the acquaintance process in rather divergent manners"
- Can anthropological techniques apply in this case? If certain demographic types may influence interaction patterns?
- "perception of disclosure" may be an interesting metric to keep track of in long-term communication -does it change it downstream disclosure or personal relationships?
- "I Hear You, I Feel You"
"We can infer that the specifics of social interaction between a chatbot and its users can affect self-disclosure length. Additionally, length of use (as measured by experiment day) and chatbot variant both might influence the participants’ willingness to disclose their thoughts and feelings to a chatbot.” "However, despite all three groups being told that the chatbot represented a counselor from their local area, sharp inter-group differences emerged in how they perceived its persona" "But I don’t have to care about the chatbot. I can just talk about myself and focus on how I feel." “This implies that the users may assume the chatbot has more intelligence than it actually does, which might lead to users not reaching out to professionals for proper help.”
- How does they presence of researchers affect the sense of anonymity and, in turn, the willingness of participants to self-disclose
- Bot disclosure level seemed to have the biggest effects on feelings discussed
- Are the 3 groups having an effect both in “presenting instructions for how this chat is supposed to go” and “what I am willing to share?”
- How did this study measure trust, enjoyment and intimacy?
- Why did trust not increase over time?
- “Together but not together”
"Messaging was seen as a way to “uphold an image where there can be no room for error, vulnerability, and a close human connection”. The applicability of live-typing for interpersonal communication was seen as a threat since it limited participants’ ability for selective self-presentation as they could no longer edit their messages in a “controlled and socially desirable fashion”
- The diverging experiences with a potential live typing feature reminds me of the togglable "read" function in messages. Also in Snapchat, they would play the typing sound and indicate a chat had been started even if you did not send anything.
- How does the role of live typing play in facilitating personal conversations where one or both parties are preoccupied with other tasks? How do these set of typing indicators compare to the current paradigm of three bubbles, stop, pop up? What is the comparative richness within the context of user patters
- Enhancing user experience with the conversational agent for movie recommendations
"1) How does self-disclosure between a user and a CA affect user satisfaction and intention to use? 2) How does reciprocity between a user and a CA affect user satisfaction and intention to use?" "Taylor and Hinds, 1985).Berschied et al. (1989) provided the basis for the intimacy measurement items that are used to measure user perception of closeness and familiarity after conversing with an agent. Trust items consisted of measures of the reliability, honesty, and trustworthiness of the agent " "However, using a real artificial 3D conversation agent, or using virtual reality, might generate a higher social presence"
- Self-disclosure and intimacy did not seem to be directly linked
- Trust was strong, but what did trust mean in this case? Trust in competency?
- Is it unerving to have a CA in a setting like netflix?
- Used a 2D avatar -> what is social presence
- Apendix A for survey items
General Thoughts:
- Online ethnography of media richness in high self-conscious individuals → lex and grinder etc? Characters? What moves people into online spaces?
→ create anonymous chat groups → do certain patterns in self-disclosure emerge? Wb is a mixed group of CA agents and high anxious individuals
→ Computed-mediated group conversations
→ How does mediums richness affect saliency?
-
Medium richness → longitudinal study?
-
Is there a long-term divergence in the way people treat CA, does that then inform the way they perceive and interact with humans?
To-read:
-
Irwin Altman and Dalmas A Taylor. 1973. Social penetration: The development of interpersonal relationships. Holt, Rinehart & Winston.
-
Ellen Berscheid, Mark Snyder, and Allen M Omoto. 1989. The relationship closeness inventory: Assessing the closeness of interpersonal relationships. Journal of Personality and Social Psychology 57, 5 (1989), 792.
-
Yi-Chia Wang, Moira Burke, and Robert Kraut. 2016. Modeling Self-Disclosure in Social Networking Sites. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW ’16). ACM, New York, NY, USA,
-
Mark, G., Becker, B., 1999. Constructing social systems through computer-mediated communication. Virtual Real. 4 (60). http://dx.doi.org/10.1007/BF01434995.
-
Robin L Wakefield, Kirk L Wakefield, Julie Baker, and Liz C Wang. 2011. How website socialness leads to website use. European Journal of Information Systems 20, 1 (2011), 118–132.
-
Richard L Daft and Robert H Lengel. 1986. Organizational information requirements, media richness and structural design. Management science 32, 5 (1986), 554–571
- look up some chat repositories on GitHub and try to see how you can integrate the live-typing package in their code ** In prog
- Read selected to-read from previous research
- Walter self-selective presentation
- Do the 50 ideas ** In prog link here: https://docs.google.com/document/d/1ZdQ8FYtDrBPFnbjiWvCx6EXXQ0wj9d5QaMl0Hf6aVgs/edit?usp=sharing
- Move all documentation into wiki
Chat Repositories
Found this repo: https://github.com/kingofthestack/react-chat-window/tree/master/demo, which is a straightforward react-based chat app. I downloaded the demo and then tried to integrate the live-typing features into it by importing live-typing lib into the node_modules folder. Then I used the function in the launch components, putting the response element in the DOM. I then encountered a "reserved keyword error" which indicated I was running into issues parsing the typescript in live typing. I tried several different solutions, primarily centering on adding/modifing the config/eslint files. I think it may also be due to the directory structure, so I will also have to look more deeply into that. At this point, it had been around 2 hours so I decided to wait until I could touch base with someone more familiar with the structure. I have yet to fix the error, but it is on my to-do for next week.
"Hyperpersonal dimensions of technology, language, and cognition"
“As receivers, CMC users idealize partners based on the circumstances or message elements that suggest minimal similarity or desirability. As senders, CMC users selectively self-present, revealing attitudes and aspects of the self in a controlled and socially desirable fashion.”
“empirical tests have shown how CMC leads to more extreme impressions than FtF (Hancock & Dunham, 2001) and more positive relations over time compared to FtF”
“There is much less ‘‘leakage’’ in CMC since there is no unwanted nonverbal indication of undesirable affect or attitude”
“Editing behavior was also coded, using the following four definitions: (1) forward deletions; (2) destructive backspaces; (3) insertions (putting in spaces, letters, or punctuation where there previously had been none); and (4) replacements, such as changing capitalization to lower case, or changing words or letters. Three coders sampled 10 per cent of the tapes in common, with inter-rater reliabilities achieving Scott’s pi = .63, which was acceptable under these conditions.2”
-
Surprising that CMC leads to more positive reactions over time
-
I think these observations may vary greatly by age group and ‘texting culture’
-
I’m interested in if the authors validated their definition of a ‘high reward sources’ / had a metric that tracked this. ** They had pre-testing that showed their strongest preference was an opposite sex student. (maybe its a comfort thing)
-
Tracked editing behavior through tracking forward deletions, destruction backspaces, insertions, and replacements
-
More personal language with desirable targets
-
Mirrored language to accommodate who they were speaking to
-
More editing more mindfulness
"Modeling Self-Disclosure in Social Networking Sites"
We showed that women self-disclose more than men, and people who more strongly desire to manage the impressions they make on others self-disclose less. We then demonstrated that social network size was negatively associated with selfdisclosure, while network density and average tie strength had positive correlations with self-disclosure. Most of the results are consistent with those found or suggested by prior literature, which validate the effectiveness of the machine learning model we proposed.”
“As a sensitivity test, we replicated the analyses reported here on de-identified, aggregated posts from Facebook users in Australia and Singapore and discovered similar results.”
-
This study primarily focuses on text communication, but in SNs they often are heavily image focused -I wonder if the influence of images as a communication medium would change some of these observed patterns
-
Used humans to manually name model-produced directory
-
H4 is “Average tie strength will be positively correlated with self-disclosure” (pdf) , does that flip beyond a certain point toward anonymity?
"How website socialness leads to website use"
“The Technology Acceptance Model (TAM; Davis, 1989), as a by-product of the Theory of Reasoned Action (TRA) (Fishbein & Ajzen, 1975), has been a prominent starting point in the IS literature for examining the formation of attitude and behavioral intentions toward IT systems. Perceived usefulness (PU), perceived ease of use (PEOU) and enjoyment (Davis, 1989) determine attitudes, which in turn influence behavioral intentions to use an IT system.”
“An individual motivated to visit a website for what is primarily a utilitarian product (e.g., home improvement products or supplies) may respond differently to social cues compared to another who visits a website for a hedonic product”
-
I feel like the use of superimposed videos may have too strong unwanted implications for ux, in turn modifying the “sociability” of the site. Why not modify the copy instead?
-
How much of these findings are due to the stage of technology maturity at the time?
-
I think retail live chat/chatbots may be an extension of their interactive guides