We use generative artificial intelligence (GenAI), specifically ChatGPT, to simulate surveys on payment tools, focusing on perceptions of privacy and benefits. To validate the responses generated by GenAI, we compare the results with an existing survey on the privacy of financial apps by Brits and Jonker (2023). By designing prompts for hypothetical respondents (hereafter generative agents) that mirror the distribution of characteristics observed in actual surveys, we find that their views on payment app benefits and privacy align with real survey results when respondents are grouped by their level of privacy concern. Privacy-concerned agents view financial apps less favorably and perceive more risks, even without indicating this tendency in the prompts. Additionally, ChatGPT reflects the stark difference between users and non-users observed in the actual survey, with users finding payment apps more beneficial and less risky than non-users, despite not specifying these features in the prompt. However, ChatGPT does not replicate the variation--measured by the standard deviation of responses--observed in the actual survey, even when we specify detailed demographic characteristics of the generative agents in the prompt to match the dispersion in the observed data. This result means that there is a possibility that minority opinions may not be reflected. Moreover, ChatGPT provides responses with a bias towards being more privacy concerned. These results suggest that GenAI has the potential to be used as a complementary tool for surveys on users' perceptions of the privacy and benefits of payment tools, rather than as a substitute for actual surveys responded by humans.
Keywords: ChatGPT; Generative artificial agents; Privacy paradox; Westin index; Survey; Payment
Views expressed in the paper are those of the authors and do not necessarily reflect those of the Bank of Japan or Institute for Monetary and Economic Studies.