The potential for generative artificial intelligence (AI) to automate customer service in the financial sector has been widely embraced. AI-powered chatbots are already a significant presence in consumer financial customer service. However, the rise of this technology brings new risks that need to be addressed.

The Consumer Financial Protection Bureau released a report in June, acknowledging the potential risks to consumers associated with the use of AI chatbots in consumer finance. This highlights the need for industry players to be proactive in mitigating these risks and ensuring the protection of their customers.

One particular concern with the use of generative AI chatbots is the potential for creating novel cybersecurity risks. Large language model (LLM) AI chatbots have been described as a “security disaster” in an article by MIT Technology Review. While this characterization may be sensational, it does highlight the fact that generative AI can amplify existing cybersecurity risks and introduce new ones.

Companies in the financial sector need to be vigilant in addressing these risks. General counsel and compliance officers should work closely with AI developers to ensure that adequate security measures are in place. This includes ensuring that AI systems are regularly updated to address emerging threats and vulnerabilities.

Another challenge posed by generative AI in finance is the potential for biased or discriminatory outcomes. AI systems learn from existing data, and if that data contains biases or discriminatory patterns, the AI chatbots may inadvertently perpetuate such biases in their responses to customers. This could have serious legal and reputational consequences for financial institutions.

To mitigate this risk, companies should implement rigorous data screening and cleansing processes to ensure that AI systems are trained on unbiased and representative datasets. Additionally, ongoing monitoring and auditing of AI chatbot interactions can help identify and rectify any biases or discriminatory patterns that may arise.

Transparency is also crucial in ensuring that customers understand when they are interacting with an AI chatbot. Clear disclosures and communication about the role of AI in customer service can help manage expectations and build trust. Customers should have the option to escalate their interactions to human representatives if they prefer, and companies should have well-defined processes in place to facilitate such escalations.

While generative AI chatbots offer significant advantages in automation and efficiency for the financial sector, companies must not overlook the important role of human oversight and accountability. AI should be seen as a tool to augment human capabilities, rather than replace them entirely. Companies should have robust mechanisms in place for human review and intervention when necessary.

As technology continues to evolve, it is imperative for the financial industry to stay ahead of emerging risks. Forward-thinking companies are investing in robust cybersecurity measures and implementing responsible AI governance frameworks. By doing so, they can harness the benefits of generative AI chatbots while ensuring the security, fairness, and integrity of their customer interactions.

LeackStat 2023