This study investigates the vulnerabilities of three large language models (LLMs)—ChatGPT-4o Mini, Gemini Flash, and ChatGPT-4o—when tested with adversarial prompts across multiple languages. The research begins by evaluating the models’ responses in English, then extends to less commonly represented languages, including French and Haitian Creole, to explore linguistic disparities in security robustness. A total of 12+ adversarial prompts, modified based on model outputs, are utilized to assess the models' ability to address ethically and legally complex questions. These prompts are designed to test the limits of the models’ judgment, security awareness, and ethical safeguards.

In addition to text-based evaluations, the study incorporates voice interactions to examine multimodal vulnerabilities. This approach broadens the scope of analysis by probing potential weaknesses in handling audio inputs, which represent an emerging dimension of LLM applications. Focusing on low-resource languages and multimodal interactions, this research seeks to uncover overlooked vulnerabilities in the current security measures of widely used LLMs.

The study builds upon previous work (Paredes et al., 2024) by introducing the Adversarial Response Scoring System (ARSS), a novel framework for systematically evaluating and quantifying the security awareness of language models. ARSS employs a scoring system to rank responses based on appropriateness, compliance with ethical norms, and resistance to adversarial misuse. These findings highlight the critical areas for improvement in LLMs, particularly their vulnerabilities to adversarial manipulation in less common languages. By addressing these gaps, this research contributes to developing more robust and equitable LLMs, ensuring their safe deployment in diverse linguistic and cultural contexts. In conclusion, the study offers insights for enhancing LLM security and ethical performance while advancing the broader understanding of adversarial testing in AI systems.