├── .gitattributes ├── Authorization-Bypass-through-User-Controlled-Parameter.md ├── DataLeakage.md ├── DataPoisoning.md ├── Improper-IAM-Models.md ├── Inadequate-Anomaly-Detection.md ├── Inadequate-DR-Plan.md ├── Inadequate-ML-Grounding.md ├── Insecure-Model-Arch.md ├── Insufficient-InputValidation.md ├── Insufficient-Logging-MLOps.md ├── Insufficient-OutputSanitization-ContentFiltering.md ├── LICENSE ├── Lacking-Env-Segmentation-Gap-MLOps.md ├── Lacking-Output-Disclaimer.md ├── Lacking-RateLimiting.md ├── Lacking-User-Feedback-Mechanism.md ├── PromptInjection.md ├── README.md ├── Unrestricted-Task-Execution.md ├── Violation-DifferentialPrivacy.md └── index.html /.gitattributes: -------------------------------------------------------------------------------- 1 | # Auto detect text files and perform LF normalization 2 | * text=auto 3 | -------------------------------------------------------------------------------- /Authorization-Bypass-through-User-Controlled-Parameter.md: -------------------------------------------------------------------------------- 1 | # Authorization-Bypass-through-User-Controlled-Parameter 2 | 3 | ## Description 4 | 5 | Authorization Bypass Through User Controlled Parameter mainly occurs in transactional Natural Language Generation (NLG) Systems, such as e-commerce chatbots and other consumer facing LLMs. An external party can exploit this vulnerability by manipulating user-controlled parameters that affect significant functionality such as purchase price or quantity. 6 | 7 | ## Extended Description 8 | 9 | LLMs or chatbots designed for secure transactions could behave unpredictably if a malicious actor manipulates parameters that determine critical values such as purchase price, quantity of an item, or transaction details. This type of failure to validate user inputs could result in unauthorized benefits to the attacker, such as purchasing at reduced prices, manipulating the quantity of items held, buying entire inventories for market manipulation, misdirecting transactions to other accounts, etc. This scenario could lead to financial losses, operational disruptions, and reputational damage. 10 | 11 | ## Potential Mitigations 12 | 13 | - **Immutable Key Parameters**: Design the system to treat certain vital parameters as immutable to user input, such as item prices, identity information or other critical transaction details. 14 | - **Input Verification**: Implement robust verification and authentication for all variable user-submitted parameters. 15 | - **Transaction Validation**: Execute multiple checks throughout the transaction process before final validation and confirmation. 16 | - **Access Control Policies**: Set up robust role-based access control. This limits the relationship between users and the parameters they can manipulate. 17 | - **Alerting Mechanisms**: Develop robust detection systems to alert about abnormal behavior patterns that might indicate attempted vulnerability exploits promptly. 18 | 19 | ## Related Weaknesses 20 | 21 | - **CWE-20**: Improper Input Validation: Not properly validating user-controlled parameters could enable attacker unauthorized benefits. 22 | - **CWE-266**: Incorrect Privilege Assignment: If user roles are mistakenly provided with the ability to edit parameters they shouldn't control, they could change the system’s behavior to their advantage. 23 | - **CWE-285**: Improper Authorization: This could occur if the system fails to correctly perform access checks, giving an attacker the opportunity to manipulate key parameters. 24 | 25 | ## Impact Analysis 26 | 27 | - **Financial Impact**: Unauthorized manipulations such as purchasing at reduced prices, increasing the quantity of items held, or misdirecting transactions could result in financial losses for the business. 28 | - **Operational Disruption**: Altering transaction parameters can lead to disruptions in the system's operations and could potentially burden the business by causing an increase in failed transactions, inventory inconsistency, etc. 29 | - **Reputational Damage**: If vulnerabilities are exploited and become public knowledge, it can lead to damage to the organization's reputation. Trust from customers, partners, and stakeholders may be eroded. 30 | - **Regulatory Risk**: If the system operates on regulated data, products, or services, a breach of transaction integrity may lead to scrutiny from regulatory bodies, potentially adding further financial and reputational impacts. 31 | - **Potential for Market Manipulation**: In more severe cases, an attacker could potentially use this exploit to flood the market with discounted goods or create an artificial scarcity, destabilizing market conditions for the affected product or service. 32 | -------------------------------------------------------------------------------- /DataLeakage.md: -------------------------------------------------------------------------------- 1 | # Data Leakage Risks in ML/AI Systems 2 | 3 | ## Description 4 | ML/AI systems are prone to data leakage, which can occur at various stages of data processing, model training, or output generation, leading to unintended exposure of sensitive or proprietary information. 5 | 6 | ## Extended Description 7 | Data leakage in ML/AI systems encompasses more than unauthorized database access; it can occur subtly when models unintentionally expose information about their training data. For example, models that overfit may allow inferences about the data they were trained on, presenting challenging-to-detect risks of potential data breaches. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Data Masking and Encryption**: Protect data at rest and in transit with encryption and mask sensitive details when feasible. 12 | - **Access Controls**: Deploy robust access control systems to ensure that only authorized personnel can reach sensitive data and models. 13 | - **Regular Audits**: Carry out frequent audits of data access logs and model outputs to uncover any leaks. 14 | - **Differential Privacy**: Apply noise to datasets or model outputs to prevent the re-identification of individual data points. 15 | - **Data Minimization**: Limit the use of data to what's necessary for training and operations to minimize the risk of leakage. 16 | - **Monitoring**: Set up real-time surveillance to identify abnormal data access patterns or potential security incidents. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-200**: Exposure of Sensitive Information to an Unauthorized Actor: Denotes the risk of accidentally revealing sensitive data. 21 | - **CWE-359**: Exposure of Private Personal Information (PPI): Highlights the dangers of leaking personal data. 22 | 23 | ## Impact Analysis 24 | 25 | - **Financial Impact**: Data breaches can lead to significant fines and are particularly costly in heavily regulated industries or areas with strict data protection laws. 26 | - **Reputation Damage**: Trust issues stemming from data leaks can affect relationships with clients, partners, and the wider stakeholder community, potentially resulting in lost business. 27 | - **Legal and Compliance Implications**: Non-compliance with data protection can lead to legal repercussions and sanctions. 28 | - **Operational Impact**: Breaches may interrupt business operations, requiring extensive efforts to resolve and recover from the incident. 29 | - **Intellectual Property Risks**: Leaks in certain fields could disclose proprietary methodologies or trade secrets, offering competitors unfair advantages. 30 | -------------------------------------------------------------------------------- /DataPoisoning.md: -------------------------------------------------------------------------------- 1 | # Susceptibility to Data Poisoning in ML Models 2 | 3 | ## Description 4 | Data poisoning is the deliberate corruption of training data for machine learning models, designed to skew the model's predictions or behavior. This manipulation can have profound implications on model integrity, necessitating protective measures for training and validation datasets. 5 | 6 | ## Extended Description 7 | Data poisoning attacks strike at the core of ML models—their data. Injecting malicious or flawed data into a training set can sway a model's decisions. The impact varies from subtle biases to full dysfunction or adversarial takeover, through either direct data tampering or indirect subversion of data collection methods. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Data Validation**: Apply rigorous validation to confirm data conforms to expected patterns and statistics. 12 | - **Outlier Detection**: Employ statistical methods to identify and manage outliers, potentially indicative of poisoning. 13 | - **Data Source Authentication**: Authenticate and secure data sources to preempt source-level poisoning. 14 | - **Model Interpretability**: Use interpretability tools to inspect and understand model behavior, which may reveal signs of poisoning. 15 | - **Regular Model Evaluation**: Routinely test model accuracy with trusted datasets to spot anomalies that could signal poisoning. 16 | - **Secure Data Storage**: Protect data repositories against unauthorized changes. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-707**: Improper Neutralization: Points to the dangers of not adequately addressing harmful inputs. 21 | - **CWE-20**: Improper Input Validation: Stresses the importance of stringent data input validation. 22 | 23 | ## Impact Analysis 24 | 25 | - **Operational Disruption**: A compromised model can malfunction, disrupting operations and leading to poor decision-making. 26 | - **Trust Erosion**: Exposure of model bias or malfunctions due to poisoning can diminish trust in the model and its operators. 27 | - **Economic Impact**: Faulty model decisions can result in financial detriment, particularly in critical sectors such as finance or healthcare. 28 | - **Strategic Misdirection**: In strategic contexts, a compromised model may guide an organization astray, potentially to the advantage of competitors or adversaries. 29 | - **Legal and Compliance Risks**: Decisions influenced by tainted data may breach regulations or ethical norms, incurring legal penalties. 30 | -------------------------------------------------------------------------------- /Improper-IAM-Models.md: -------------------------------------------------------------------------------- 1 | # Improper Implementation of Identity & Access Control for ML/AI Model Systems 2 | 3 | ## Description 4 | Machine Learning (ML) and Artificial Intelligence (AI) systems, due to their critical and often complex nature, require stringent identity and access control mechanisms. An improper implementation in these systems can lead to overly permissive agency in model interactions. This might allow unauthorized individuals or entities to access, manipulate, or even control the model's operations, parameters, or outputs. 5 | 6 | ## Extended Description 7 | As ML/AI models are increasingly integrated into decision-making processes, their access control becomes a focal point of security. An improper control mechanism could permit unauthorized training data uploads, access to intermediate model layers, alteration of model parameters, or theft of proprietary model architecture. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Role-Based Access Control (RBAC):** Implement RBAC to ensure that only authorized individuals can interact with the model based on predefined roles. 12 | 13 | - **Multi-Factor Authentication (MFA):** Enforce MFA for critical operations or administrative accesses to the ML/AI system. 14 | 15 | - **Regular Review and Audit:** Periodically review and audit user access rights, ensuring that no excessive permissions exist and obsolete permissions are revoked. 16 | 17 | - **Logging and Monitoring:** Maintain detailed logs of all access and operations on the ML/AI system. Use anomaly detection to identify and alert on unusual activities. 18 | 19 | - **Encryption:** Use encryption for data in transit and at rest, ensuring that even if unauthorized access occurs, the data remains protected. 20 | 21 | ## Related Weaknesses 22 | 23 | - **CWE-285:** Improper Authorization - A failure to ensure that a user is given the right level of access. 24 | - **CWE-287:** Improper Authentication - When the system doesn't verify the user's identity correctly. 25 | - **CWE-288:** Authentication Bypass Using an Alternate Path - Bypassing authentication mechanisms through alternative methods. 26 | 27 | ## Impact Analysis 28 | 29 | - **Data Breach:** Unauthorized access could expose sensitive training data or proprietary model details, leading to significant information leaks. 30 | 31 | - **Model Sabotage:** Malicious actors could alter model parameters or training data, causing the model to behave unpredictably or in a biased way. 32 | 33 | - **Operational Disruption:** Unauthorized actions could halt model operations, resulting in operational disruptions or downtimes. 34 | 35 | - **Loss of Trust:** Security incidents related to ML/AI models can severely damage the trustworthiness of the model's outputs and the organization's reputation. 36 | 37 | - **Economic Impact:** Data breaches or model misuses could result in significant financial losses, either through the misuse itself or through fines and penalties from regulatory bodies. 38 | 39 | --- 40 | -------------------------------------------------------------------------------- /Inadequate-Anomaly-Detection.md: -------------------------------------------------------------------------------- 1 | # Inadequate Anomaly Detection 2 | 3 | ## Description 4 | 5 | Inadequate Anomaly Detection often occurs when an LLM system fails to effectively flag or act upon abnormal user interaction patterns or unusual parameter manipulations. This lack of adequate anomaly detection mechanisms could enable potential malicious activities to go unnoticed. 6 | 7 | ## Extended Description 8 | 9 | Intelligent systems like LLMs often deal with scattered and complex data. In such environments, it's crucial to have robust anomaly detection mechanisms in place to swiftly spot and address suspicious behaviors. The absence of effective anomaly detection can lead to delayed or no response to unauthorized actions, and potentially expose the system to various security threats like unauthorized transactions, bot interactions, or parameter manipulations. These security incidences, if unchecked, could result in significant financial losses, operational disruptions, and reputational damage. 10 | 11 | ## Potential Mitigations 12 | 13 | - **Implement Anomaly Detection**: Institute an effective anomaly detection process that monitors user interactions and detects unusual activity. 14 | - **Robust Logging and Alerting**: Implement adequate logging of user activities and robust alerting mechanisms to be notified immediately when there's any suspicious behavior. 15 | - **Continuous Monitoring**: Set up continuous system monitoring to track trends, detect patterns, and promptly identify any anomalies. 16 | - **Regular Review**: Frequently review the system logs to spot any abnormal patterns that might have been ignored by automated systems. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-20**: Improper Input Validation: The inability to validate and restrict user inputs might lead to anomalies being unnoticed. 21 | - **CWE-778**: Insufficient Logging: The lack of detailed operational logs can prevent the discovery of anomalies. 22 | 23 | ## Impact Analysis 24 | 25 | - **Financial Impact**: Unlimited abnormal activities could result in financial loss. 26 | - **Operational Disruption**: Failure to promptly notice and address anomalies could disrupt normal system operations. 27 | - **Reputational Damage**: Incidents arising from unchecked anomalies could erode the trust of customers and other stakeholders. 28 | - **Security Threat**: Anomalies could be a sign of security threats, and if not addressed, may lead to more serious security issues. 29 | -------------------------------------------------------------------------------- /Inadequate-DR-Plan.md: -------------------------------------------------------------------------------- 1 | # Inadequate Disaster Recovery Plans for ML Systems 2 | 3 | ## Description 4 | Machine Learning (ML) systems' integration into operational frameworks is crucial for business decision-making and automation. Lack of robust disaster recovery (DR) plans exposes these systems to extended outages, risking data loss and service disruptions. 5 | 6 | ## Extended Description 7 | ML/AI ecosystems, which encompass data pipelines, training regimens, and inferencing processes, can be significantly impacted by disasters, hardware malfunctions, cyberattacks, or software errors. The absence of a detailed DR strategy, specifically designed for ML intricacies, makes system restoration a daunting and fallible task. 8 | 9 | ## Potential Mitigations 10 | 11 | - **DR Documentation**: Create a comprehensive DR plan detailing steps for ML system restoration, with regular updates as the system evolves. 12 | - **Regular Drills**: Conduct disaster simulations to evaluate and refine recovery procedures. 13 | - **Backup Routines**: Systematically backup all data, models, and settings in secure, diversified locations. 14 | - **Failover Systems**: Establish redundant or cloud-based systems for immediate switchover during primary system failures. 15 | - **Continuous Monitoring**: Utilize tools to promptly detect and address anomalies or failures, with automatic recovery or alerts. 16 | - **Training**: Train staff thoroughly on DR protocols. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-688**: Function Call With Incorrect Variable or Reference as Argument - Indicative of potential system errors. 21 | - **CWE-255**: Credentials Management - Critical for managing secure ML pipelines and data access during recovery. 22 | 23 | ## Impact Analysis 24 | 25 | - **Operational Disruption**: Extended outages can severely interrupt business processes and continuity. 26 | - **Data Loss**: Insufficient backup measures risk losing vital training data, model insights, or configurations. 27 | - **Financial Consequences**: System downtime can inflict financial damage due to halted services or operations. 28 | - **Reputation Damage**: Inability to quickly recover ML systems can undermine stakeholder confidence. 29 | - **Compliance Violations**: Non-compliant DR strategies may breach regulatory standards, attracting fines or legal repercussions. 30 | -------------------------------------------------------------------------------- /Inadequate-ML-Grounding.md: -------------------------------------------------------------------------------- 1 | # ML Models with Inadequate Grounding 2 | 3 | ## Description 4 | Machine Learning models are designed to identify patterns and make predictions based on their training data. However, without sufficient grounding—essentially being anchored in an accurate representation of real-world facts—they may produce invalid or unpredictable results, which can have serious repercussions. 5 | 6 | ## Extended Description 7 | Grounding in ML is how well a model's predictions align with real-world truths. Inadequately grounded models may give outputs that are consistent within their own logic but are nonsensical or incorrect when applied to real-world situations. For instance, a chatbot might generate a grammatically perfect but factually incorrect or illogical response due to poor grounding. 8 | 9 | ## Potential Mitigations 10 | - **Robust Grounding Techniques**: Use techniques to ensure outputs are well-correlated with real-world truths. 11 | - **Continuous Model Training**: Regularly update the model with new, diverse data to better reflect current scenarios. 12 | - **Validation Datasets**: Employ datasets that test model predictions against factual truths. 13 | - **Feedback Loops**: Enable user or expert feedback on inaccurate outputs. 14 | - **Human-in-the-loop**: For critical applications, combine automated predictions with human review. 15 | - **Domain Knowledge Integration**: Infuse the model with expert knowledge in relevant fields. 16 | 17 | ## Related Weaknesses 18 | - **CWE-693**: Protection Mechanism Failure: Overlaps conceptually with failures in grounding as a protective measure. 19 | - **CWE-834**: Excessive Data Exposure: May occur due to inadequately grounded outputs. 20 | 21 | ## Impact Analysis 22 | - **Misinformed Decisions**: Incorrect predictions in high-stakes fields can have dire consequences. 23 | - **Loss of User Trust**: Consistent reality-disconnected outputs can erode confidence in the system. 24 | - **Operational Jeopardy**: Poor decisions based on weak grounding can disrupt critical operations. 25 | - **Legal and Ethical Implications**: Untruthful outputs could lead to legal and moral issues. 26 | - **Increased Overheads**: Constant correction and monitoring due to inadequate grounding can reduce efficiency and increase costs. 27 | -------------------------------------------------------------------------------- /Insecure-Model-Arch.md: -------------------------------------------------------------------------------- 1 | # Reliance on Insecure Model Architectures 2 | 3 | ## Description 4 | Machine Learning (ML) models can harbor inherent design or architectural vulnerabilities, paralleling traditional software risks. Deploying these insecure architectures can compromise dependent systems. 5 | 6 | ## Extended Description 7 | As ML/AI increasingly integrates with everyday activities and critical decision-making, ensuring model robustness is essential. Analogous to code vulnerabilities in software, ML models may possess flaws in their design, training, or structure that are exploitable or lead to undesired outcomes. These could stem from training phase oversights, susceptibility to adversarial exploits, or utilizing outdated models, all with potentially extensive repercussions. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Model Evaluation**: Routinely assess models for weaknesses with methods like adversarial testing. 12 | - **Stay Updated**: Keep abreast of scholarly and industry updates on vulnerabilities in standard model architectures. 13 | - **Architectural Review**: Critically analyze a model's architecture against industry benchmarks before deployment. 14 | - **Retraining and Updating**: Consistently update and retrain models with current, secure data. 15 | - **External Audits**: Engage in intermittent independent security reviews of ML architectures. 16 | - **Fallback Mechanisms**: Establish alternative processes for continuity if a model malfunctions. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-693**: Protection Mechanism Failure - Indicates a general security assurance deficiency. 21 | - **CWE-664**: Improper Control of a Resource Through its Lifetime - Relevant if models lack ongoing maintenance or updates. 22 | 23 | ## Impact Analysis 24 | 25 | - **System Vulnerabilities**: Inherent model flaws can render systems attack-prone. 26 | - **Compromised Data Integrity**: Vulnerable models with write permissions might distort or damage data. 27 | - **Unreliable Outputs**: Non-robust models may yield unreliable or incorrect results, affecting decision quality. 28 | - **Operational Disruption**: A model's breakdown or breach can interrupt critical workflows. 29 | - **Loss of Stakeholder Trust**: Observable inconsistencies or security gaps can diminish stakeholder confidence in the system or its managing entity. 30 | -------------------------------------------------------------------------------- /Insufficient-InputValidation.md: -------------------------------------------------------------------------------- 1 | # Insufficient Input Validation in AI Interfaces 2 | 3 | ## Description 4 | AI systems are highly dependent on the quality of input data, which can vary from structured datasets to dynamic user inputs. Insufficient input validation makes these systems susceptible to anomalies, affecting their performance and security. 5 | 6 | ## Extended Description 7 | The integrity of input data is critical, especially for AI systems engaged in real-world applications. Inadequate input validation can result in issues like model drift, data breaches, or even complete system compromise. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Input Validation Routines**: Ensure inputs are rigorously checked for conformity to expected formats, ranges, and types. 12 | - **Schema Definitions**: Define the expected data schema clearly to prevent any ambiguities between structure and content. 13 | - **Whitelisting**: Enforce a predefined list of acceptable inputs, discarding all that do not conform. 14 | - **Boundary Checks**: Inputs must be checked to ensure they are within safe and expected limits. 15 | - **Regular Audits**: Periodically examine input validation protocols to adapt to new data trends and potential threats. 16 | - **User Education**: Inform data providers on the correct input formats and the importance of data integrity. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-20**: Relates to the risks involved when inputs are not validated correctly. 21 | - **CWE-89**: Highlights the dangers of SQL injections due to poor input handling. 22 | - **CWE-74**: Focuses on the implications of passing unvalidated inputs to other system components. 23 | 24 | ## Impact Analysis 25 | 26 | - **System Crashes**: Erroneous inputs may lead to unexpected system behavior or crashes. 27 | - **Data Corruption**: Compromised inputs can degrade the quality of datasets, impacting AI decision-making. 28 | - **Injection Attacks**: Vulnerabilities like SQL or script injections become a risk with poor validation. 29 | - **Model Manipulation**: Attackers may exploit input handling flaws to alter model outputs. 30 | - **Operational Disruption**: Frequent input validation issues can cause operational inefficiencies or downtimes. 31 | -------------------------------------------------------------------------------- /Insufficient-Logging-MLOps.md: -------------------------------------------------------------------------------- 1 | # Insufficient Logging & Monitoring in ML Operations 2 | 3 | ## Description 4 | Comprehensive logging and monitoring in ML/AI systems are essential to ensure transparency, traceability, and accountability. The absence of such mechanisms or their insufficiency can compromise the system's security posture by making it challenging to detect, analyze, and respond to anomalous or malicious activities. A lack of non-repudiation further complicates the verification of actions, possibly leading to unauthorized changes going unnoticed. 5 | 6 | ## Extended Description 7 | ML/AI models often function as black boxes, making their operations opaque to users. Without proper logging and monitoring, it becomes exponentially more challenging to identify the root causes of unexpected outputs, diagnose biases, or trace unauthorized or unintended use, such as prompt injection attacks or modifications. Such shortcomings not only affect the model's reliability but also expose it to various security threats, from data poisoning to backdoor attacks. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Comprehensive Logging:** Implement detailed logging of all interactions, including data input, model parameter changes, and output requests. Ensure that logs are immutable and timestamped. 12 | 13 | - **Real-time Monitoring:** Use monitoring tools that provide real-time visibility into model operations and generate alerts for suspicious activities. 14 | 15 | - **Integration with SIEM Systems:** Integrate logs with Security Information and Event Management (SIEM) systems for centralized analysis and correlation. 16 | 17 | - **Periodic Reviews:** Conduct regular reviews of logs to identify patterns, anomalies, or potential security threats. 18 | 19 | - **Log Protection:** Ensure logs are stored securely, with restricted access and encrypted if necessary, to prevent tampering or unauthorized access. 20 | 21 | - **Backup and Retention:** Maintain backups of logs and define a suitable retention policy, considering both operational needs and compliance requirements. 22 | 23 | ## Related Weaknesses 24 | 25 | - **CWE-778:** Insufficient Logging - The software does not attempt to record any security-relevant information when such behavior is unexpected. 26 | 27 | - **CWE-223:** Omission of Security-relevant Information - When the software does not record or display information that would be important for security-related decisions. 28 | 29 | - **CWE-250:** Execution with Unnecessary Privileges - This can further exacerbate the issues if logs are manipulated by users with excessive rights. 30 | 31 | ## Impact Analysis 32 | 33 | - **Compromised Incident Response:** Without proper logs, incident response teams might struggle to identify the cause, source, and extent of a security breach. 34 | 35 | - **Forensic Challenges:** Inadequate logs can hinder forensic investigations, making it difficult to ascertain the sequence of events leading to an incident. 36 | 37 | - **Regulatory and Compliance Issues:** Insufficient logging can lead to non-compliance with industry regulations, resulting in potential legal implications and fines. 38 | 39 | - **Increase in Successful Exploitation of Vulnerabilities:** If malicious activities go undetected due to poor logging, vulnerabilities might remain unpatched, exposing the system to further attacks. 40 | 41 | - **Loss of Reputation:** A perceived lack of transparency and traceability can erode trust among users or stakeholders, affecting the organization's reputation. 42 | -------------------------------------------------------------------------------- /Insufficient-OutputSanitization-ContentFiltering.md: -------------------------------------------------------------------------------- 1 | # Inadequate Output Sanitization & Content Filtering in ML Outputs 2 | 3 | ## Description 4 | Machine Learning (ML) models generate outputs based on their training and the inputs they receive. In the absence of rigorous sanitization and content filtering, these outputs can be unpredictable, inappropriate, or subject to exploitation. Outputs that do not meet expected standards or pass necessary checks pose security and operational risks. 5 | 6 | ## Extended Description 7 | ML models, especially those interacting with real-time user inputs, are prone to adversarial attacks or may produce outputs with harmful content. In situations where ML outputs drive user interactions, decision-making, or content generation, the lack of proper output sanitization and filtering is a significant concern. These issues can range from benign non-useful outputs to those that are manipulatively dangerous. 8 | 9 | ## Potential Mitigations 10 | - **Output Validation:** Implement validation processes to ensure outputs meet expected norms, standards, or constraints. 11 | - **Sanitization Routines:** Develop and apply methods for cleaning or neutralizing potential harm, inappropriateness, or unexpected elements in outputs. 12 | - **Content Filters:** Utilize content filtering to scrutinize outputs against lists of undesirable or hazardous content, acting upon any identified issues. 13 | - **Feedback Loops:** Enable systems or users to report problematic outputs, using this information to improve model accuracy. 14 | - **Continuous Model Retraining:** Conduct regular model retraining with enhanced datasets to minimize unsanitized outputs. 15 | - **Review & Monitoring:** Continually evaluate output samples and monitor for anomalies, particularly for critical applications. 16 | 17 | ## Related Weaknesses 18 | - **CWE-116:** Improper Encoding or Escaping of Output – Concerns output interpretation that can lead to exploitation. 19 | - **CWE-79:** Improper Neutralization of Input During Web Page Generation (‘Cross-site Scripting’) – Pertains to outputs resulting in XSS vulnerabilities. 20 | - **CWE-20:** Improper Input Validation – Addresses vulnerabilities introduced by inadequate validation of any data process, including outputs. 21 | 22 | ## Impact Analysis 23 | - **System Exploits:** Vulnerable outputs may enable attacks such as XSS or SQL injection in web applications. 24 | - **Misinformation:** Incorrect outputs from ML models can lead to poor decision-making or spread of misinformation. 25 | - **Operational Risks:** Inappropriate content can damage reputation, erode user trust, and create legal issues. 26 | - **Data Poisoning:** Poorly managed inputs can adversely affect the model, causing a downward spiral in output quality. 27 | - **User Trust:** The credibility of a system can be compromised if it consistently generates unsanitized or offensive outputs. 28 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 basicScandal 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Lacking-Env-Segmentation-Gap-MLOps.md: -------------------------------------------------------------------------------- 1 | # Lack of Environment Segmentation in ML Operations 2 | 3 | ## Description 4 | In the world of ML/AI operations, the sanctity of environments - be it development, testing, or production - is paramount. When these environments lack adequate segmentation or isolation, it opens the door to unintended interactions or unauthorized accesses. Such weak segmentation can allow malicious actors or even accidental actions to cross boundaries, potentially affecting critical components, leaking sensitive data, or altering model behavior. 5 | 6 | ## Extended Description 7 | As ML/AI models evolve from development to deployment, they transition through multiple environments. Each environment has its unique purpose and sensitivity level. Without proper segmentation, there's a risk that experimental changes in a development environment might unintentionally propagate to a production system. Similarly, if a testing environment gets compromised, an attacker might gain insights or pathways to compromise the more secure production environment. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Clear Trust Boundaries:** Define and document the trust boundaries between different environments. Ensure that each environment has its distinct set of credentials and access controls. 12 | - **Network Segmentation:** Use network-level controls like firewalls, virtual LANs, and access control lists to restrict traffic between environments. 13 | - **Containerization:** Utilize container technologies to encapsulate processes, ensuring that applications in one environment cannot interfere with another. 14 | - **Role-Based Access Control (RBAC):** Assign roles specific to each environment, ensuring that users or processes can only access the environment relevant to their role. 15 | - **Regular Audits:** Conduct periodic security audits to ensure that the segmentation mechanisms are functioning as intended and to identify potential weaknesses. 16 | - **Environment-Specific Monitoring:** Implement monitoring solutions tailored to each environment's specifics, ensuring anomalies or breaches are quickly detected. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-653:** Insufficient Compartmentalization - Failure to separate or compartmentalize resource pools to control or limit damage. 21 | - **CWE-668:** Exposure of Resource to Wrong Sphere - When a resource is accidentally exposed to another sphere, leading to unintended resource access. 22 | - **CWE-272:** Least Privilege Violation - When a process or function has more privileges than needed, potentially affecting other environments. 23 | 24 | ## Impact Analysis 25 | 26 | - **Data Leakage:** Without proper segmentation, sensitive data from a production environment could leak into less secure development or testing environments. 27 | - **Model Tampering:** Malicious actors might manipulate a model in a less secure environment and leverage weak segmentation to propagate these changes to production. 28 | - **Operational Disruptions:** Unintended interactions between environments can lead to operational issues, such as system downtimes or degraded performance. 29 | - **Increased Attack Surface:** Every environment without proper segmentation becomes a potential point of entry for attackers, thereby increasing the overall attack surface. 30 | - **Loss of Stakeholder Trust:** Breaches or visible system errors due to poor segmentation can erode trust among users, clients, or stakeholders. 31 | -------------------------------------------------------------------------------- /Lacking-Output-Disclaimer.md: -------------------------------------------------------------------------------- 1 | # Absence of Disclaimer in ML/AI Outputs 2 | 3 | ## Description 4 | The ML/AI system provides outputs, predictions, or decisions without any accompanying disclaimer clarifying the potential limitations, uncertainties, or risks associated with the generated results. This absence might lead users to over-rely on the system's outputs without considering its inherent limitations or the context of its training data. 5 | 6 | ## Potential Mitigations 7 | 8 | - **Disclaimers**: Always accompany ML/AI outputs with clear disclaimers that provide context for the generated results. 9 | - **User Education**: Educate users on the potential risks and limitations associated with the AI/ML system's outputs. 10 | - **Human-in-the-Loop**: In applications where the risk of misinterpretation is high, consider including a human-in-the-loop to validate and contextualize AI outputs. 11 | 12 | ## Related Weaknesses 13 | 14 | - **CWE-693**: Protection Mechanism Failure (This is a broad category, but the idea of not having a protection mechanism, such as a disclaimer, fits this CWE in a conceptual manner.) 15 | 16 | ## Impact Analysis 17 | Absence of a disclaimer can lead to: 18 | 19 | - **Misunderstanding**: Users might misunderstand the capabilities and limitations of the AI/ML system. 20 | - **Over-reliance**: Users might over-rely on the system's outputs, potentially making critical decisions based solely on AI predictions. 21 | - **Liability**: Organizations deploying the ML/AI system might face legal or reputational challenges if users feel they were misled by system outputs. 22 | -------------------------------------------------------------------------------- /Lacking-RateLimiting.md: -------------------------------------------------------------------------------- 1 | # Absence of Rate Limiting in ML/AI Interfaces 2 | 3 | ## Description 4 | Machine Learning and Artificial Intelligence interfaces, when exposed without rate-limiting measures, are prone to overuse and abuse. The lack of rate limiting can lead to resource exhaustion and service disruption due to Denial-of-Service attacks, which may be caused by malicious bots, automated scripts, or system misconfigurations. 5 | 6 | ## Extended Description 7 | Rate limiting is crucial to prevent a system from being overwhelmed by too many requests. It is a critical security and operational measure that ensures the ML/AI systems' computational resources are not exhausted, by controlling the number of requests from an entity within a specified time frame. 8 | 9 | ## Potential Mitigations 10 | - **Rate Limiting Protocols**: Employ algorithms like token bucket or leaky bucket to manage request volume. 11 | - **IP-based Limiting**: Restrict requests based on the requester's IP address. 12 | - **User Authentication and Tiered Access**: Authenticate users and assign request limits based on user levels. 13 | - **Monitoring and Alerts**: Set up continuous monitoring and alerts for abnormal traffic patterns. 14 | - **Captcha or Challenge-Response**: Use Captchas or challenges for web-facing interfaces to block automated bots. 15 | - **Geo-based Rate Limiting**: Apply different request limits for regions with known malicious activities. 16 | 17 | ## Related Weaknesses 18 | - **CWE-770**: Pertains to the allocation of resources without limits or throttling. 19 | - **CWE-400**: Addresses the risk of uncontrolled resource consumption. 20 | 21 | ## Impact Analysis 22 | - **Resource Depletion**: Excessive traffic can cause slowdowns or outages. 23 | - **Denial-of-Service Attacks**: Attackers could disrupt the service for legitimate users. 24 | - **Increased Operational Costs**: High traffic can lead to higher costs, especially in cloud-based environments. 25 | - **Degraded User Experience**: Service quality for regular users may decrease due to high load. 26 | - **Potential for Exploits**: High volumes of traffic can reduce the scrutiny of inputs, leading to possible exploits. 27 | -------------------------------------------------------------------------------- /Lacking-User-Feedback-Mechanism.md: -------------------------------------------------------------------------------- 1 | # Absence of User Feedback Mechanism in AI Systems 2 | 3 | ## Description 4 | A vital aspect of refining and ensuring the reliability of AI systems is to have mechanisms that gather feedback from users. When systems lack this essential component, it diminishes their adaptability and responsiveness. Without the ability for users to report erroneous outputs or unexpected behaviors, the system remains ignorant of its potential flaws and vulnerabilities. 5 | 6 | ## Extended Description 7 | AI systems, by their very nature, evolve and improve based on feedback. Without a structured and user-friendly mechanism for end-users to provide feedback, critical insights into system anomalies, biases, and potential improvements remain untapped. Over time, not only does the system fail to adapt to changing needs or correct its errors, but user trust and confidence in the system also diminish. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Feedback Loops:** Design and integrate easily accessible feedback loops that allow users to report system errors, unexpected outcomes, or suggestions. 12 | - **Feedback Analysis:** Periodically analyze the feedback to identify common issues, trends, or areas of improvement. 13 | - **User Education:** Provide guidelines or tutorials to users, helping them understand the importance of their feedback and how to provide effective insights. 14 | - **Prioritize User Feedback:** Ensure that feedback from users is given due importance in system improvement strategies and development roadmaps. 15 | - **Feedback Response System:** Implement mechanisms to notify users when their feedback results in system enhancements or fixes, further encouraging participation. 16 | 17 | ## Related Weaknesses 18 | 19 | - **CWE-693:** Protection Mechanism Failure - This emphasizes the failure of protective mechanisms that don't consider feedback as a crucial aspect of system integrity. 20 | - **CWE-352:** Cross-Site Request Forgery (CSRF) - Without user feedback, vulnerabilities like CSRF might remain undetected in systems with web interfaces. 21 | - **CWE-200:** Exposure of Sensitive Information - Users might identify sensitive information leakages faster than automated systems, highlighting the importance of their feedback. 22 | 23 | ## Impact Analysis 24 | 25 | - **System Stagnation:** Without feedback, AI systems might remain stagnant, failing to adapt or improve over time. 26 | - **Erosion of User Trust:** Users may lose trust in a system that repeatedly produces errors without any avenue to report or rectify them. 27 | - **Prolonged Vulnerabilities:** In the absence of feedback, system vulnerabilities might persist longer, increasing the risk of exploitation. 28 | - **Decreased System Utility:** An AI system that doesn't evolve based on user feedback might become less relevant or useful over time. 29 | - **Loss of Competitive Edge:** Systems that don't incorporate user feedback might fall behind in comparison to competitors that do. 30 | 31 | -------------------------------------------------------------------------------- /PromptInjection.md: -------------------------------------------------------------------------------- 1 | # Vulnerable Prompt Injection in ML/AI Systems 2 | 3 | ## Description 4 | The vulnerability of ML/AI systems to prompt injection attacks arises when external entities can influence or inject malicious prompts, leading the models to generate unintended or harmful content. 5 | 6 | ## Extended Description 7 | Prompt injection in ML/AI systems can manipulate a model's output. Similar to traditional injection attacks in software systems, attackers can craft specific prompts that, when processed by the model, produce outputs beneficial to the attacker. These might include misleading information, inappropriate content, or outputs that exploit downstream system vulnerabilities. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Input Validation**: Ensure that all prompts or inputs to the models are strictly validated against a set of accepted values or patterns. 12 | - **Output Sanitization**: Even after input validation, sanitize the outputs from the models to remove or neutralize potentially harmful content. 13 | - **Restrict Prompt Flexibility**: Use fixed or restricted prompt structures where possible to minimize the risk of malicious inputs. 14 | - **User Role Management**: Limit the ability to modify or inject prompts based on user roles, ensuring that only trusted users can influence model inputs. 15 | - **Monitoring and Logging**: Continuously monitor model interactions and log all prompt modifications or insertions to trace potential attacks and respond promptly. 16 | - **Escaping Special Characters**: Neutralize special characters or sequences that might have special meanings for the ML/AI model. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-74**: Improper Neutralization of Special Elements in Output Used by a Downstream Component ('Injection'): This highlights the generic risk of injection attacks, where external inputs can influence system behavior. 21 | - **CWE-20**: Improper Input Validation: Underscores the need for stringent validation routines to prevent harmful inputs. 22 | 23 | ## Impact Analysis 24 | 25 | - **Misleading Outputs**: Systems may produce information that is incorrect or serves the attacker's intentions. 26 | - **System Exploitation**: Crafted prompts can cause models to produce outputs that result in exploitable behavior in downstream systems or applications. 27 | - **Reputation Damage**: Generating harmful content due to prompt injection can erode user trust and harm the reputation of the ML/AI system or the organization. 28 | - **Legal and Compliance Risks**: Producing misleading or harmful content, especially in regulated sectors, might lead to legal consequences or compliance breaches. 29 | - **User Data Risks**: If the system processes or acts upon user data based on injected prompts, it may reveal sensitive information or make incorrect decisions based on malicious outputs. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # LLM Integration & Application Findings Templates 2 | 3 | Welcome to the **LLM Integration & Application Findings Templates** repository. This collection of open-source templates is designed to facilitate the reporting and documentation of vulnerabilities and opportunities for usability improvement in LLM integrations and applications. 4 | 5 | ## What is LLM Testing Findings? 6 | LLM Testing Findings is an open-source initiative aimed at fostering a deeper understanding of large language models, their capabilities, limitations, and implications in various fields, particularly cybersecurity. The project is an evolving compilation of findings, tools, and methodologies developed by experts at Bishop Fox. 7 | 8 | ## Project Description 9 | 10 | The integration of large language models (LLMs) into various applications introduces new challenges in maintaining security and optimizing user experiences. This repository provides a structured means for testers, developers, and security analysts to report findings comprehensively. 11 | 12 | ## Getting Started 13 | To begin using this repository, clone it to your local machine: 14 | 15 | `git clone https://github.com/BishopFox/llm-testing-findings.git` 16 | 17 | ## How to Use These Templates 18 | 19 | Each template is crafted to address specific issues within LLM integrations and applications. To use these templates: 20 | 21 | 1. **Select a Template**: Identify the template that corresponds to your finding. 22 | 2. **Fill in the Template**: Provide all requested information within the template to ensure thorough documentation of the issue. 23 | 3. **Submit Your Report**: Share your completed report with the relevant stakeholders or project maintainers for further action. 24 | 25 | ## How to Contribute 26 | 27 | Contributions are welcome and encouraged! To contribute: 28 | 29 | 1. **Fork this Repository**: Create a personal fork of the project on GitHub. 30 | 2. **Modify or Add Templates**: Make changes to existing templates or create new ones that could benefit the community. 31 | 3. **Create a Pull Request**: Propose your changes through a pull request, and provide a summary of your modifications or additions. 32 | 4. **Await Review**: Allow time for the project maintainers to review and merge your contributions. 33 | 5. **Feedback and Discussions:** Join our [Discussions](https://github.com/BishopFox/llm-testing-findings/discussions) forum to share your thoughts or ask questions. 34 | 35 | ## Acknowledgements 36 | 37 | A special thanks to all contributors and community members who have participated in this project. Your insights and collaboration are invaluable to the success and growth of LLM Testing Findings. 38 | 39 | ## Contact 40 | 41 | For any additional questions or information, please email us at [contact@bishopfox.com](mailto:contact@bishopfox.com). 42 | 43 | 44 | ## License 45 | 46 | All templates in this repository are provided under the [MIT License](LICENSE.md). Your contributions are assumed to be under the same license. 47 | 48 | 49 | ## Community and Support 50 | 51 | Questions, comments, or need assistance? Open an issue in this repository, and a maintainer will assist you. 52 | 53 | Thank you for your contributions to enhancing the security and usability of LLM integrations and applications. 54 | 55 | - **Discussions:** Join the conversation in our [GitHub Discussions](https://github.com/BishopFox/llm-testing-findings/discussions). 56 | - **Social Media:** Follow us on [Twitter](#https://twitter.com/bishopfox) and [LinkedIn](#https://www.linkedin.com/company/bishop-fox/) for the latest updates. 57 | - **Blog:** Dive deeper into our findings on our [official blog](#https://bishopfox.com/blog). 58 | 59 | --- 60 | *This project is maintained by Rob Ragan [rragan@bishopfox.com](mailto:rragan@bishopfox.com) & the awesome team of passionate hackers at Bishop Fox. Committed to excellence in LLM integration security and usability.* 61 | -------------------------------------------------------------------------------- /Unrestricted-Task-Execution.md: -------------------------------------------------------------------------------- 1 | # Unrestricted-Task-Execution 2 | 3 | ## Description 4 | 5 | The problem of Unrestricted Task Execution in Large Language Model (LLM) systems arises when external entities can manipulate the model's task encoding capabilities to perform arbitrary tasks, far beyond its original design and purpose. 6 | 7 | ## Extended Description 8 | 9 | LLMs, intentionally programmed for specific tasks, may fall prey to manipulation when task specificity constraints are reprogrammed, nonexistent, or if input controls are bypassed. Under such situations, an unexpected scope arises, enabling an attacker to exploit the LLM well beyond its defined purpose. The potential ramifications extend from generation of unforeseen or even harmful responses leading to reputational damage, up to escalating computational demands. If the scope and intricacy of tasks are not in line with resource allocation, it puts a strain on computational resources and could lead to unexpected cost overruns. 10 | 11 | ## Potential Mitigations 12 | 13 | - **Task Specificity**: Solidify task specificity by enforcing stringent constraints regarding the scope of tasks the LLM can execute. 14 | - **Input Control**: Implement robust control over user inputs to prevent manipulation and control of the model's intended function. 15 | - **Output Validation**: Introduce an additional layer of validation to assess the relevance and appropriateness of the responses against the model's intended purpose. 16 | - **Monitoring and Alert Systems**: Implement comprehensive monitoring and alert mechanisms to identify and respond to unintended task execution. 17 | - **User Role Management**: Assign roles and manage users to limit the level of interaction based on the level of trust and authority of each user. 18 | 19 | ## Related Weaknesses 20 | 21 | - **CWE-285**: Improper Authorization: The LLM inadvertently allows unauthorized users to manipulate task prompts, leading to execution paths that diverge from its intended purpose. 22 | - **CWE-284**: Improper Access Control: The issue manifests due to a lack of stringent access controls that can limit the LLM's behavior. 23 | - **CWE-250**: Execution with Unnecessary Privileges: It is crucial to limit the LLM's abilities and prevent exercise of unnecessary privileges. 24 | 25 | ## Impact Analysis 26 | 27 | - **Reputation Damage**: Unpredictability and potential misuse of the LLM could lead to reputation damage. 28 | - **System Resources Usage**: The LLM may consume additional system resources while performing unintended tasks causing performance impacts and/or cost overruns. 29 | - **Functionality Misuse**: The LLM may be used to perform actions that it was not originally intended to undertake resulting in misuse. 30 | - **Increased Attack Surface**: The broader the task-performing abilities of the LLM, the larger the attack surface. 31 | - **Loss of Functional Control**: Deviating from its intended utility, the LLM may become an uncontrolled tool if not managed properly. 32 | -------------------------------------------------------------------------------- /Violation-DifferentialPrivacy.md: -------------------------------------------------------------------------------- 1 | # Violation of Differential Privacy in ML Training Data 2 | 3 | ## Description 4 | Differential privacy is a strategy to share dataset information while protecting individual participants. A breach implies that ML training data may compromise personal data confidentiality, undermining privacy guarantees. 5 | 6 | ## Extended Description 7 | Real-world data is vital for ML efficacy, yet it raises significant privacy issues. Even with overt identifiers removed, subtle data analysis or data merging can divulge sensitive personal details. Differential privacy counters this by introducing noise into data or outputs to prevent isolating any individual, but lapses in this approach can lead to privacy invasions. 8 | 9 | ## Potential Mitigations 10 | 11 | - **Differential Privacy Implementation**: Incorporate differential privacy methods and tools in data handling and model training. 12 | - **Data Anonymization**: Strip datasets of direct identifiers. 13 | - **Regular Audits**: Perform ongoing checks to maintain differential privacy norms and detect potential privacy breaches. 14 | - **Data Usage Policies**: Define and enforce strict rules on data accessibility and utilization. 15 | - **Training on Aggregated Data**: Prefer aggregated datasets over detailed records to lessen individual data exposure risks. 16 | - **User Consent**: Guarantee informed user consent for data usage and potential privacy risks. 17 | 18 | ## Related Weaknesses 19 | 20 | - **CWE-359**: Exposure of Private Information ('Privacy Violation'): Highlights the importance of personal data confidentiality. 21 | - **CWE-200**: Exposure of Sensitive Information: Accentuates the broader issue of accidental information disclosure. 22 | 23 | ## Impact Analysis 24 | 25 | - **Legal Consequences**: Disregarding privacy can trigger data protection law violations, attracting substantial fines and legal challenges. 26 | - **User Trust Erosion**: As privacy concerns mount, breaches can significantly diminish user trust, which is hard to restore. 27 | - **Economic Impact**: The financial ramifications extend beyond fines to potential customer attrition and revenue loss. 28 | - **Reputation Damage**: A reputation for data negligence can have enduring adverse effects on an organization's image. 29 | - **Ethical Implications**: Privacy breaches have ethical ramifications, particularly for companies in sensitive fields like healthcare or finance. 30 | -------------------------------------------------------------------------------- /index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 🦊 LLM Testing Findings 7 | 8 | 39 | 40 | 41 | 42 |
43 |

LLM Testing Findings

44 | 45 | 119 |
120 | 121 | 122 | 123 | --------------------------------------------------------------------------------