In addition to having an inventory of existing tools in use, there also should be a process to onboard and offboard future tools and services from the organizational inventory securely.
AI security and privacy training
It’s often quipped that “humans are the weakest link,” however that doesn’t need to be the case if an organization properly integrates AI security and privacy training into their generative AI and LLM adoption journey.
This involves helping staff understand existing generative AI/LLM initiatives, as well as the broader technology and how it functions, and key security considerations, such as data leakage. Additionally, it is key to establish a culture of trust and transparency, so that staff feel comfortable sharing what generative AI and LLM tools and services are being used, and how.
A key part of avoiding shadow AI usage will be this trust and transparency within the organization, otherwise, people will continue to use these platforms and simply not bring it to the attention of IT and Security teams for fear of consequences or punishment.
Establish business cases for AI use
This one may be surprising, but much like with the cloud before it, most organizations don’t actually establish coherent strategic business cases for using new innovative technologies, including generative AI and LLM. It is easy to get caught in the hype and feel you need to join the race or get left behind. But without a sound business case, the organization risks poor outcomes, increased risks and opaque goals.
Governance
Without Governance, accountability and clear objectives are nearly impossible. This area of the checklist involves establishing an AI RACI chart for the organization’s AI efforts, documenting and assigning who will be responsible for risks and governance and establishing organizational-wide AI policies and processes.
Legal
While obviously requiring input from legal experts beyond the cyber domain, the legal implications of AI aren’t to be underestimated. They are quickly evolving and could impact the organization financially and reputationally.
This area involves an extensive list of activities, such as product warranties involving AI, AI EULAs, ownership rights for code developed with AI tools, IP risks and contract indemnification provisions just to name a few. To put it succinctly, be sure to engage your legal team or experts to determine the various legal-focused activities the organization should be undertaking as part of their adoption and use of generative AI and LLMs.
Regulatory
To build on the legal discussions, regulations are also rapidly evolving, such as the EU’s AI Act, with others undoubtedly soon to follow. Organizations should be determining their country, state and Government AI compliance requirements, consent around the use of AI for specific purposes such as employee monitoring and clearly understanding how their AI vendors store and delete data as well as regulate its use.
Using or implementing LLM solutions
Using LLM solutions requires specific risk considerations and controls. The checklist calls out items such as access control, training pipeline security, mapping data workflows, and understanding existing or potential vulnerabilities in LLM models and supply chains. Additionally, there is a need to request third-party audits, penetration testing and even code reviews for suppliers, both initially and on an ongoing basis.
Testing, evaluation, verification, and validation (TEVV)
The TEVV process is one specifically recommended by NIST in its AI Framework. This involves establishing continuous testing, evaluation, verification, and validation throughout AI model lifecycles as well as providing executive metrics on AI model functionality, security and reliability.
Model cards and risk cards
To ethically deploy LLMs, the checklist calls for the use of model and risk cards, which can be used to let users understand and trust the AI systems as well as openly addressing potentially negative consequences such as biases and privacy.
These cards can include items such as model details, architecture, training data methodologies, and performance metrics. There is also an emphasis on accounting for responsible AI considerations and concerns around fairness and transparency.
RAG: LLM optimizations
Retrieval-augmented generation (RAG) is a way to optimize the capabilities of LLMs when it comes to retrieving relevant data from specific sources. It is a part of optimizing pre-trained models or re-training existing models on new data to improve performance. The checklist recommended implementing RAG to maximize the value and effectiveness of LLMs for organizational purposes.
AI red teaming
Lastly, the checklist calls out the use of AI red teaming, which is emulating adversarial attacks of AI systems to identify vulnerabilities and validate existing controls and defenses. It does emphasize that red teaming alone isn’t a comprehensive solution or approach to securing generative AI and LLMs but should be part of a comprehensive approach to secure generative AI and LLM adoption.
That said, it is worth noting that organizations need to clearly understand the requirements and ability to red team services and systems of external generative AI and LLM vendors to avoid violating policies or even find themselves in legal trouble as well.
