Artificial intelligence (AI) is rapidly reshaping the cybersecurity landscape, offering both promise and new problems for IT teams. On the plus side, AI offers powerful tools to strengthen security postures. It can detect intrusions by recognizing unusual patterns, analyze source code for hidden vulnerabilities, and process vast streams of surveillance data faster and more accurately than human analysts.
These capabilities empower organizations to act more quickly and decisively.
On the other hand, AI introduces a new category of security risks. Poor data classification and labeling practices can lead to unauthorized access, particularly when sensitive information is used to train AI models. The widespread adoption of generative AI (genAI) tools increases the chances that users will inadvertently expose confidential data.
Generative AI can create fictional narratives for use in highly targeted “spear phishing” emails on a large scale. Its coding capabilities can be used to create more sophisticated malware and embed it in open-source software, which is a prime vector for software supply chain attacks. The cost of those attacks is expected to triple, to $138 billion, by 2031, from $46 billion in 2023 1.
The “black box” nature of large language models can make it difficult to trace the origin and authenticity of AI-generated outputs. Adversaries can exploit that ambiguity to create prompt responses that encourage users to divulge sensitive information. Users may also unintentionally disclose intellectual property that can be incorporated into the model and shared with others.
IT leaders need to implement guardrails around AI usage to maximize its benefits while minimizing the risks. The following strategies provide the best protection.
- Champion key AI principles to act as a north star, ensuring that you meet your commitment to maintain customer trust today and in the years to come.
- Institute rigorous data classification and labeling programs before any information is used to train or prompt models. Limit access to training data through role-based controls and just-in-time privileges. Encrypt sensitive data sets.
- Adopt a “trust but verify” AI usage policy. Build secure sandboxes for generative tools that don’t connect to sensitive or proprietary information. Ensure that models can’t be trained with user prompts. Log interactions for auditing purposes.
- Create a training program for all AI users. It should encompass best practices for prompting, vetting results, avoiding inadvertent information disclosure, and detecting “deepfake” images and audio that impersonate real people. Revise existing end user training to cover safeguards against increasingly sophisticated spear phishing attacks.
- Take advantage of AI-powered tools in the security operations center, but keep humans in the loop. Machine learning should be used for tier-1 triage and anomaly spotting, but analysts should approve preventive or remedial actions.
- Guard the software supply chain. Demand software bills of materials; run continuous code integrity scans; and favor signed, reproducible builds. Generative AI can be used to scan open-source code for hidden vulnerabilities.
- Stress-test AI models against adversarial abuse. Conduct red-team exercises, prompt-injection tests, and bias evaluations. Model updates should be tested before being put into production. Review and update established patch processes.
- Train developers on secure AI coding patterns, and conduct security reviews of any code created by a genAI copilot.
These measures help organizations adopt AI quickly and at scale with confidence while protecting against the new breed of AI-enabled threats.
1 Software Supply Chain Attacks To Cost The World $60 Billion By 2025, Cybercrime Magazine, October 3, 2023