Navigating privacy and information security in government’s use of AI

Earlier this month, the Partnership for Public Service AI Center for Government® hosted a conversation on privacy and information security with two public sector artificial intelligence leaders:

• Bishop Garrison, founder and principal, Orange Court Strategies; former senior advisor to the secretary of defense, Department of Defense
• Marina Kelly, chief information security officer, Raleigh, North Carolina

Over the course of the discussion, the following themes emerged:

AI security tools can protect or, without proper oversight, cause harm

Bishop suggested that creating an AI safety tool can result in an AI security threat. When he helped create an AI-based detector for “deepfake” videos—deceptive videos that use AI to fraudulently replicate a person’s likeness and voice—he was mindful of the risk.

“[To] create a proper deepfake detector, what you’re also doing is looking for opportunities to generate synthetic content yourself,” Garrison noted, describing how replicating the processes to create deceptive videos can be necessary for training security models on how to properly identify deepfakes.

Given this need to simulate creating deepfake videos, Garrison further elaborated on the privacy factors his team considered during the model’s training process. These factors included whether more sensitive training data had proper privacy waivers and how bias affected the model’s video classifications.

Garrison emphasized that, for technology like the deepfake detector, proper oversight and governance are what determine whether the technology operates as a useful tool or a potential security threat.

“At the end of the day, it’s a tool. A hammer could be a weapon, or a hammer could hammer nails,” he said.

Transparency and human oversight build public trust in government AI tools

Marina Kelly walked us through her team’s development of JAMES, a threat intelligence reporting agent built to deliver actionable reporting on cyber threats to critical infrastructure in Raleigh, North Carolina.

It was of utmost important to Kelly that her team prioritize public trust as JAMES was being developed.

“We wanted to make sure that what we built aligned with the values of the city of Raleigh,” Kelly said. “So, we wanted to make sure that … we were being transparent and honest about the work we were doing in this space.”

Another critical factor in ensuring trust was factoring in human oversight in both the development and use of JAMES, according to Kelly. She stated that while “AI can be a force multiplier [for] small teams … you still have to have that person there, using their judgment, reviewing what is being given by the agent.”

Leadership is essential for clear expectations when using AI for security purposes

Both panelists emphasized that leadership is vital to ensuring the responsible adoption and use of AI.

Kelly emphasized how the fast-paced nature of AI as a technology has already rendered existing risk management frameworks for digital and AI-driven threats obsolete. In this context, Kelly described how “leadership is essential to help cut down on the noise, to set priorities, to calm people down and to set guardrails.”

Garrison agreed. “It’s incumbent on leadership to ensure that [it is] vocalizing exactly what [its] expectations are in the use of these technologies,” he said. Otherwise, “you’re leaving it to folks who may or may not be well trained or knowledgeable on these tools.”

Want to hear more insights from these leaders? Check out the recording below.

---

Continuing the conversation

The AI Center for Government® champions AI innovators across all levels of government. If your agency is working to lead AI well, we’d love to hear from you. Join us as we highlight real-world AI use cases and convene public sector leaders from across the country to share tools and insights to lead confidently in the age of AI.

We’re here to help!

• Sign up for our newsletter.
• Follow us on LinkedIn.
• Get in touch! Email us at [email protected].