Source: James Brown via Alamy Stock Photo

COMMENTARY

Ever since large language models (LLMs) like ChatGPT burst onto the scene a year ago, there have been a flurry of use cases for leveraging them in enterprise security environments. From the operational, such as analyzing logs, to assisting detection of phishing attacks, to the more mundane, like rewriting documentation.

While there's been a lot of focus on ChatGPT, I have been testing Google Bard for rewriting and simplifying old security documentation that needed a touch-up. Most notable is the dreaded security policy. You'll be hard-pressed to find anyone who loves writing (or even reading) security policies. But as they form the skeleton of most enterprise security frameworks, they are quite an important bit of documentation.

So how does Google Bard stack up to ChatGPT for rewriting security documentation, and specifically security policies? Before I answer, I'll share some tips for getting started.

Best Practices for Using LLMs to Write Security Docs

First thing first: Remove any proprietary data or personally identifiable information (PII) from your documentation. As policies are generally high-level, there shouldn't be much of this.

Next, write the prompts you'll feed into the LLM with the policies you want to update. Here are a few prompts that work well for Google Bard:

Now that you have your prompts, the LLM can start ingesting your policies or procedures.

Helpful Bard Features That Aren't in ChatGPT

Google Bard has several useful features that are not available in ChatGPT.

One, it understands that it's writing a security policy so, while it always follows the prompt's directives, it will also change suggestive language to authoritative language. For example, it will change "should" to "must," which is important in a policy. This is a nice feature that ChatGPT lacks.

Bard also has a neat "draft" feature that can be easy to miss. In the top-right corner of the generated document, there's a "view other drafts" button. By clicking the button, you gain access to two alternative texts generated by your prompt (to give you three drafts in all).

You can move between the three drafts and pick the one that best suits your preference. If you're unsatisfied with any of the drafts, just click the "regenerate drafts" button to the right of the three boxes, and it will generate three more options. While ChatGPT can regenerate options in unitary fashion, it won't present them in the user interface like Bard does; you have to regenerate them individually.

Once you pick the draft that suits you, you can modify it again by selecting the "modify response" icon (highlighted below) at the bottom of the draft:

This gives you options to make your document shorter, longer, simpler, more casual, or more professional.

The "Simpler" option prompts Bard to reduce word count, simplify language, and shorten sentence length. "More casual" isn't appropriate for security documentation, as it produces almost comical directives like "don't do that, man!" This is probably not what you want for an enterprise security policy. The "More professional" option makes sentences longer and words more complex, effectively pushing your policy towards "legalese." These options impact the tone and readability of your document, so play with them to your heart's content.

Bard has a couple of other neat options that don't exist in ChatGPT. The "Google" button at the bottom of the draft can quickly dig up (via Google search) a comparison of what you've written. If you paste in a physical security policy, for example, it will search for something like, "What is the purpose of a physical security policy?" or "What is a physical security policy?" Hopefully, you already know what your security policy is for.

Once you're done, you've effectively got a nice, shiny new security policy without superfluous language and that's readable to the common mortal. You've also saved yourself a huge amount of time. You can export it directly into Google Docs (no Microsoft integration yet), copy it directly, or share it with a link.

Google Bard's Advantages for Writing Security Documentation

What's the resource gain on using this method? After running it through 300 pages of documentation, the answer to that is "significant." It takes an hour or so to manually proofread a single 10-page policy, remove excess verbiage, tidy up grammar, remove duplicates, and improve readability and formatting. The Bard approach reduced it to minutes.

This effectively compressed weeks' worth of work into a few hours with significant resource savings. And most important, our policies are now readable and understandable to a layperson. While I still had to review the policies at the end to tidy up sentence structure and formatting, I found that Google Bard is a very good companion for rewriting security documentation that, at this time, has several advantages over ChatGPT.