GenAI.mil – Making the Best of Us Better and the Worst of Us Worse
Recently, the Defense Department launched Google Cloud’s Gemini for Government as the first of many programs comprising the Pentagon’s new Artificial Intelligence (AI) platform, GenAI.mil. Secretary of Defense Pete Hegseth wrote that “AI tools present boundless opportunities to increase efficiency, and we are thrilled to witness AI’s future positive impact across the War Department.”
AI can be a powerful productivity tool. Secretary Hegseth offered examples ranging from drafting correspondence to analyzing drone footage. Outside the military, large language models (LLMs) and other AI tools take notes during meetings, transcribe speech, and serve as all-purpose assistants.
But as any AI user will tell you, the technology is hardly flawless. In 2023, a judge sanctioned a pair of New York lawyers after discovering the two had cited fake court cases, thanks to a ChatGPT hallucination. Several companies have also seen employees feed sensitive customer and proprietary data to AI applications, resulting in security violations. And who among us hasn’t seen social media flooded with garbage content ranging from AI influencers to Shrimp Jesus. It’s become so ubiquitous that one dictionary named “AI slop” as its Word of the Year for 2025.
As the military begins adopting AI tools into its daily workflows, it’s worth watching how the private sector handles this technology. Despite promises from the tech industry, most businesses simply are not seeing a return on investment from AI technology. Others bemoan that a deluge of AI-generated “workslop” only creates more work for managers who must correct poor products. A study from AI company Anthropic found that many employees hide their use of AI at work. Some respondents claimed they brushed their AI use under the rug for fear of being replaced entirely by AI, while others admitted to using AI to complete routine tasks quickly and spending the rest of the time zoning out at work.
Like workers everywhere, the military will have to adapt to this new technology in its everyday life. However, large language models (LLMs) have several shortcomings that could adversely affect military leadership and organizational culture. Organizations can mitigate these challenges through thorough training on Large Language Models, a deceptively simple technology that requires immense effort to master.
A Brief Note on Artificial Intelligence
Google search results for the term “AI” exploded tenfold following the public release of ChatGPT in November 2022. But although AI has rapidly spread into our daily lives, few understand what this technology truly is. Although there is no formal, established definition for “artificial intelligence”, the term generally applies to technologies that mimic many human functions, including decision-making and content creation. One popular form of artificial intelligence is a large language model, a type of Generative AI that can mimic human writing patterns by training on large amounts of text. The most famous, of course, is OpenAI’s ChatGPT. Others include Google’s Gemini (now part of the DOD’s GenAI.mil suite), X’s Grok, Anthropic’s Claude, and Meta’s Llama. Although there are many forms of AI, many of which will undoubtedly have military applications, this article will focus on LLMs.
AI is Not a Replacement for Leadership
A quick Google search shows that leaders across the Armed Forces are using ChatGPT and other LLMs to help them write evaluations, awards, counseling statements, and memos. Indeed, it is tempting to use LLMs for personnel actions. These are often tedious and can take up an inordinate amount of time.
Large Language Models excel at proofreading and cleaning up language in administrative documents. Let’s face it, Army writing was bad to begin with, and it’s only gotten worse with lapses in American education and the military’s over-reliance on PowerPoint bullets. And unlike the old days when specially trained clerks produced much of the Army’s correspondence, we’ve given everyone a laptop and expect them to write like Hemingway.
Not to mention, it’s tempting to outsource personnel actions to AI because there simply isn’t enough time to keep up with all the work the military expects of service members. Counseling, awards, evaluations, and other personnel actions are often the first items leaders push off their plates when they get overwhelmed.
We’ve all seen the results. Awards and evaluations are often late. Although few will admit to the practice, many leaders copy and paste achievements from one award or evaluation into another. (I know this because my first PCS award in the Army – mailed to me two years late – was clearly an award written for another officer several years ago with my name substituted in his place) A landmark study from 2015 also found that many leaders do not conduct their required quarterly counseling with their subordinates, instead fabricating dates for it.
Many private-sector managers have begun using AI to help them draft performance reviews, and not just because the process is faster. Managers can offload the emotional burden of providing honest feedback to employees onto chatbots.
A recent episode of the satirical cartoon “South Park” shows just how problematic outsourcing human interaction to LLMs can be. In the episode “Deep Learning”, the students show little remorse about using ChatGPT to write essays. Likewise, their teacher did not feel guilty about using the program to grade students’ AI-generated homework. The real kicker, however, was when the boys used chatbots to have emotional conversations with their girlfriends. Indeed, the practice of using ChatGPT to engage with women on dating apps, as well as the disappointment when a man struggles to make small talk in real life, has become so pervasive that there’s even a term for it – “Chatfishing“. (One woman was wary of the practice because, claiming she’d been “chatfished” into bed already once before. Truly, we live in dystopian times.)
Put yourself on the other side of AI-generated counseling. How much faith would you have in AI-generated feedback, even if it were completely correct? It’s safe to say that some subordinates would simply prefer no feedback to AI feedback, even if the content was valid. Not to mention, AI use in performance reviews is a liability nightmare. Who’s looking forward to IG complaints over bias in AI systems? And you can bet the IG isn’t looking forward to the headache involved in questionable AI detection tools.
Human beings generally use automation to perform tasks that they’d rather not do themselves, freeing up time for more critical work. The use of AI is a powerful indicator of an organization’s priorities – using AI overwhelmingly for personnel actions gives subordinates the impression that they are not a priority.
AI is Not a Replacement for Your Staff. Or Your Therapist.
“Chat and I are really close lately,” said Maj. Gen. Hank Taylor, the top U.S. Army officer in Korea, during the annual Association of the U.S. Army convention. Although Taylor noted the technology’s use in predicting logistical shortfalls, he also spoke of using the chatbot to make personal decisions, even encouraging his subordinates to do so.
Maj. Gen. Taylor is not alone. During a recent conference, OpenAI founder Sam Altman claimed that young people often used ChatGPT to help make major life decisions, adding that “It has the full context on every person in their life and what they’ve talked about.” Despite OpenAI’s clear warnings against the practice, some users have turned to large language models for mental health and relationship advice – often with dangerous results.
The military is a people business, filled with personality conflicts, bureaucratic turf wars, and petty tyrants. There’s no shortage of counterproductive leadership, ethical lapses, and tricky personal problems that leave even the best of us dumbfounded. It’s tempting to confide in and vent to ChatGPT over personal problems. It’s always there. It’s nonjudgmental. It can even “remember” previous chats, serving as a long-term confidante. One study suggests that ChatGPT may be the largest mental health resource in the country. The study’s survey respondents claimed they turned to LLMs for mental health support, citing the technology’s accessibility, affordability, anonymity, and lack of waitlists.
Much has been written about the effects of large language models on young people’s cognitive functions. But few have considered how this technology can affect senior executives’ decision-making. This demographic has unique vulnerabilities to the harmful effects of LLMs and fewer guardrails on their behavior.
Senior leaders may be more susceptible than the average person to the harmful effects of AI chatbots because they may have more exposure to the technology. Early research from private industry suggests that senior executives are far more likely to use AI and LLMs than lower-level employees, though the reasons aren’t entirely clear. One theory may be that senior executives are more pressed for time and may turn to LLMs for assistance. Another theory might be that lower-level employees might be more reluctant to use AI programs for fear that the technology might replace them. Senior executives may also have greater confidence in LLMs’ abilities than lower-level employees, who may be disproportionately affected by hallucinations and AI slop.
Additionally, military leadership is inherently lonely, a key risk factor in AI-induced psychosis. One senior military leader even confided in me that the higher in rank you get, the less honest the people around you become, highlighting the fact that high-ranking officers often perceive the reality their subordinates want them to see. Military leaders often feel they cannot show vulnerability in front of subordinates, which can be perceived as weakness and undermine credibility and confidence.
The prevalence of AI chatbots as mental health companions, combined with the unique vulnerabilities of military leadership, will undoubtedly prompt many military leaders to use them for venting and seeking advice. Unfortunately, this can be a recipe for disaster. Chatbots only know one user’s side of any situation, and even then, many users, consciously or unconsciously, control the information they share. Users can also modify their questions or requests to “trick” chatbots into giving them the answers they want. Researchers have also noted LLMs’ tendency towards “sycophancy” – the tendency to spout false information or give bad advice that aligns with a user’s preferences. This is even more dangerous, given that Army senior leaders tend to be less willing to seek out alternative viewpoints and debate those with differing opinions than the general population, according to one Army War College study.
In short, Army senior leaders now have their own personal “yes man” on call 24/7. LLMs will always take your side in a workplace dispute (“You’re absolutely right to feel that way.”). The LLM sycophancy program is so pronounced that ChatGPT told one user that his purported plan to “sell shit on a stick” was “not just smart – it’s genius!” Although OpenAI claims it has since modified its models, other chatbots face the same challenge.
Of course, sycophantic subordinates have always existed. And Bad Idea Fairies have been leading senior leaders astray long before LLMs. But without proper education on the harms of Large Language Models, senior leaders risk falling into AI-induced psychosis. And staffs may struggle to contain them. It takes a thorough appreciation of one’s own susceptibility to AI’s worst effects, as well as a keen understanding of the inner workings of Large Language Models, to avoid AI-induced psychosis.
Finally, there’s a practical reason to avoid sharing one’s deepest thoughts with chatbots. OpenAI founder Sam Altman warned, “People talk about the most personal shit in their lives to ChatGPT… If you talk to a therapist or a lawyer or a doctor about those problems, there’s legal privilege for it. And we haven’t figured that out yet for when you talk to ChatGPT.”
Conversations with commercial LLMs are not private. Law enforcement can access LLM chats with search warrants, as was the case with the U.S. Army soldier who used ChatGPT to help plan a bombing in Las Vegas on New Year’s Day 2025. Communications on government platforms have even fewer privacy safeguards. As a standard warning banner on government communication systems states, communications on government information systems are not private, save for legitimate privileged communications with lawyers or clergy members. Even members of the general public can request access to correspondence carried out on government platforms. An adage about government emails holds just as true for Large Language Models: “Don’t tell anything to GenAI.mil that you wouldn’t want to read in the New York Times.”
Users Need Training on AI
Large Language Models seem easy to use at first. Simply type in a request, and your favorite chatbot spits out a convincing-sounding answer. But LLMs require considerable effort to master.
I say this as someone who spends considerably more time with ChatGPT than the average person, according to the program’s 2025 annual statistical roundup. ChatGPT can easily turn a 0% product into a 70% product in a few seconds. It’s certainly efficient, even if the quality is lacking. Where the technology shines, however, is in transforming a 95% product into a 100% product. The only problem is that it takes significant effort to eke out those last five percentage points. And even then, you’ll need a 95% product to begin with. Users require a keen understanding of the subject material to combat AI hallucinations. It also helps to know your goal for any given product to filter out responses that simply don’t fit your end state. Finally, LLMs require expertise in prompt engineering to produce a high-quality final product.
Most users do not have formal training in LLMs. Despite the stereotype that young people will instantly grasp how to use AI because they are “digital natives”, there is a growing sense that young people, even those who may spend hours each day online, lack basic computer skills and digital literacy. In other words, the ability to copy and paste your homework prompt into ChatGPT and then copy and paste the response into a Word document does not necessarily make someone an expert in LLMs.
I’ll be the first to admit that I absolutely hate the thought of more mandatory training. I also realize that nearly every agency in the Pentagon bureaucracy loves to insert itself into Professional Military Education at all levels, often in one-hour blocks of instruction that many students snooze through. Not to mention, I also shudder at the thought of giving some poor soul an additional duty as an “AI advisor”. That said, the services could offer optional coursework and certificates on LLMs, perhaps with incentives to complete the course, such as promotion points for junior enlisted service members. It’s not much, but it’s a start.
Our Boring Future
There are plenty of hyperbolic statements about AI. Somewhere between the dystopian vision of Terminator’s Skynet and the utopian belief that ChatGPT might cure cancer next week is the banal reality that artificial intelligence tools replicate human beings with all their flaws. They have the capacity to turn great work into exceptional work. They can also perpetuate laziness and out-of-touch leadership.
In short, it makes the best of us better. And the worst of us worse. And that’s the boring future we live in.
(Author’s note: The opinions expressed in this paper are the author’s own.)
Check out all of Small Wars Journal’s great content.
The post GenAI.mil – Making the Best of Us Better and the Worst of Us Worse appeared first on Small Wars Journal by Arizona State University.