Redesigning your organization for AI

Ever since early humans had to decide who hunts, who gathers, and who stays behind to make sure the fire doesn't go out, we have been trying to find more efficient ways to work together.  

Flash to 2024, and generative AI evokes the invention of the wheel: It has the potential to profoundly change how work happens, but we're still a long way from seeing what it will ultimately become.

More than two years after OpenAI unveiled ChatGPT 3.5, gen AI is finding a home in the workplace. One recent study predicts that 80% of US workers are likely to have at least 10% of their workloads taken over by large language models (LLMs), while roughly 20% may see AI handling more than half of their work. 

There's no doubt generative AI technology will change how people work, just as 24/7 access to the Internet has. Exactly what that workplace will look like, though, is a matter of some debate. 

“Companies are shifting from rapid experimentation with gen AI to long-term adoption and execution,” says Dennis Woodside, CEO and president at Freshworks. “Having people and teams that are informed and adaptable is crucial for success in today's workplace. That means being as innovative with your organization design as you are with the technology.”

Time to rethink the org chart?

The first modern org chart. New York and Erie Railroad, 1855.

New schools of thought are emerging around various workplace adaptations that AI will require. One of those—raised in a recent Harvard Business Review report—suggests that organizations should redesign their workflows around dialogue, with AI chatbots taking part in nearly every important conversation, and conversations driving work forward. The authors envision a world where "AI and humans perform a task together, learn from and improve each other, and continuously optimize a process at the user level in near-real-time."

Ethan Mollick, a professor of management at the Wharton School, goes even further, predicting that LLMs will require organizations to reinvent themselves. In a recent essay in MIT Sloan Management Review, Mollick predicts that LLMs will drive "a fundamental shift in the way work is done, organized, and communicated," with entire tasks being outsourced to the algorithm in the cubicle down the hall. 

Others make a case that chatbots might even end up running your organization day to day. Nearly half of C-suite executives believe AI chatbots could perform most or all of a chief executive's duties, according to a September 2023 survey by online learning platform EdX. (Even 49% of CEOs agreed.) 

Will organizations need to undergo a radical redesign to accommodate gen AI? Will your new work buddy—or even your boss—be a bot?

The notion that gen AI will reinvent how organizations are structured is a great vision of what the future could look like, but it's not something we're likely to see anytime soon, says Eric Sydell, CEO of Vero AI, creators of an AI-infused analytical engine that can make sense of both numeric and text data.

"The idea that organizations are just going to spontaneously restructure themselves around AI is not going to happen," he says. "That kind of fundamental change often takes generations to occur."

But gen AI could lead to significant changes in how startups and small teams operate. 

Leaner, meaner teams

Most enterprises are still in the tinkering stage of gen AI, more focused on individual productivity than organizational disruption, says Thomas Davenport, a professor of information technology and management at Babson College. Right now, few are even thinking about how AI may require them to restructure their teams.  

"Gen AI has the potential to change multiple aspects of work related to content generation," he says. "But I see little or no systematic exploration of entirely different organizational structures. There's no empirical evidence for it." 

The rise of generative AI is unlikely to cause a massive structural upheaval at the largest firms, at least for the near-future, agrees Jess Lantis, vice president of people operations for Guru, an AI-powered knowledge management platform for teams. But it could have a greater impact on companies that are launching today.

Early-stage companies may stay smaller longer, with AI initially supporting key roles such as marketing or web development that eventually need to be filled full-time by humans, says Lantis. Larger organizations could end up flatter, relying on smaller, more fluid teams. In either scenario, employees will be able to spend less time hunting down information and more time putting that information to work. 

"You won't need layers of managers on managers, or as many highly specialized experts," says Lantis. "People can be generalists and access the information they need by asking an LLM, and that will impact headcount and org design. But I don't believe gen AI is going to turn org charts on their heads."

Read also: Technology with nothing to hide

The death of silos? 

Gen AI may have its greatest impact by erasing functional silos, making specialized information and expertise available to everyone within the organization, notes longtime HR technology advisor and researcher Josh Bersin, principal of The Josh Bersin Company, which recently released its own LLM for HR professionals called Galileo. 

"These silos exist because each functional area is so complicated that it needs its own team of specialists," he says. "Once that data is democratized by AI, those teams may just go away. People can see what's going on in different parts of the organization they were unaware of before."

Bersin says AI will accelerate the trend toward dynamic organizations, which rely more heavily on cross-functional teams organized project-by-project based on the skillsets of the individuals, not their job titles. 

"AI will break down functional silos better than any other technology," he adds. 

Gen AI could also end up shifting the balance of power within an organization by surfacing hidden talent. As Mollick notes, "your company's AI skills might be anywhere." People showing the greatest facility with writing gen AI prompts and finding useful applications for these tools may rise faster and further, even if their job titles might not reflect those skills.

This is a key reason why enterprises like Accenture, Cisco, and Skillsoft, among others, are encouraging employees to experiment with GPT tools, sponsoring AI hackathons, and rewarding employees with spot bonuses and other incentives. Per that EdX survey, 8 out of 10 executives believe employees with advanced gen AI skills should be paid more and promoted more often.

"People who know how to use these tools will have an advantage, because they'll be operating at very different productivity levels," adds Lantis.

Trust never sleeps

Mollick argues that gen AI chatbots will quickly become a kind of employee (albeit one that never calls in sick or steals your yogurt from the snack room refrigerator) and should be managed as such.

Bersin says he's already seen this happening in organizations that are deploying Galileo. 

"If you're an HR business partner supporting a bunch of line managers who start using Galileo, your job just changed," he says. "Galileo can answer their questions just as well as you can. So now your job is to keep the AI up to date and help it serve more people. You've become the manager of the AI."

And while no one is ready to call a chatbot "boss," such systems will also be deployed for performance coaching and mentoring, Bersin adds. 

But organizations will need to carefully weigh what tasks they're willing to hand over to the bots and what's better left to puny humans, says Mark Campbell, chief innovation officer for tech advisory firm Evotek. 

It's one thing to use a copilot to take on a relatively low-risk chore, such as writing an internal policy manual or figuring out how to distribute workloads across your data center, says Campbell. It's quite another to use an LLM to write your security automation scripts. 

"Is this code really secure, or did it come from the People's Republic of Software?" he asks. "If I don't have the skills to write the script myself, how would I be able to check it?"

There's also a danger in getting too attached to the notion of chatbots as people, he warns. Once upon a time, a calculator was someone who was highly skilled at math, and a word processor was a person who entered text into a computer terminal. Now they're just tools.

"When you anthropomorphize AI, you risk dragging in a lot of human characteristics they don't possess," he says. "Nobody talks about how to manage Microsoft Word. It's a tool we use, and some use it better than others. I think the same will ultimately be true of AI."