Talk to the spreadsheet

Generative AI is expanding its utility in the workplace, thanks to new copilot tools that make business apps more powerful and easier to use—especially in service operations.

For IT leaders, that means getting productivity-boosting AI into the hands of support agents or employees more quickly and easily. To show how, Girish Mathrubootham, founder and CEO of Freshworks, and Marco Casalaina, vice president of Azure AI products at Microsoft, led a discussion and demo in a recent webinar.

Here are some highlights of their conversation:

DIY copilots 

"One of the challenges with ChatGPT and large language models is that they are limited in time with respect to their knowledge," said Casalaina. For instance, when Casalaina queried ChatGPT about the heating system of his new car, a Hyundai Ioniq 6, it didn't know that the car model existed. Besides the lack of recent data, LLM problems with hallucination—incorrect or fabricated answers—are already well-documented. 

Using simple English commands, however, Casalaina showed that anyone can build their own copilot on top of GPT-4 (or any LLM)—one that can provide expert information on, say, a Hyundai Ioniq 6. They would simply upload the car's user manual into the copilot, enabling it to surface the most up-to-date information and answers. 

"The hottest new programming language is in English," said Casalaina.

What Casalaina demonstrated was an AI technique called retrieval augmented generation (RAG), which improves the accuracy and reliability of an LLM by combining it with additional data sources. Such advances can help enterprises avoid hallucinations and safeguard sensitive or proprietary data as they build out their own use cases, optimizing the value of their software investments. 

Tap into gen AI with pictures and voice

Many professionals have tinkered with gen AI text prompts for basic tasks like creating a marketing email or summarizing a document. But more intuitive modes of communicating with an LLM are emerging. 

As Casalaina explained, users can upload a photo of a product to find out how it works or how it’s used, or share a screenshot of a support ticket with an agent to get it resolved. Gen AI-based copilots work in the background to gather the needed info from different sources. “Just using images and voice to actually help everybody understand what the problem is—and not just that, get answers—that’s going to be game-changing,” said Mathrubootham.

Be the boss of your spreadsheets

Casalaina called out another promise of GAI: autonomy, which allows LLMs to answer complex queries or even write and execute Python code with very little prompting required from users. 

Pulling out a sample spreadsheet from a personal project, Casalaina showed how autonomy uses chain-of-thought reasoning to handle complex queries. He asked ChatGPT to show him the top 10 ticket buyers of a charity event he managed several years ago involving 75 San Francisco restaurants and sort them based on different attributes. 

Recalling how he used to spend hours painstakingly doing such analysis with pivot tables, Casalaina simply spoke to the spreadsheet and got his answer in seconds. 

“When we think about things like prompt engineering, it's going to change very rapidly as this type of chain-of-thought reasoning comes into play,” said Casalaina. “This is something you can try today.”