Episode Intro by Host:

Nirupama: Hi, you’re listening to Coefficient, a brand new podcast series brought to you by TheMathCompany. And I’m Nirupama, your host—a journalist, researcher, and podcaster—currently the content lead for thought leadership and podcasts at TheMathCompany. In every episode of Coefficient, we deep dive into compelling topics related to the present and future of data analytics and AI for business intelligence.

In the first two episodes, Cofounder and CEO of TheMathCompany, Sayandeb Banerjee, helped us understand what it means to be a truly data driven business and shed some light on how enterprises can get there.
In the previous episode, seasoned experts at TheMathCompany demystified retail media marketing. They spoke about what it means, who it is for, why it is important, and also how to get started.

This is episode four, and today we are talking about a very important topic that I’m pretty sure all of you have heard about a lot in the last few months: Generative AI.

Now, enough has been said about just how powerful this technology is and the tremendous potential it has to transform the way we work as individuals and organizations. But now it’s time to start talking about how to implement this technology as an enterprise and make the best use of it. And that’s what we’re going to do today.

Our guests for today are Sourav Banerjee and Srivatsa Kanchibotla, who are two leading voices in any discussion related to generative AI here at TheMathCompany. I’ll just take a moment to introduce the two of them.

Sourav is the Head of Innovation at TheMathCompany. He has 18 years of experience working in data science. And here he leads a team that builds capabilities in Natural Language Processing, computer vision, forecasting, and just all things exciting and path breaking.

Srivatsa is Principal at TheMathCompany. He’s worked in, and set up high performing data science teams in a wide range of industry verticals over his experience spanning 15 years. He’s been following the generative AI space since 2017. And he’s played a major role in building AI capabilities here at MathCo.

On to the episode.

Episode Conversation:

Nirupama: Hi, Sourav. Hi, Srivatsa. Thank you so much for taking the time to join this podcast. I’m really excited to talk about this very hot topic, right? Before we get started, with the discussion, Sourav, I first wanted to ask you to help clarify a few things.What is generative AI and what are LLMs?

Sourav: Sure, yeah. I think generative AI is a class of models which basically, unlike traditional machine learning models, focuses more on learning the underlying representation of the data rather than predicting a specific category or specific number or so on. These models are typically trained on large amounts of language, video, or voice data. And large language models are a subset of generative AI, which is typically trained on, a large amount of, you know, trillions of tokens of, natural language data. GPT is one of the most famous examples and ChatGPT is, a Large Language Model, which is very famous, which is a model which was created by OpenAI that focuses exclusively on human-like conversation.

Nirupama: Thank you. This is really helpful. I think most people now would know these things, but it’s important to sort of set the context. Now let’s get into the, you know, main conversation. AI has been around for a while now, right? And chatbots have existed. But, you know, in the last few months after the launch of ChatGPT and other key advancements in Generative AI, it just feels like things have exploded, in this space, in AI. So, I want to start by asking, why all this hype and is this hype justified?

This is the question I want to ask Srivatsa.

Srivatsa: Sure, whether the hype is justified or not, the binary answer kind of depends on your perspective, but, language models and Generative AI has become definitely useful. A lot of opportunities exist for adoption of this technology, at a personal level, at an enterprise level, and even to a certain extent at the societal level. We can adopt and see benefits, from this technology, right. The recent hype, I would say is a result of basically investor money going towards startups and other, investment opportunities that have been explored, and that has caused quite a stir in the startup community, in the investment and private equity community, so on and so forth.

So, to that extent, the hype is definitely real, but there are certain things, legal and ethical considerations, that have to be thought through before adoption of AI can reach the masses, if you will, or reach the societal level. So, the hype is definitely real, especially in the smaller community of the startup community and the private equity venture capital community. It’s definitely real. A lot of dollars are flowing in that direction. Many startups are being funded and quite a few ideas are coming to fruition as well.

Nirupama: Got it. It’s kind of now accepted that because of, you know, how much people have been talking about it. And of course, like how you said, there is a lot of investment going into this, that, it is an important thing that we should all be, paying attention to and also try and start adopting because we have been talking about this for a long time. I want to ask you sort of, what is the potential of generative AI in the enterprise setting? Because, you know, we all, when we first got to know about ChatGPT, we got to know about it as a tool for personal use, right? So in the enterprise setting what is the potential and maybe if you could give me a sense of, the different types of business functions that it can help and the different use cases, that generated AI can have.

Sourav: I think it’s, it’s still early days. So, I would say while there are use cases – emerging, enterprises to figuring out, you know, how they can best use this technology. But having said that, I think the potential is enormous. In fact, I would, I have seen forecasts of billions to trillions of dollars of productivity as a result of Gen AI.

I mean, numbers are debatable, but I think the potential is enormous and, eventually I think it is going to reach each and every function in some way or the other. As of today, we see use cases all the way from where people are trying to access information in a more easy to understand standard manner versus also trying to summarize, their KPIs, summarize their metrics and, provide more intelligence in an easy to use manner to businesses, most functions will benefit. Largely we see a lot of use cases in marketing, where there is an opportunity to automate some of the content or at least augments of the content creation, there is customer service where there’s a lot of data on, customer conversations. There is opportunity to make the customer service process more easier for customers.

Legal, and also, you know, software development is another area which is sort of ripe for a lot of augmentation through Generative AI. I already use a lot of, you know, ChatGPT 2 for day-to-day coding work. In our customers, we are seeing certain interesting use cases emerge. Some of them are, some variants of NLP or natural language querying, which would have been existed, which have taken, you know, seen more, just seeing more traction between this new technology and some newer use cases. We see some use cases in the area where, you know, customers want virtual analysts for their business.

We are also seeing a lot of interest in healthcare where, you know, market research kind of use cases, also, interestingly, there are a lot of use cases in how to make data accessible and data more easy to consume for business users using Generative AI. So yeah, it’s exciting.

New user cases coming up and we are also learning, day by day.

Nirupama: Yeah. It, I mean, I know you just covered some of the basics, but it just still looks like it’s used in like pretty much every business function. The next question I have is for you, Srivatsa. In the first and second episode of this podcast series, we spoke to Sayan and he spoke about what it means to be a data driven business. And he talks about data-driven decision making and becoming a truly data driven enterprise. So, I want to understand, what is the role of Generative AI in facilitating better and, maybe faster decision making.

Srivatsa: Typically, enterprises face, or the reason why they talk about this topic is, enterprises work typically in silos, especially really large enterprises and Generative AI has the capability to kind of break down those barriers and enable accessibility and conversational interaction with your data, if you will. So, for I would say the last 10 years or so, breaking down the silos that exist within an organization was a key focus area for business leaders.

And, my feeling is that generative AI will help a lot in breaking down those barriers, in helping people collaborate, across functions, and discover & use, data better because of the conversational interface, that it can provide, right? So that would definitely help, in organizations kind of being, data-driven, using, all of the data that they have available in their enterprises, having the ability to discover it faster interact with it better and in a conversational manner.

And I think that’s what would, eventually break down the barriers, not that a lot of barriers exist today, but Generative AI would definitely ease that path and it will make it easier for business users, business translators and others who are not particularly have a data background to kind of easily discover and use the data that’s already available in the enterprises. Generative AI will also provide a capability to use different kinds of data, not just data that is present in the enterprise, but get some data from outside the enterprise, mix it with the enterprise data, and do some interesting analysis, interesting visualizations, all of these in a conversational manner.

I think that’s where the push from Generative AI will come to making enterprises data driven.

Nirupama: So, like I’m understanding that, you know, whatever dashboards are doing now, right? Like. taking, you know maybe that is something that is very statistics heavy, data science heavy, research heavy and something that’s too technical and sort of simplifying and giving you something that is understandable for a non-technical audience.

So, you’re saying generative AI, you know, because I can also ask things in a conversational manner.

Srivatsa: Correct. The generative AI can be, like a co-pilot or a co analyst, along with you, it can help with the disambiguation of certain topics.

It can help you discover, so if I’m looking for a sales metric, what is the best place to find a sales metric. Can I retrieve some of that data? Look at some trends, plot some charts, do a simple analysis along with an AI assistant, right? The AI assistant will help you with disambiguation of certain terms. What is the best metric to judge a linear model, so on and so forth. So it can, for now, at least it can be side by side with an analyst or with a business translator and help them through the process of doing an analysis and extracting value from data.

Nirupama: Right. Yeah, that sounds really helpful and really promising for business leaders. So, the next, you know, angle of conversation I want to take is, about implementation.

So, we have understood the use cases, and we have understood how it helps with data driven decision making. Now let’s talk about, how to implement it. So, this question I have is for both of you. Everybody is now somewhat convinced that, businesses do need to adopt generative AI. So, how do they go about it? How do they start?

Sourav: I can start with my thoughts and Srivatsa can add on. So, essentially there are multiple ways and multiple choices today that exist, and both in the forms of models as well as model providers, right? So essentially, there is a host of open source models that are now coming up almost every week. And there is obviously Microsoft, which is, which started all of this and then Google and, Anthropic that are coming up with their own closed source models, which with, some value added layers on top of it.

And to make things more complicated, I think, on top of that, there are frameworks, like orchestration frameworks that are coming up, which makes it easier to use LLMs and you know, embed them with applications. So, so obviously a lot of choices. So, so I think the way to start is use case dependent.

First and foremost, I think it is imperative to get the right use cases in place. which is a high value use case with enough, like potential ROI and also enough users who will see the impact of it. In most cases, I would suggest they can, GPT 3 or GPT 4 kind of a model experiment with it, get to a proof of concept fast, you know, understand the feasibility of your model capabilities, what it can do, what it cannot do.

And then, you know, once you have, we are convinced about the capabilities, then, I think it’s, it’s time to invest a lot more on your custom element infrastructure, which essentially is a model that contextualize to your business, maybe even, you know, use open source to bring some of that into the enterprise and then scale with that.

So that’s, you know, generally what I would do, obviously there are some apprehensions stills about, you know, privacy of LLMs and hallucination. Those are problems that still keep coming into our discussions. Not all of them are solved, but I think, I still think that whatever I said is probably just going to go and get started with the LLMs.

Srivatsa: Yeah. So, I would say not everyone understands cell phones or how the internet works, but everyone does understand how to use it in their daily lives. There, how people can integrate cell phones or internet into their business. So, AI will play a similar role where you might not, I mean, if you’re a business and you’re not in big tech, you might not fully understand the, the internal workings of large data models or generative AI. But adoption should be high on the agenda, or high on the priority list nonetheless, right? Some way you have to, figure out to use it and keep abreast with the advancements. It might be the case that, AI is not mature enough right now to directly be applicable in a business use case. But just like Sourav mentioned, there are concerns around security and hallucinations, and there are workarounds for it as well, right. And the paradigm that is being, used most is that of a copilot, we don’t leave AI alone in the room, or a learner’s license, right? AI will always have like a human being or an analyst along with it. So, the copilot, co analyst paradigm is what is gaining the most traction.

And I think that’s fair because of the issues that AI inherently has, you can’t let it run amok alone in a room or in an enterprise use case. There are guidelines and there are workarounds and there are guardrails that you can put in place to kind of control the behavior of  the AI and make it work for you in some, well defined use cases and exploring those options, and building some amount of  AI literacy in your organization along with data literacy, right, I think that is almost inevitable at this point, right? Because in a few years if the AI revolution does live up to its promise and you as an enterprise are not a literate in the sense of at least being able to use those models and apply those models instead of understanding the under the hood details, it will be a competitive disadvantage.

Nirupama: So you sort of touched upon, some of the concerns that are there and the guardrails that are required. That’s something that I want to delve into a little bit deeper, but we’ll come to that. Before that, I wanted to, you know, I’m curious to know both of you, you’ve been talking to so many business leaders about this, right? You know, are there any myths or, misconceptions that you see people have, business leaders have, about implementing this technology, that you want to bust or any fears that they might have that you want to ease.
Srivatsa: I would say data privacy concerns are very legitimate but it’s also a myth to a certain extent because some use cases can definitely be done using either open source model or closed source model without exposing any of your data.

In fact, I would say a common myth or a misconception in enterprises specifically and, clients that we work with is the data requirements, most people seem to think that they have to bring their own data to be able to play with generative AI, but that’s not necessarily the case. There are specific situations where you might need to provide your company information to the large language model for it to be very relevant and very customized to your business scenario, there are plenty of use cases where without exposing any of your data to the large language model, you can adopt generative AI. I think that’s one common misconception. The other one, is the what I would call the machine learning way of thinking. In machine learning, more data, training data, these are commonly used terms, which people understand, and they’re trying to interpret Generative AI in terms of training data, in terms of accuracies, and in terms of some metrics for explainability and so on. Some of these might not apply to the generative AI, or Generative AI has kind of transcended those issues, if you will. And yeah, like I said, the more there is literacy and more of these myths that tare broken down, the adoption will steadily increase, speed up, based on how soon we are able to kind of tackle these issues.

Sourav: Yeah, I think, somewhat related to Srivatsa’s second point, because of the hype that ChatGPT has created and because of its phenomenal ability to converse with you, I think there’s that expectation that somehow ChatGPT will be plugged into my data and learn certain business.

Trends from my data and start answering my business questions, automatically. To some extent it is true. It is better than anything that was there before. I mean, the best way to think of ChatGPT, or any GPT model today, is an infant that knows English very well, but, but it needs to be taught skills to actually understand your data and essentially you know, extract inference from your data. So, there’s a lot of work to be done from where ChatGPT is today to, for it to become a really true, trustable assistant that that knows your business and understands your data. So that’s you know, that’s something I think businesses and truly even us are figuring out like, what it means for us to really, you know, start adopting for real enterprises cases.

Nirupama: So now I want to come to, you know, the question that we sort of addressed earlier, which is, you know, the risks that people have been talking about, they’ve been writing about, and, you know, there have been discussions, you know, with the adoption of generative AI in, you know, not just in businesses, but they’re also talking about it in all facets of society.

But I want to talk about, you know, when it comes to, businesses, right? When it comes to adopting Generative AI in businesses, what are the risks that are involved? That we need to genuinely be thinking about and also, you know, if you could talk a little bit about, careful, responsible implementation and you know, what are the guardrails that we need to put in place for that.

Srivatsa: Sure. Let me take a shot. So, I was having this conversation with Aditya and I was telling him that, ChatGPT and AI can probably make decks fairly soon and his reaction was, okay, so if something was wrong, who should I blame? Who do I catch? So, I think that question is very much out in the open.

There are quite a few risks, first of all, of more practical concern is what kind of data was used to train these models, because you might use proprietary images, proprietary information and texts, to kind of train your language models and it opens up, basically, the LLM providers are people who make large language models, it opens them up to lawsuits. And if enterprises use those models, there might be downstream effects. So that’s a serious concern that has to be addressed. Adoption of AI can have certain negative behaviors, but that is at the level of any technology. Any technology when it’s newly introduced might change human behavior and introduce it in parts that are not healthy and that risk is always there.

Enterprises need to be also careful with respect to customer facing AI. If you put AI in front of your customers and it starts answering questions wrongly or making up facts or providing harmful information. One common use case in enterprises is if there is a conversation interface and someone asked for a KPI or a metric to which they typically would not have access to. And then the AI start showing that information,  that’s an issue. There are a few of these issues that, like I said, at an enterprise level, like at a society level, including like, how will it change human behavior and so on. But I feel we are equal to like rising to the challenge as well. There will be guardrails and legislation and ethical guidelines that will be, either they’re already in place or discussions are ongoing to kind of create, those structures which will help us, kind of adopt this technology in a safer manner. At the enterprise level, which is most of our clients, I would say the, be careful with customer facing applications, like first try internal applications, where you have complete control over what answers the AI is exposing and to whom, right. And then slowly, think of customer facing applications, where the AI is directly interacting with your customers.

Nirupama: Yeah. Thank you for taking us through that. I’m sure this is a, you know ongoing discussion, which, you know, will improve as we learn newer things about, how to use this technology in businesses.

So the last question I have is, which is something probably that, you know, you cannot avoid when we talk about AI. One thing that people talk about everywhere is, you know, how it affects people’s jobs, right? Is AI going to replace a lot of jobs? We know that AI will, you know, adoption of AI, especially generative AI, which has so much potential, it will undoubtedly reshape jobs and skills in the future.

So, could you elaborate a little bit on this transition? This transition that will happen in, you know, how we work and how people need to think about their jobs and skills and also how organizations can prepare their workforce for this change. I’d like to ask this question to you first, Sourav then Srivatsa, if you have something to add, please feel free.

Sourav: Obviously there are a lot of concerns about how it will impact jobs. I’ve seen large numbers anywhere from a few millions to 300 million, large numbers make headlines. I am not sure how accurate they are, but you know, nevertheless, some jobs may get impacted, unfortunately. On the positive side, it will also create new jobs. So, to make AI work or to make digital transformation work, new roles have to be created and we are already seeing a lot of newer roles. For example, there’s a designation called prompt engineer, which I have been seeing people are hiring for now. Which never existed, like, 2 years back.

But one thing that is for sure, I don’t know the magnitude of job loss or job creation, but one thing that I think will happen inevitably is people’s jobs will get, you know, influenced or touched by AI in everything. What that means is basically you will probably have augmentation of AI, maybe in each and every thing. So, we are seeing augmentation in, you know, legal, marketing, sales, you name it and there’s AI augmentation pretty much everywhere. So, people will need to be AI literate, if you may call it, so to understand how to work with AI, what are the pros & cons, when to use, when not to use, and things like that and then practical example where, I’ve seen in the past is, whenever this, AI technology is used and inserted in the workflow typically the newer people who don’t have so much experience, who are still learning, find it very useful versus people who are very experienced and SMEs in that area find it to be a nuisance and, an annoyance sometime because they will say, ‘Hey, I already know whatever they’re saying the recommendations from AI are not that great.’ So, those kinds of things will happen. We’ll go through adoption changes, but, eventually I think AI augmentation is probably coming in every sphere. It also means that all of us probably need to upskill ourselves in our own jobs to be better than the AI and, I’m sure we will be better than it and adapt. That’s my take on it.

Srivatsa: I don’t know if it’s different, but, at least until now, most technological advancements resulted in the automation off the boring and the tedious and the cumbersome kind of jobs, right?

But with this, I think, even the creative knowledge-based sectors, will probably have some level of automation, right? A very sarcastic joke or a great painting, or the nice piece of music was always a human endeavor. It’s what made us human and it was some of our greatest aspiration as humanity was art, but now AI can generate art. Did it actually think it up? And all of those philosophical discussions aside, among the creative and knowledge based job sectors, if you don’t upskill, or if you don’t move up the value chain, because AI will eventually reach there and try to automate away, I don’t know if I should say your job, but it will automate away the creative and knowledge-based industry jobs as well. And it’ll start lower in the value chain, right? So, there is, some amount of literacy definitely needed, like Sourav mentioned, but people should look to kind of enhance whatever it is that they are doing through AI and basically find ways to accommodate and adopt AI quickly because it not just automates the boring and the cumbersome, but it automates the creative and the knowledge based industries as well.

So, I think that’s a critical thing to keep an eye out.

Nirupama: Yeah, you did say not your job, but you were talking about marketing. So, yeah, it looked like, you know,

Srivatsa: I meant your job.

Nirupama: I know, I know.

Sourav: There will be an AI podcaster soon if it is already not there.

Nirupama: I’m pretty sure it is already there.

Sourav: I’m sure you will be, better than, contextualize the conversation much better.

Nirupama: Yeah, yeah. And yeah, I get it. I get it.

Srivatsa: There are standup comics. who basically start their set with some ChatGPT jokes nowadays.

Nirupama: Oh, interesting. Okay. Yeah. So, yeah.  I guess it’s all about, , you know, us learning how to make the best use of AI? Like you all mentioned AI literacy along with, you know, we used to talk about data literacy.

So also, AI literacy. There are new jobs, like sort of mentioned, prompt engineer, things like that. So is there anything, you would like to say to business leaders about how they can think about this, you know, about their workforce, how to maybe encourage adoption, healthy adoption of AI, and maybe even, you know, skilling for being a literate.

Srivatsa: Yeah, absolutely. There, so most enterprises already have some track of learning and development. Immediately an AI sub field or an AI course can be added to that. Which specific AI course will depend on what kind of use cases they are looking at and what kind of company it is. But if your organization already has a learning and development track or a function that basically upskills, then, AI should definitely be added to that.

People individually can also look into some aspects of AI literacy. It’s a dense, technical topic that spreads across multiple fields of mathematics, et cetera. So it is dense, but without getting into those weeds, how can you use it? How can you increase your productivity because productivity is a large, significant theme of AI. Because of its natural copilot paradigm, it can definitely improve productivity for anyone, developers, marketers, so on and so forth, right? So, exploring those opportunities, definitely. For business leaders, I would say, While there are concerns around privacy, et cetera, at least getting started with the governance framework, some way of qualifying use cases and deciding which use cases are useful, which use cases are not useful, plus which use cases are technically feasible and which use cases are not.

With the concerns in mind, with sharing data, without sharing data, whatever it is, business leaders need to get started or should already be in the process of developing some kind of a governance structure and a roadmap for AI adoption. Apart from that, I think they should be a little bit more excited. Not that they’re not already, but some of the excitement versus action ratio, I would say, should be a little bit biased towards action. Most people are talking about it, but are unsure really of how to step into the field. And that’s why you have, TheMathCompany, as well to help you take that journey.

So, a bias for action in AI adoption, I think that’s my advice to business leaders.

Nirupama: Thank you so much Saurav and Srivatsa for taking the time. This was a really helpful conversation for me. I learned a lot about, you know, Generative AI. I had read about it, but you know, but about how to think about it from the enterprise point of view.

Sourav & Srivatsa: Thank you.

Thank you for tuning into this episode of Coefficient. Nuanced and grounded conversations about new technologies always excite me because they help me to learn, and they also push me to put some of these learnings into practice, wherever I can. I hope this was one such conversation for you.
For those who want to learn a little bit more about generated AI, I will add links in the description.

(To know how you can adopt Generative AI in your enterprise, sign up for MathCo’s Gen AI workshop here: https://bit.ly/unlockgenai 
If you want to take a step back and understand the basics, here’s a primer on Generative AI by Gartner.)

We have a lot more interesting topics and important conversations lined up for future episodes. If you like this conversation and would like to listen to more such conversations, then I urge you to follow and subscribe to the podcast in whichever platform you listen to your podcast. It’s also on YouTube.

Whenever we release the next episode, you will get a notification. Until then, goodbye.