Data Science Hangout | Prabha Thanikasalam, Flex | Calculating ROI with the Business

You're as a data person, you're in a marathon. It's not six months. It's not a year. You're in a, you know, if you want it, this is a career you need to be learning every day, every week, every month.

Thank you, Brian. I love finding these kind of questions where everybody has different thoughts and tips on them, and we can kind of dive deep into them. And Millie, I saw you had a great recommendation as well, if you wanted to chime in too.

Yeah, right now, I'm like in a data science fellowship. So, I definitely know the feeling of like learning and feeling like you're getting sidetracked. So, these days, I've been using the Pomodoro timer. I think like the standard is that you're like, you have focus time for like 20, 25 minutes, and then five minute breaks in between each, and like each little section, and you can do like up to 12 in a day, if that's your goal. At least that's what I have. But yeah, I have like section time for like my project work, or if I want to like dig deep into something. Like right now, I'm like trying to like work on my own like little professional website. So, I set these little tasks, and do like a solid two hours of work is like I think my limit right now, but it's like a good way to like get a lot of stuff done in a little bit of time, and feel like I'm making progress.

I like it. So, personally, that is a challenge for me also, holding attention to do data science work. Actually, in the new year, I did make a resolution that I will not keep my email inbox open all the time. So, there are some days where I am adhering to my resolution, and I only open email two, three times a day, and there are some days where I fail, but that has helped. You know, at work, email has become like a chat. People expect responses in a couple of minutes, but I've told my, you know, close collaborators that this is my plan for 2022, and it is even in the last one and a half weeks, I see that this helped. I'm able to do more focused work.

Measuring business value and ROI

Thank you, Amziani, for opening up that discussion too. I see Libby, you had to ask a great question earlier as well. Would you want to ask that one live?

Sure, can you hear me? Yep, I can. Awesome. Okay, so I was wondering if, and I hope I asked this question in an eloquent way, if I don't, please feel free to ask me to elaborate. A lot of the stuff that you were talking about was demand sensing, demand planning, and like, you know, reducing costs for sourcing and stuff like that, and I was wondering if there has been any analysis done on planar processing, like when you have scenario planning and you have software that planners are using, has there been any analysis on the way that planners use the software so that processes can be made more efficient and timing can be saved and stuff like that? That's been really, really interesting to me right now. I'm moving into a role where I'm looking at customer data, customer interaction data, and so coming from where I was this summer, I was at Canaxis, thinking about the way that planners are actually clicking and using and the troubles that they have with their workflows. Has there been any look into that or is there value there?

Canaxis is actually one of the, is the planning tool used at Flex. I think the, I think, are you asking about from a process, is your question from a process perspective? Yes, right. Okay, so from a process perspective, this, let me talk about, you know, not specific to Canaxis or not specific to the Canaxis usage at Flex, but generally using software and adhering to business processes and data analytics and a software application helping in that business process. At an even higher level, it is change management. So in a large organization or in, when you talk to an experienced domain expert who's done their job for a long time based on experience, intrinsic knowledge, just having done that job for five, 10, 12, 15 years, it is change where now the data analytics says, or a piece of software or a piece, a computer says, here is the right answer. Just go execute on this right answer, right? So that fundamentally is change management and it comes, you know, there are many layers to it.

There is the individual not accepting is probably a wrong word. Individual saying, huh, this, this piece, this computer thing can help me do my job better. Or the other reaction could be, I know how to do this job. What is this computer thing telling me? So that's the starting point at an individual level. And then you can, then that set of individuals come to like a team organization, et cetera. So it is fundamentally change management at individual level. It is when it's an individual. Now that at a team level and an organization level, it is change management. And that is dependent on, you know, some teams do it well. Some companies do it well. Some do not, you know, in my experience, just saying, Hey, here is a new piece of software, use it for your work is a sure way to hit a dead end very soon. So that has to be sold both at the say at executive level and also at execution level, the people who are actually doing the day-to-day job.

Does that address your question? I mean, it fundamentally is change management and in some extreme case, you just have to take away the old way of doing things and say, it's not an acceptable, right. That's the very extreme, like the individual is not able to do their day job the old way. That's the most extreme case. That's the forced case of change management. And that happens. And sometimes it needs to happen.

Yeah. Thank you. Some of, part of my question was kind of, is there any analysis done on like adoption rates? Like how many planners are using the tool versus how many are actually changing suggestions that are given to them by an algorithm due to some kind of aversion to that? Like you were saying, like, I know my job better than this computer does. We do, we capture data on multiple levels of data capturing going on in our set of tools. The first one, and in each level shows, let's say a maturity level. The first one is, let's say we introduce a piece of analytics or application. There are potentially say 20 people who do that function, who are that functional group. How many of the 20 use it? How often do they use it? And we know that the expected usage is a hundred times a week because we know the amount of usage or the amount of times that a business process needs to run in the team or in the company. So then you can start measuring from users and usage, what percentage of the usage is happening in the software or analytics in that business process. So that's the first level we do measure for our applications, usage and usage.

Then the second level maturity is, it's almost like A-B testing, so to speak. Are the results, you know, what are the KPIs, metrics and results that are the outputs of the process? These are not this is not model accuracy. This is not analytics metrics or model metrics. These are business process metrics. Is it changing? Is it better? Then that builds up, that is the effectiveness. Sorry, that's the, that's a measure of the effectiveness of the process as it relates to this common goal, right? So that's the next level of maturity.

Now, these data points start giving us, start giving two, I think there are two, if it is all good, then there are two very, say high value data points here. One, the business process is enhanced because of the introduction of the software or analytics. Then the second thing is the closed loop process. There are going to be aspects where humans make decisions, which the software doesn't know about. I mean, that's the difference between model and reality. Now, once, developing that closed loop process is really hard. But the more you're able to do that, the better this engine or piece of software becomes. But to be able to do it, the chicken and the egg problem, to be able to get everybody to contribute to that closed loop, we first have to prove that the process is enhanced by doing so. So those are the multiple layers of, say, or multiple hurdles that are crossed. And it is very important to have, to collect metadata on your product, on the process, on the business outcomes.

That's super helpful, Prabha. Thank you so much. And also, I think taking that a bit further, I saw there were some anonymous questions that came in too, around working with the business. And one was, how do you measure, understand the business value of your work? So earlier, when you say millions, for example, or especially if a data scientist is lower in the org chart, how do they go about measuring that impact?

Actually, it's a simple answer. I go back to the first thing I said. We are partners. I do not accept the statement that a data scientist or an IT person is lower in the org chart. To people that I work and have built trust with, I can say it to their face. In a very extreme case, sometimes I just say, hey, look at your paycheck, look at my paycheck. It says the same company logo on the top. So we're colleagues. And we are partners. It is sometimes a hard statement to make, but I have made the statement to my colleagues. And I think we are not a service organizer. I think we have to fight for our rightful place as equal partners. So I will not accept the premise that a data scientist or an IT person is lower in the org.

Now, I think the first part of the question was, how do we measure success? It's a very simple answer. Dollars. At least in a company or in an enterprise. The capitalism and the company exists to make money. So there is no more common agreement on what success means. Analytics is a means to an end. If I have three project proposals and I have to prioritize what my team works on, I give a lot of weightage to who has which business team who made the proposals have thought through this problem of what do they want us to do? Are they able to clearly articulate how a model or a piece of software will help them make their business case better? What are the KPIs that they want to measure? I mean, the software application will measure it for them, but they need to tell me where they are. The business teams are the experts in saying what KPIs are important for a given business process. What are the data and analytics can can potentially help them to measure what is known to them, but also can help them. You know, they know that, you know, today, many business teams know that something is important, but they don't have an easy way to measure it. And data analytics and software can help them. So they can give them the known known answers. In some cases, it can also give them the known unknown answers going forward.

And I think those are all intermediate steps to final value, which is dollars, right? So if there are multiple projects in front of me, I look at the maturity of the request or the business thinking that goes from request to analytics model to KPIs to dollars. And if a business team comes to me with a more well thought out and articulated proposal, I'm, they get the resources or, you know, that is a key point in my decision making for prioritization.

That's super helpful, Prabha. And I was wondering if you do also give feedback to people on their proposals, like, oh, we didn't prioritize this because you didn't make this clear. Yeah, yeah. That's, that's, you know, they, they, they lose out, there is penalty for them not doing this initial work that they are not the first in queue.

Building trust and establishing partnerships

I, I know you mentioned, like, it's simple, it goes back to the partnership with the business. But I will say, making real, like those relationships and partnerships probably isn't simple for all of us. So I was just wondering, like, what are the most important factors? And this is a question actually earlier on Slido, that help you establish your

Time is a very key component. You know, the, my partners today, when my closest partners in the business teams today were not my partners, say, three, four, five years ago, they were not this, they were not the same partners that they are today that they were, say, even a year ago. So, you need to give time. I mean, it's with any relationship, it gets stronger with time and trust. And so, building trust is, time and trust are very important. But again, demanding to be, to not get subservient, right? As a service, you're not a service provider, you are a partner. Establish ground rules with your partners. And like with everything else, I mean, the value you provide, the results will speak for themselves, right? The more, if you're, if you're bringing value to them, and then they want more of it.

I think the, I think getting that trust and as, as soon as, once that partnership is established, a key push, a key, I think another parameter here is the data science team and the IT team become sort of like more controlling of our own destiny, because then we can do the hard skills data science work. And if that is good, then we control our own destiny. So, there is a soft skills portion, which is like engaging with them, establishing a working relationship. That is a soft skill thing. Then we get to control our own destiny for a little bit. And then the hard skills kick in, hard data science and analytic skills kick in to establish and grow.

Thank you, Prabha. And on this, this topic as well, Mark, I see you put a great point into the chat as well, if you wanted to jump in and maybe share that.

Yeah, sure. Hi everyone, my name is Mark. I work for a health system as a data analyst in Houston, Texas. And what I mentioned in the chat was that, you know, sometimes part of like the business relationship between the teams, like the business teams and the data science teams, part of that good working relationship is understanding that, you know, what the work you do may only be like one of many factors that play into decisions by the business teams. So, I gave an example of like a model says do X and they do Y instead. And oftentimes there's a really good reason for that. And just understanding what that is, is part of like an ongoing conversation.

One example from our work is that we make recommendations on, so we receive a lot of patient satisfaction surveys. We make recommendations to different hospitals in our systems for, you know, what individual units should focus on for the upcoming quarter based on looking at like the variable importance on what's driving those patient satisfaction scores. So, we get down to the nitty-gritty, but like the business leaders, one thing we found is that they like to sometimes ignore that recommendation based on like the most important variables because they want to have consistency across all the different units in their hospital. So, that was really helpful for us to understand is that sometimes, you know, a model says, hey, you should focus on like trying to improve this one specific question. But by providing, you know, all of these different, you know, objectives, it makes it harder for them to improve overall. So, having that clear consistent, you know, one singular message has been super helpful. Thank you, Mark. That's really helpful.

Thanks for the comment. And it is the analytics and decision are two separate steps and they don't always have to be in lockstep with each other.

Siddiqui, I apologize if I'm pronouncing your name wrong as well, but I see you had your hand raised earlier and I just wanted to make sure we didn't miss you. Hi. So, you can call me Masro. So, I want to add a point. Masro, nice to meet you. Same. Very nice to meet you. So, how to make a good partnership? So, I would like to add three points. The first point would be customer first. So, whenever you are having a partnership, you meet with them as a customer. And then, what I try from my side that I always think for them, you know, if you think for them, this person is always thinking for me and he's adding value to my business, to my team, and they will always come to you when they have a project per se. So, that's a customer first approach. It works everywhere, outside, inside. Second is more like a consultative approach. So, most of the time, the other partners, the stakeholders, they don't understand what they want from you. So, they will come with a random idea. They have this problem and they want a solution. So, rather than thinking from their head, you should keep a consultative approach. So, when you do that, they think, okay, he's actually adding value to me and I did not think from this point of view or vantage point. So, then they start trusting you and you build trust with them. And one, what is it for me? So, most of the time, when we work with like IT, they will always have a problem, you know. So, if you come with an approach that you also have something, if you help me, you will get this X and Y and Z kind of achievement or addition in the project. So, when you think from that angle, they will definitely help you. So, these are three points which I would like to add here.

That's awesome. Thank you so much for sharing those two and I see you're putting those in the chat as well. Rachel, I'd like to add something to what Mustru said, right? Definitely. This is presenting to executives and business team, right? Data scientists and IT people, we have to talk the business language when we present and put the results of the models and the analytics in the perspective of the business language, what it means for the company, what it means for their business process so we have to learn their language, translate data model and analytics to their language to talk to them and I think both sides need to make that effort a little bit. And so that's very, that's a, you know, Mustru touched upon this in using other, say, criteria and values but it's very important. I like Christian's comment too, and use colors, lots of colors, they like colors.

Common traps for data science teams

Prabha, the other question that I saw come in from Slido was, are there common traps that data scientists or data science teams fall into? It's a broad question. Measure, yeah, so we, you know, many of you may have seen like, you know, data science is three different, you know, this Venn diagram, there's the coding part of it, there is the math part of it and then there is the business domain part of it, right? So, make sure that you are, you know, you're not, you know, you're not, you know, make sure that all three are covered in a project, right? So, and if that's the capability side of it. The other big trap is when, say, executives come and say, hey, can you do something cool with artificial intelligence or machine learning? So, that is a solution looking for a problem and that's a trap, right? Because that has the potential for like, it has the potential for being, find me a needle in a haystack kind of a thing and you go keep looking for it and it is ripe for that kind of an engagement to get to project creep, right? So, that's actually a very, I'd say that's the very big trap to fall into.

I think it is connected to this finding a needle in a haystack thing. Set a deadline or clear criteria when to stop investing in a project. I think it's a, it's an off, you know, it's a next step to that second point, but I think it's a hard thing to do, but it is almost like a sunk cost thing, but it is a definition and criteria for having that stop.

Thank you. If you don't mind, just hop in, just add to that comment. I would also say one of the challenges is that sometimes data scientists always want to do the most interesting model to them to help them learn, but it's also sometimes like taking a sledgehammer to try to hang a picture, right? So, choosing the right model for the right situation, I think is one of the traps or downfalls that they fall into. That's a great point, Scott.

Delivering value and calculating ROI

Yeah, I think maybe some of this maybe had been tackled already, but I think sometimes it can be really difficult and I was just more curious about the trust and value sort of link, and are there specific things, sort of concrete, maybe almost like a recipe that you're following that, or if you can turn it into a recipe for others to follow, what your advice would be?

Dollars is the win all thing. That's the, if you have that you have everything else. The, if you are not able to get a clear way to dollars, then business KPIs. What matters to your constituents, or your partners. That, so it's, if I have to rank them, so to speak, in order. Dollars, if you are able to get to that calculation of dollars, great. That's the best case. The second best case is business KPIs, what your partners care about. Then, much, much. So, one and two are fairly close to each other. The number three lags behind in a very far away land is model metrics, how good the model is. So, that's how I think about it. Very accurate, cool model does not create value.

Dollars is the win all thing. That's the, if you have that you have everything else. Very accurate, cool model does not create value.

Yeah, I think Christian also put in there. The failing fast or delivering value quickly because you could have this conversation and come back three months later and as a business executive I've moved on. I'm already thinking about other things and you haven't demonstrated that you can actually add value. Quickly, so that feels like an important part as well. Yes.

Recommended resources

One thing that I love to ask everyone probably is, are there certain resources or whether that's books or podcasts, or other events like this that you attend that you'd recommend to us all.

I think from a business perspective value I can recommend a few books that I've read in the last few years that are not data science book, but it expands my focus of thinking of value and things like that. I would say, top of the list would be Prediction Machines. That's a book written by a few professors from, I think, Toronto, or University up in Toronto. Another book, I would highly recommend for business value or business value focused reading would be Measure What Matters. That is a book written, I forget the name of the author, but it takes a lot of concepts from productivity management that comes from Andy Grove and early years of Intel from Andy Grove and how high output management. I would say Prediction Machines and Measure What Matters. They are non-technical books but help to develop thinking and approaches and methodology for value creation.

Awesome. Thank you. If no one's going to jump in, I have one other question I want to ask you. I know you said focus on dollars, especially with selling internally and showing business value, but how do you actually go about doing that? How do you get to the point of understanding exactly what the dollar value is associated for each project?

That is an agreement that I make with my partners at the beginning of the project. We come, we make, you know, just has to be a straightforward calculation. It has to be A times B times C or something like that. And then we can clearly measure the A and the B and the Cs, right? So for a given project, and if we are not able to get to that upfront, we are not able to agree on the A's and the B's and the C's and the D's, then I know there is a lesser chance of success. If my partner or the business team is clearly able to say, these are the A's that multiplied by the B's gives us the value. And the current value of A is X1. And if we increase that to X2, that can just get plugged back into the equation. So that is part of that initial scoping and definition of success.

Because there are cases where, you know, take this pricing predictions, right? So there are cases where, you know, the model says something and you're able to go get negotiate and get a better price. It's really hard to at that point. Say, how much of that better outcome for flex is attributed to model versus the person who's doing the negotiation. So those are all like really hard. And my advice will be spend less effort on splitting the pie at that point. And if you are partners, then you got that value or money together, right? And if you haven't agreed upon A times B times C calculation, then that helps. You don't have to have a contention point at that time of how to split the dollars. So as much as possible, create what, how will you measure success before you start doing the work.

Awesome. Thank you so much, Prabha. That's extremely helpful. I just had a something I wanted to ask the group as well before we go. And just wanted to ask everyone if you would feel comfortable inviting someone to like join us at the next hangout. Feel free to share the link with them which I'll put in the chat, but I'd love if everyone would be able to reach out to someone from an underrepresented group in data science. I love to welcome other people into the conversations as well. And even if that's just one person that we all invite, I think that would make a huge difference. And just wanted to say that. So thank you all for thinking about that.

And thank you, Prabha, so much for jumping on and sharing all your insights with us. So thanks for inviting me and for the audience. Thanks for joining and listening and giving us the time. Thank you all. Have a great rest of the day.

Data Science Hangout | Prabha Thanikasalam, Flex | Calculating ROI with the Business

Transcript#

Excitement about data science leadership

Flagship projects at Flex

Supply chain and COVID

Building partnership with the business

Staying focused when learning data science

Measuring business value and ROI

Building trust and establishing partnerships

Common traps for data science teams

Delivering value and calculating ROI

Recommended resources

Featured software#

rstudio