Should you build or buy AI? | Jay Timmerman | Data Science Hangout

Welcome back to the Data Science Hangout, everybody. My name is Libby. I'm the Data Community Manager here at Posit, and for those of you who are new around here, Posit is the company behind awesome data science tools for R and Python. You might know us as the folks who used to be called RStudio, and we make things like the RStudio and Positron IDEs, Shiny, and other great tools. The Hangout is our weekly get-together, every Thursday, same time, same place, where we chat about all things data science across all of our different industries.

If you are watching or listening to this as a recording in the future, hello, you're really missing out on the chat. We'd love to have you live, so look for a link in the description to add our call to your calendar.

We definitely want to hear from you, so please jump in the chat, introduce yourself. Say hello, mention your role, where you're based, your LinkedIn profile link, or a link to your website, somewhere where people can find you, and something that you do for fun as well. You can also share resources, job openings, as well as any roles you happen to be looking for right now.

This is going to be a conversation with a data science leader that is powered by your questions, and we are using Slido again today. It's a great way to see all the questions and upvote the ones you're most interested in.

I am excited to be joined by our featured leader today, Jay Timmerman, Head of Data Science and AI Platforms at Biogen. Welcome, welcome Jay. Great to be here. I'm happy to talk to you guys about my experience at Biogen and doing data science. And just so we kind of are clear, I'm representing myself. I'm not representing Biogen here, so I'm happy to provide my personal experience with using Gen AI and how it can be used.

About Biogen and Jay's role

So Biogen, for folks that aren't familiar with it, is a large pharmaceutical organization headquartered in Cambridge, but I'm actually based in North Carolina in RTP. Biogen's kind of claim to fame is they created some revolutionary products in multiple sclerosis. And so those products came out in the early 2000s and the more recent years been pivoting more into rare orphan diseases.

So one of our major products is called Spinraza. It's for kids that have spinal muscular atrophy. It's a devastating neuromuscular disease. And their product has really just changed the standard of care for folks like that. There's another product we have released recently for Friedreich's ataxia, another sort of devastating disease that is actually changing the standard of care. And then more recently, we launched a product for Alzheimer's. So it's cool to work in a company where they work on really hard things and, you know, there's evidence that we're able to bring them to market.

So I sit in one of our, I work in IT, and I sit in one of our enterprise services organizations. And so I get the opportunity to work with everybody, right? I'm not aligned to just research. I'm not just development. It's really kind of interesting to be able to support the entire organization horizontally.

So the types of data we get to work with are all over the place. It could be a lot of corporate function data, like, you know, looking at expense data. We can work in our pharmaceutical operations group. We might look at vendor invoice data and customs declarations. In a commercial organization, some real world evidence. So it really is massive in terms of the scope of data that we have access to.

So in terms of fun, I have two kids. I have a, my son is five and my daughter's three. So most of my fun activities are spending time with them. My son just started t-ball, so it's kind of exciting to be able to, you know, get him to learn how to hit the ball and play catch in the yard.

Vibe coding and code oversight

So vibe coding is like, so these LLM enabled code generation tools are becoming more powerful. There's things like Cursor, which is like an IDE that is used to help generate code. And there's also other ones, I think Windsurf is another one. And so the idea is these things are becoming increasingly powerful.

For us, we don't do that. We're not in that kind of business. I think if you're trying to experiment with yourself, see what you can create, I think that that's kind of a fun experiment. However, I would caution, if you don't know what you're doing, you have to expect that the quality of what you're creating is probably not great. And also the security components of this. I've seen examples where, you know, embedding keys and code, kind of a big no-no, the vibe coding could do that. So I think for exploration, I think it's a reasonable thing to do if you want to see what these tools are capable of. But putting it in front of external facing people, I would be extremely careful.

The way that my team operates is we are very focused on generating repeatable components. So I'm less interested in someone creating some crazy new capability and like it has some crazy functionality. And I'm more interested in people building a component that other people in the team can reuse. So for that to work, everyone needs to be able to understand how the code operates.

The rise of the head of AI role

I think head of AI is increasingly a legitimate role that is not just a manager AD role. I'm seeing it more at the VP, at the executive level. And I think the reason for that is there's a recognition that this capability, if the capability got no better, like if suddenly chat GPT or whatever just stopped, there's still a gigantic opportunity space of how to learn and apply the technology, but it's going to continue getting better.

One of the things that I see a lot of is software as a service vendors sort of popping up, providing kind of like niche solutions. And what I've seen is basically people want to wrap, they basically build a tool that's a wrapper on open AI and they call it a product and they want to charge you money for it. And so having this sort of head of AI, people that are responsible for being that expert kind of prevents this like, I call it like AI grift. You basically prevent these grift organizations from getting in and start extracting money. When in reality, what they've built is unsophisticated. They don't have any intellectual property. They don't have some secret data. They basically just glued a bunch of off the shelf stuff together.

Having this sort of head of AI, people that are responsible for being that expert kind of prevents this like, I call it like AI grift. You basically prevent these grift organizations from getting in and start extracting money.

I think the AI lead part is really, there's really two components to it. One is a platform component. When you build AI stuff, I call them workflows. I think workflows is the right way to think about it. You need a set of technologies to support that thing, but then you also need a team that can deliver.

The way that this happened is there was enough groundswell of impact. Like there was enough evidence and, you know, like, so imagine you're an executive, right? You starting to see things bubble up like a couple of examples. You're like, oh yeah, that is useful. And then you're also reading in the news, CIO magazine or whatever. Hey, you have an AI capability. And I think that these two things together are starting to create pressure to create in these separate functions.

Data privacy and LLMs

Working with healthcare data, have you noticed any changes in how you think about and manage data privacy as LLMs and AI become more mainstream?

Identified patient data has existed for a long time in systems. And in many ways, we already had processes in place to keep data private, to protect it. And when you add in the LLM component, it's really more of an extension of that. It's not like something completely new. Sure, the AI has extra stuff. I think the concept of responsible AI, sort of monitoring how it outputs, that is sort of new. But the data privacy stuff, I don't really see that. I think it fits within how data was already being managed today.

I try to focus on keeping this as kind of an extension of things we already have. I think if you think about it as this brand new capability technology, it makes it less clear how to like act, like how to manage it. And if you think of it more, it's like a new piece of technology. In many cases, you probably have like policies in place that can kind of cover it.

Training models with secure data

So I think when you say training, the thing that comes to my head is you mean either building an LLM from scratch, which nobody does anymore, except the major research labs, where you're talking about fine tuning, which is you take an existing model, you give it a new set of data, and you try to like use gradient descent and deep learning to get it to respond in a way that you like.

I think that historically fine tuning, like historically as in like two years ago, one year ago, that was a legitimate technique to try to get the model to fit your output more. I think that I am more skeptical of those techniques now. I think that there are things like retrieval augmented generation, which is, again, I don't know if folks know about what this is, but the cheat, the hack way to get new information available to the model is you basically take your content, the stuff you care about, and you like, you don't try to train it, you don't try to put it into the model. You just sort of like put it over to the side in like this like database thing. And then when you ask a question, you have this mechanism that like looks through that document database and pulls the right piece out. To me, that is the more practical approach to it.

Dealing with hallucinations

When I talk with people about building a gen AI workflow or solution or whatever, I say that this is an intrinsic part of the solution. You can't box it out. It's a feature of the LLMs. You will be unable to physically remove hallucinations from it. Now you can minimize them, right? And that's what the goal is, is you built a solution that minimizes that problem, but you need to have a solution that's resilient to the hallucinations.

So when you frame it that way, it kind of gets people's minds turning in a different direction. So you're like, okay, we need this secondary verification system. That could be a human in the loop thing, or maybe what it generates, if it's wrong, it's not that big a deal. If you have a question and answer chat bot or something like that, and it's informing you on the HR procedures or what your benefits are or something like that, the risks of the hallucination there are probably lower than if it's like, I just messed up something in the bioreactor, what do I do next, right?

So maybe you have the solution extracts information, and you can verify with independent information. So when you do that, yeah, the model might hallucinate, but it'll at least focus you in on the thing that it hallucinated on. So if the data matched 90% of what was in a source system, and 10% is wrong, well, now you're inspecting 10%, you're not inspecting 90%. So I think framing it that way has been very effective for us. And just like attacking the fallibility head on, just not hiding it and then coming up with really good ways to mitigate it as part of the solution.

Building repeatable solutions and using Posit Connect

When I think of repeatable solutions, I'm thinking in the context of like a chatbot, right? Like let's say you want to build a chatbot on documents. How do you like, what's the code to do that? So I think of those as, when I say repeatable, I mean like repeatable design patterns.

There's some of these frameworks that exist now to help make this better. So LangChain is one of the most popular ones. There's also something called Llama Index. And then, you know, you're starting to see that the big research labs like OpenAI are starting to embed some of those capabilities within their SDK.

To me, the jury's still out on a specific framework. We do use LangChain, but specifically, we use their LangChain expression language because it's supposed to be more robust. I don't really think of it like a container problem. It's really more the amount of code you need to write to build one of these solutions is actually pretty low. And something like Posit Connect product plug works really well for this. So imagine you're building some chatbot that you want other people to use. Sure, you could use Streamlit, but you could also do it in Shiny, like Shiny for R, and you could deploy it to Posit Connect and it could achieve the exact same thing as that.

I'm an old R user, I guess. So I graduated from grad school in 2013 with my master's in analytics. And that's where I really fell in love with the products, saw the tidyverse sort of develop, saw Shiny get created, see the development of Posit Connect. So I think that I am purely from the R ecosystem. I'm an R guy, like through and through.

The Posit Connect product specifically is extremely useful because I think someone, the previous guy was talking about containers. One of the things I like about Posit Connect is it automates a lot of the DevOps stuff away from you. You don't have to care about it. You just, you bring your requirements file, you bring your contents file and it just builds it for you and deploys it and you get all the role-based access control. So it really simplifies the deployment process. And when you're trying to like rapidly build something that people can use, that makes a big difference.

Talking to stakeholders about AI and data readiness

Number one, you, the person dealing with this, need to have enough knowledge of how these tools work to be able to answer it effectively. I think that there is a prerequisite that you understand what is and is not possible with the technology, and that requires you to have tried and used it in a variety of different ways. So, like, I guess my message to folks is try to use this stuff, get a feel for how it works, what it's good and bad at, and then it'll more prepare you for these types of discussions.

The way I always frame this is, again, I spend a lot of time trying to explain AI and the way it can be applied, and the way I do it is there's inputs and outputs, and typically I say if you can give the model the adequate context, it can probably answer the question, right? The problem is a lot of business process is not written down. There's a lot of context that may not be in the database, right? Like, there's a number, but a lot of times there's the context behind the number, and the model is incapable of knowing that.

A lot of times what I do is I don't try to, like, push them off. I'll try to deeply understand what they're trying to achieve, and there may be a smaller component you can solve with this. Maybe you don't solve the whole process. Maybe you only solve one piece, and then that actually helps you get, like, build momentum because it'll be faster to build, and then you'll have evidence that it works, right? And so, I think I try to scope them down smaller, and then you get that critical mass that can expand it.

Data science teams and AI

My team, my team are data scientists, historically trained, classically trained, you know, regression, all that stuff. And now they're very effectively running gen AI projects. And the way I look at it is, this is like a tool in the tool belt. Like AI is not, while it's new and shiny and everyone wants to get into it, data scientists are uniquely positioned to be effective at this. The difference between a data scientist that does shiny apps and a gen AI developer that builds AI workflows is an LLM call.

The way you frame the problem is the same. You have a business problem. You're trying to figure out how to frame it and execute a series of steps to generate a desired output. Before you had shiny apps, maybe have an algorithm. Now we have an LLM to kind of stick in there, but all of the other things around it are the same.

Will all data science teams turn into AI teams? Maybe, potentially. I think that if you don't and someone does this, that will be bad. I think if you push off AI as a data science team, you're at risk of being marginalized because the capabilities of the models and the tooling is increasing. And so, what do we do when the tool can do many of the things that we can do, right? You don't want to be stuck as the manual verification step. You want to be building the tools because you know what the model should be doing, right?

The difference between a head of AI and a principal data scientist, to me, they're fundamentally different. Principal data scientist is a lead. They're a tech lead. They're a person that understands the landscape of technology. They have experience managing teams, but they're not setting the strategic direction. The head of AI, to me, is a person that manages teams. They're understanding how to position AI, the technology within the business, developing relationships with the stakeholders, kind of having higher oversight on the products that the teams are building.

Build versus buy

The way I look at this is, I have a team that's focused on delivery. We have access to all the same tools that every other vendor has in the universe, right? Like, it's not like some SaaS company has some magical access to models that we don't have. We're all starting from the same level. So, my thing is, okay, if I see a technology, we know the technology is very, very powerful. What is the vendor bringing to the table? Are they bringing intellectual property or expertise or something like a more mature product that is truly differentiated from Jay slapping a wrapper and a streamlined app together to solve a problem?

And as you start to interrogate what the tool does, in many cases, it falls down. Like, a lot of times the solution is a React front end on a rag chat bot. And you start to ask, well, did you fine tune it? No. Are you using some kind of proprietary technology? No. It's like, you just have a fancy interface on something that's already available.

So, I think that when I think of build versus buy, it's like, how long would it take to build that internally? What are the functional requirements of it? Are we supporting 10 people or 10,000, right? Do we have the skills internally to build it? How quickly is the answer needed? So, I think that it's really a balance of power of the product, what's our internal capability, and then it will help you make a more informed decision.

If your organization has no people building GNI solutions, then they're extremely susceptible to this because there is no alternative. There is no person to build the Streamlit app. They're just, this is the solution and there's nothing else. So, whatever the price is, is the price of the product. So, I think that's another thing we can do to defend our companies is build this capability so we have a viable alternative to buying something that's not that great.

Career advice and stakeholder engagement

I give a lot of unsolicited advice. That's unfortunately a flaw in my personality. I think when I try to solve a problem, I try to come at it with coherence. And I think in the AI space, this is very important. Because there's so many pieces moving and there's so much technology, a lot of times if you just anchor on, does this make sense? Can I explain this to another person and it makes sense? And if you can do that effectively, that is a very useful skill because it's actually a tool you can apply to any situation, right?

A lot of times if you just anchor on, does this make sense? Can I explain this to another person and it makes sense? And if you can do that effectively, that is a very useful skill because it's actually a tool you can apply to any situation, right?

There's an important thing that is different or that is uniquely different about Gen AI versus traditional data science. Stakeholder engagement is the difference between success and failure. It actually doesn't matter how tangible the problem is. If you don't have that stakeholder engagement to sort of direct you on what looks right or wrong, that is where it's failed for me. It's not even like, can it solve the problem? But if they won't show up to meetings or they don't give you the good guidance, the product that you produce will not be very good.

Human-centered language for AI

What is your opinion on using human-centered terms for AI? So, learning, thinking, reasoning, when you're describing the operations of large language models.

I avoid them. What I do is I try to frame this again. I try to use simple language and frame it in the context of things people already understand. So, what I tell people is, an LLM is a text computer. You have a CPU in your computer, an LLM is a text computer. And if you can provide information to the LLM, it will be able to compute and output that way. That is a more like intuitive way I try to explain it versus, yeah, I can learn and reason.

I try to treat it like it's just another technology. And I feel like that gives you credibility because you're not using jargon. Like, I really avoid jargon. I try as much as possible to avoid jargon. And that's what, you know, for the RAG example, right? Like, could I have said embeddings, vector databases? You know, like, I just said, no, it's like this like thing next to the model where the documents go. And then, you know, it just goes over there and gets it. That's about the level of understanding people need to have to use the products. It doesn't really matter how much depth you have there because the application of the technology is where the value is.

Well, I think that we should probably wrap up and give everybody a chance to hop to their next meeting. We have Bill Wilkins next week, SVP of advanced analytics with practical applications at Safety National Casualty Corporation. Please come and join us next week. We love having you here. Jay, this has been a fantastic discussion. If you would like to go find Jay, he's Jay Timmerman on LinkedIn. Jay, thanks so much for joining us. Absolutely. It was a pleasure. Thanks, everyone. Appreciate the questions.