Posit Academy Overview

Hi, everybody. Happy Tuesday. Hope you're having a great day. Welcome to today's Posit Enterprise Community Meetup.

Over the past few months, we've featured a number of stories from amazing pharmaceutical leaders at Roche, AstraZeneca, GSK, Eli Lilly, Pfizer, Janssen, all sharing their perspectives on the adoption of open source. One of the common questions that I've seen across many of these meetups, webinars and data science hangouts is how teams are training the next group of users and questions about best practices for those learning.

I'm so happy to have Garrett Grollemund, our Director of Learning at Posit joining us for today's presentation. Garrett is an award winning R instructor, the author of Hands-On Programming with R and co-author of the bestseller, R for Data Science.

I also wanted to say thank you to Devin Pastor, one of our principal solutions engineers who works with a lot of our pharma customers at Posit, who is helping out in the background will also join us for the Q&A. You may see Devin answering a few questions in the YouTube chat as well.

So with that, you can use YouTube for questions, but you can also ask anonymously through the Slido link that I'll pull up on the screen here as well. And we'll copy that over into the chat too. But with that, thank you all so much for deciding to spend some time with us. And I will pull Garrett up on stage here with me and turn it over to you.

Well, hello, everybody. As Rachel said, my name is Garrett. I am the director of learning at Posit, but I've been working with Posit, formerly known as RStudio, for about a decade, mostly teaching people how to use R, to write code, to do data science, and the other things that you do with R. And now more and more Python too.

Many of the clients that I've worked with were people who were trying to migrate from SAS to R or from Excel to R or you might just say nothing to R. So they're trying to train teens to learn how to code, do reproducible work, and they do program-assisted data science. What I discovered is you can teach adults, no matter any age, to do this very effectively. But whether or not the training ends up being a success depends on how you do it. And that's going to be the theme of the talk today.

How should you train your colleagues to adopt R or Python or any code language to do what they're formerly doing in SAS, Excel, or on their own?

Why workshops and online courses fall short

Now for a lot of those 10 years I spent training people, I was teaching workshops. I'm going to try to convince you today that workshops, standard training of all varieties, really is not an effective way to retrain or retool a workforce to use code. It's also true with online courses. But I'm going to show you how you have options beyond these things to do a very good job of it.

When I joined RStudio 10 years ago, it was a very different world. A lot of people didn't know about R, or if they did, they had these impressions of R that were not very complimentary. So we had to spread the word about R, and we decided to do that by teaching people to use R in the tidyverse, which we were creating at workshops. We also experimented with other partners to do video courses, online courses, MOOCs. If there's a way to teach data science, I have done that.

What I quickly noticed teaching data science was that teachers out there really aren't that good. The workshops you see at conferences are not so great. And even some of the online courses, I really questioned how effective they would be. And after thinking about this, I realized the people who become experts at coding or data science did that by devoting years of their life to becoming that expert. But they didn't have a chance to devote those years to being a good teacher, or even find out what made teaching work.

Serendipitously, my background is in psychology. I have a PhD in statistics, but that's from doing psychological research, I work my way over to writing code that supports statistics, and now what we call data science. And my background in psychology alerted me to the fact that there's many things that you can do to make training more successful, and I thought, well, there's these theories that are well established, like cognitive load theory, multimedia learning theory, all the rest. They all apply to learning. And if we apply these to the data science training environment, we should be able to make a much better workshop.

If there are any trainers in the audience, I strongly recommend learning about these things, and there's a book that makes it really simple to learn about the research. All the most important educational research is covered in this book here. It's called How Learning Happens by Paul Kirshner and Carl Hendrick. So that's something you might consider if you're developing your own training programs.

Anyways, I took this and I put it into my workshops and my training programs, and it worked amazingly. People love the workshops that I was creating. I started to win awards like crazy. Correspondence was sent to me from the American Statistical Association because I'd come to teach at their JSM conferences, and they'd poll all the students, their statisticians, and the workshop that was highest rated amongst many, many workshops would always win an award, the Excellence in Continuing Education Award. Well, I won it three years in a row, and that was unprecedented.

But I thought, wow, this is great. I've cracked the code. I can teach a really effective workshop. These students are telling me in every workshop that learning R is going to be life-changing for them because they could see how it would be so much easier to do the things they're doing and they'll actually be able to do some more.

The forgetting problem

So I thought, the world looks pretty good, but I still have a background in psychology and I started exploring educational psychology, the literature of psychology that's related to education, and there are some disturbing signals in there, and I'll share some of those with you.

For example, the best we could tell based on research is that students only remember about 56%, so about half of the content, that you tell them in a lecture. And that's if you test them immediately after the lecture. And we expect that this drops off from there as time goes by.

Not only is this disturbing, the students who are being tested here are undergraduates at college who we might think of as professional learners. If they can only manage to remember half of what they learned in a lecture, then the distracted adult let's go right back to all their other work responsibilities, probably go learn even less.

Next other studies, here's a study that looks at procedural instruction. So this is more like teaching someone how to do a procedure, whether that's using a machine or fitting a certain model or something that you do, it's a behavior. And they find the same numbers immediately after the training, where the students are doing the thing, only 60% of them can reproduce it. And then six months later, that drops to 40%. One year later, that drops to 30% and so on.

And most trainers I discovered weren't aware of this at all. But you can look at the students being trained and kind of get the sense that this is exactly what is happening.

I went back to my workshops, and I continued to teach, and I continued to get rave reviews. And I realized what all the knobs were to make student satisfaction really high in a workshop. But I also realized they don't really relate to the stickiness of what the student's learning. And anecdotally, I started to check back in on some of my students, and I realized, even though they love the workshop, and they could do lots of great things in the workshop, they weren't doing those things a year later or six months later. To me, that seems like a training failure.

To me, that seems like a training failure. And I'm not going to speak for everyone, but I almost feel like it's a dirty secret of the training industry that you don't get terrific outcomes as a trainer.

And I'm not going to speak for everyone, but I almost feel like it's a dirty secret of the training industry that you don't get terrific outcomes as a trainer. I've heard people talk about marketing and say, you know, we know half of what we do works. We just don't know which half. Well, it seems like that might be the case for training as well. But we're dealing with a field where people aren't trained to be trainers. So I think there's lots of upside to do much, much better. And that's what we set out to do at RStudio, which is now Posit.

And the way we fix this is by looking into why this happens. So some of you may have seen this graph before. It's called the forgetting curves or the ebbing house curves based on the gentleman who first postulated it. This summarizes a body of research that goes all the way back to the 1800s, has been verified in many different fields. It's just a fact of human life.

The way to look at the graph is these blue lines are someone's ability to do something, be that to do a task, to remember vocabulary, to take a test. It's just their ability to do that thing that they learned. And the first blue line shows how that ability changes over time.

What happens consistently is your ability to do something drops off after you first learn it. And especially if you never use it again, your brain basically cannibalizes the resources that devoted that ability and uses them to do other more important things. And eventually you can't do that thing anymore.

But you don't have to settle for that outcome. If you wait until that thing starts to get difficult for you, and then you practice it and do it again, you're essentially relearning it through your practice. And you do it again, and you jump up to the next blue line. So your ability started to fall off, but you practice, and now you can do it again pretty well. And then it starts to fall off again. But it doesn't fall off as fast as it did the first time if you had never practiced it. And if you repeat this process and you practice again, you'll be on the next blue line, and then the next blue line.

Whenever you don't use something, your brain does start to lose its ability to do it. But the more you practice the thing, the more your brain decides to conserve those neural networks and retain that ability. Now if you've practiced something enough times, you've learned to ride the bike and it stays with you forever.

You could sort of see why this doesn't play out in a workshop that you take for half a day or two days or what have you, and then you never really see it again. These practice sessions work because your brain is building new neural circuitry in between. That means you have to go to sleep, nighttime sleep, in between these sessions to really build up a robust neural network.

How the brain builds skills

All right, so we could see in the data that's how that works, but we also know from research and study why it happens. This is your brain. It's a collection of neurons that are joined together, and your abilities and the information you know and the memories you have are all encoded into your brain as networks between these neurons.

The network is set up so when one neuron in the network fires, the other neurons fire as well, and they trigger whatever behavior needs to come together to make this experience that your brain is recreating for you. We can imagine having two networks that do two separate things in our brain. One of these we use, every time we use it, those neurons fire together, and then your brain wires them together a little more strongly, because this is how the brain works. You probably heard neurons that fire together wire together. Every time you use it, that network gets stronger and stronger, whereas the one you don't use gets weaker and weaker.

Your brain has important things to do. It's not going to maintain a network that doesn't do anything useful until eventually over time you'll have a very, very strong network at the thing you were doing, so strong that doing it no longer takes conscious thought. It's completely automated. When one of the neurons in the network fires due to some cue or trigger, all the rest just automatically cascade, and you juggle the ball, or fit the model, or interpret the data. Whereas that other neural network that we never use, it doesn't even exist anymore. It's gone.

This is the biological phenomenon that explains the forgetting curves. Like I said, this wiring together occurs while you're sleeping, so this is something that happens with lots of repetition over a long period of time, weeks, let's say.

If we go back to our model and think about, do we want our colleague to be able to do this new skill, pick a skill where the answer is yes, and then consider, what is going to determine whether or not he or she could do that six months from now? You'll discover it's very little to do with the initial instruction we give the colleague and everything to do with the amount of practice they've had shortly after that instruction.

How do we train to take advantage of this? How do we train to make people who can use R as well as SAS six months from now?

Something else to consider as well, just watching a lecture or watching a video on your computer about doing something probably builds a neural network, but it's a neural network that's related to sitting still and watching a video unfold. It's actually doing the thing that builds a neural network so you'll later use to do the thing. We have to be conscious of that in training too, and unfortunately a lot of training doesn't really focus on the student actually doing the things. It's more, hey, watch this video.

Well, the revelation we had at our studio is that data science is a skill. If you want to learn to do good data science with code or otherwise, you have to practice it and learn it as a skill. A great analogy would be playing the piano or some musical instrument. Those are skills as well.

There are things you need to learn like how to read notation, rhythm, music, but you also need to sit down and practice the piano if you want to play a song on the piano. You wouldn't expect to do this well by watching an online course or videos online. You wouldn't expect to learn to play the piano well either by attending a short workshop that lasts for a day or two days. I mean, it might help, it might get you enthusiastic to do the practice you need afterwards, but at the end of the day, if you want to play the piano, you do need to practice over time. And usually with some sort of feedback from an expert who's mentoring you as you play the piano. And the same is true for data science. We need to use practice over time to learn to do it in a new way.

Well, what we find or what you would find too, if you research the training communities, students don't complete online courses. They don't stay very engaged with them even when they do complete them. And that being someone who's designed many online courses, I can tell you, it's very hard to design an online course that multiple students can come to and be challenged by without a teacher there overseeing the process. You sort of have to design for the lowest common denominator, and that's not enough to help the rest of the class learn.

When we look at really well-supported massive online courses, like the Johns Hopkins or Coursera MOOCs, we see that it's typical for about 10% of students to finish those. I've also had the privilege in my life to look at the data for some online data science training providers because we were partnering with them, and what we saw was something that was very similar to a gym. Lots of people pay for a subscription to go to the gym, but at the end of the day, they never show up and go to the gym. They're just sort of paying.

And the missing element for these online courses, whether it's a MOOC or this gym model, is motivation.

So lots of people think education works this way. There's an educator. They have the information that the student needs. There's a student. We just need to transfer that information over, and then the student's now an expert. But that's not how education works at all. Information is cheap and easy, and you could get it from the library or YouTube or wherever you want it, but if you want to learn to use that information, you have to do something after you acquire it, and this is the hardest part of learning something.

You have to practice it in a loop over and over again, and it's easy to start practicing things the wrong way, and as a student, you might not even know you're doing it, and then you're just building neural networks that encode mistakes. It's not going to work out well. You need to practice and get feedback, maybe some coaching. Practice itself is hard work, and all of that work requires you to be motivated. It's hard for busy adults to stay motivated at one more form of busyness, but this blue loop here is what matters if we want our colleagues to become excellent R users or Python users or data scientists. Most training environments ignore the blue loop. It happens after the training's over.

Most training environments ignore the blue loop. It happens after the training's over. So what we decided to do is focus on that blue loop to make it easy to do the hard work of learning, to do it as efficiently as possible so you know it's paying off, and to give you the coaching that keeps that practice loop where it should be so you're practicing the right things instead of spinning off into mistakes.

So what we decided to do is focus on that blue loop to make it easy to do the hard work of learning, to do it as efficiently as possible so you know it's paying off, and to give you the coaching that keeps that practice loop where it should be so you're practicing the right things instead of spinning off into mistakes.

How Posit Academy works

How do humans actually learn skills? Might seem like the training industry is botched, and if you want to help your people transition over to R, you have a hard time of it, but it's not actually the case. We learn skills every day in modalities that are very familiar and that lots of people have confidence in.

So we use pianos as a metaphor earlier. If you wanted to learn to play the piano, you would find a piano coach or music teacher and they'd give you some one-on-one feedback, and it'd be a process. But every apprenticeship also works this way, every internship, every residency. If the thing you're trying to learn happens to be a physical skill, we call it coaching. These things work all the time. We need an experience like this for using R, for doing data science.

So how should your analysts learn data science or code or R or Python? We've put together an experience that looks like this, and this is my answer. This is how it should look. If you train with Posit Academy, the first thing you have is a mentor. A mentor is someone who knows how to do data science with R, and they're going to help you. They're going to coach you to learn it.

We have two types of mentor at Academy. One is the type that we provide. We provide experts to help people learn, and the other type is a mentor that the company who engages us provides. They act as an apprentice in the course experience, an apprentice mentor. But they're another resource for students, and the cool thing about apprentice mentors from the company is after the experience is over, that apprentice is still at the company, still acting as a mentor, still forming the node of a support network within the company, so you build up a community of practice.

The next thing you have in addition to your mentor is group mates. We've designed a cohort-based learning experience where no one learns in isolation. They learn in small groups of five to seven students who are all trying to learn the same thing as you are. So as you go through the course, not only do you have an expert who has your back, who's coaching you, you also have fellow travelers who you could discuss things with. You could work through problems together, and you may not even realize it, but you can hold each other accountable and motivated as you go through the process.

So those are the people involved. Next we give you a project to do. So an Academy course is not a course. It's an apprenticeship. You're going to do something very real and very close to what you do at your job because that's ultimately what you want to learn how to do. We want you to see the value of everything you're learning, and the best way to do that is to make it similar to your work, which is very valuable to you.

So an example project would be comparing different treatments, maybe with asthma, maybe with some sort of disease. If you came to us from finance, we'd give you a finance project and whatnot, but we give you something that is close to what you're going to do, and you're going to do it under the guidance of the mentor and with the support of your group mates.

Each project is scoped to cover what we think or we have found over the past decade is essential to learning R. So that begins with importing data, exploring the data with plots and visualizations, wrangling your data, analyzing it. If you're familiar with R, I'm talking about dplyr, tidyr, joins, that sort of thing, but also fitting models to your data. And then how do you summarize your work to communicate with someone else, and how do you do that reproducibly so the next time you have a similar project, you're already way ahead and you start with your previous work versus making it up from scratch. We focus on R Markdown and now increasingly Quarto, which is the next version of R Markdown.

So you're going to do all of that to complete your project and do that real life thing that you will presumably now be able to do at your job and create a lot of value with. But how do we make that happen? We break the project into 10 progressive milestones. Each week you work on one of these milestones. When you get to the 10th one, you finish the project, you have your portfolio piece, which is a report that shows the conclusion of all your data science work.

But each of those milestones build up to that in sequence. So first you learn the basics of working with R, how to troubleshoot, what to do if you get stuck and you need help, or what to do if you hear of a new package or a new function and you want to teach yourself how to use it. We teach you how to teach yourself. That's a big part of working with R. Then you learn visualization, then you learn data wrinkling and joints, and so on.

Think of the project as the skeleton or the backbone of the course. Most courses have a syllabus, but we have a project, and it functions a little bit like a syllabus. You're going to need to do all these real-life things to finish the project. We divide into our milestones, and then we prepare tutorials that teach you what you need to do to do that milestone.

So if you have to fit a model in R and you're just learning R, you don't know how to do that. We're going to assign you tutorials from the library tutorials we've written that will show you how to fit models in R. These tutorials are designed to teach through exercises. We've written grading code to give us a type of artificial intelligence that will look at the code you submit, and if it's wrong, they'll give you advice on how to fix that code. It doesn't focus on unit tests or making sure your result is right, it looks at what you're writing and tells you how to change that, because that's what you need to learn.

And then at the end of that week, after you've learned your tutorials, done your milestone, we have you get together with your group and present what you've done. We ask you for each milestone to recreate something, a step in your project. If you've studied the tutorials, you should be able to do that recreation. But then we ask you to teach yourself something new to create a personalized or extended milestone that you could present to your groupmates. This creates a lot of cross-pollination when the groupmates get together, they learn from each other, but it also helps you practice teaching yourself new parts of an ever-expanding free and open-source language like R.

The group sessions, we hold them over Zoom or Microsoft Teams, looks something like this. You share your screen, you're going to be working with the RStudio IDE, which is a real tool many day scientists use. What's in Posit Workbench, you also end up working with Posit Connect, so you learn how to use the real tools that you can start using the day after academy ends. In fact, most students start using them well before academy ends at their work to create value.

Your groupmates give you the accountability. You have to show up with something to show them. You also meet with them during the week one time for a co-working session to work through trouble together. But they're more than just the people who make sure you did your assignment. After the course is over, they're your future support network. They're the beginnings of a community of practice that you've established by learning R inside your pharmaceutical company.

So a typical week, do some tutorials, complete and personalize a milestone, present to the group. In between, you have a co-working session with your groupmates to help out. It's optional, but we strongly encourage people to go, and most people do go. And then you have the communication channel like Slack or Microsoft Teams where you can discuss the project and answer each other's questions. And your mentors are there too to answer questions. Your mentors also have each of the in-person sessions. Repeat this until you finish your project, and that's what the academy experience looks like.

Results and student outcomes

So does this work? Can this teach someone who doesn't know R to use R and do real data science with it? The answer is yes. We've taught about 600 students so far using this method. We have about a 95% completion rate. This is for a course that lasts 10 weeks, 12 weeks normally, because we put in some breaks for holidays and stuff just based on how the year works out, and the students stick with it.

People are really excited to be students in this sort of experience. They're really excited to actually apprentice on something, learn it well, and have a mentor there to help them through any question that comes up, any challenge that they have, or anything they want to explore that we didn't put in the course, but they know they're interested in.

So our students like it, but our customers are actually companies. We're trying to help companies create capacity to do work that didn't exist previously in their company. And it works there, too. This is probably the most on-the-nose quote from a previous customer. James Wade shared this at a previous RStudio Enterprise Meetup.

And he said, they were very interested in whether this was working, so they went back and they surveyed the students they put through Academy six months after they left Academy. And they found that 16 out of the 17 students were still writing code at least once per month or more frequently, using what they learned in Academy. And that 17th person, by the way, was promoted to a manager, so they no longer had to write code, but they were convincing their direct reports to start using R and write code based on what they did at Academy.

And they found that 16 out of the 17 students were still writing code at least once per month or more frequently, using what they learned in Academy.

Now I don't know how you feel about that, because you're not a trainer, but perhaps based on what you saw at the beginning of my talk, that this is unheard of in the training community, 16 out of 17 survey response. But it's not unheard of out of our customers. This is par for the course, but James went on record about it.

We have taught pharmaceutical companies. There's a quote on our website, paws.co.uk slash Academy, where AstraZeneca talks about how great their Academy experience was. People notice the difference and they sign back up for more cohorts of more students.

Anyways, thank you for listening to me. You may have noticed I'm not a pharmacist. I'm an educator, and I geeked out on education, but I hope after hearing this, you get a sense of what you need to do to successfully transition your colleagues to R or to Python from wherever they're coming from. These principles work in every circumstance, and if you avoid them, you're going to have fleeting gains at best.

Demo and Q&A

This is the student landing page or an example student landing page for the Academy experience. Now, remember, so much of the experience is about that social motivation you get from your group and your mentor and talking to them outside of this platform here. But when it comes time to do your learning in a tutorial, this is where you go. We have a syllabus on the side that's divided into different milestones. You can see first we're just learning R, then we're importing and visualizing our data, adding variables. These things are progressive. They build on each other until eventually you're getting models and writing your own reports and publishing them.

Within each week, we have the lessons that you take and then the milestone that you do. You go into the milestone file. We provision the IDE with the different files that you would need or that you might find at your job if you were doing this type of task. And we have an R Markdown document that guides you through the task.

This is the first milestone. It's kind of like a hello world in some ways, but we gave you some data, we want you to open up, we want you to look at it, we want you to recreate something. In this case, we give them an image of a plot and we say, there's a function that does this. It's the hist function. We did not tell you about it this week, but we did tell you how to read help pages and learn about functions. So can you learn about this function and make this plot? And then we give them an extension, which is, okay, let's push this a little further. Maybe you could change the color, maybe you could scour that help page and learn about your options and try to apply it.

I see Ben had asked, thanks, Ben, for the question over on YouTube. At what point in the process would the ability to creatively solve problems be expected? Curious where the complexity of practice should be introduced.

We don't discourage creativity for solving problems at any point. As the milestones get more complicated, they sort of become more divergent. So in the second week already, you'll be making plots and each project is guided by a motivating question, which might be as simple as, you know, is there a difference between these two treatments? Or it might be, you know, what factors predict this thing?

And normally within those questions, there's multiple relationships you might search for or discover, especially if it's what factors cause this, who knows? So we encourage the students to be curious, explore as many relationships and data, maybe even make their own new variables or features to explore and visualize with different types of plots.

So I would say as soon as the second week, there's lots of room for experimentation, exploration, reporting new things back. Also that extension piece allows people to really learn a lot of different things. For example, I've had a student who thought, I like the plot I made, but this would look really interesting animated because there's something happening over time. Ours free, it's all free and it's all very accessible online. So he went out and taught himself the GG animate package and brought that back. He was able to do that because we talked about how to learn new packages.

So when a new customer comes and talks to Academy, we try to find the project that fits best with them. And one of them is, modification of diet and renal disease. So this, sometimes we have to be clever about getting around HIPAA and regulations and stuff. So this is data that's actually been published in scientific literature. So it's not going to trouble for using it.

And it is a clinical trial that looked at four different treatment groups on renal disease, so kidney disease, and the treatment groups varied the level of blood pressure medication the patients took and also the amount of protein in their diet. So we had four different groups and ultimately we run an ANOVA to determine whether or not there's a difference between the groups.

These are the 10 milestones in the project. So first they're going to create a histogram of the measure that we're interested in, how much your kidneys can filter, the measure of kidney health. We progressively get them from using Basar. So first we just want to make sure they could use our functions. Then we start teaching packages like ggplot2, that's the graphics package in R. And they group and summarize by patient to reveal things that you wouldn't find in the data as it is, but you can derive from the data. We join in a second data set that they can work with. We tidy and pivot tables.

We start reporting our results in a way that makes it useful to turn in assignments, but also to turn in reports or decisions to people that your team is working for. We work with different types of data such as strings, dates, times, depends on what sort of data you would work with at work. That's what you'll work on on your project, but we do teach you how to work with each type of data in the tutorials.

Here's some of the artifacts students make over the course of 10 weeks on this project. The way this project looks to the students, though, this is sort of an executive summary that I could share with my colleagues. What the students see is what I was showing on Posit Academy.

This is the actual MDRD project. So we get to Milestone 6, MDRD stands for Modification of Diet and Renal Disease. You see we're talking about globular filtration rate. This is the project. In week 6, they get this milestone. Run the code below to make the table. This is not the code you'd write to make the table. It's just code that delivers off the table we made. So now they have to figure out how to take their data set, which they're very familiar with, and fit a model that returns this table here. And then they can check to see if they did it right, if it looks the same. And then they can extend that, changing maybe the variables in the model or whatnot.

I see a lot of questions that are coming through from both Slido and on YouTube. But one was, can you share more details on how Academy was structured for customers in life sciences? Or are there any measures of student engagement and overall success in life science?

So, first of all, Academy is not structured specifically for life sciences. It's structured specifically for people who need to learn R or Python to do what they do. We've become experts in teaching R and Python. And what we found is it's not very different whether you need to use R to do this or to do that. It's not even that different if you're coming from SAS or Excel. If you're coming from SAS, we can use the knowledge you have about SAS as a springboard to connect to R. But at the end of the day, we're not going to give you a translation guide that shows you how to ask where the bathroom is in R. We want to teach you to teach, to speak fluently in R, and that's what we achieved in 10 weeks.

So our structure looks the same for life sciences as for other industries. You need to know how the mental models of the R language, so it's divided into functions and objects, and this is what those mean. You need to know where to get your help from, whether or not you're going to use clinical trials data or preclinical trials data, the R language operates the same way. You need to know the tools that execute your code. You need to know procedures for backing out of something that didn't work and finding the right path forward.

Then there's a lot with data that is also remarkably similar across different fields. So we make sure you know how to get your data into R where you could use it. If you have really big data, we'll teach you how to get R to the data in the database using the dbplyr package. If once you have your data, you need to know how to join tables together, R works best when you put everything in one table before you analyze it, we teach you that, and that's field independent. Tidying your data, R works best if your data is in rectangular format or tidy data format, we teach you that. How you make a plot is the same.

There's so many commonalities across the industries that really, there's just one and a half or so weeks that are different from task to task at this level, and that's what the projects capture. Each project teaches the scope of all those things. It's the scope of what you'd find inside the book R for data science, by the way. But it takes the payoff step, what I call the payoff step, the model that's valuable to you with the type of data you use, and the tweaks that are idiosyncratic to the type of data you use, and it matches those to your industry.

So how do we structure things for life sciences? We made projects that looked at data that life scientists or life science industries use, and that asked questions and answered the questions that life scientists ask an answer of that data. We don't train anyone to be a life scientist, we train them to be a life scientist who uses R, but we assume we're meeting them as life scientists already. Now we just show what, how to use R to do the things that they're interested in.

I think part of the flavor that I can add that Garrett really hit home throughout the entire webinar is that to frame Academy against some of the leading questions that you'll often see is that groups are coming with very differentiated needs, you know, even in very specific areas like just clinical programming, right? For some groups, we have this large set of SAS programmers, and we're trying to convert them to R. For other groups, they're trying to do novel analytics that they've never done before. So they're coming from like a clean slate.

But ultimately, a lot of the patterns across companies across groups are still consistent. And so part of the beauty of Academy is bridging a lot of the pedagogical techniques, but recognizing the fact that the more domain specific they become, the more they're going to immediately resonate.

And so I think, you know, with respect to like the analyzing clinical trial data elements, this is absolutely a call to say that, you know, we have been engaging with companies interested to do that, looking to develop a program that fits what type of work that they are trying to do. But it is a conversation to have, it is not a kind of a routine, hey, everyone is going to go through these same motions, and hopefully they'll come with the same outcome. Instead, it really is about saying, what is your group trying to do?

And so that's why we really look forward to being able to continue to extend those teaching engagements and really are looking for people to broach what are they interested to learn. And the answer in most cases is not going to be like, no, we won't teach that is a conversation about how are you trying to, to empower the group and what.

And that's one of the observations that in kind of my last decade or so of interacting with groups is, even if they crack open a book, or they crack, crack open some courses, when they start to work on their own projects that they can't, you know, copy and paste some code snippet to work off of, it's easy to get frozen as soon as something doesn't work. And you don't really know where to go. And groups that go through Academy, we see things like they're enabled and empowered to be able to troubleshoot to diagnose and to kind of move off of the path that was covered, even within the given context.

And I see Devin, you just answered in the chat too, but Ramil asked, are there elements of building data pipelines and best practices, foundations for clinical trials?

There's kind of like two elements to it, like to me, when I think of the conversations around data pipelines, we think of like the orchestration components of like, I have a variety of things, I have some raw data, it gets processed a bit, you have some derivative data that you're going to then tabulate and visualize and do some other modeling activities potentially against. You know, Academy won't necessarily cover like how to deploy that data pipeline with whatever orchestration system you might use, whether that be like RStudio Connect or Posit Connect or some other pipeline, but inside the pipeline, the analysis activities, the managing and munging of the data, the visualization, the tabulation, those are the core concepts that are covered with that. So you should be set up to be able to think from like, I have raw data, I have insight objectives on the other end, like, what are the pieces that need to stack together to do that? Those are well within Academy's blue house.

I did also mention in the beginning, when I was introducing the presentation that we've recently had a number of great meetups and data science hangouts featuring different pharmaceutical leaders, and a lot of the questions from those events have been targeted to learning and how to teach groups of new users who might be moving from SAS to R. My colleague Isabella recently put together a great blog post that kind of highlights each of those different leaders' perspectives on moving to open source in the pharma space.

I appreciate the messages of support that have been shown up in the chat. I've seen all those, and thank you very much, everyone. Thank you all so much. Have a great rest of the day.