People Analytics at different stages of company growth | Adrian Perez | Data Science Hangout

Hey there, welcome to the Posit Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12pm US Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.

I would love to introduce our featured leader today. We have Adrian Perez, Head of People Analytics at GitLab. Adrian, would you like to introduce yourself? Tell us a little bit about what you do and something you like to do for fun.

Sure. So, hi everybody. I'm pleased to meet everybody. And I've been to probably a couple of these hangouts. So I think they're a really cool way to create community. And I always just kind of love the conversation. So I'm really excited to be here today. Like she said, my name is Adrian Perez. I currently work in People Analytics. And I've been doing that for about 10 years.

My background is in statistics. I did an undergraduate degree in math, did a master's degree in statistics. And initial goal was I really wanted to be a teacher. I really wanted to work in the college setting and be a professor. And I started going to find whatever business problems I could solve using data. I started working as a consultant for a while. I've worked in market research. I've done marketing analytics. And I worked for a couple of financial institutions before I started working at HEB. And for those of you who don't know HEB, it's a local regional grocer here in San Antonio or in Texas.

And I started the People Analytics function here or at HEB. I was hired as just an analyst. And I quickly realized a lot of cool things you can do with People Analytics. What People Analytics is is really data analytics, business analytics in the HR space. It's everything from, you know, building data models and structures to also building out dashboards and doing advanced analytics, helping with experimentation. And one of the cool things I get to work on, I've learned a lot in this job, is also like survey design and then survey analysis of the items that you have and how do you leverage statistical techniques to make sense of that and create these latent constructs.

It can get very nerdy and it's also very, very useful for the business. So I built out a team there and then I made a shift over to the tech world. Now I'm at GitLab. And I was charged here with kind of the same thing. We're a growing company. We didn't have any People Analytics function, so I got hired on. And I work with one other person on my team, but we also are a hub and spoke model here. We have a central enterprise data team that manages all of our tools, all of our data warehousing, access controls, roles within Snowflake. We use Snowflake. And then I lead what's called a functional team. So we have like product analytics, we have marketing analytics, we have sales, we have engineering, we have people.

Team size and the cost center challenge

Awesome. You mentioned direct report. Do you have an enormous team? Are you guys small and mighty? What does that look like?

So, you know, I think one of the struggles in a lot of analytics roles, in my experience, they're directly related to in some way, like the front end revenue side of the business, right? When you're in People Analytics, you are not making any money, right? You're not providing any money for them, maybe insights and helping them do the business. But you're on the administrative side, you're strictly a cost center, fighting for headcount in any anybody else who might work in one of those kind of expense departments or divisions, probably understand the struggle of getting headcount in those positions is very difficult.

So I was actually the expectation was we were growing really fast. We wanted to build out this team really big. I hired a fantastic direct report. And if any of you find him on LinkedIn, his name is Shane McCormick. He is phenomenal with building data models and data structures and translating user requirements into dashboards that are very useful and helpful. And we just kind of hit the ground running, took off and the headcount slowed and stopped. So it's just been the two of us as the company's growing for the past, you know, four years or so.

Minimum company size for people analytics

Is there a minimum company size before people analytics becomes worthwhile? If so, how large would you estimate a company needs to be before people analytics becomes really worthwhile?

That's a fantastic question. What do you think? Yeah, no, I think that's that's when I struggle with, too. I think it just depends on your comfort with data and the drive for it. Right. I would say to have like a proper people analytics function with at least a couple of headcount, you need to probably have a few hundred people. There's lots of different things that you can look at. But part of people analytics is the talent acquisition side. And whether you have a company of ten or you have a company of ten thousand, like you're getting a lot of applications and there's a lot of things to look through there.

I read an article when I first started and there was somebody who was at a small company. I think they had like 50 people. She was the only HR person and she was doing HR analytics, people analytics at her company and spreadsheets. Like you don't have to do anything fancy with Tableau or Power BI or using shiny and building dashboards. It could be as simple as just a few reports, maybe like an hour that you're building out in an automated fashion, getting your reports exported once a week and then having a process in Excel where you're leveraging the data and putting insights together for leaders.

And then you can focus on like the experimental side. Right. That's something that I think is really impactful on the on the talent acquisition side of, hey, we're trying to recruit this certain way. What if we do an intervention and try to recruit this separate way for a control, for a group, for a randomized controlled experiment? I'd say there's probably no real like size that says, oh, you can do it if you have data like analyze it. Right. I would say probably having a true functional like group, maybe like two to three hundred people is probably where you're fine to go ahead and have like at least an individual who is responsible for this.

Types of analysis in people analytics

Can you share some specific types of analysis or modeling tasks that are done in people analytics? And what are the top three to five questions that business generally wants answered from people analytics?

Yeah, so a general question that I get a lot is we want to know if this thing worked, whatever that thing is, whatever we did. And unfortunately for that question and any and anybody else has been asked that question when you're asked after the fact, it's very, very hard to tell them what worked and didn't. Like getting that into the business's mind of like I have to be brought in sooner in the process to be able to help you design and organize and structure the data is critical.

Another one that's a top question is like they want to understand the inflow and outflow of their headcount. How many people have we hired? How many people have we lost? How many people have we promoted? So it's kind of like a cyclical thing you're looking at. And retention rates is a big part of that. So with that comes things like just making sure you're defining it properly and documenting it somewhere so folks aren't going out there in their rogue ways of creating their own.

And so those are the other one is a lot of reporting asks. Everybody wants reporting and that's just the nature of I think business analytics is everybody wants to be able to make up their own mind and not be told what they should focus on. They want to look at a report, it should shine out to them, this is what I need to look at and it's what I need to investigate next. And the truth of the matter is like that only goes so far. Like reporting can only give you so much even if they're savvy enough to ask questions of the data.

Survival analysis case study at HEB

Something I often mention, it was a really fun one to do and I didn't actually get to do it but I was leading a statistician on my team who went through it. We at HEB had a method of recruiting and when you would apply to a store, you literally apply to that store, a store admin would look at your application, bring in people, interview them, say it sounds great, can you come in tomorrow, meet with my store leader at this time. You can imagine it's kind of a long process, right, and you're getting somebody who's maybe a high school student who's going to be checking at the cashier or bagging groceries for, you know, maybe like five, six hours a week.

So they would go through a lot of this and there's a lot of time in that and the recruiting team had always wanted to take that and centralize it. Let us handle all those applications. We can have them apply to like several stores at one time and we can move them where we need to and it would streamline the process and there was a lot of resistance. But when the pandemic happened, they were like do it.

And what they wanted to do was understand the impact. It was about six months later, everybody was kind of saying, hey, maybe we should go back to the way things were. We didn't really have a measure for quality of hire, like is this person performing well or not, but we did want to look at retention. How long are we keeping them because those are very high turnover roles. So we ran a survival analysis and we kind of pinned the same time the previous year when we were running the old process to when we flipped.

And what we saw was, you know, looking at these two groups, this new method actually provided just as good if not better quality candidates. There was a more streamlined process allowed for us to hire folks quicker as well and it gave a whole ton more time back to the stores. And this was used by our director recruiting for all the store leaders to look at in the presentation she gave out. And they immediately said, okay, great. We surrender, like you go ahead and do this from now on.

And what we saw was, you know, looking at these two groups, this new method actually provided just as good if not better quality candidates. There was a more streamlined process allowed for us to hire folks quicker as well and it gave a whole ton more time back to the stores.

It wasn't the most complicated analysis, but the person I was leading had never done a survival analysis before. She had to kind of go learn it. She ran some regressions in there, was able to provide a compelling argument. And then we changed the course of the business really, which I think for the better, to be honest.

And for anybody wondering about scale at H-E-B, how many stores were at H-E-B when you were there? Over 300. And now I think they're probably approaching close to like 350 or something like that. So they have a lot of stores. We've had a hundred and when I was there, 120,000 employees. I think they're probably close to like 130, 140. And there was four of us, the people.

Managing HR professionals and data they don't like

How do you manage HR professionals when they don't like the results that the data showed them? And relatedly, do you see HR in general becoming more or less interested in data analytics?

So to answer the second question at first, I'd say that they're getting more interested. Folks, I think are realizing the need and people analytics. When I first started, it was as obscure. Oh, it's kind of neat and nice to have shirts as it can do all these things, but can it really? There's a lot of skepticism. I think folks now are just kind of accepting it. There's a lot more companies with people analytics functions out there. So when I have new like VPs or directors that join in the HR space here, I'm like one of the first people they reach out to within like that first week or two for a chat to find out like, hey, where are we in the state of people analytics, which is really cool.

And as far as if you tell them something they don't want to hear, it's the same as any other business leader. I mean, if you, I used to work in marketing and we were doing consulting. And the way this would work is somebody who was in charge of a specific type of marketing campaign or a marketing method was in charge of working with the consultants. And when we'd run the model, there'd be a marketing campaign that we would say, oh yeah, that doesn't do anything. It adds nothing. And inevitably it would always be, well, I'm responsible for that. You can't take that out of the model.

I try to stay away from, no, the path you're going is wrong. Do this instead, because really I'm a consultant in this. They're going to come to me if they need me. And that's my job is to help them learn more, not to tell them you're wrong. This worked. It didn't work. But by and large, it's a delicate thing that you have to just kind of make sure you realize they see you as a partner, not as an obstacle.

But by and large, it's a delicate thing that you have to just kind of make sure you realize they see you as a partner, not as an obstacle.

Applicant tracking and recruiting operations

Does that include implementing third-party applicant tracking systems, keyword analysis, and how much of that work is done on the front end before they actually enter the company as employees?

Yeah. So when it, I don't do anything like the actual, like I'm not building any kind of AI tools or leveraging other tools that are helping sourcers or recruiters skim through resumes and then find and match skills or anything of that sort. What I'm helping with mostly on the HR, on the recruiting side is operations, right? How many people did you have apply to this role? How many of them went through the next stage? How long did it take them to go from that stage? We're trying to understand scheduling. How long did it take to schedule after we had the offer verbal? How long did it take you to get a written offer to them? Especially in something like tech, you work in a very competitive space. Somebody, if they're interviewing with you, they're typically interviewing with several other people.

On the other side, we'll also help them measure their campaigns, right? So they go and they will target certain groups like universities. And we help them try to understand what's the effectiveness of those things or things like referrals. So I'm looking at a lot of metadata to help them understand where can you narrow your search? Because it's just like sales. In sales, you're going to say, well, who are the highest valued prospects that you want to go after? I can't tell you what to look for, how to win those people. That's kind of your job.

Advice for first-time managers

You mentioned you hired and manage a direct report. And you're also in People Analytics. So I think you'd be the ideal person to ask for management advice. I'm about to be a first-time manager of a small team in data science. I'm really freaking out. So any advice you're willing to share, both from your own experience hiring and managing, but also from your subject matter expertise in People Analytics.

Yeah. No, well, I guess there's, like, first, congratulations, because that's a big step, especially if you've been an individual contributor. So there's, like, I guess two things that I think about when I think about leading somebody. There's the business that you have to manage, right? And everybody's got their work they're doing. They've got it organized and structured. But the other piece of it for me has always, because you're right, like, looking at the data, if you want to retain strong employees, you have to have, they have to have a good, strong relationship with their manager. Like, that phrase, you quit your manager, not your job, is not a phrase because it's not true, right? Like, that is real.

The thing I've always followed, and I don't know if it's the best or not, is I've reached out to the old leaders that I had that I know were great. And I've asked them, like their advice, like, I'm kind of struggling with this thing, what would you do? Like, I think you already been a little nervous about it is like a good sign of humility. And I've always appreciated managers who are humble, and can say, I don't get that I don't know that. We're partners in this, right? So I treat everybody that works for me. It's like a partner.

I mean, I hire people who are who are always going to be doing things I can't do. And that's just like, why I'm hiring them, right? Why would anybody hire anybody, I'm not going to hire people to do the things I already know how to do. I'm paying somebody to do something that I can't do, or maybe if I could, I just don't have the time. So I'm there to guide them, not even necessarily in the technical aspect all the time, to guide them with how do they be as effective as they can in what they're doing.

The other thing I lean on to, and I've used this example before. But I just love sports. And I love team sports. And my whole life growing up, I would play team sports. And I didn't care who my team was like, of course, you want to win and all that. But I want to play with fun people, not just great players who are jerks to everybody. Right. And it's the same thing here. And I want what when you do that, you get kind of a collective group all the time, and you leverage whatever strength they have in that moment on the court or on the field or whatever.

I translate that to my work life was like, I'm not going to force somebody to do something they hate, and don't like and it's just because I need it. If I'm looking for something I need to be pushed, as far as like our strategic, like pushes for strategically, I will probably be hiring towards that. And I will pose it to like the group of who I lead, like who wants to take this on, like who finds interest in that. And as I get to know them, I learn what they lean towards what they're good at, and what they're probably not good at.

I have somebody in my last job, this guy was phenomenal building assessments and surveys and doing really deep statistical analysis, hated dashboards, hated dashboards, hated data management. And he probably built like one dashboard for me out of necessity the whole time he was there. But everything else he did, man, he did it phenomenally, like no one else could do it. So I wasn't going to lean on him for anything that I just needed to get done unless I absolutely had to.

I leaned on him for the things he did really well, and he enjoyed, and I tried to make sure to elevate his work. So it's not like I mean, it's a journey. And I think as long as you realize that when you're going into it as a journey, and it's a hard one, and just be honest with yourself too, because some people take it on, and they're leading, and they're like, I really don't like this as much as I thought. And you know, I've seen a lot of folks take steps back into individual contributor roles, because that's just what's right for them. For me, when I went into it, I didn't think I would like it, but I needed to grow my career in this function. So I was like, sure. And I love it.

Tech stack at GitLab

So what tools do you use at GitLab?

So, you know, I am not as good at programming as he is in SQL. He's not as good at programming as I am in R. He never used R before he joined. And he never used R before he joined. He's learned it out of necessity. We use R. Snowflake is where we house all of our data. And so Snowflake has the web UI where you can write SQL. We do a lot of it there. I don't, I've tried VS code a couple of times, just it's too many tools. I'm like an R and I'll just write SQL when I need to. And he, we don't use Python, but at the company, a lot of folks use Python for the developer side and everybody uses Python.

And then we have our data science team that uses Python. They use Jupyter notebooks. And then Tableau, like Tableau became our kind of BI tool for the organization. And then we also leverage, I'm not sure how many of you out there familiar with the dbt to kind of keep our data modeling model building set up and organized. And then we use GitLab to kind of manage all like the structure, the version control, everything we need to with that.

And then also you mentioned that you don't like VS code. I hate VS code too. Have you tried Positron? Well, I was actually starting to toy around with Positron and then errors and things came up. So I don't get as much time as I used to, to tinker with tools, but that's next on my list. Cause I didn't have a problem with VS code. It just, it's like, I don't, I don't need it. Like I just don't write enough people and I'm not like pushing commits or I'm not doing any commits and then pushing changes to the, the, the base code.

I would encourage anybody who has not played around with it. Go play around with it. I, when I first started using VS code, I was like, I just can't, I just, I just couldn't get into it. And Positron was a totally different story because it's built for the interactive type of like data analysis that I like to do. It's got a variable pane. It's got a plot pane. It's got a console there for me and my source. Like it's amazing.

Sports analytics and measuring process over outcome

How do you think that this would work in the sports industry, especially for a soccer team?

Yeah. Well, if, if, if whoever wrote this question wants to speak up, if you do HR, or if it's just like analyzing sports, I mean, there's a lot of carryover, like between like what I do. I mean, if you're doing sports analytics, you're often trying to understand the value of a player. Right. And that's typically what a lot of times we're trying to do when you're looking at like talent assessment things, like what's the value and impact of an individual.

I took an executive education in people analytics at Wharton four or five years ago now, I can't remember. And one of the professors that ran that, his name is Cade Massey. It's C-A-D-E. And then Massey is M-A-S-S-E-Y. And he's like all about sports analytics. So he actually had someone who ran the people or like the analytics function of what's the team in Philadelphia, the Phillies, baseball team. He came in and he was talking to us about how they analyze all these different kind of sets of the data and what they look at and prioritize when they're recruiting a player.

And one of the coolest parts that I learned about him when he talked about this is you really got to measure the process, not the outcome. So there's challenges with every sport when it comes to that, right? In hockey, if you want to see how well they shoot, then you may just look, okay, how many shots did they take? And you want to measure, there's not a lot, right? There's not a lot of data. So you have to measure like maybe the quality of the shot they took. How many shots on target? Where were they on the field, on the ice when they did it?

And one of the coolest parts that I learned about him when he talked about this is you really got to measure the process, not the outcome.

And it's like it may not work for everybody every time, but if you got to play the long game and over time it's going to work for you, right? And that's how these sports teams do it with their athletes they're looking at. They're trying to understand how they impact the game based on their behaviors, not just outcomes. So I love it. Like sports analytics is kind of like my next like if I could just switch into that world like that'd be amazing.

Privacy and data governance

How do you manage to respect privacy of the data you work with?

Yeah. I mean, so we have a data governance group at GitLab, but that director was just hired like a year ago. So we've had to manage this on our own. Since I've been here up until recently, but even so like every group's privacy, it's very different, right? And then there's on the people side, things that people might not even think are like sensitive pieces of information, but are absolutely sensitive. You know, like even location, like the city somebody lives in, right? Like we're all over the world and we don't necessarily need, like, there's no need to know that information.

So one thing that we wanted to make sure we did was we were the first group to ask for them to do dynamic masking. So when you get to Snowflake, you have a role assigned to you, right? And ours is like people analyst role or something like that. And it knows because of that, I can access these tables, I can't access these. And it also knows because of that, I can see these fields. And if I don't have that role, it's going to come out and know for me.

So we had to work to figure out, okay, how can we make this data as useful for us to be able to provide the things? If we do it in aggregate, it's not a problem. But if we show the individual levels of like birth date or I don't know, even sometimes email can be considered sensitive if it's just exposed. So various fields will have in there that we have to hide. We can see them, others can't. And if we aggregate those things into fields that are going to come in reports, it's no problem. But every time we're adding a new field, we go through that process.

And especially if it's a new data source, we have to include our privacy legal advisor to come in and look it over. The data team works with us to ensure that we're following their protocols. And then we have double checks to make sure that we're following our own special like protocols to mask what we need to. So the highest, utmost priority is not exposing this. The other thing is limiting who can see it, like number wise.

And transparency is the other piece, right? Like I'm not sharing with everybody, but I will share with you, this is what I do have access to. So folks know and they're aware. So when they're providing something like their gender identification, it's not like in some box where they're like, I guess I have to. We're very clear with you do not have to. It's a voluntary thing. And if we're going to use it, we're using it for X, Y, Z. So transparency is super critical to gain that trust. But I take that very seriously.

Rapid fire questions

So there's no way we're going to get to all of them, but if you would be up for a rapid fire round, I would love that.

Madison's question was, have you done any work on skills analysis and or leveraging graph databases in your people analytics and data science work?

So maybe if you can clarify Madison, but I haven't done too much with that. I think having access to the data is the critical thing there, like understanding who's in what meetings, who's chatting with who and all that. And again, back to the sensitivity piece, like a lot of folks just don't feel comfortable with like anyone knowing and having that. But we've explored the idea of like a survey where you go out and you, in the survey, you're asking like, name the people you work with most frequently in a week, right? Like the five people, name people you reach out to if you have a problem, whatever. In mass, you can get that and analyze and understand like, oh, so so-and-so's reached out to a lot. This person's reached out to by five different departments on a weekly basis for whatever, right?

And then the other question with skills, I've done a little bit of work with that. There's this thing called, it's like a Reasec model and it's like an interest-based model that you can do. And the government has created this and they built it. And it's a way to kind of match your interests with skills or with jobs. And then you learn the skills that are matched there. And so what we did was we took that government piece and we kind of replicated that in an internal system, in an internal form at HEB, because a lot of the people that worked at HEB, again, are hourly employees who are like students still and trying to figure out what they want to do with their life.

Yeah. Especially with AI and the rise of AI and chat GPT and CLAWD and like being more prolific, just kind of more prominent in like work. We're leveraging that a ton to understand like concepts that would just take way too long to analyze. Otherwise, like we could do topic analysis and things of that sort before. But now I can literally tell it like, give me the top 10 topics you're seeing about X or what are the top most common topics you're seeing and tell me how many times you're seeing that and then give me excerpts. It can do it all super fast. So that's made it a lot more, I guess, widely available to look at these qualitative things, both from exit interviews and like current engagement surveys.

Is people analytics considered a subset of human factors or is it the other way around?

I don't know. I can't say I know enough about human factors to say which one's a subset or the other. My wife is an industrial engineer. And so maybe what she's taught me about like human factors, I guess they're related for sure. I don't know if I'd say one is a subset of the other, but I think they just have different ends in purpose, right?

Is it possible to break into people analytics with data experience, but no HR experience?

Yeah, absolutely. I think so. I mean, I, I grew, I'd never worked in HR before in my life. My wife had, I just knew HR as the once a year, a talent assessment thing we do in the engagement survey. And then I got the role because of my analytics background. And then I just kind of dove into it and like, you know, feet ahead first. So absolutely. I mean, you find a role that needs somebody that does SQL, you find a role that's posted to somebody needs SQL and dashboarding and you know it like go for it.

Career advice

Well, Adrian, you have shared so much wisdom with us today. Do you think in the minute that we have left, you could leave us with a piece of career advice that has impacted you or that you like to give others?

Yeah, well, that's interesting. I was talking to my wife about the other day and we were talking about how we finished college and we just had kind of started doing life, right. And you just tackle the next thing that's in front of you. Like we, we got jobs and we got apartments and then we needed more space. And I think it's it, what we've learned is like, we took one thing we learned earlier in life and we're still applying the things that work and we've shed the things that don't work as we're going, like you kind of refine. And that's the way I've also approached my career. I've done all kinds of jobs. I've had terrible bosses. I've had fantastic bosses.

And what I've learned is I just, even if it stinks and it's terrible, I've learned something from it. I had one job where I was there for like a couple of months and it was really not a good fit. And I felt awful leaving. And when I left, I realized how much I learned in those few months, even though it was awful, both about what I wanted for my future, what I didn't, what I could apply analytically, what I still need to grow all kinds of stuff because it was just like drinking through a fire hose and it was miserable. But at the end of it, I like, I grew and I learned from it. And I think just taking every situation for what it is has been super helpful for me in my career. Cause whether I realize or not, like something I'm doing now will help me in the future, whether it's going to help me realize what I want to avoid or what I want to go to.

Oh, awesome. I agree. Learn what you don't want. It's super valuable. Well, Adrian, this was fantastic. Thank you for hanging out with us, sharing all of your wisdom. Everybody was so wonderful to see you. Don't forget to take the survey and we will see you next week.

Bye everybody. Thank you so much. I really appreciate the questions and the attention. It was awesome being able to see everybody and I'll hopefully be here next week or next hangout as well.