Data Science Career Growth | Pallas Horwitz | Data Science Hangout

Hey there, welcome to the Posit Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12 p.m. U.S. Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.

And with that, I would love to introduce our featured leader today, Pallas Horwitz. She's an analytics consultant and a professional development coach. Pallas, thank you so much for being here.

So excited to be here. Thank you for inviting me.

Yeah, of course. And I learned about Pallas through a mutual friend of ours, Mark, and Mark is not here today. Mark, if you're listening later, we miss you. He's hiking right now, but he messaged me and said to say hello to everyone and he wishes he was here.

Yeah, he's hiking the Grand Canyon, which is amazing. Well, Pallas, if you could introduce yourself for us, that would be great. A little bit about you and something you like to do outside of work for fun.

Okay. Well, I'm Pallas. I've been doing data things for about 13 years now. Spent half my career in the gaming industry and half my career in tech more generally. And I've definitely worked my way up starting entry level and then growing into a lead and then manager and then director. And it's just interesting to see, you know, how things have changed every five years. There's new trends, new tools. And outside of work, I'm a big movie buff. I like really old movies, like 1930s, 1920s. And all that stuff's available on streaming now. YouTube's great for that stuff. So I've watched a lot of old movies.

Reframing "failed" data projects

One of the questions that I had initially for Pallas was, you know, a lot of data science projects actually fail. And a lot of like analytics initiatives fail. It's something that as data scientists, I think we've all dealt with. And so one thing I've struggled with when I'm writing my resume, or I'm talking about my experiences, there's always this question of like, well, what impact did you have? And I'm thinking, oh, well, that project got canceled or that project failed, or that whole initiative got canned. Is there a way you can think of to like some advice you could give for people on how to phrase or structure things or approach things in that situation?

Yeah, I think a lot of times when we think about impact, we think, therefore, the company made this decision, they launched this feature, they made this amount of money. And there's a broader definition of impact, actually, when we try something out from a data science perspective, and it doesn't work, well, we actually just learned something new. We learn this data isn't predictive of this outcome, we learn this technique falls short. And sometimes the absence of something working saves the business time, it means we don't need to invest further there, it means we can ask other questions, we can focus on other hypotheses. And that in and of itself is valuable, you are moving the research of your company forward when you find out something doesn't work, because it means, well, someone else doesn't need to do that. You've saved time, you're moving forward. And so I think, you know, when we think about these research studies, or these analysis techniques, it's not just about what answer did you get? But what are your business stakeholders care about? And how do you move the conversation forward. You know, our role as data people is to help business leaders make informed decisions with confidence. And so if our data can't inform this, the solution, well, that's okay, then they can rely on their business intuition. And that's valuable for the business stakeholders to know. And so it's not a failure, it's still a learning, it's just a different flavor of learning.

And so it's not a failure, it's still a learning, it's just a different flavor of learning.

As you're saying all this, I'm sitting here trying to think of things that I could have done differently, or ways that I could have talked about all of these failures differently. And sometimes they were failures that were completely outside of the data product itself. They were failures on a leadership level or strategic level, failures of communication. But I still learned from those. I think that's a great takeaway.

Project management frameworks for data work

We have an anonymous question: what data project management framework have you found most success using?

So the truth is, you know, these projects are broken into phases, we need to understand the question, we need to decide on our technique. And then we actually need to do the analysis and iterate, etc. And, you know, I've worked on product development teams that use agile, I've worked on teams that use Kanban. And none of these are actually a great fit for data projects, because there is so much ambiguity. And so it's really, it's not that there is a best path here, it's about understanding how to chunk up your work and set expectations. Because if you know how to set expectations with your stakeholders, the actual project management framework itself is less important. But the framework is there to drive decisions, drive timelines, drive prioritization. And for data projects, it's often better to have a conversation, and then square peg round hole it into whatever process your company is using. But I've never seen a process that fits perfectly and seamlessly where you don't need to have that stakeholder conversation.

Investing in your own skill development

One thing I also am curious about is something that I struggle with personally, advocating for taking the time to like sharpen your own axe. But my question is, instead of like, hey, how do I convince my manager that I need time to sharpen my axe? How do I convince myself that it is okay to say, I need several hours a week to skill build when that doesn't have a product at the end of it to hand over to somebody?

It's a little bit goes back to our conversation about failure. And it's only failure if you don't take into account the learning opportunity and the fact that you're moving the conversation forward. If you are hyper laser focused on, I have to figure out the root cause of what drove this metric, or I have to just project out, forecast, whatever thing I'm forecasting. You're not thinking more holistically. And, you know, as AI becomes more and more popular, AI is able to, you know, point to these different techniques and methods. And what actually AI can't do, though, is add that, you know, human je ne sais quoi. And, you know, if we think about how can you be the best creative thinking human that is going to connect insights better than the people in the AI, you know, models around you, investing in your skill set is going to make you better. And sometimes it's not just about getting explicit permission from your manager, because if you want to spend two hours a week, your manager is not going to know what you're doing for two hours a week, you're going to be able to slide that in just fine. But it's about recognizing your impact doesn't come from just spending more time in Python and R, typing at the keyboard, trying to fit this model. Your impact comes from your ability to think holistically. And you're not going to be able to think holistically without taking a step back and figuring out what do I need to invest in for myself. It's invest in yourself for the sake of helping drive investment in your company. And you have to put your career first because no one else is going to do it for you.

And you have to put your career first because no one else is going to do it for you.

Communication failures between data and business teams

Given all the different teams and all the different people that you've worked with, what repeated forms of communication failures have you seen? And how can you now overcome them preemptively?

Well, there's a lot of usage of words like failure in these questions. And I really try to move away from that mindset. You know, things are not so binary in terms of success and failure. But I'd say the constant throughout my career, regardless of whether I'm at a startup, if I'm working on slot machines, hair extensions, you know, helping out Facebook in India, the common thing is, people who are drawn to data careers, think differently, oftentimes, than the business people around them. And that means they also communicate differently. They focus on different things. I've been called pedantic many times in my career. Sometimes I take that as a compliment.

But it's, I fixate on the details, and I'm paid to fixate on the details in ways that no one else around me is. And so the biggest theme in my career is, we're talking past each other. The business stakeholders don't fully understand my skill set, my tool set, and what I can actually accomplish. And until I really took a step back and spent more time talking to my stakeholders, I didn't really understand what they were trying to accomplish. And so it's really about having that conversation. You know, we all would love to just focus on coding, and not have to worry about the human element. But the truth is, those human connections and those human relationships, understanding what matters to the people around you, that's actually what's going to help you drive your impact forward. That's how you're going to build trust.

That's how, you know, when your model doesn't produce the results you were hoping for, when you have that relationship, and you have that trust, and you have that understanding with your stakeholders, that's going to give you grace. And they're like, oh, you worked really hard, and now we know this methodology doesn't work out, and you can try this other thing. But that doesn't work unless you talk to each other. And there's this art to understanding, what are the questions my business stakeholders meant to ask? Because sometimes they don't even know how to word their questions. Because as I said, they don't understand our skill sets and our tool set most of the time. And so how do you learn how to translate what they meant to ask you for?

Going deep on domain knowledge vs. staying technical

This question is, as an R expert, R coding, programming language, I'm often encouraged to learn more about business context. Should coders go deep into domain knowledge or rather focus on building strong bridges with business experts? I feel I'd rather stick to what I do best and just code in R.

So, I mean, that's a hard one. This is a really personal choice. Thinking about what you should be doing is a bit of a trap. Because, you know, I guess in the first three to five years of your career, you have to check a bunch of boxes, and it's about learning how to be effective and do effective analysis, find statistically significant results. And so you should be focusing on your hard skills those first three to five years.

From there, when you play to your strengths, you're naturally going to be more inspired, more curious, and you're going to work harder than the people around you. And so if you focus on what you should pick up, well, all of the other people who are naturally inclined to that are going to do it better than you. And so if you force yourself, well, everyone says I should be doing this in Python, and then you force yourself, well, the problem is there's going to be the next generation is going to be doing that much faster than you, and they're going to work harder than you and get further ahead than you. And so it's not about what you should or shouldn't be doing.

But it's about where do you see yourself in five to 10 years? When the next technological sea change occurs, what are you doing now to make your skill set and your career trajectory robust against the next technological change? You know, when I first started writing data pipelines, it was all in Bash, and I was managing a Hadoop server, and it was miserable. And I should have become an expert in Hadoop and Bash, and that skill set would serve me not at all today. It does help me have empathy for my data engineering partners. But if I had just invested several years in becoming really good at that, it wouldn't have helped me get to where I wanted to go. And so there isn't an easy answer here, but just recognize, if you're only bringing one thing to the table, eventually the technology is going to move past that one thing. And so how are you going to future proof yourself?

Communicating your skills to the market

My question is, given all the ambiguity around data-related titles and role definitions, what are some of the ways that you think a data professional can best communicate their skills to the market so that they can actually land in a role that makes sense for them?

So one thing you can do is, if you just spend a lot of time on LinkedIn and if you can find people who are already in the roles that you want to have, how are they talking about themselves? If you can reach out, even if companies aren't hiring, if you can network with people who run teams who have the type of role you want to have, reach out to that person, especially if they're not hiring, because that means they aren't getting a lot of outreaches, and just see if they'll have a casual coffee chat with you. You can make a guess and blindly have hypotheses about how you should market yourself, but the best way to test your hypothesis is talk to the people that you would sell your experience to and find out what lands the most with them. Let them tell you what they're looking for and what the market wants.

Translating stakeholder requests into real insights

Pallas, when you were talking about helping people translate their ideas into a language that resonates with stakeholders, I was wondering if you could give us a specific example or one that like comes to mind for you right away.

I think a common theme is stakeholder asks for dashboards. I want to be able to slice this dashboard by date, country, blah, blah, blah, blah, blah, like 10 to 15 micromanaging slices the data. And whenever I get those sort of requests, my initial thinking is, yeah, you don't know what you're actually looking for. You just want to ask for everything just assuming if I work for a week or two, putting together every different slice of this dashboard, you'll finally have that magic aha moment. And so you don't actually know what you want. And so whenever I get a very detailed ask that has like 10 to 15 different sub segments in it, that tells me let me go have a conversation with that person.

What are you trying to accomplish with this? What are you going to do with this data? What are you going to do if the data is informative and it validates your hypothesis? What are you going to do if the data negates your hypothesis? What are you going to do if the data is muddy? And then I check to see, do I get different answers? Because if the stakeholder tells me, well, if the data is positive, I'm going to do X. And if the data is negative, I'm going to do X. And if the data is blurry, I'm going to do X. Well, now we no longer have a data driven decision. This stakeholder already knows what they want to do. And so understand why they want to do it that way. And then think about what are the pitfalls?

Well, the problem is they're not thinking about this long term holdout. And this idea they have, they're not taking into account all of the users that aren't going to have access to that feature. So instead of just providing them a dashboard of all the 10 to 15 different cuts, I'm going to also provide them information on how things are performing over and above the holdout so they can actually see what level of incremental value they're driving. They didn't ask for the incremental value functionality. They didn't ask for the holdout visibility. But that's going to actually tell them if they're on track to accomplish their goals. And so you have to understand what they want to accomplish. And then you have to use your own statistical knowledge to figure out, based on what's in my toolkit, what can I easily provide them that will actually answer their questions?

You know, it's pretty common for the first time I pitch ideas for them to be rejected. And I often think of in a business context, when someone tells me no, that means that this idea is not yet a fit for where the business is. And so once I think about, okay, this idea is not yet ripe, what are the things that are holding the idea back? What do they care about right now? How can I turn a not yet into a yes? And that, again, you have to empathize with your audience and understand what they care about to get that no to a yes.

The job market and building your network

So I'll speak from personal experience. I was unemployed for almost a full year. Imagine trying to live on your savings and almost being ready to empty your retirement account after a year because, frankly, it's just hard to pay for things on unemployment. So I very quickly, I have, like, a lot of coding background, but I've also just been like a figure it out as I go person and I will handle the task at hand and whatever's on fire, I put it out and that kind of deal. In the job market, I got several applications that asked specifically, how much experience do you have in health care? How much experience do you have in HR? How much experience do you have in this? And depending on what I was applying for, if I put less than one year, my resume went into the dumpster.

If you are experiencing this, I feel for you. My heart goes out to you. It is not easy. Cry yourself to sleep, rock yourself, whatever you need to to get through it. Call mental health lines if you need phone a friend. But it is really rough. I highly suggest I'm not saying lie, but I think it's worth going into an interview and saying, I know your company does this. I haven't done a lot in that. But here's what I've studied in just prepping for this interview. And I want you to know that I can learn the domain knowledge very quickly. But these are the tools that I have ready to go. That's going to take your company to the next level. All that being said, I got to second round interviews doing that, and it was great. But what got me the job was knowing somebody. So build your network.

Keeping up with a rapidly evolving field

Hi, Pallas. So my question is, given how quickly the data science field is evolving, do you have any best practices or tips on basically keeping up or staying ahead with the data science field?

I'll be very honest. One of the reasons I went into management and leadership was because staying on top of the latest coding trends, it was something I should be doing, but it was not a natural strength of mine. Whereas understanding the needs of technical stakeholders and business stakeholders and connecting those dots, that was something that I knew I was particularly good at and the people around me weren't. And so if you think about the things that you do better naturally than everyone around you, and then you think about, well, how is AI going to magnify that? How can I leverage AI and what's coming next to enhance my strengths? That's not a very specific answer, but that's like a lot of what I do in my coaching practices, helping people understand their strengths and then figure out how do you now future-proof yourself. And make sure that when that next generation, because I'm a millennial, and so Gen Z came up and they were hungry, and next Gen Alpha is going to come up and they're going to be really hungry too. And so what are you going to do that's unique to you that no one else can be as you as well as you can? Like you're the best at being you.

What drew Pallas to coaching

Let's move to Cecilia and she would like to know what drew you to coaching after such a strong technical and leadership career?

I mean, the favorite part of my job was having one-on-ones with my team. And I was laid off two months ago on April 1st. And I had to think about, all right, well, the market's in a weird spot. What do I want to be, how do I want to be spending my time? Well, in my job, my favorite way to spend my time was having one-on-ones with my team. Wait a second, coaching is just all one-on-ones. I'd have to figure out how to meet people and how to connect with people. But there are ways that I don't need to find a company to pay me to have one-on-ones with people. I can just talk to the people directly. And if I'm providing value and they want to meet with me and I want to meet with them, we'll figure out how to make it work. And so, you know, I'm only two months into my coaching career, but I'm really enjoying it so far. And I've had a lot of fascinating conversations. And, you know, I am seeking my next role. Coaching was just something that was going to help keep me busy while I was unemployed. And it's possible my coaching takes off and then full-time employment doesn't make sense. We're having a bit of a race condition here to see which happens first.

Stakeholder management in practice

When you're interacting with business stakeholders, is there any particular fundamental structure you use that you recommend during meetings? And what's usually the most crucial agreement or insights that you hope to take out of those meetings?

So the first thing I do, if I have a particularly difficult stakeholder that I'm not winning over, I'm going to ask them for a casual coffee chat. I want to get to know them interpersonally. I want to know what they do on the weekend. I want to hear about their pets. I want to see photos of their pets or their kids or whatever. I want to build a connection where they're going to give me the benefit of the doubt. Because once they see me as a human, they're going to be more inclined to be patient with me, to listen to what I have to say, to not jump to conclusions. And I'm going to be doing the same for them because now I care about them as a human as well.

The trap is to think of your stakeholders is just like barriers or obstacles or another subsystem or just like a technical component and their people. And it's rare for, especially for technical people, it's rare for a technical person to reach out to a business stakeholder and say, hey, I just like to get to know you better and about your career and how you got to your position. And when you actually show interest in people, the dynamics of the conversation changes and then you're having a dialogue as opposed to people just shutting you down.

The evolving career landscape

How do you expect the career landscape to change in the next few years? And also like, how have you seen the career landscape change because you've been in data for a really long time?

I mean, when I started my career, I did most of my work in Excel when I started my career. And then we moved to automation and I learned SQL and I learned Python and I'd used R a bit in college. And then I learned some machine learning techniques and then it became Bayesian. And then it turned in more into deep learning. I was like, oh, that's wizardry. I'm not ready to learn. I'm gonna hire someone else to think about that stuff. But I've seen the sophistication grow and grow and grow. And now with AI coming, there's almost like this critical point where just being a tech, it used to be the more technical and the more statistical wizardry you knew, the easier it was to get a job. And now I think there's this pendulum swinging the other way, which is if you're just a technician, it gets harder unless you're one of those few that have the PhD that are working on the LLMs, helping OpenAI and Claude and ChatGPT and all of that stuff. If you're not in that niche domain, just being a technician probably isn't enough.

And so how are you expanding your impact in terms of maybe as a data scientist, you wanna expand your impact, but you want it to be technical still. And so you wanna demonstrate, well, I can do all of my data engineering as well as all of my data science, as well as all my analytics engineering and dashboarding, et cetera. And so you wanna demonstrate that you have extra value because you don't have just the typical data science skillset. Maybe you're a technician, but you also wanna learn how to do some software development as well, or you wanna show that you can build apps with AI. How can you be more than just the person who writes models in R and Python? How can you be something plus that?

GitHub profiles and job candidacy

So I have a friend that works in the industry and I know it's a sample of one, but when he's evaluating candidates, he looks at their GitHub profile. So how much does your GitHub profile weigh into job candidacy? And do you find it to be more important than other tangibles?

I think it really depends on the role and the type of company you're interviewing with. There are companies where absolutely that's a non-negotiable. And you need to have that, and you need to have recent commits in a language that they care about working on a problem that's similar to the problems they work on. And so this goes back to a question from earlier, which is, talk to the people that are in the roles you want to have, talk to the hiring managers, especially the ones that aren't currently hiring because they're not swamped with LinkedIn requests. But talk to the people who are managers at companies you want to work at, and they can tell you how important that is.

Over my career, I've only had two roles where I actively checked the GitHub commits, and it was because it was a really niche problem I needed them to solve. And I needed to know that they had very, very relevant domain experience, but I haven't checked GitHub profiles for most of the roles I hire for. But that's not true everywhere. It really just depends on the actual industry itself and the type of company you want to work at.

Showcasing confidential work

So I'm relatively new to data science as sort of a professional pathway, so I'm still learning conventions and norms, and I often kind of struggle to figure out how to showcase skills and data-related achievements when the projects that I want to highlight are confidential. So just thoughts about strategies or suggestions for demonstrating expertise without violating confidentiality.

Yeah, that's hard. You know, you can talk to the company where you're doing these projects, and say, you know, hey, I'd like to showcase my work. What if I scrub all the labels, scrub all of the identifying context? Is there a genericized version of this you'd be comfortable with me showcasing? They're gonna say yes or no, and then they're gonna nitpick with you over how much you need to mask the details of the analysis. And if they do say no, well, I bet that there's a hobby or a small business near you where you could apply a similar technique to solve a similar problem. And you can go in from the get-go saying, hey, I wanna showcase my work. Or do an analysis on movies or sports or something like that. You can always do a pet project that repurposes the technique you used, but doesn't actually rely on their data. But usually the easiest route is figuring out what's the obfuscated version of what you did that the company would be comfortable with you going to the market with.

Mentoring when you're early in your career

So my question was, how can I continue to be a leader and mentor in a space when I'm now sort of at the bottom? Like I used to mentor freshmen when I was in college. I used to coach synchronized swimming. So I had a lot of these leadership mentor opportunities, but now I'm in my first full-time job and it's in data, which is not my actual degree. But how can I continue to use those mentoring skills even if I'm sort of like towards the bottom of the chain per se?

You actually have a really important accomplishment in this market, which means you have your first data job and there are a whole bunch of people out there looking right now and they have not gotten their first data job. And I bet they'd love advice about how you actually accomplished getting your first data job. How did you transition into this career? How did you get to a place of paycheck stability? That's really, really valuable. And so don't discount just what you've accomplished in the past six months to a year. There are a lot of people who are trying to accomplish those things and haven't figured it out yet.

Explainability and decision-making in the AI era

With the increase in vibe coding and LLMs writing code, do you think that explainability and interpretability in data science have risen in importance? So as data scientists, like should data scientists be better decision makers than they are coders?

I'd like to say the answer is an easy yes, but I'm hesitating and I'll explain why. I have been in situations where it's really hard to make the business decision and no one knows the right answer. And so they turn to the person who can produce the most sophisticated math. And so while I do think interpretability, explainability is really important, there is always going to be that one category of problems and let's call it 20% where no one knows the right answer. So they're gonna defer to the person who can make the most complicated math and be a little intellectually intimidating. I don't think that's the way you can actually build your career, but I don't think that category of problem is going to go away. So I do think in general, explainability, interpretability is very important, but that one category I think is always going to exist because when humans don't know, we don't like the unknown. And so that thing we understand the least that seems really confident, let's just go with that.

Closing advice

Like what's the number one most memorable piece of career advice that you can leave us with after today, Pallas? Or just something that comes to mind that was really impactful for you personally.

The one that was told to me that was, so I was in the gaming industry at the time, making video games. I'm sure there are people on this call who work in healthcare, and this advice does not apply to you guys, but it applies to everyone who's not in healthcare, which is I adjusted an analysis. The results were awful. The product was going to die. We were going to have to pull the plug. I felt so responsible that I produced the analysis that was going to kill the product. And a friend of mine told me, it's just an effing video game. No lives are at stake. And I think sometimes, we devote so much of our waking lives to our analysis and understanding the data that we forget to have perspective. And again, this is not for the healthcare people. This is for everyone else. Lives are not at stake.

And a friend of mine told me, it's just an effing video game. No lives are at stake.

As I was asking you the question, what's one that made a really big impact on you? Mine is sort of the same. I had a manager say, Libby, there are no analytical emergencies. Like you can chill out. It's okay. Cause I'm very hard on myself. That's the feedback I get from managers. Like Libby, you're so hard on yourself. Thank you so much, Pallas for hanging out with us. This has been wonderful.

Thank you. No, this has been great. And it's a shame that I wasn't able to like read the chat as well and answer more questions. Cause this is how I love spending my day. This is why I've opened a coaching practice because there's nothing I enjoy more than just like thinking about things, talking to people and then helping them learn how to think and talk as well. So thank you so much for including me and thank you all for all of your wonderful questions.

I mean, I'm just getting started right now. I have some evening availability for folks, but yeah, I have a link to my calendar on there. By all means, sign up, you know, we'll talk for 20, 30 minutes and talk about what are you looking for? Is this a great fit? What sort of structure do you need? I do do a sliding pay scale for folks depending on where they're at their career. And sometimes it's a fit and sometimes it's not, but you know, I think I'm a friendly person. I'm not going to bite, you know, come chat with me and we'll see what makes sense.

Everybody, if you would like to save the chat, there are three dots at the top of the chat box. You can click those and then click save chat and you can take all the resources with you. Thank you for hanging out with us on this beautiful Thursday. And next week, I hope that you come hang out with us because we are going to have Karen Healy and Jonathan McPherson, two of our keynote speakers for Posit Conf that are coming up. And if you know either of those guys, you know that it's going to be a really, really interesting discussion. So come hang out with us. Can't wait. Have a wonderful week. Thanks for being here.