Jason Foster @ Marathon Asset Management | Data Science Hangout

Hey everybody, welcome to the Data Science Hangout. I'm Rachel and I lead Customer Marketing at Posit. Posit is the open source data science company building tools for the individual team and enterprise. I just started adding that because I noticed some people actually found out about the Hangout without knowing about Posit yet. But thank you so much for hanging out with us today. The Hangout is our open space to hear what's going on in the world of data across different industries and to connect with others facing similar things as you. We get together here every Thursday at the same time, same place. So if you're watching this as a recording in the future and you want to join us live, there'll be details to add it to your calendar below.

I have a really fun announcement today. You may have seen on my LinkedIn already, but I'm thrilled to make it official and have Libby here and joining me as my partner at the Hangouts. And so Libby and I first met at a Data Science Hangout, I think back in July 2021. So like one of the very first ones and I can't believe it has been that long. And I know many of you already know Libby very well from the Hangouts and beyond. But Libby, do you want to say hi and introduce yourself too? I would love to. I'm Libby. And yes, Rachel hasn't been able to get rid of me for three years. So she just said like, well, just invite her in. I am a data science freelancer right now. I really, really enjoy teaching and mentoring. And that is what I get to do. And I'm also starting as a Posit Academy mentor. My first cohorts will be next month. So I really, really love this community and really hope that we can do cool new fun stuff. Happy to be here. Me personally, I'm a big maker and a big reader. I made this shirt. I actually dyed this fabric myself and sewed this shirt. So you'll see me wearing lots of me made clothing. But yeah. So excited to have you here, Libby.

I know other people love connecting with each other in the chat. So I always like to add this reminder here. If you do want to connect with others, I encourage you to say hi in the chat and introduce yourself, maybe your role or where you're based, something you do for fun, or if you want to share your LinkedIn so you can find each other after. But to continue just my very quick intro here, we're all dedicated to keeping this the friendly and welcoming space that you all have made it over the years. And we love to hear from you no matter your years of experience, titles, industry or languages that you work in. It's 100% okay if you just want to listen in today. Although we love getting to hear from you live. So you could raise your hand on Zoom to ask questions and I'll call on you to jump in. You can put questions in the Zoom chat and just put a little asterisk next to it if it's something you want me to read instead. Otherwise I'll call on you to jump in. And then third, we do have a Slido link where you can ask questions anonymously too. So with all that, thank you for spending time with us today. I'm so excited to be joined by my co-host Jason Foster, Director at Marathon Asset Management. And Jason, do you want to kick us off with introducing yourself and sharing a little bit about your role?

Jason's background and journey into finance

Yeah, no, thanks for having me today. I've been an avid listener for most of the data science Hangouts this year. I think really what piqued my interest was the Wes McKinney interview, I think back in March of this year, given his Python background. And that's primarily what I use professionally. So definitely motivated me to join these. I've learned a lot from everyone you've hosted so far. So thanks for putting this on. And hopefully other people in finance want to join going forward and want to share their journeys and how they're approaching quantitative finance, data science, Python, et cetera, and R too. Always interested in what other people are doing as well.

A quick, quick intro about myself. I actually did not live, I was not born in the US, I was born overseas, lived overseas for until I was about 10 years old, and then came to the US there. And what's interesting after that is I actually went to high school in Alaska, that's where I graduated from. And I wanted to be close to home. And I liked math. And I wanted to do something in the business world. And the closest big city to Alaska is Seattle. And fortunately, I didn't go to PositConf, but I wish I did. But that's where I did my undergrad in math and finance there. And actually, the moment that I hit the ground in Seattle, my family moved to the Middle East, back overseas. So it turned from a short flight to a pretty long flight again to go visit them, visit home.

So I did math and finance in undergrad. And then I was fortunate enough to have a few different internships. And one of them was at an asset manager in Seattle on the quantitative research team. And this was back in 2009. And I was lucky enough to sit next to a local university professor in statistics. He was more like a consultant and really gave our team the theoretical academic boost that we needed. And I was able to learn a lot from him, including R and R package development. We had an internal R package library that we could use. So I was learning a lot about finance and R and programming stats from the team. And I really was motivated to learn more. And after that, I did so. And I went to grad school. And I got a master's in financial engineering. And I think one interesting thing is this was a while ago. And it's so similar to what data science would be now maybe in finance and math if you just applied it in that space to finance. But back then, it was called financial engineering, financial math, computational finance. Any one of those really is all the same.

So I did grad school in New York. And I was fortunate enough to get an offer from the same firm in New York to join the risk and quantitative analysis team there, too, as a risk manager primarily. Covering different types of portfolios and investments and working with different types of portfolio managers, multi-asset, fixed income, et cetera. And then in 2016, 2017, I presented a couple of times at R and Finance on an R package that I've developed. And it's available on CRAN. It's called Roll. And it's just doing basic rolling statistics, but super fast.

And there's a lot of ways that I've made it fast over the years. And I can talk about that, too, and some of the people that I've interacted with over the years and some of the feedback I've gotten and the lessons learned. It's interesting to have a package available on CRAN for everyone to use. And then our firm made a pretty strategic shift to Python. And so we switched professionally a lot of my work over to Python, and so did everyone else at the firm. And one of the more interesting projects that I led towards the end of my previous employer was on a term I've heard a few times here, too, is on citizen developers. And that's when you have a bunch of people that aren't technically employed to do professional software engineering, creating packages or libraries for their day-to-day job just to make their lives a lot easier. And that's challenging in itself. Motivate people whose job it is to do something they're not supposed to be doing, but it helps them. So you have to find some time to convince people to spend time on that for them and for others.

More recently, I wanted to kind of get my hands a lot more dirty again and have some more impact on some of my font skills and apply them and do more programming and quantitative work. So I've joined about a year ago a smaller asset manager, still not small at all, but smaller. And I've been able to use what I've learned in terms of scaling processes from a risk management perspective, quantitative framework to other types of portfolios and portfolio risks specifically there now, too.

What risk management means

So the way to think about it is maybe from the other side of the table, which is the portfolio manager side. And you think about the investments that they want to make and the style that they have and how are they going to put risk on the table or money in terms of investments to stocks or to bonds? And then how do you kind of quantify the various outcomes of those exposures, those risks, those bets that they're taking in the same way that they would take it? And you want to be able to explain to them the risks that they're taking in the same way that they take risk. Two different teams might have different approaches and I'm not going to want to use the same language or presentation of how they're taking risks. I'm going to want to tailor it specifically there too.

So just to maybe really apply a little example is, let's say you really like a stock, pick your favorite technology company that's based in the U.S. And just by saying those few terms, you can start thinking about different exposures that a company like that might have, you know, just general market exposure, just general equity market exposure. And then they're going to also have some U.S. equity market exposure. They're going to have some technology exposure. So those are three pretty high level risks already. And you have to figure out how much exposure or weight do they have to each of those. And then you can start thinking about, OK, what happens in this scenario? If that happens, you know, technology, you know, continues or doesn't in its current trend. And then you can start having those conversations with the portfolio manager and ask questions along the lines of, does, you know, does this surprise you? Is this what you'd expect? And it's more of a two way dialogue. And it can get kind of fun in a way, because especially when you have events coming up like elections, you can start thinking about who's going to, you know, what kind of impact each candidate have in terms of their policies and start thinking about, OK, what are they going to do here or there? And then what's my exposure to that factor?

Moving from R to Python

That's an interesting question because I professionally do a lot mostly in Python programming, but I still have this R package that I'm maintaining. So, you know, kind of personally on the side, nights and weekends, chipping away at that package as well. A lot of what I'm doing is getting questions from the portfolio management team or other teams. And we ideally want to be able to use the systems that we have in place to answer their question. And to be honest, they just have a question they want to answer. They don't necessarily need to know how you're coming up or which tools you're using to answer that question. It's more fast paced in that sense. So really, whatever it is that gets the job done the fastest is really all that matters.

R and Python, in my opinion, are so similar. And I've heard it mentioned a few times that maybe there's the whole tension because they are so similar and people are comparing them because of that. If they were so different or special from each other, then they wouldn't have those conversations. It'd be obvious which way to go, but it's maybe not. And that's why. So in my mind, I think they're so similar, there's really not much differentiation. And I've been a pretty big adopter myself of doing a lot of stats in R or in Python, and then doing a lot of the graphing in R with ggplot as well. So I've been a pretty big adopter of the Quarto framework more recently. And I've been using RStudio for a very long time, actually. I remember hearing about it way back in 2011 when it was presented at R and Finance. So it's been good to see it evolve over the years to really support the two languages, R and Python, especially.

If your whole firm switches over to Python, for me personally, it was not hard. Like I said, I think R and Python are so easy in terms of transitioning and answering very similar questions with both. And you have to keep in mind, though, one thing is who is going to support the system in place at your firm. So if all of the technology teams know Python and they don't know R, and you want to ask them a R question, it's going to be very challenging to get much out of them or support or traction on your problem that you're facing. And really, if you're trying to get your job done, you want to just do what is maybe easier and more seamless to do. And that's one of the selling points of switching to something that's well supported within your firm, whether it's R or in Python as well. And it's not so much of an uphill battle to get what you might need out of them or to do your job within the day.

I think for me personally, it goes back to moving around a lot as a kid and just being in different environments. You just kind of have to adapt and try to use what's in your environment to your benefit as much as possible.

Helping teams transition to code-based tools

One interesting observation was I saw a lot of people jump from VBA directly into Python, less so from R into Python, which was interesting. And some people switching from MATLAB into Python and a bunch of folks just skipping the whole R element entirely, which was interesting to me. And in terms of training, one thing that I did initially was set up some exercises for people. So, putting together, you want to use formats that are similar to what people would use internally. So, creating a PowerPoint deck that is going through exercises that you would normally need to do for your day job and asking questions along the way. So, for example, in the risk world, you can think of, in terms of risk, a lot of people would think about volatility and standard deviations. So, if you wanted to calculate a standard deviation of a return series, how would you do that in Python and going through the simple example there? And then your firm might have some custom tweaks to how they calculate that. So, how would you do that and how would you maybe tap into these APIs that you would use internally?

You think of it like asking a question and you have some questions and you take out the answers a little bit and you make them do and solve some of the problems themselves by using Python or whatever language and to really show them what they need to do to get their day job and to make it have a bigger impact on what they need to do. So, it kind of motivates people when you show them and guide them through exercises in whatever language that you're doing. And that motivates them to learn the language and then they can also see how it benefits them specifically. So, putting together exercises was a big, big one. And then also just hosting internal education sessions. Those can be a little more challenging because some people just learn differently and they might have different paces, which is totally fine. And when you give people exercises to go do on their own time, they might learn a little bit however they need to learn and they come ask you questions whenever they need to. So, there's pros and cons to both approaches, but I have done both. And I would say, really, at the end of the day, just showing people how to do their work in a language and then that really motivates people to learn it even more on their own time.

No, that was not technically a part of my role. I like to think about and say you have your nine to five, and then this is more your five to nine type job. And some of the advice I've been given over the years, something I always keep in the back of my mind is people don't always know what you can do until you show them. And they also don't know what they want until you show them. So, this would be an example of that.

People don't always know what you can do until you show them. And they also don't know what they want until you show them.

We had an internal Python package that was created by a group of risk managers, and no one asked them to do that. They just did it. And then they released it internally, and it caught on like fire, and everyone wanted to use it. Everyone wanted to adopt it. It really started to impact everyone's roles. And then that's when senior management will see the impact of that kind of work. And then you can start making the case that you should spend more time professionally working on or spending some of your day job working on that as well. But it takes a bit of showing for that to happen. I find that people are pretty receptive if you show them something that benefits them. And after that, it's kind of on you to use that to your advantage to make it more formal of your role there too.

Communicating with non-technical stakeholders

A very long time ago, and I think you still do it today, but you could hook R into Excel directly. And that was something that I did a long, long time ago. But it was so cumbersome and painful and really hard to share that as well. And I think I've heard more recently that Python is available in Excel now as well.

In finance, there are different types of investors. There's quantitative portfolio managers, there's fundamental qualitative portfolio managers. I myself have always worked more with the more qualitative portfolio managers myself. So I've never had the luxury, I guess, of speaking the same language as a quantitative portfolio manager would probably speak. And I learned that quite early on in my career to take the audience into account. So, you know, imagine coming from, I was in undergrad, undergrad school for too long, almost seven years actually. So imagine coming from a quantitative background with that many years of school into a real world setting and working on interesting quantitative projects only to present them and to find out that they have no impact. And you're communicating to an audience that doesn't either understand or find what you do useful. They might find it really interesting, but you've realized that you've spent a lot of your time working on something that's not going to have any impact.

And really, from that moment on, I had to take a different approach in order to have impact. And really, once you figure out how to have that rhythm and speak the same language with the different teams, it's even more rewarding. And one way, though, I still, you know, maybe a secret, in a secretive way is behind the scenes, I'm still doing quantitative work. But when I'm presenting and talking to portfolio managers, I'm going to leave out a lot of those details, and to only mostly focus on the business impact and business value, and tailor in a way that speaks the same language as them, as much as possible.

One of my more interesting analyses is it sounds really simple, but it's similar to optimization, but it's random portfolio weights. So you can just generate a bunch of random numbers and allocate different, you know, investments to those weights. And to see a bunch of different outcomes for different portfolios, you can kind of use that, you know, scatterplot of outcomes and compare how you're doing versus a bunch of random portfolios, kind of like monkeys throwing darts at a dartboard, or typing on a typewriter to see if they can write out Shakespeare in a way. And are you better, better than them, or not? So you just, are you really far off base from a bunch of random numbers? Or are you actually pretty competitive? But when I present something like that, I'm not going to talk about how I actually generated those numbers. Because, you know, technically, what you're trying to do there is not, you might think it's really easy to do that, because you might just want to generate random uniform numbers. But actually, if you look at the scatterplot of those results, you'll see that they have different clusters. So you have to do something technically much more mathy than that. So it's really tempting to just generate uniform random numbers, but really you have to generate more exponential type distribution in terms of numbers to normalize. But I'm never going to want to talk about that level of detail with a portfolio manager. I'm going to focus on, okay, I'm just going to generate some numbers, show you the results, and we're going to start having some conversations. And it's really about bridging the gap between how they take the risk. You're kind of in the middle, and then you have another side, either the models you're using, or the models in the system, and the quants that are developing them. So you have to speak both languages is maybe one way to think about it, understanding the audience.

Convincing portfolio managers to listen

I think it goes back to maybe listening to them first, and not come out of the gates too fast in terms of what you're thinking and viewing from your independence risk perspective. I think it's really important to understand their style, their views, and then to also think about what the client wants and what the in or in stakeholder wants for whatever product you might be working on. So you kind of want to have a big picture of the different areas of really who's making the decisions from the portfolio side, then who's being impacted with the results on the end user side as well. And once you have that, and you kind of really understood how everything works together, you want to be able to speak the same language. And once you're able to do that, I think they'll start listening more to you. And one unfortunate thing is it just takes time.

So the portfolio manager and other people, they need to trust you. And you might be the smartest, most brilliant, quantitative person in the world. But if no one's listening to you, even though you're right, that's not a good outcome. And you want to be able for people to listen to you. And I think with that, time is the biggest factor. And then when you do have some time under your belt, you're able to communicate and to talk in the same way and have similar impacts to what a portfolio manager might be thinking there as well. So I think it's a bit of speaking the same language over time and understanding essentially what and also what the end user client is expecting as well. And if you can frame questions and points in that way and make it, frame it in a way where it's really about the client and really about the end user, I think you could maybe speed up a little bit more and to really have that conversation earlier on as well. I think everyone wants to do the right thing at the end of the day. And if you can frame it in that way, then you might get some earlier adoption as well too there.

Data, quantification, and crowded trades

I've worked on a number of projects over the years that required crunching a lot of data. And that's also one of the reasons why I felt the need to create the role package in R. It's available on CRAN for everyone on this call to go download afterwards. Because at the end of the day, I wanted to calculate basic statistics, like z-scores. And I wanted to have some sort of weighting scheme attached to it. And I wanted, and I had missing data. And if you wanted to just roll apply or roll some other function in R, you're rolling some function that you're kind of tied to how that function behaves. It might accept weights, it might accept missing values, maybe not both, or maybe one or the other. So you're kind of at the mercy of that function. And I wanted something that could support those features. And I wanted to be able to do that fast because I didn't really know the right window size or the weighting scheme. So I needed to be able to iterate quickly and create some sort of shiny app is what I did there too. And you can toggle between those parameters very quickly to get a sense of the different z-scores of the data yourself there too.

There are a bunch of concepts in finance that aren't really taught in textbooks. And one big project that I worked on, and it's public information, it's been published externally before, is some work on crowded trades. And that's something that is a very easy concept to explain, but it's a really difficult concept to quantify. So everyone probably can figure out what that means just by the word crowded trades, but it's when a lot of people are in the same trade or investment and you want to know that. And especially from a risk perspective, that's very important because you don't want to be the last person standing when everyone has left the room. And you can have a pretty significant drawdown and lose a lot of money. So we want to be able to make sure that we can at least track that quantitatively, have some conversations about that. But how do you quantify something like that that's not a textbook? And that's really where it goes to talking to people about their experience in those types of situations. And especially if you're starting out in finance or something else, some other domain, if you don't know something, it's definitely useful to go talk to people that have experienced those situations and say, hey, you know, in these types of scenarios, what have you noticed in the data or what data would you use? What has the experience been in these situations? And then you start to see where people are thinking and where their minds are going when you ask those questions.

Talking is one thing, but then going and finding the data is another thing. And you have to go talk to either data vendors or come up with the data yourself. But you get probably better insights if you can find data that's maybe more proprietary or something that you've developed yourself. I like to go to a lot of finance conferences and I've heard some really smart asset managers talk about quantitative asset managers talk about how they like to hire Ph.D. level people to do basic regression and data cleaning. And the question is, why would they want to spend their time on that? And really, though, that's that's the hardest part is some of the more basic work. And if you can find somebody with a Ph.D. mentality who can get their hands dirty and wants to get their hands really dirty with data cleaning and linear regressions, then that's that's even better in a way.

So I had to find data that would suit the question I'm trying to answer. And I got that from talking to different people, trying to find the right data sources and then putting everything together in aggregation of some way and some sort of dashboard of sorts. And, you know, one thing at the end of the day, though, is it's you can send out just a spreadsheet of numbers or a PDF of numbers or a shiny with just a grid on it. But really, what's impactful is when you can tell the story with the data and to kind of not really tell people how to think, but really what to focus on and they can draw their own conclusions. And that's also a big challenge because you I mean, some people will read data and they'll draw their own conclusions or it's really obvious just to regurgitate the numbers on the page. But if you can find how different data points tell a bigger, better story, that's when some people start to listen to you even more. And you realize they realize that you have more insights to say than what's just shown on the paper and you can really connect the dots.

Really, what's impactful is when you can tell the story with the data and to kind of not really tell people how to think, but really what to focus on and they can draw their own conclusions.

Working across teams with poor relationships

So it definitely depends on the different types of different types of teams and the different types of people on those teams. One thing that I've noticed over the years is I like to be just as passionate about my work as maybe other people would do their job. So if you can try to be more partner-like with them and really show them, try to put yourselves in their shoes is another way, I guess, of thinking about it. So if you think about, okay, if I was some person who just got this email request from this person, this other team, you know, what does it mean? Why should I care about it? And they might have a pretty negative response to you off the bat. But if you can kind of put yourselves into their shoes, maybe you could reframe that email or that ask in a way to really get a better outcome. And that's something that is challenging to do. So think about it from the other person's perspective.

You can kind of draw parallels in a way to, you know, open source development, you know, you can get, I think of it like anybody today or tomorrow can go open up an issue request on my GitHub and write whatever they want on it. And maybe it makes me feel great, or maybe it doesn't. You know, I really shouldn't try to take it personally. Anyone can do what they want. But trying to then take a step back and to really think about it from their perspective and maybe reframing the question in a way that's better and maybe leads to a better outcome would be a better scenario for everyone. And maybe even reframing into a way of they're asking a question and you could be getting a better outcome if they've asked a slightly different question. And then maybe even you are motivated to tackle the problem, even though it came off very negative at the beginning there too. So I like to think of it, I guess, from a different person's perspective. And they can ask anything at any time. And you want to keep an open mind and maybe just take a deep breath, take a step back and to think about how can you solve the problem more generally than maybe what they're asking in a very narrow sense at the same time too.

LLMs and the future of coding in risk management

I was expecting a question around LLMs and maybe I'll take that question to the secret question around LLMs in a way. I use, I personally pay for Copilot. And I find it's very useful for my coding to get things done quicker, both in a number of ways. One is I might do something one-off very sloppily. And when I have time, I'll go back and want to do a faster, better job, more efficient job. And it can help me solve that problem, maybe reformat my code, my function entirely. That's been pretty useful from that perspective. But of course, I have to double, triple check everything that comes out of it.

More recently, what I've been doing is almost, it's a little cumbersome in a way, tedious, but almost going function by function in my package and essentially doing like a code review of sorts. And to really make sure that I'm doing things efficiently, I could have done something differently. And if the suggestion is A, and I'm going to review it, I'll take it serious for a little bit. But if it makes sense or doesn't make sense, I'll just move on from there. So maybe I wouldn't take everything so seriously from the output perspective, but it's definitely interesting to see some of the suggestions sometime and to make you question what you've done.

When I do some interviews and ask people questions, I am always refreshed when some people say, I don't know. That also means people are being honest and they want you to trust them. And you appreciate that from a risk perspective. I have not come across the case yet where chat GPT says, I don't know, especially when asking coding questions. It's a very confident system at the moment. It would be a little more trustworthy if it could say, I don't know, sometimes that would be pretty useful. So I would take everything from it with a grain of salt, but I found it useful from a coding review perspective.

I definitely think there's maybe some adoption in the risk management space there in terms of thinking about different regimes or different, that's a similar environments that you're in. So is today's environments, you can quantitatively do some of that today using big macro data, but can you use something more sophisticated like a LLM type model or some other model to figure out the regime you're in and to think about the different types of outcomes and scenarios and to just create a better toolbox from a quantitative perspective.

Community involvement and professional growth

I think it's interesting. I went to RFinance physically for the first time in 2015, and I would always read the presentations online for the years before then. And I was always so impressed with the quality and the content. And I was very lucky and enjoyed going there in person. And that motivated me to submit a talk for the next year. I got a lot of value out of going there physically and meeting the people behind the packages that I'd use on a day-to-day basis. It's really interesting and it's useful to meet them and to hear their thinking, especially things that are not posted online, and you get to learn about what they're thinking or what they're doing. Just, I learned a lot just from going to conferences and then having some confidence to present the following year to get some feedback from everyone out of it was very useful.

I definitely got some feedback after a couple of my presentations about different approaches to what I'm doing to make it even faster. So, you know, I was able to benefit from that feedback and make the package even faster over the years. I'm not going to say it was easy, quick thing to do, but it got done eventually. And it had a lot of impact. So, I appreciate that advice. It just, you know, I have limited time nights and weekends to contribute to that work. So, it's one of those passion projects where you know what you want to do. It's just finding the time sometimes to do it can be quite challenging. So, the feedback was quite impactful and, you know, Hadley at one point opened up a couple issues on my GitHub to request some functions a long time ago. So, just getting that was quite exciting to see.

More recently, you know, some of the data table folks, seems like that package has gotten re-energized significantly over the last year or so with some more funding. So, they're doing a lot of work in that space and they're presenting at a lot of conferences. And it's a very optimal optimized package to do a lot of things. So, I'm a big user of that. And I've been lucky to, they do have a rolling function now, froll. And so, I do talk to the developer on the data table team who's working on that. And we share, you know, lessons learned about different approaches and learn from each other that way. So, it's been exciting to talk to somebody who, he doesn't go to the R and finance conferences. So, I don't know if I'll meet him in person, but it's been nice to see and to read and talk to him over the years about what he's thinking. And I'm always impressed with that package.

You know, mine is pretty fast and theirs is going to be a little bit faster. So, they're a very amazing bunch of folks who are working on that package. So, I've definitely learned a lot, even though, you know, maybe they might be a little bit faster. I still learned a lot from them. And I think that's something that is important for everyone to do and just to keep learning. And, you know, I started with R back in 2009 and then switched to Python. And now, who knows what's next and with all the LLMs and everything there too. So, kind of an unanswered question at the moment. So, excited to see what's next there too. But definitely, continuous learning is something that I enjoy doing and keeps you engaged on what's coming up next. So, it's something I recommend for everyone. And going to conferences, presenting, even posting what is useful for you would be interesting to see. Something, maybe not even something that's relevant for your professional life, but even personal life would be interesting to see. Like, something that's actually useful for you is something that would be useful for others to hear and share about and to learn from as well.

Standing out in the job market

I remember getting stacks of resumes and going through them. And after a while, a lot of them look very similar. So, it is very, very hard, but rest assured, some real people are actually looking at them. But that being said, step one is really just to have the qualifications from a quantitative perspective. So, something quantitative is always useful. That's kind of like a starting point. And then I personally like to see some sort of passion or motivation for the role. And then you have to think about what is something that would make you stand out? And it goes to one maybe easier thing to think about is, what is something that's typically actually challenging for a quantitative person to think about? And really, one thing that is maybe more of the soft qualitative skills. So, you can think about what's something that they've done in that space, whether they're really following the markets and can they really explain what's going on in the markets and talk about the markets.