What is DevOps? And advice for those just starting! | Alex Gold @ Posit | Data Science Hangout

Welcome to the Data Science Hangout, everybody. Thank you so much for being here with me. And I don't have Rachel today. She is enjoying a nice vacation. So it's just me. And I'm also really nervous because I've never done this by myself before. So be extra nice to me.

And I hope that you have a wonderful time. This is a space that we have really kind of cultivated together. And it's an open space to hear what's going on across data science. We have people calling in from all over the world. We chat about data science leadership. We connect with others who are facing similar problems. And it's every day on Thursday, same time, same place, most Thursdays, every once in a while. We will miss one for a holiday or for conference. We are all really dedicated to making this a friendly and warm and welcoming space. And we love hearing from you no matter your years of experience, your title, your industry, what language you work in.

Everybody is welcome. If you're watching this recording in the future, and you want to hang out with us live, there's going to be details below for you to add it to your calendar. So we really enjoy connecting with everybody in the chat. And we want to encourage you to just briefly like say hello, drop a link to your LinkedIn or to your personal website.

I have found over the years hanging out in the Hangout that oftentimes I will say hi to somebody and then I won't get their LinkedIn or their personal website or anything. And their name will be like Amy Smith and I'll never find them again. So let people know who you are and where they can find you. You can also share in the chat any roles that you might be hiring for. Please, please do that. And if you're looking for a role, let everybody know what you're looking for and what you would like to do and see if we can make some connections that way.

There are three ways to jump in and ask questions today or just share your own experience if you want to chime in with something. So you can raise your hand on Zoom and I will call you to just unmute and ask your question. You can put your question in the Zoom chat. Isabella and Curtis are behind the scenes making sure that all of those questions get to me somehow. Thank you so much Isabella and Curtis and Randy's here. And then you can also put an asterisk next to your question if you might be somewhere noisy or your mic doesn't work and you would like us to ask that question for you. We would be happy to do that. We also have an anonymous way to ask questions which is Slido. So Isabella has put a link in the chat. It says you can ask anonymous questions here. We will keep track of that throughout and try to grab any anonymous questions that we see.

All right we are joined today by our co-host Alex Gold. I'm so excited to talk with him. He leads solutions engineering and support teams at Posit. So welcome Alex and I would love to get started by just having you introduce yourself. Tell us what you do and a little bit about yourself outside of work. Yeah absolutely. It's great to be here. I was messaging Libby ahead of time that we would count this as a success as long as neither of us vomit on the camera. So like that's the bar we're setting. We're gonna nail it. It's gonna be fine.

So yeah I lead the solutions engineering and support teams here at Posit. My background is in math and econ. I am an econ PhD dropout. Sort of wound my way into data science and was a data science manager for a while and then came to Posit about five and a half years ago as a solutions engineer, was an individual contributor and then sort of moved back into management here at Posit when an opportunity presented itself. Outside of work I live in Silver Spring, Maryland right outside DC with my wife and our puppy and our four-month-old which has been an adventure for the last little while. And other than the data stuff I do I like doing landscaping and house projects and have been a martial artist for 24 years. So that's my big out-of-work hobby.

Wow what kind of martial arts do you do? I've done a bunch of different things. The main thing I've done is tang sudo which is a Korean punching and kicking thing kind of like taekwondo if you know anything about it. Also Japanese small circle jiu-jitsu and I've spent a few years each studying tai chi and bando which is a Burmese martial art but mostly tang sudo.

Background and path to data science

That's amazing. Well okay don't cross Alex. He'll come for you. Okay well I would love to hear a little bit more about your educational background. So you said you were in math and economics. What led you the data science route and what made you decide to drop out of the PhD program?

Yeah so yeah my background it was the university I went to. I went to Wesleyan University in Connecticut. We didn't have minors or something. I don't remember exactly the deal but it was like this joint major of math slash econ. It was like one major that was both of them so that's what I did in undergrad and then I spent a while working in like the DC think tank world doing policy analysis and that kind of thing and I spent a bunch of years doing that and you know did a lot of different kinds of policy analysis. Some really quantitative some much more qualitative and realized that like the thing I had always loved about economics was sort of that it's this funny mashup of like math and people.

You know especially if you're doing for any of you who have econ backgrounds right like I was doing exclusively microeconomics mostly like what you would broadly classify as like labor economics and so it was all about like people and numbers and it turns out that like if you add some computer stuff in there you basically have data science and at the time data science didn't really exist as a field but you know when I was a few years out of school it started becoming sort of more of a more of a real thing. So the the PhD part was sort of in the middle of my think tank time when I was like oh if you want to like move up at a think tank in DC you kind of got to go get a PhD so I did that. I started it. Decided it really wasn't the right career path for me and was very lucky to have a lot of support from family and friends which was really meaningful and dropped out after a year with my master's which was actually a great deal because I got a master's degree for free because my program was funded.

So if any of you have gone to grad school, a piece of advice I got from a professor when I was still in undergrad they were like when you go to grad school do a PhD program and only go if it's funded and then if you don't want to finish you get your master's for free in a year and it turned out that was great advice because that's that's how it worked. To be fair I'm not sure I learned a lot in that master's degree because it's all like first year econ PhD stuff which like Curtis right like you have a PhD in econ you know that first year. It is not real material it's just it's just prep for for the rest.

But still nothing to sniff out. Free master's degrees. I am still paying mine off. Yeah it opens doors even if the the stuff you learned is it's a credential and that is valuable in certain ways.

What is DevOps?

I'm curious about so you're you went from a sort of research think tanky role into data science. Yes. And you felt like that was kind of a natural progression. What is DevOps though? Because you're a solutions engineer what does a solutions engineer do? And you just wrote a book about DevOps so can you talk about what solutions engineering is or looks like for you every day? What kind of problems you solve?

Yeah so you know I think an experience that I had and I think many data scientists have I'm sure many people in this room have had this experience is like as you get more serious about doing data science sort of like all roads lead down the stack right you start you're like yeah I'm gonna like build a sweet machine learning model or whatever and then you're like oh but the data sucks so I got to clean the data and then you're like oh but like the database sucks so I can't even clean the data because the database is bad and you're like oh actually the server that's bad. So I my first sort of data science job I ended up managing RStudio server on a Linux server for that organization and it just became really clear that like that was I mean for me I really enjoy that a lot of people do that work and they're like oh this is this is no fun I do not like this right and those people go back to doing you know or try and go back to doing data science I often think that's a losing battle but for me I found I really enjoyed that stuff sort of further down the stack I found I was really interested in like how to manage servers and networking and data and all this kind of stuff and that was how I ended up here at Posit as a solutions engineer that's that's what a lot of the work is of doing solutions engineering is this sort of connection between right how would you use the Posit products as a data scientist and how do you set them up so that they work well for for the people who want to use them.

And so that was a really great fit for me because I obviously cared a lot about what data scientists do day to day but I really enjoyed like working on a Linux terminal and figuring out how to make it not suck for them so you know that's that's sort of the answer of like how I got to the point of being a solutions engineer and uh why I wrote my book DevOps for Data Science it's available for free online do4ds.com but sort of there's this like broader question of like what is DevOps and like to me I think like DevOps is this funny thing where it's like it's a general purpose thing about making computer code not suck when you put it into production because right you have like in like the 90s you have the agile revolution where everybody's like oh we're not gonna like spend years defining requirements then it gets super out of date and then we write a bunch of code and it isn't good we're gonna do the agile stuff where we like deliver small batches frequently right I think probably a lot of people in here have heard of what agile is and probably aspire to it I think agile is always aspirational but like it's great but then there's this question of like okay we're writing the code in this agile way but then it has to like go live right the code has to go from being on my computer I wrote it I wrote a little agile thing I like showed it to the customer the customer's like this is great and then you have to like put it into production and this is where the DevOps part comes from.

And so DevOps is it's a really mushy term because it's a set of like it's it's not there are tools involved but it's not just tooling it's also processes and people and like a philosophy that basically you try and put things into production in a way that works well with agile so DevOps is a super mushy term and there are also like a gajillion companies out there that sell DevOps software and so like shockingly they're like DevOps is the thing we do wow crazy and so like I think there's a lot of tension on the word DevOps it's it's very overloaded but it is also a real thing right it really is this idea that you can deliver not just create but deliver software in small increments right and I think there's a lot that as data scientists we can learn from those ideas and and one of the the reason that I really wanted to write this book was that like you can't just be like oh DevOps and then just like translate it right like you can't just be like oh now just do that for data science because data science is a fundamentally different practice than software engineering we're not software engineers right we're doing a different thing.

data science is a fundamentally different practice than software engineering we're not software engineers right we're doing a different thing.

And so uh you know having done the work I'd done with just dozens and hundreds of companies seeing how they were implementing data science um I wanted to spend some time uh you know digesting those thoughts and then spitting out a book so that's that's what I did.

Relationships and working across teams

So Mike I like all I can say is that like I think it's exactly like always the right thing to do is to try and take that step back and go hey why are we trying to do that and you know in your organization maybe you have authority to like just give somebody that hardware to show them where to get that hardware but one of the ways the relationship between data scientists and other parts of the organization can go awry is when data scientists are really hungry for more more more more more without necessarily having a lot to show for it and so I think like your question of why do you really need this you know you may have the question but even more so the like person over on the it team who's responsible for controlling cloud spend who is going to see this 128 core machine that's been up for nine months and has cost several thousand dollars and they're like what is going on here.

And maybe that's worthwhile for whatever you're doing but before you do that you should probably make sure that you've convinced yourself at least that you can you know yourself either you or or you know whoever that that actually is a worthwhile even if you have permission making sure that it's a worthwhile you know endeavor is is I think a great use of time.

Yeah and I think you know this is a completely unsatisfying answer but to me so much of this is about relationships right like we are humans working with humans and it's so tempting especially as technical people to want to treat the other humans as like the other side of an API where we can just like query the API and get what we need and and I think sometimes actually if you can get your organization to the place where you have either a literal technical API or a like team to team level kind of understanding that's actually really cool but a lot of times relationships are what allow you to build that out and and get you to the point where like that actually is is possible right.

I think one of the really important concepts for a data scientist is getting the permission and the ability to have a sandbox where you can play with real data right and so like getting that sandbox is often a years-long endeavor particularly in a big big organization right like it's years of approvals and building it out and hardware and that kind of stuff but then once you have the sandbox right then it's like wow this is this is magic and so it is not easy and often requires a lot of just relationship building and and that sort of thing but uh if you can get it done it's it's an incredibly valuable thing to have.

Data science backgrounds and hiring

I think it is incredibly valuable for people to have a really strong background in like sort of what I would describe as classical statistics basically or something like it to go into data science and and the reason what my one of my first bosses out of out of college he he had this mantra that it's all about the data generating process and I love this this like idea that like whatever you are trying to do in data science you are somehow trying to derive some truth about the data generating process and so having a really strong understanding of what a data generating process is and how you model it I think is incredibly valuable right.

And some of that is statistics some of that right like there are all these newfangled machine learning things right but like basically all they do is you're removing and here's my my economics background right it's like in in you can specify basically the parameters of a model then you estimate the parameters of that model right like if you're talking about like a linear regression your parameters are just your betas right and yeah I completely agree I gave a talk a few years back about like you should just do linear modeling and I still stand by that that talk but I think like right you're at so so like in a linear regression you're estimating the betas like that's all it is right if you're talking about like a neural net what you've done is you just you've removed any assumptions about the shape of the underlying data generating process but now you're estimating millions of parameters instead of like seven and there are trade-offs there right but my opinion at least is that if you can't form a reasonably good hypothesis that you will never be able to check about what the data generating hypothesis what the generating data generating process looks like like my opinion and again I am biased is that you have no business throwing it all into a black box model and pretending that what comes out is meaningful.

Because I just don't think that there's um like you you want to have at least a theory even if you can't test it on what is going on here what am I trying to make sense of that is happening in the world and that is not to say that like a data science degree doesn't do that I know from my experience like statistics and econometrics degrees really get into that point of like what is the underlying distribution that you are trying to model what is the underlying process.

And so like I don't think that all those things are a waste of time that's this is again like this is me just like giving opinions I have no like standing I'm just sharing opinions but like you asked so that's my opinion is that is that you know as a um you know as a data scientist you're trying to make sense of the world in some way and so if you if you believe that just throwing it into a black box model is is enough to me like that's sort of an abdication of responsibility you're kind of like the model takes care of it and and the reality is that the model is gonna and this is right like there's there's a whole field of literature here on like stretching into the ethics and and stuff of of data you know science and and data usage but a lot of it to me is there's always implicitly a model of the world and if you're not thinking explicitly about what that model is that you're trying to like bring to light it's getting it's coming to you and you might not realize it.

Can R be put into production?

I mean that was an easy one right um no i think i think that's complete nonsense i think you know again there's a question of who is putting it into production and what do you mean by production by production right if you are visa and you have to serve millions of um credit card authorizations that's probably i mean probably thousands per second right you have to like do fraud checking thousands per second across the entire like probably r is not the right language or that like but you need to be using like c++ or rust for that right you need to go like real far down the stack for that i mean we see r in tons of production contexts here and and the other thing that i would just say is that like often the implicit second half of that sentence is you should use python like that's often the implication is like you cannot put r into production and you should use python i think python is a great language i think that r and python have a lot of trade-offs against each other it is not clear to me that python has such a dominant even today that there is such a dominating um advantage for python and production systems relative to r uh other than that whoever you're talking to probably knows how to put python in production and probably doesn't know how to put r into production so um you know on for for most of the um at the scales that most of us are working at right r in production is plenty performant your code is going to matter far more than whether you were to write the same thing in python.

Um and uh uh you know i think um i'm trying to think of anything else to say on this riff like i've just said it all like i i i think i really do think it's it's hogwash i think r has a place in our production stack right it it shouldn't i also don't i see some people who try and do everything in r i've occasionally been that person i think that's also a mistake but there's it is absolutely the case that r can be a part of a really robust production stack and and is at a lot of large organizations.

Advice for those starting in DevOps

Yeah just muck around with stuff i mean i i'm i'm very firmly of the belief that having some mental models is helpful as a place to start so you can understand like that was sort of why i wrote the book is like i think it's helpful to have sort of a framework for for what what you're doing but the way to learn this stuff is really to play around with it right my book is like so thin on all of these topics there are books like every chapter in my book there's a book not just a book like books on each of those things and so you know what what i would say here is find something you want to do and then try and do it and you're gonna get i mean it's it's not that dissimilar from learning a programming language right uh you you start trying to do things and you hit roadblocks and you figure it out.

Um and and go along the way so like for me one of my first big tasks that i did that was like devopsy was for my when i got married i wrote my own wedding website and then i really wanted to be able to collect rsvps on the wedding website but i was posting the wedding website in on github which doesn't like it was a public repo because i wasn't paying for it and so i needed a way and i wanted to load the results into a google sheet and so i needed basically i just needed a place to like store my and there are better ways to do this now but this was a few years ago now uh i mean i basically need a place to like store my google credentials and like turn an api call from my wedding website into a row in a google sheet and so i like had to figure out how to use aws lambda to like receive this stuff from my wedding website and like push it out to a google sheet and it wasn't a very complicated process but i learned a ton just by having to like make a thing that ran in the cloud and didn't break and like it was it was like seven lines of python code or something right the the actual coding was trivial but there was a fair bit of sort of devopsy stuff in terms of figuring out how to how to make that that work.

Um and so that that's that's my advice people is like find something you want to do whether it's a side project or a work project and and get started like y'all are here because you're like smart people who can learn technical stuff to me the biggest barrier is do you have a reason and i often find that it's it's like somewhat helpful to read books and and that kind of thing to give you a lay of the land a framework for what's going on but if you want to really understand it there's no substitute for just doing some stuff um so that that is just find a project you want to work on um and and then you know figure out how to do it after bashing your head against the wall a few dozen times because that's inevitable.

Writing a book in Quarto

Yeah that's an interesting question um i've had a draft blog post here for a while that i i keep meeting to post and haven't gotten around to something about having a four-month-old gives you like not that much free time to like write blog posts and stuff. Um for me personally uh i have never been a very enthusiastic writer and did not consider myself a particularly good writer um so the main thing i learned is that i am less bad of a writer than i thought perhaps even good at times i feel pretty good about the book hopefully if you read it you think it's like reasonably readable.

Um and you know i learned a lot about how i write and the way my writing and my thinking is really tied up together um i think i when i started the book i thought my i thought that my ideas were way more baked than it turned out they were and forcing myself to write a book ended up revealing a lot of weakness in in the thinking that i that i had and forced me to really get precise in a way that i didn't have to before that.

Um and that's really tied up in writing for me for me i am a like i mean some people are different but like i am an out loud thinker i i do my best thinking out loud and it turns out that writing also provokes the same kind of thing and so for me it was really really impossible i did outline my book but like i couldn't really know if that outline was any good until i started writing and understanding whether like okay i wrote this chapter now i'm trying to write the next chapter but i can't really write that chapter because i needed to introduce something in the last chapter that i didn't actually get to and actually these chapters are in the wrong order and they need to be like split up and recombined anyway.

Um and so i i think that was the biggest learning for me was understanding that like i could create something written that i think is pretty good but it was a lot of work it was really hard and that the fact that i don't really like my first drafts was not uh forever damning of my ability to write something good with enough time and um willingness to go back and polish uh which which took a lot a lot of time.

What tool did you use to write your book? I did all in Quarto i i i think the i think the very very beginning i started in our markdown and then Quarto was out soon thereafter and i switched over and oh my god it was so much better it's like they took i mean they they literally did take what they learned from 10 years of writing our markdown and we're like oh let's just include the batteries this time and um in general i found the Quarto book writing experience to be really great um and uh obviously the the responsiveness of the Quarto team when i had questions and concerns was was awesome um you know posted stuff on github and that kind of thing um but i had a really good experience i wrote it in in RStudio um using Quarto which like is not the place where most people write books but honestly like i think i think it's really easy to get preoccupied with tools and and you know what and it's just like you just got to write a lot of words like that's that's the thing is it's just a lot of words and so um i tried really hard to not let my tooling get in the way of just doing a lot of writing because that's really what i had to do.

Transitioning from individual contributor to manager

Yeah i i love this is like so my role now right mostly what i do when i'm not writing a book which is not which is all the time these days thank god done with that um i um right i manage a team and and now i like manage managers so like i think this is a um uh this is something that's near and dear to my heart.

One of the things about management is that it is a totally different job from doing data science there's a completely different craft and it really is a craft um one of the things that i i think is really interesting about management but that is has been frustrating to me personally right i think many of us probably in this room uh identify as like a smart person like that's a thing that we value about ourselves um and one of the unfortunate things about being not unfortunate but one of the things about being a manager is that smart actually doesn't matter nearly as much as like showing up the right way every day it's it's really a craft it's not a a uh sort of knowledge-based thing right i've read a lot of books about management and they help like this much relative to just sort of like showing up right every day and and caring about your job.

smart actually doesn't matter nearly as much as like showing up the right way every day it's it's really a craft it's not a a uh sort of knowledge-based thing.

And so when i'm talking to people who might be interested in management and trying to decide whether it's the right path for them right to me a lot of the questions that i try and ask people and try and figure out are like are you more interested in creating a great work environment for other people than doing the work right uh if you still want to do data science on a day-to-day basis you should not be a manager when you as a manager right you're in my opinion your job is to create an environment where other people can do their best work right your job is to create a place where they have the right level of sort of managerial oversight and that level is not zero contrary to what like certain engineering places will tell you like managerial oversight and assistance is really helpful.

In particular i think a lot if anybody has read casey newport's book um world without email very good book uh but he really distinguishes between the work and the workflow right and and how the work of management in a lot of ways is to get the right people together and design the workflows so they can go do the work where the work is the really like hard thinking about what to do. To be very concrete right like if if we are working with a customer or a prospect at posit who doesn't have posit professional tools right the work is how to figure out or how to help that customer understand what value the posit professional tools would bring to them uh how does it fit in to their existing data stack how would they use it how would it benefit them right that's the work the workflow is like do they do a proof of concept or not how do we sign the paperwork for them to you know uh actually acquire the posit products how does a person from my team get assigned to work with them.

And so to me if you want to be in a managerial role you have to be really passionate about that question of helping other people have a really great day at work and a great year and a great career and care a lot less about like oh i want to go write a bunch of code if you if you still really want to write code it's probably not the right moment.

I will say that like i think a small fraction of people really get joy from management and management is a slog like relative to individual contributor work where like you build a model and it works and then you're happy and you like you know uh uh publish a dashboard and it works and you're happy like that's that's great um as a manager the wins are way further apart and like it's way more of a slog in between and so you really have to be excited and and find joy in helping those other people win and and really feel that in in what gets you out of bed in the morning and if if that doesn't get you out of bed in the morning probably not the right fit for you.

Now there are organizations where like the only way to move up is to move into management and that just kind of sucks like increasingly i think there's a recognition in a lot of organizations that having very senior individual contributor roles and also manager roles is the right it is the right way to do it some organizations have really recognized this and built out those like very senior individual contributor roles and other organizations really haven't and unfortunately if your organization hasn't sort of recognized that there are there should be paths to leadership and advancement that are not managerial roles that's a really hard spot to be in.

What managers look for in hiring

Um i would say um this is something that i select for really heavily in in hiring um and i think it is crucial for solutions engineering and also for data scientists and that is curiosity like number one by far is curiosity um i i tell people when i am in a hiring process i make about 90 of my decision based on the questions the other person asks me um i i basically think of the like part where i ask them questions which is for me i split my my interviews about in half so i spend about half of it asking them questions and then like leave their questions for about half and to me and this is especially true in roles where right like a solutions engineering role where you are working with people um and i think most roles you work with people i don't know many where you actually don't um to me i'm actually much more interested in what is this person learned over the first you know over reading the job interview and the job posting and that sort of thing um and then what questions do they have on the back of that are they asking good questions are they asking questions that like are like way off topic and and are clear that they like haven't understood what we're doing here or are they asking questions that really get to the heart of the work that to me is is really important.

i make about 90 of my decision based on the questions the other person asks me.

And so um i would say this is a quality that is very well developed on on the team that i'm a part of because we we really pay a lot of attention to it in in hiring um but i i think to me you know curiosity will kind of take you everywhere and it is it is something you know the other question is that there's a lot of things that are really hard to select for in hiring like it's really hard to assess how good is somebody at working with other people like you can you can get a get a vibe for it but like it's pretty hard to assess you can pretty well assess in an interview whether somebody is a curious person and whether they'll be able to bring that curiosity to work and so i think it is something that is unusually easy to assess in an interview and generally a very positive sign if somebody seems curious and and and by curious i mean not just like oh i have a lot of questions but like thinks about what they're hearing and then asks good questions as a result that really gets the heart of what what we might be talking about in in a job interview.

Mixing R and Python in the same pipeline

Yes absolutely the question is where do you make them meet right like where where do you force them to interact you can with Quarto have them like in interleaving in different um uh chunks in one document i would argue to do that would be a mistake it is just like a lot of headache that i don't think you need to try and render one Quarto document with both r and python.

I think in general something that data scientists can learn and this is i think it's chapter three in the book or something um one thing that data scientists could really take from software engineering um this is sort of devops but more just software engineering is like really thinking about the structure of the work that we're doing and how to create clean divisions between different parts of a project and so for me i would say like you absolutely can use both r and python in one pipeline i would recommend if you're doing that like they should communicate via api through json or something else right maybe you write to like a database or a you know parquet file or something but like you want a pretty clear line of delineation i think between them because if you're doing tasks that are different enough that it's best to write them in different programming languages you want sort of a clear handoff and so designing a project so that there are clear handoffs from one stage to the next and if you want to do cross languages from language to language i think is really important.

And i think one flaw that many data scientists have is that we just don't have training as software engineers and so one thing that i think is worth spending some time thinking about is like where are there opportunities to create a clear handoff and abstraction point in the project that i'm doing so i in again in the book i talk a lot about like there's this concept in um in software engineering of three layer apps right there's the front end the business logic and the data layer and like you can get like there are all kinds of like different ways to think about this and like oh no the three layer app sucks oh no it's great oh no it sucked like whatever i don't care but having any sort of framework that you're trying to hit in terms of how the parts of your project fit together i think is is really really helpful right it's it's like it's like the metal right it's like how you take your spaghetti code and turn it into functions it's the like one level up version from that is how do you structure the actual components of a project um i think it's something that that you know many data scientists should probably do more of and again if you're going to go cross language probably a good sign that you should have some real handoff.

Awesome all right well we've made it to the top of the hour alex thank you so much yeah we're good great no passing out that was a real is a real uh possibility um there are some questions that we didn't get to answer everybody's questions were amazing thank you i might sneak into alex's dms and ask him a couple of these just so i can put the answers on linkedin so find me on linkedin i don't even have my own linkedin as a link i'm the only libby heron on linkedin so type my name and then you will find me um and maybe i'll post some extra answers from alex um later today thank you so much everybody for coming i hope you have a wonderful day alex thank you so much and thank you again curtis randy thank you bye everybody.