Meetup | The Looma Project | Why I Stopped Worrying and Learned to Love the API
videoimage: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
There we go. Hi, everybody. So nice to see you all today. Welcome to the Posit Enterprise community meetup. This is our last meetup of the year and I don't know how the year went by so fast. I'm Rachel and I lead our customer community here at Posit. Welcome to the meetup. This is a friendly and open environment for teams to share use cases, teach lessons learned, and just meet each other and ask questions.
If you've joined us here at the meetup before, maybe put a two into the YouTube chat if this is your first time joining. Special welcome to you. Put a one in the chat. For those who have been here before, I encourage you to help welcome everybody in over in the YouTube chat as well. Together, we're all dedicated to making this space inclusive and open for everyone, no matter your experience, history, or background.
During the event, we will certainly have time for Q&A. So you're able to ask questions on YouTube Live and also anonymously through Slido, which I'll share on the screen here in just a second. So you can use that Slido link there to ask questions anonymously, but also for some of the questions that we'll have during the presentation. I actually did just put a poll over there to start with and just wanted to ask everybody, do you deploy your own APIs today to just get a feel for where everybody's at?
So if you want to jump over there to the Slido, it's just POS.IT. And our team can put in the chat too. I know Davis, our presenter, has a few questions for you that we'll show through Slido as well. To address one of the popular questions up front, yes, the recording will be available and it will actually be ready as the presentation ends. The exact same YouTube link.
But with all of that said, thank you so much for joining us here today. I am so excited to have Davis DeRodes here, Director of Data Science at the Luma Project, at Luma, whose work focuses on the intersection of human-centric content and sales. Davis's focuses are full stack data science, causal inference, data quality, explainability, and R. And Davis, I will pull you up here onto our virtual stage. Thank you so much for joining us.
Thanks for having me, Rachel. I really appreciate it. Also, thanks to everyone commenting in the chat and deciding to come here. I know holiday times are around and things are hectic, so it means a lot to come and have some fun together. Also, I heard this is the last one of the year, so hopefully we end out on a high note. So I'll just dive right in. Again, I'm Davis DeRodes. I'm the Director of Data Science here at Luma. I am a small data science team leader, so it's just me, a senior data scientist, and a senior analytics engineer. So a pretty small team, so if there are questions later or discussion topics later for small team data science, I'm all game for that too.
Introduction to APIs
But at first, we're going to discuss the main topic at hand, which is APIs. So the title I came up with for this talk is why I stopped wording and learned to love the API, which, for those who don't know, is a joke or a pun from the Peter Sellers Stanley Kubrick film, Dr. Strangelove, or How I Learned to Stop Worrying and Love the Bomb. Again, I always love a good pun and good wordplay. However, the real point I was trying to get at was APIs are the bomb. They're great. They're fantastic.
In my data science journey, I definitely started more in the stats kind of experimentation realm. It kind of became more data engineering-y back-end focus. And definitely in my earlier data science experiences, APIs were always a hassle. Someone was like, oh, the data's in the API. I was like, please, no. It was always an annoying thing to query and to get the data from and to structure it properly. It was just really annoying. The tool was really annoying. I was using the time as well. And also, the other side of the coin too is deploying APIs. I've always found super annoying as well with Docker and all the headache around that. And normally, even in my classes, we wouldn't even deploy the API someplace. It would just be creating the API and just letting it host locally.
So, again, I'm very much now a big fan of APIs, especially my current work at Luma. I feel like I use them all the time. And I just want everyone to kind of feel the joy, especially if you're on the more statsy side of data science with APIs.
About the Looma Project
Before I kind of go into the main kind of elements of this talk, I do want to give some background of kind of what we do at Luma. So, Luma is a retail media startup located in Durham, North Carolina, which is where I'm at. And we seek to be the leader in human-centric point decision content distribution. What that looks like today is, if you can see in the kind of image in the back, it's our product called Loop, which is a network of tablets and grocery retailers. So, basically, tablets, you know, in your local, we're in Harris Theater, for example, in your local grocery store, playing content related to the products that are on the shelf.
So, here you can see that this tablet is mounted on a beer cooler and it's playing films about the beers that are in that cooler. And those films are very human-centric, right? So, they're about the people who made the beer, you know, the stories behind the products and the people who enjoy them. So, that is what Luma is. And the data science team is really trying to optimize Loop and then also kind of evaluate how efficacious it is. Is that a word? I hope it is.
Audience polls
Now, I know we have a couple polls going. But the first kind of question I want to ask to folks is how annoying is it to pull and store data from APIs? So, if someone's like, hey, got this data for you. It's in this API and you need to have it consistently in your data ecosystem. How annoying from like a scale of one to five or I'm not sure what the metrics we created were, but do you find that?
It looks like we're seeing kind of annoying is starting to win here. That would have been my guess. Kind of annoying. Of the worst is a pretty strong sentiment.
Kind of annoying is still taking the lead. 56% of people have said kind of annoying. All right. Fantastic. Great. That's also great to hear. I would personally answer that question someplace between pleasant and delightful. So, hopefully the stuff I'm going to discuss a layer will hopefully maybe shine some light to maybe a more positive experience, possibly.
The second question I want to ask, too, which given by that response, I'm assuming this one and the first question that Rachel asked is, how easy is it to create and deploy an API?
Which I believe, Rachel, around the first question, most folks do not deploy an API currently. Is that correct? Yes. So, the first question I asked in the intro, yes. A lot of people haven't yet deployed an API. But I'm seeing right now for how easy is it to create and deploy, we're going from the worst. Now we're jumping over to no opinion. 50%.
We're topping out at 50% for the no opinion. Others skewing towards the worst are kind of annoying. That is great to hear. So, hopefully throughout this talk, this will be a hopefully useful, helpful talk to hopefully move you at least out of ug and kind of annoying to either no opinion or maybe even pleasant. For creating and deploying APIs with Posit Connect, I'm at delightful. I probably create too many APIs now, if you ask my team. APIs and Slack bots are the two things folks tell me I make too much of.
Talk agenda
So, our agenda today for the rest of the talk, I divided it into three sections. First is querying external APIs. Just basic calling REST APIs. Best practices for me, at least, for exploring, formatting, and organizing those queries. And also production lines and those extractions to put it into your data environment using Posit Connect and also a really great workflow that leverages the exploration, the formatting you did in the first step.
The second section is going to be on functional API request generation with R. Basically, a bunch of fun use cases for if there's folks in your company doing things manually in other systems that you know there's an API and you see the right query to, you know, create the thing they're doing and automate that. You can do that with Posit Connect. I'm going to show you guys a really cool example with something that we call score at Luma regarding evaluating the films that we create with Typeform. And also integrates with Slack as well.
And the last thing is creating and deploying APIs with Posit Connect. Just that last question we just asked you guys, right? Both sharing data externally, a great use case for APIs. If you have technical, external stakeholders, there's probably no better way to share data than with an API. And then also ingesting data. So, if you need to bring in data from folks, APIs are also a great way to do that as well, especially with external technical stakeholders. And also even sharing ingesting data with technical internal stakeholders. So, if you have an engineering team and you need to send data over to them to populate in a front end, an API is probably the preferred way to do it, honestly.
What is an API?
Just a quick background, too, just for folks who are really unfamiliar with APIs. There's a great article that I have kind of cited here below regarding kind of the framework of what is an API. Which a great metaphor I've heard is an API is a waiter. So, you, the requester, are like the customer at the restaurant, if you will. And then the API is the waiter. And the recipient is the kitchen, right? So, you could be like, I like a ham sandwich, right? The API takes that to the recipient and then brings you back, hopefully, a ham sandwich or tells you, hey, we're out of ham or you asked for ham the wrong way or something along those lines.
So, that's kind of a really great metaphor I like. The thing is, too, is that things on your side can change. So, for example, your back end can change. For example, like you create a new table. But as long as you're matching the same schema or matching the API format the same way, if you're the recipient, for example, it's all good. So, the API is that really great middle point to translate the needs of your requesters and what you have on your back end.
The API is that really great middle point to translate the needs of your requesters and what you have on your back end.
Exploring API requests in RStudio
Hopefully, y'all can see my R terminal currently. So, first thing I'm going to do is library tidyverse, of course, as all great R scripts start with. And there's a link I would love for y'all to see in GitHub regarding just like a list of just free APIs out there. It's a really fun list of just free data out there that you can pull from the API, these various APIs. One I did find on that list was this aviation API. I know nothing about planes. Maybe this is a really terrible example. But it is completely free. There's no Gordon Ramsey. No one's going to tell you you're not authenticated. So, it's a completely public API for you to get data regarding airports.
So, normally, when you go to an API, you'll see some amount of documentation. And this, for me, is very around the mill for API documentation. So, you can see here in this API. Basically, you'll see these different URL requests for the function. So, you see right here, right? This is a get request for all the charts for a specific airport. And you can get the server URL here. So, basically, you're going to be the main thing you want to basically query the API or get a response from the API.
And then we need to specify which airport we want to do. So, here, you see the query parameters. And with a standard REST API, you're just going to add a question mark, the parameter name, and then equal sign, and then whatever you want to put in. So, as I mentioned before, I'm in the Raleigh-Durham area. So, I picked our local airport, RDU. And then when you run this, you're going to get that. And the main package we're going to be leveraging for running API requests is this HTTR package here. And when you get back, it's going to be a lot of gobbledygook. Like, this is not very helpful. It's a lot of the headers, you know, whether the response was 200, which means it was accepted. If you get a 403, Gordon Ramsey, that means it was forbidden and you're not authenticated properly. If it's a 404, that means it was not responsive, which is really weird. So, hopefully, you get 200s.
The magic function you're going to want to use is this HTTR content, which is going to extract the content from that response you got. So, now, if I look at the actual content, okay, KRDU, that's the airport code for Raleigh. And then we get a beautiful list in RStudio to explore. And for me, this is really great UI, in my opinion, because you can just go through here and look, okay, this one, this is around the right structure, right? This is looking good.
All right. We can do one more step further, which is we can put it in a tabular format. So, there's one or two ways to do this. I'll show you guys the first way to do it, which is if everything is really flat, so there's no nested list or anything, you can just use map DFR. And just get tabular data from the API, just like that.
So, if I go to the API, I think weather was over here. Yep. So, I went over here. I get the server URL here. And, again, this should be pretty standard in most of the API documentation you'll see. And then it had the parameter of the airport code again, so I put it here. So, I'm going to get that response. It's going to give me something pretty gobbledygooky, but then I got that content.
And the one thing I want to show you here is that these are normally JSONs that are coming back, and they can have nested structures, as a lot of JSONs do. And you see here, the sky conditions has this nested aspect here. So, if we do map DFR, it's going to come out weird, and we don't want to do that. So, another way to do this, and this is the way I do it with complicated queries, is I start by just making the weather content a tibble. And you see here, it's just going to most likely be just like a one row, one column list inside of a tibble. And we're just going to unnest it wider. And we're going to get, you know, something that looks pretty normal. But we still have this nested list here in the sky conditions. But as any tidyverse pro knows, you can just do more per functions to get to what you want. And now, we have two different rows here.
All right. Got to get them to slide. There we go. So, just to show kind of comparison, like the RStudio experience. I put R experience, but really RStudio experience. On the left is just so much better than from the experiences I've had running in other environments. So, for example, that's just my terminal. Right? You just get this long JSON. You don't know what's why. It's kind of complicated to look at. Versus this is something you can go through all the objects and see the structure really clearly. So, personally, RStudio is my favorite tool for exploring APIs.
Structuring and productionalizing API queries
So, okay. So, now, you've done your exploration. You know what you want. All right. So, what's next? I recommend structuring your kind of API query. So, when you're just getting data. And two types of functions. So, for one, query generation. So, first, just a function to just generate the query that you're going to want to run. Or the request you're going to want to run. Just a function up top. This is really helpful, too, for debugging. Especially if you're using non-REST APIs. So, for example, our team has an internal PostgresQL API that sits on top of yeah. The database sits on top of a GraphQL API. That's it. Got my QLs confused. GraphQL API. In which case, those queries are much more complicated. They're much longer. And it's nice to be able to easily run a query, run a function, get the query, and send it over to them for them to help me debug.
The second type of function is to format the output for what you actually want. So, just actually getting the weather response, in this case, in the format that we want. So, that includes running the API call, extracting the content, and then formatting it using tidyverse functions. And so, if you see the bottom, that's an example. And now, with this functionalized, I can run it for any airport I want. For example, the Atlanta airport, which is where my family lives.
So, okay. So, now you have functions to get the API request you want. Great. So, what's next? Next is productionalizing that and getting it into your data ecosystem. So, as always, our packages are fantastic. So, absolutely make a package including all your functions, all those query and get functions. They are really great. Make an RMD or a Quarto with all the basically things you want to run and ingest. I recommend publishing it on Posit Connect and scheduling its generation. And then you can get the email notifications from Posit Connect, or you can add a Slack bot. I love the Slacker package. I probably have way too many Slack bots on our team. But then you can get a nice little message every morning letting you know that, hey, the ingestion went through. And, like, you got your data in your warehouse properly.
But also, nice thing, too, is when things go wrong, I can start my morning off with messaging my engineers being like, hey, did something change in the API that made things wrong? The whole purpose of the API is that hopefully it's static. So, especially for public and really big companies, hopefully it doesn't change unless there's a large version switch. However, it is really good to know when APIs do change, especially internally, so you can service that to those teams.
Dynamic API posting: the Score Typeform project
Next is dynamic external API posting with R. And our example I'm going to flag as our score typeform surveys. So, the background for this score typeform project was we need to send surveys internally asking employees to watch and review different films. We needed a UI to upload and manage those films, a click button kind of solution for folks to generate a typeform survey, and then also send it out to everyone via Slack. And then also ability for users to say, I'm busy, I can't review this film right now and decline it.
And we were able to do a large majority of this project with just Shiny and APIs, and plumber APIs, and also just typeform Slack APIs as well. So, we see here there's three main kind of functions I was discussing. So, first, uploading a film. That was also used APIs, the Vimeo API. So, all that involved was having a button to upload it into Shiny. We've all seen that before. A button, like an action button, basically, to then send it to the Google Cloud Bucket, and then a call to the Vimeo API, which has functionality to pull it from the Google Cloud Bucket. And again, all of that inside Shiny. So, really press a button, upload the film, push it to Vimeo, which is a really useful UI.
The second thing, cool feature, was sending out surveys. So, basically, okay, we have the film in our environment. We have the Vimeo link for it. Now we need to send surveys out to folks. So, basically, pressing a button, it's going to post typeform. Send an API call to typeform and post a new typeform, basically, for all the questions that folks need to answer regarding the film. It's also going to embed the Vimeo link that we created in step one in all the different slides. Write that metadata about where that survey is located into BigQuery. We're a GCP stack, too.
And then three, we then sent a post request to the Slack API to send out a message to everyone who we asked to score the film. And the last thing is, basically, folks being able to, like, confirm that, like, yes, I'll score this, or no, I'm too busy. And that is the confirm survey acceptance step. So, first, we actually created an API using Posit Connect and plumber to take in a response from the Slack app so that when you press yes on the Slack bot, it then sends a message to our plumber API hosted on Posit Connect. And then that does two things. It writes the response to BigQuery and then notifies the score back in Slack that it was accepted or declined. That all uses an API.
So, I want to show you guys first. We're going to skip the first one because it's pretty boring and more about Vimeo. But the second one I want to show you all, which is sending out the surveys using our tool. So, I just want to show you guys the kind of main kind of problem we're solving here with that second step is that creating a type form manually can take, like, a super long time, right? Basically, right, it's folks, you know, going through every single slide, relinking that Vimeo link. It would take forever and wouldn't even be possible really for human purposes. Also, I'm not sure if you guys ever used type form before, but it's a lot of drag and drop, click and stuff. So, you have really complicated logic or other complicated needs in your type form, you really wouldn't be able to do it feasibly at scale for human creation, basically.
So, RStudio is a great way to basically automate that kind of survey generation process.
So, I'm going to go to this brand here. So, this is a brand we work with and this is a film that we made. And we're basically going to select some people that we'd like to score the film. So, I'm going to pick myself. I'm going to pick my analyst engineer and my data scientist. And then this is all in Chinese. I'm going to press this button, which is going to send me a notification on Slack with a deadline for me to score this film.
Okay. And if I go to Slack, we'll see. And yes, this is Martin Scorsese photoshopped on top of Thor the God of Thunder. He gave me a message saying, hey, we'd like you to score this film here. And it's saying I can accept or decline. And it has a link to the type form. Which was just created with that API with the film that would like me to watch on every single tab. So, having a human actually go through and add this link to every single tab is just not doable. However, with the type form API and APIs as a whole, you can create that process right there.
And again, this is all, as I showed you in the earlier architecture diagram, there's really nothing else really involved besides external APIs and Chinese. And just to circle the loop a little bit, I'm going to press accept here. Because I'm a good employee. I'm going to accept it. I'm going to get a message back saying, thanks. I'm going to mark you down as accepting the score request. Please complete the survey as soon as you're able.
And so, the code really behind what I just did, the posting the request to the type form API and creating that survey, was really that code block there is a really important one. And it's just using your favorite paste function, either paste or glue or whatever. And basically just copying and splicing that Vimeo film link with the standard API request needed to generate that survey on type form.
Creating and deploying APIs with Posit Connect
All right. Now we're going to move to the third section. And then now we're going to show you guys creating and deploying APIs in Posit Connect, which I would put delightful. I think the way Posit Connect has created the infrastructure for deploying former APIs, it's click button easy. I just want to share that kind of joy.
So, first I'm going to go over this kind of one use case of receiving data. Just to kind of circle the loop on a score thing. So, as you guys saw, when I press that accept button, there's a Slack app function, which I can show you on this next slide, called interactivity. So, basically I put our API here with this hyper parameter of payload here. Very similar to what I showed you guys before. This is something I have deployed on Posit Connect. And basically when I clicked yes on that app in Slack, it basically ran that API request with the payload of he accepted, right? And then I just have our code running in the background to basically do all the things it needs to, update the metadata in BigQuery, and then also send that thank you message back. So, that's generally how it works. It's very simple, as you see down here. Just one function API, basically one function, one request API.
Before this said score response status none, but now it says accepted. Because I pressed the accept button on the app, it went to our API, and then it updated BigQuery, and it also sent another request to the Slack API to send me a message.
All right, but the main event, though, is actually showing how to deploy. Hey, Davis, these APIs are really cool. Can you show me how to deploy and make one? And absolutely. So, you see here, this is a plumber R script. For those who haven't made one, it's really easy to get started in RStudio. There's literally a plus plumber API. When you click that, it will just make a new one for you. It looks pretty similar to this. All you need to do is load in your libraries that you would use for this function. So, for me, it's plumber, LoomLighter, and then, as I mentioned before, we're a GCP stack on the backend. So, a big R query. And then, here, you just put your different functions you want for your API.
So, for here, I have two functions or two requests. So, first is a get request for my pizza rankings. And it's just like R functions. So, you just write your normal R function down here. So, here, it doesn't take any parameters, right? I just want to give this list of a data frame of pizza rankings. These also are all real pizza places in Durham. And this is my official ranking. So, definitely highly recommend Hutchins Garage for pizza.
And the second one, which is much more exciting is... And by the way, this is a really simple remedial example. But if you have external or internal technical stakeholders who need data that's in your data ecosystem consistently, this is a great way to share it. Because I can change the source of this. I can make this a BigQuery table. But as long as it's returning pizza rankings and restaurant and ranking here in some format, then it's all good. And my downstream users are not going to be impacted. So, that's the real importance of an API is that if you are sending people technical stakeholders, right? Because, yeah, the business stakeholder isn't going to be able to use an API. We're not trying to solve that problem. But if you're working with other technical teams that need data in your environment, this is the way to do it.
That's the real importance of an API is that if you are sending people technical stakeholders, right? Because, yeah, the business stakeholder isn't going to be able to use an API. We're not trying to solve that problem. But if you're working with other technical teams that need data in your environment, this is the way to do it.
So, I can try out this function here, my pizza rankings. I can see here if you run this request locally, you'll get this JSON with my pizza rankings. And then if you go down here, I can submit I'm going to submit one on behalf of my dog. His name's Theodore. North Carolina, one of the United States. And execute. And you'll see here you'll get a response saying thanks for your response. And the database will be updated with Theodore's response.
So, okay. That's great, David. You got the API working locally. It was really easy. But how do you get to Posit Connect? Just like you do anything else. Press the publish button. That's going to do all its package dependency magic. And it's going to deploy on Posit Connect. Presto. And it's live.
And on top of it, too, I made a little dashboard just in our markdown. So, I already submitted. If I refresh this, you're going to see a second submission. That was just my submission. Yeah. And you get the raw responses. You're going to see me and my dog.
And Davis, I was just curious. So, those links that we shared where it's the terminal and for R, is there a way to share that with somebody? Like, what's the best way to share that with someone that's not as technical? Honestly, I wouldn't share it with someone. That's a great question. I wouldn't share it with someone not as technical. Yeah. This is kind of a fun example of what you can do with APIs just conceptually. However, yeah, for the business stakeholders that I personally work with, I wouldn't have them working in terminal or R. I would be creating other ways like Shiny dashboards.
API authentication and security
Authentication is really important with APIs. You want to make sure that the right people can have access to your API, both for asking for data, but also posting data. So, one thing is that when you create packages for accessing APIs that require authentication, make sure to never hard code your API keys, because someone could take them and impersonate you and possibly do bad, evil things. So, store them in external files, like on your computer or something, or in your R profile as a global variable. This is a nice safeguard. And also, when you publish to Git and things like that, then your keys won't go there, which is also good, too.
And for those who don't know, the HTTR package has a default place for you to put your authorization code in. It's like a hyperparameter called authorization. And the standard format, generally, is bearer, and then your API key. Also, be careful about public APIs. So, for this one, it's all fun and games. But you don't want people uploading or sending things to your environment that you don't want. So, always try to do as much error handling if you have a public API. And also, if you can do a private API, do that.
Posit Connect makes it really easy to create authenticated APIs. All you have to do is, just like on a normal thing or document you publish on Posit Connect, switch the sharing settings to all users logon require. And then folks can just use an API key in Posit Connect to authenticate.
Q&A
Yeah, great question. So, I try to read the document. If I'm really, really stuck, I try to read the documentation as much as possible just to sort of see kind of the thought process going on. Another thing too is I really do think that the example I showed with making a table and unnesting, there have been very few times where I've run into like serious problems with that. And our internal GraphQL API is really, really complex and really, really nested. So, I think if you follow that formula, things should work out pretty well just with the unnesting wider and longer. And the third thing too is if you're using like a really common API, another solution kind of third door is we use Fivetree internally, which is a great way to just get your data into your data ecosystem pretty easily, just in a standard format.
I've come across with that just a little bit. So, there's a couple of things you can do. One, depending on the API you're pinging, pagination, I'm not sure how folks say, some people say pagination, some people say pagination, tomato, tomato. Pagination helps. So, if you can break up that query into smaller queries, even if the code is less efficient, so even like a dumb map or a dumb for loop of just going through everything just so it's smaller, that'll help a lot too. Also, if it is like a GraphQL API and it does have built in pagination, that's going to both help if you have internal users, their kind of compute, and also it's going to speed up your time as well.
And then as for the APIs that I deploy, I have run into that a couple times. It was actually interesting during the demo, normally the Slack API, when it receives calls, expects calls coming in in like I think a three second or like less interval, and it only comes around four or five seconds. And as a result, it gives like a little like warning, like, you're expecting a response. Another thing you can do too is increase the timeout, time on your API, so it's active more often. And so, that's kind of more awake and can take responses more frequently. However, you will increase your compute depending on where you're hosting your Posit Connect instance.
Another question over on Slido, I think was from PK, and I might need some help understanding what Bear tokens are. But the question is, how has your experience been creating APIs that use authentication, like Bear tokens? I think Bear were, I do like the concept of Bear tokens. Are they, I'm not sure, maybe are they actually called Bear tokens in some circles? I've never heard that term before. But yeah, as I mentioned before, with Posit Connect, it's really, really easy. Outside of Posit Connect, and with Posit Connect, as I mentioned before, it's just switching those sharing settings and sharing like the Posit Connect API key. I even did that just a couple weeks ago with an external technical stakeholder, and it could not have been easier. Outside of Posit Connect, to borrow the phrase, it's a bear. It's just, I've never had that full experience before, but just, I can't imagine like Docker late on time authentication, depending which platform you're using, the rules are much more strict and much more complicated.
Awesome. I'm going to share a link in the chat too, I think is a helpful example with code as well. Our solutions engineers created this example with bike prediction, but it shows how pins and plumber and Shiny and Vetiver are all tied together too. That's another really cool thing that we're actually, we're not fully built yet, but we're also using Vetiver and having our modeling be completely off the APIs. Our senior data scientist has been mostly working on this, where we're sending our data to the API and getting the response back and pushing to the query.
That's great. Let me jump back over to YouTube and see, there was a question from Lass, do you have any experience consuming streaming data with WebSocket protocol in R? No, actually, my experience with streaming is pretty limited, straight up. So no, I do not have that experience, but best of luck. Hopefully some other folks on the chat can help you.
I see Ralph just asked, when I query APIs, it's often hundreds of calls in a for loop. Any ways to use cleaner code than a big for loop? Yeah. Anytime you're using a for loop, you can probably be doing it in map. The catch with map sometimes, though, is that if you use a map underscore a certain format, you might get errors based off of your response not matching that format. So you start with just a raw map, as I showed you. I used that map DFR for that one example, but you can just swap that to a map for your API call function, and that's a bit cleaner than just a for loop. But also, I'm also a fairly big proponent of not for loop shaming. If it works, it works. For loops are readable, but map is generally the cleaner alternative.
Thank you. I'm going to throw a few more questions at you here. I see a few others coming in. I see Lisa had asked and also said, thank you so much, Davis. This helped decomplicate things a bit. And how did you become more comfortable with API documentation?
So that's a great question. A little bit of time. So a little bit of over time and space. I personally just find projects where I really understand the API, like what's asking. So I have a set personal project for my fantasy football league, where I'm pulling data from the ESPN API to get historical data and modeling for a fantasy football league. That way, when I see the different objects, I know generally what the web UI is like and generally have a better understanding of the subject matter. So I try not to get bogged down as much with the parts I don't know. That's a really great way to get more familiarity and more experience with APIs. So I'd say just, again, like all things, probably not the most helpful answer, but more experience and more practice and just getting interested in just doing side projects. Yeah, the main thing is just, I also got to say, too, doing it in RStudio and just being able to see the data right there in the list also is a really big help, too.
And it was what options do you have for API authentication besides Posit Connect? How do you integrate other auth systems with plumber? Yeah, I wish I had a better answer. I know the way at least I would do it for GCP is I would be looking at ways to get Google's OAuth integrated in there. So I would just start literally Googling like OAuth plumber. Yeah, but again, that's the thing, too, I mentioned before is a part of the reason I'm going to be honest here, part of the reason I do find this process delightful is because of having the Posit Connect API support. So, yeah, I wish I had a better answer. But I would start looking at Google OAuth if you're in GCP. Amazon, I'm not sure what it is. And you can also there's some really dumb ways you can do it where like a really dumb way to authenticate would to be have a parameter of your request to be authentication. And then probably not secure, probably a terrible idea. But depending on your use case, it might be okay.
But I don't one was are there specific packages to utilize APIs and Shiny, or just HTTR and Shiny reactivity? I am sure there are out there. I, but out, regardless, I was able to accomplish kind of everything, kind of shouldn't talk without those. Really, HTTR is the really fundamental one. There are probably other better ones out there. But but possibly.
But thank you all so much for spending time with us today. And spending with us at all the meetups the past year. I can't believe this is the last one of the year. Wishing you all happy holiday season and Happy New Year. We do have our last data science hangout of the year this Thursday. So there's one more community event. JJ layer, founder and CEO posit will be joining us this week. Thank you again, Davis for an awesome presentation. Thanks so much for having us and other one has a great holiday season. Bye, everybody. Thank you so much.
