Workflow Demo Live Q&A - May 29th

Awesome. Well, let's get started here, and people can join us as they do. Thank you all so much for joining us today, and thank you, Isabella, for such a great session. I'm going to pull Isabella and Charlotte here on stage with me.

As a reminder to everybody, we have these Workflow Demos the last Wednesday of every month, and they are all recorded, and so I'm just going to share the link in the chat with you right now. So if you want to add the monthly recurring demo to your calendar, you can use that. But we now have 14 – I can't believe we've been doing it this long now – 14 different workflows shared as recordings, from building a model annotation tool to pins workflows to deploying APIs, and you can see those all here in the playlist on YouTube as well I just shared in the chat.

But also just wanted to mention real quickly, if you are new to Connect or just curious to try it out after Isabella showed you everything that's possible, you can request a free evaluation from our team, and I just put the short link for that on the screen there, and I'll put it in the chat too.

And then before I jump into Q&A, I also wanted to let everybody know, if you're interested in learning more about Quarto, there are three Quarto workshops at PositConf this year in Seattle in August, and so there's one on building a dashboard, on introduction to Quarto, and on Quarto websites.

Okay, cool. I'll share the link to the workshops there too. But also, if you are thinking about joining us at the conference in Seattle in August, I want to also invite you to join us with the Posit team at next Thursday's data science hangout, and so at that hangout, we're going to answer any questions you may have about PositConf, from workshops to talks to advice for first timers, and it's also a great opportunity to meet other people ahead of time in the chat who will be there in person too.

Introductions and Q&A setup

For the Q&A room here, if you'd like to ask any questions anonymously, you can use the Slido link that I shared earlier, which I'll share again here. It's pos.it slash demo dash questions, or just feel free to type right into the YouTube chat.

And before we jump into the questions, I thought we should go around here and just say hello and introduce ourselves as well. I didn't do that yet. I'm Rachel Dempsey. I lead customer marketing here at Posit. I'm based in the Boston area, calling in from our office today. And I'm also joined by Charlotte, helping with the Q&A, and Isabella.

Hi, everyone. My name is Charlotte Wickham. I'm on the developer relations team at Posit, and I work really closely with Quarto and the Quarto team.

Hi, everyone. I'm Isabella Velazquez. I'm also on the developer relations team here at Posit. And thank you so much for watching. Looking forward to your questions.

PositConf virtual participation

So I pulled a few of the questions over from Slido already that we can get started. But while we just talked about conf, I thought this would be a good one to pull up first. Someone said, thanks for presenting, mentioning Posit Conf. It's a good hint to ask again when virtual participation will open.

I would say very, very soon. Keep an eye out on the Posit blog, and you'll see the announcement again very soon.

Quarto dashboards vs. Shiny apps

Awesome. Okay. One question we saw come in during the demo was, what suggestions do you have for developing a Quarto dashboard over a Shiny app?

Great question. I think it is kind of a balance of the simplicity of what you're looking for, what you hope that your audience can see and do, and then also the resources that you have available for actually publishing and sharing your content. I would say there's definitely a spectrum of very, very simple kind of quick prototypes all the way up to very complex, robust, scalable, interactive apps.

Quarto dashboards definitely can hit on that kind of like, if you need something quick and simple, and Shiny apps can become, again, more complex depending on your needs. One nice thing about Quarto dashboards is that you can add interactivity with Shiny. So if you're kind of looking for that middle ground of not doing a full-blown Shiny app, but you also need toggles and filtering and things like that, Quarto dashboards are still an option for you.

Yeah, I would just add that, you know, there's sort of this definite like deliberate consideration on both the Quarto and Shiny team side that it should be easy for you to move, right? If you start in a Quarto dashboard with your prototype, it's relatively simple to then add in some Shiny components. And if that gets to the point where it's just not doing what you need it to be able to do, then you can move up to a full-scale Shiny app.

Chaining dashboard refresh to data refresh

And one that someone just asked was, is there any way to chain the refresh of the dashboard to the refresh of the Python script? So use the presence of new data as a trigger.

Yeah, I think you could get really creative in terms of how you would like to chain things. Of course, today's demonstration was very simple, like schedule the script, schedule the dashboard. But PositConnect has various options for kind of like this automatic refresh, for example, Git-backed publishing. So anytime the PositConnect sees a change in a GitHub repository, it can automatically schedule that refresh of, say, the dashboard and things like that. Of course, there's various other tools and suites of tools for continuous integration. So my answer would be yes. But I think, like, as you want to make it a more complex kind of chain, it really depends on, again, the tools that are available to you and how exactly you want to do it.

Virtual environments in Python and R

And it was, can you say a bit more about creating a new virtual environment? Is this a Python thing? Would it be relevant to our own development?

I would say in my experience working with Python, virtual environments are very, very important and helpful. So they create this kind of isolated package of, like, your packages, your Python installation, and make it a lot easier for whenever you need to reload your project or you're sharing your project across environments, like if you're doing it locally or on POSIX Workbench, like you saw in the demo.

So, again, my understanding is there are tools that can do the similar sorts of tasks in R. One example is the R package, which creates kind of a list of the packages that you're using in a particular project, the version. And so that way, if you are working with other people on other projects, you can ensure that everybody has the right packages installed of the right version, things like that.

I would say in my experience, when I first started using R, I was working as a solitary data analyst, and I knew exactly what package versions I needed. And as things became more collaborative and more complex, as more people got brought into the team, then it became very important to make sure that everybody was aligned in terms of their packages and their package version.

One other thing to mention about Conf is that there is a workshop called What They Forgot to Teach You About R. And I actually took that workshop in Seattle back in 2018. And that's when I learned about RMV and how to kind of create these isolated environments where collaboration is a lot easier when you need it in R.

Permissions and row-level security

This one is, can you set different hierarchy of login details? So like head of service sees data set one, manager sees data set two, or employee sees data set three.

Yes, I believe so. You can create different kind of groups for permissions on Connect. So in the demo, for example, you saw anyone, specific users, or just me. And in that specific users, you could kind of create these different groupings depending on the permissions you need.

I think that ties into, there's another similar question that was, when using Quarto dashboards, can I have row level security? Or is it just a static interactive HTML, which all users would see the same information?

I'm not super familiar with the term row level security, but my general understanding, no, like the dashboard that you create from a Quarto dashboard is an HTML document. So the level of permissions you're going to be able to put on that sort of more controlled by where you're publishing it. You could imagine potentially publishing different versions of the dashboard to different endpoints on Connect and then controlling the permissions that way.

Reactive Quarto dashboards

One was, can we create reactive Quarto dashboards or do we need to move to Shiny? So, for example, if we want the value boxes to change dynamically.

Yeah, you can create interactive Quarto dashboards. The one in the demo today was just a static one, but you can start embedding pieces of Shiny in for when you want to, say, have a dropdown that filters a plot and things like that. And as mentioned earlier, as your project becomes more and more complex, it's an easy transition to a full Shiny app when you need it.

Comparing dashboarding tools

How do you compare or choose dashboarding tools among TyPy, Quarto dashboards, Streamlet, and Dash Plotly for production and scalability purpose?

Yeah, I mean, I think it just, again, comes back to what you have in terms of resources. One nice thing about Posit Connect is definitely Quarto dashboard, Streamlet, and Dash. You can publish all three of those on Posit Connect if you need. I also have not heard of TyPy, but I think it, again, just depends on the goals of the project that you want. And I think, yes, sort of like tool chain that you have readily available in terms of the dashboarding tool that you ultimately choose.

Advantages of using pins

What's the advantage of using a PIN first, just loading the data directly?

Well, I really like using PINs because, well, it enables to do that sort of refresh, as you saw on Connect. You PIN the PIN, and then that script that creates that PIN can be automatically refreshed. And because it's like the certain file format, it's very small, it's portable, and PIN also allows... There are a few functionalities that I actually didn't get into during the demo, but one of the things is like PINs has versioning. So say, like, I want to go back to a previous PIN, I can go ahead and call that specific PIN that I need and put it in whatever project that I want. I love how versatile PINs are. So PINs can be data, they can be models, they can be APIs. It's so, so nice in terms of having that flexibility, the things that you want to do.

Yeah, and then with that flexibility, it's like, as you saw, you can pull the data from an API, clean it up, and you never have to save the CSV somewhere. Remember where you put it, remember what version you were using. It's all on Connect. All, you know, labeled, documented version. I use PINs a lot in my projects.

And I guess I'd just add, like, just to make it really clear, like, you could have it in a CSV, right? But when you read in a CSV, there's often a lot of things you have to do to get the data exactly the way you want it. Like, make sure your dates are written as dates. If you've got factors, make sure they've got the right labels or have an order. Like, you don't have to worry about that with PINs. That's one of the advantages. And then the other thing I just mentioned is, if you did go to Shiny, you could have your Shiny app, as it's sort of set up, read from a PIN. And that way, it's always going to grab, like, the up-to-date data. If you've got that PIN on a schedule, you're going to grab the latest one.

And I guess I'd just add, like, just to make it really clear, like, you could have it in a CSV, right? But when you read in a CSV, there's often a lot of things you have to do to get the data exactly the way you want it. Like, you don't have to worry about that with PINs. That's one of the advantages.

Is there a Quarto book?

Somebody asked, when will there be a high-quality print book, such as those authored by Hadley, which covers this material and the use of Python in these Posit products? Is there a Quarto book?

There is an in-progress Quarto book. Unfortunately, it's not authored by Hadley Wickham, but it's authored by me and Mine Chinca Rundell. So hopefully we meet that bar. And, yeah, keep an eye. Like, it's in progress. We don't have a lot to show you right now, but we'll let you know.

Using the same pin across documents

But somebody had just asked, can you use the same pin through different Quarto documents?

Yeah, absolutely. And again, that's one of the great benefits of pins, too. Say you're working with someone else and you're like, hey, I want you to also be using the latest version of this data all the time. You can share your pin with them. And they like every time that they're running their project, they have the latest version from the pin. And every time you run your project, you're running the latest version from the pin. So you can use it in different places with different colleagues.

What's exciting about Quarto

What is something that you're really excited about with Quarto or maybe something that came out in the latest release or just something you're excited about for the future?

So my personal most excited thing about Quarto is the integration with TypeScript. So that's T-Y-P-S-T. That's like a new tool open source for generating PDFs. It's sort of like a LaTeX competitor. And as someone that's always struggled with LaTeX, it's that point in LaTeX where you need to dive into a style file like that. I don't really get it. I hate it. But TypeScript makes it so much easier. The language of types is just so much more familiar to someone like me that's coming from R. I expect the same from someone coming from Python. But it makes it so much easier to customize a PDF. So I think that's going to be really exciting for anyone that has to make PDFs and has to style them in a specific way. I think that integration is going to be a game changer.

It's an in-progress integration. So types are still maturing and developing. There are a few things that aren't quite there yet. And the Quarto integration with it is still developing. But I think that's a really exciting place to keep an eye on.

Yeah, I think one thing is just generally the styling in Quarto is so robust. And also there's a lot of new exciting things coming up for styling and branding. And I love just playing around with different variables and tweaking things to make it look exactly the way I like. And so really excited to continue doing that and then also use the new tools available in BSLib or other packages to make the Quarto document of your dreams.

Publishing Quarto dashboards without Posit Connect

Somebody asked, can you create and run a dashboard without PositConnect as a static HTML, for example?

Yes. So if you go to the Quarto website, Quarto.org, there is a section on publishing and you can see various options there. Yeah. And just to follow it, like if your dashboard doesn't have any shiny components, you can still have sort of widget components like Plotly plots or interactive tables. Those do just produce an HTML file. You can host that wherever you want. PositConnect is just a great way to schedule it.

Favorite Quarto examples

Do you have a favorite or a Quarto example that comes to mind that has been most impressive to you?

Yeah, I love this website. And usually when folks are like, oh, what kind of website can I create with Quarto? I point them to this one. It's World World Data Science. And I just think it's an amazing showcase of just how custom you can make your Quarto website look. And the creator of this website actually gave a talk in the last PositCon. So the video recording is on YouTube to hear about his journey from journalist to programmer.

Yeah, but one thing I've been doing with that. So I'm going to be teaching the Quarto websites workshop at comp. So I've been looking at a lot of examples of what people are doing with Quarto. And I'm still sort of amazed. One of the things that really amazes me is the things people can do with these things called listings. So a listing in a Quarto website is essentially a way to generate a page automatically from some content. And that content could be other Quarto files. It could be just some YAML, some structured data. And people just do really amazing things with these. The sort of classic thing is you create a blog. A blog is just a listing. But people do really cool things to showcase their projects or to list out all the talks they're giving.

It's like one of those Quarto features that you read about on the website and you're like, oh yeah, I get it. And then people have run with it and they've done some really, really beautiful, amazing things.

I love that too. I won't say that my example is beautiful. But I did use that listings functionality to build a data science community builder site. And so I was pulling together a bunch of resources from other community builders. And I found the listings site to be really helpful for that.

Well, I know we just hit the top of the hour here. So I just want to say thank you so much, Isabella, for an awesome demo. And thank you so much, Charlotte, for also joining us here for the Q&A. I'm really excited to read through everybody's comments and thoughts and see how you all use some of these workflows and see some of the dashboards that you all build as well.

But if you have any suggestions for us for these workflow sessions, please feel free to put them into the comments. Or if you ever want to just email me directly, I can put my email there in the chat as well.

Thank you all so much. Anything else to add, Isabella or Charlotte? I just want to say thank you. Thank you so much. Thank you all for the great questions. Hope to see you again at next month's workflow or at the data science Hangout, or last but not least, at the conference in August. Have a great rest of the day.