Visualization Design Lab

ReVISit 2 Paper Wins Best Paper Award at IEEE VIS

Mon, 10 Nov 2025 00:00:00 +0000

Running online visualization studies is now standard practice in VIS and HCI research. Yet the process remains fragmented: researchers stitch together survey tools, custom web code, logging scripts, analysis pipelines, and ad hoc debugging workflows. Users of reVISit already know this story: reVISit consolidates this ecosystem into a single open framework that supports the entire experiment life cycle – from design to dissemination.

To inform the academic community about the new developments in reVISit 2 – which we already described in these blog posts we wrote an academic paper about it.

Positioning reVISit in the Ecosystem

In the paper, we first situate reVISit among existing study platforms. We compare it to commercial survey systems, domain-specific research tools, and library-based frameworks. While survey platforms excel at rapid deployment, they rarely support sophisticated interaction logging or fine-grained experimental control. Academic tools often address specific domains or slices of the workflow, but lack long-term maintainability or broad adoption.

ReVISit 2 is designed differently: it treats experiment design as programmable infrastructure. A JSON-based domain-specific language (DSL) models sequences, blocks, counterbalancing strategies, interruptions, skip logic, and dynamic control flow. On top of that, reVISitPy provides Python bindings that allow researchers to generate complex study configurations directly from notebooks. The result is a framework that emphasizes expressiveness, reproducibility, and ownership over one’s experimental stack.

We also describe technical advances in reVISit 2, including first-class Vega support, automated provenance tracking, participant replay, and improved debugging tools such as the study browser. These features aim to tighten feedback loops during piloting while preserving transparency during dissemination.

Putting it to the Test: Replication Studies

To demonstrate that these capabilities are not merely architectural, we conducted a series of replication studies.

Each study highlights a different capability of the system. In one, we implement adaptive and staircase-style designs to evaluate visualizations of correlations using dynamic sequencing logic, showing how complex control flow can be expressed directly in the study configuration. In another, we integrate think-aloud protocols by embedding audio recording and transcription into browser-based experiments, allowing researchers to capture reasoning during interaction rather than only after the fact. Finally, we demonstrate provenance tracking and replay by instrumenting interactive visualizations to capture detailed interaction histories, enabling fine-grained participant replay and qualitative analysis. reVISit 2 also provides deep linking to specific trials or moments in user studies, to aid in dissemination and transparency. For example, this link takes you to the exact state you see in the above image.

Across these replications, we recruited hundreds of participants and reproduced key findings from prior visualization studies. Just as importantly, the studies surfaced practical lessons about counterbalancing, recruitment logistics, and ongoing challenges in deploying sophisticated designs online. The replication work serves as both validation and stress test for the framework, and as real live examples for studies (including the data) for the community to learn from.

Across these replications, we recruited hundreds of participants and reproduced key findings from prior visualization studies. Just as importantly, the studies surfaced practical lessons about counterbalancing, recruitment logistics, and the realities of deploying sophisticated designs online. Together, they function both as validation and as a stress test of the framework. They also remain publicly accessible – complete with study configurations and data – serving as concrete, real-world examples that the community can inspect, reuse, and learn from.

What Users Told Us

We also interviewed experienced reVISit users to better understand how the system performs in practice. Several themes emerged:

Tighter development loops. Users appreciated the integrated development environment and the study browser, which makes it possible to jump directly to specific trials without stepping through an entire study.
Expressiveness over convenience. While the DSL requires programming knowledge, users valued the flexibility it affords – especially for mixed designs and adaptive sequencing.
Learning curve trade-offs. ReVISit inherits complexity from modern web tooling (e.g., React, TypeScript). This can be a barrier for less technical researchers, but it also enables deeper customization and extensibility.
Open infrastructure. The ability to fork studies, inspect core code, and maintain version stability was frequently cited as a strength, particularly for reproducibility.

Overall, feedback confirmed that reVISit is most effective for technically oriented research teams who need more than a survey builder.

Recognition at IEEE VIS

We were very honored to receive an IEEE VIS Best Paper Award for this work. Zach Cutler presented the paper on the main stage in Vienna.

This recognition reflects years of iterative development, community feedback, tutorials, documentation work, and, most importantly, the researchers who have trusted ReVISit in their own studies.

ReVISit 2 is not the endpoint. It is infrastructure for a research community that increasingly relies on sophisticated, reproducible, browser-based experiments. We look forward to continuing to build it together.

Learning with Hands: Making Complex Chart Types Accessible

Fri, 08 Aug 2025 06:00:00 +0000

Tactile Charts as Templates for Chart Learning

Tactile charts—charts you can explore with your hands by touch—are not new. They have long been used to represent data, helping BLV readers grasp spatial patterns and relationships. But they have limitations: they take time and resources to produce, can’t easily be updated, and are often harder to access compared to digital alternatives such as alt text.

So why do we use them? Because when it comes to learning a chart type—building that mental model that helps future interpretations—touch can be a powerful sense. And unlike data-specific tactile charts, template charts don’t require frequent updates as they are learning tools, not meant to communicate a specific dataset.

Working closely with our blind collaborators and following an iterative design process, we created educational tactile template charts for four complex chart types frequently found in scientific publications: UpSet plots, clustered heatmaps, violin plots, and faceted line charts (inspired by genome browsers). Each design is accompanied by digital exploration instructions to support independent learning. You can see our final tactile model designs and accompanying exploration instructions on our accessible website.

Do Tactile Charts Help?

We tested our designs of violin plot and clustered heatmap in an interview study with 12 BLV participants. Each participant learned two advanced chart types—one using our Tactile+Text method (a tactile chart with a textual description) and one with Text-Only. Afterwards, we gave them new datasets described only with alt text to see if they could apply what they learned to new datasets differently between the conditions.

The results were striking. Participants not only preferred the hands-on learning experience of tactile charts, but also reported that tactile charts helped them build a much clearer mental model of the chart types they learned. Tactile charts allow them to “picture” layouts and chart element shapes more vividly. Several described the experience as like being able to see the chart. Participants also use tactile charts as a reference in their minds to interpret new alt text descriptions more confidently.

Why Does This Matter?

In our interviews, participants spoke openly about the barriers BLV individuals face in visualization education and communication, and their desire for equal access to information. They saw tactile template charts as a way to bridge that gap, offering a more effective and inclusive approach to visualization education.

In a world increasingly driven by data, data literacy means a lot, such as job opportunities or the ability for equal information acquisition. By using tactile template charts as educational tools, we can equip BLV learners with transferable knowledge—knowledge that carries over to new contexts. Our hope is to empower BLV individuals to be better able to interpret unfamiliar visualizations and to participate fully in discussions with sighted collaborators in academic, professional, and civic discussions where data plays a central role.

We make our models available on the project website, so that anyone with access to a 3D printer can make their own template charts.

Sometimes, the best way to learn a chart is to feel it.

This work was generously supported by the Chan Zuckerberg Initiative.

VDL is Moving to Austria

Wed, 30 Jul 2025 06:00:00 +0000

After an incredible decade at the University of Utah, the Visualization Design Lab (VDL) is entering a new chapter: the lab is partially relocating to Graz University of Technology (TU Graz) in Austria. This will be a gradual transition: only Alex will be moving, while the rest of the team remains in Utah. For the foreseeable future, the lab will operate across two continents.

VDL’s new home in Austria will be the Institute of Human-Centred Computing at TU Graz.

Alex will be hiring a PhD student and a Postdoctoral Fellow in Graz. Check out the positions page for details.

A Brief History of the Visualization Design Lab

The Visualization Design Lab was founded when Alex joined the University of Utah and teamed up with Miriah in 2015. From then until 2021, Miriah and Alex co-directed the lab, leading numerous projects and mentoring many students.

In 2021, Miriah moved to Linköping University in Sweden, where she founded the Visualization and Interaction Design Group.

In 2025, Alex returned to his alma mater in Graz to continue VDL’s research mission at TU Graz.

This website will reflect the new phase of the lab in Austria, while continuing to preserve the history, people, and publications from the Utah chapter. For archival purposes, the final version of the Utah VDL site will remain available at
https://vdl.sci.utah.edu.

Making Data Visualizations Talk: Accessible Text Descriptions for UpSet Plots

Sat, 07 Jun 2025 06:00:00 +0000

Scientists love data. In fact, we love data so much that when you open up any journal article, chances are many of the pages are scattered with data visualizations. That is not just because we like pretty pictures (although we do!). Data visualizations are a powerful tool to show trends or patterns in the numbers that may be difficult to explain in words alone.

But what if you can’t see the data visualization?

Most charts are entirely inaccessible to people who are blind or have low vision. These users often rely on screen readers to read the contents on a computer screen aloud. However, those tools usually come up short regarding data visualizations. Instead of describing the chart, they might just say something like “Image of Figure 1.” That leaves the figure caption as the only source of information; yet captions almost never explain what the data is actually showing. Imagine trying to understand a chart with a title and maybe a vague summary. That is the reality for many blind readers.

One chart type where this really matters are UpSet plots, shown in the figure above. It is a popular way to visualize how different sets overlap. Think of it as a more organized way to show set overlaps than a traditional Venn diagram. These plots are used across disciplines, especially in computer science, bioinformatics, computational biology… practically any data-heavy field. But UpSet plots are visually complex and nearly impossible to interpret without sight.

That is where our work comes in. We developed a system that automatically generates screen- reader-accessible text descriptions for UpSet plots. But not just generic summaries like “there are bars and dots.” These descriptions highlight key structural features of the plot, including the largest intersections, distribution of elements, and other patterns typically interpreted visually. The goal is to give screen-reader users meaningful access to the same insights a sighted reader would glean from the visualization. It basically provides the plot with a narrative voice.

What makes our system unique? This is not just a one-off tool for a specific dataset. We analyzed 80 UpSet plots from published papers and identified recurring data patterns, such as skewed intersection sizes or dominant sets that shaped the narrative focus of our text descriptions. These insights informed a modular, rule-based pipeline that produces both brief summaries and extended descriptions. As shown in the annotated example below, the generated text incorporates multiple layers of information, from basic chart elements to statistical summaries.

But do these text descriptions actually work?

To find out, we conducted in-depth interviews with 11 blind or low-vision screen reader users. And these were not quick usability tests! They were in-depth conversations where participants walked through the descriptions, shared what made sense and what did not, and suggested changes that made the description stronger.

These interviews revealed valuable insights into how experienced screen-reader users navigate complex text. For example, our participants emphasized the importance of structure, recommending using bullet points to help navigate longer descriptions. They also highlighted the need for high-level takeaways before discussing fine-grained details.

After this process, Maggie McCracken, one of the lead authors of our study, realized how powerful participation itself could be. As a psychology researcher, she has worked with many participant groups, but this one stood out for how genuinely enthusiastic they were. They were not just excited about the tool but about the fact that this type of research was happening at all. Several told us they were entirely used to being left out of data visualizations and that being part of the process was meaningful. And that says a lot, considering the data they were reading was about COVID symptoms, which is not exactly edge-of-your-seat material.

So yes, making charts more accessible helps blind readers. But more than that, it improves how we all think about communicating data.

If you are curious, you can try it out for yourself at https://upset.multinet.app. You can also upload your own data and generate descriptions instantly.

This project serves as a reminder that accessibility is not just a technical challenge. It is about asking who gets to be included in scientific conversations. Accessibility is not just a bonus feature but part of that conversation. Charts included.

This work was generously supported by the Chan Zuckerberg Initiative.

Max Lisnic Successfully Defends Dissertation

Thu, 15 May 2025 11:00:00 +0000

Max Lisnic successfully defended his dissertation on “Designing Resilient Visualizations Toward More Accurate Data Discourse”. Max was co-advised by Marina Kogan and Alex Lex. The committee was completed by Kate Isaacs, Vineet Pandey, and Crystal Lee.

Max will join WPI as an assistant professor this fall!

Congrats, and good luck with your next steps!

Reading Between the Lines: The US Computer Science Graduate Admission Process

Wed, 09 Apr 2025 06:00:00 +0000

The time between submitting your application and hearing back can be stressful, in particular if you don’t hear back for a long time. This post is meant to help you read between the line, so that you can judge what a non-response from a grad program means. I’ll talk mostly about PhD admissions, but sprinkle in some tidbits for MS admissions. I’ve previously written about what you can do to get into grad school.

It’s March or April. I haven’t heard back at all. Is there still a chance that I will get accepted?

To be blunt: it’s unlikely. US PhD programs often have three piles: definite admits, students on the waitlist, and rejects.

Unfortunately, only the definite admits will hear about a decision early, everybody else is kept in purgatory for a while. Our CS program informs definitely admitted candidates as early as late January or the first half of February. Domestic students are invited to visit campus in mid-February; international students are invited to participate in a virtual event at the same time.

So, if you haven’t heard back from a PhD program by the end of February, it means that you’re not in the “definitely admit” pile. There might still be hope though: you might be on the waitlist.

For MS students, the timeline is usually a bit longer: you should expect to hear in February or March.

Candidates on the waitlist are meant to fill slots that free up as students in the “definite admit” pile reject an offer of admission. Waitlisted candidates are considered promising students, but ranked slightly below others. Not every school has a waitlist; Utah only introduced one a few years ago. You might be told that you’re on a waitlist, or you might not hear back at all. Sometimes, waitlisted candidates may even hear back the latest: definite accepts go out first, rejects go out next, but wait-listed candidates may be in purgatory for a while. Whether a slot opens up when you’re on the waitlist is difficult to predict – it typically depends on whether other candidates who are vying for a slot in a particular area / with a particular professor accept or reject an offer.

Most schools don’t have a waitlist for MS students.

Can I e-mail admissions to learn more about whether I’ve been accepted?

Typically, an e-mail to a generic admissions contact or the director of graduate studies will not help you learn about your status. We usually don’t respond to these inquiries because we wait for official channels to give you the news instead.

Can I e-mail the professor I want to work with to learn whether I’ve been accepted?

You have better chances (though still slim ones) to get a response about your status from a professor you’ve interacted with, or who you listed in your letter as a potential advisor. Generally, individual faculty have a lot of influence over the admissions process. If they want to work with you (and have the money / time), you’re almost certainly going to be admitted.

This doesn’t hold for MS students: since most MS students typically focus and coursework and don’t interact closely with individual faculty members, it’s unlikely that a particular professor will know anything about your status, and professors (outside of the admissions committee) also don’t intervene on MS admissions decisions.

I’ve been rejected by a program. Can I ask for feedback on how to improve my application?

You typically won’t get individualized responses to an inquiry about what you can do to improve your application. That’s partially because it would take a lot of effort to respond (Utah’s CS program gets more than 1000 grad applications), and partially because people worry that you might use any advice in litigation against them. If you’d like feedback on your application, you should reach out to your letter writers who know you personally.

I’ve been accepted by a program, but haven’t heard back from my ‘dream school’ yet. When do I have to make a decision?

All major grad schools have agreed to use the same deadline by which students must accept an offer of financial support (that usually comes with a PhD offer): April 15. It’s strongly advisable to accept any offer you may have by that deadline. You might still be admitted if you respond later, but there are no guarantees anymore. Typically an offer letter lists a specific deadline.

We (at Utah) try to give offers to students on waitlist sooner, as we learn from rejections by other students.

I’ve accepted an offer but now I received a better one. Can I walk my acceptance back?

It’s not great but it happens. No one can force you to attend a PhD program, so yes, go ahead and rescind your acceptance, but do it as soon as possible.

I’ve been accepted to multiple programs. Should I decline offers I won’t take immediately?

Yes! The sooner the better. You might give a waitlisted student a chance to get admitted.

Professors keep ignoring my e-mail. What can I do?

I’m sorry, it sucks to be ghosted. But most professors delete a lot of e-mail from prospective students. Here’s how to get a response:

Write only targeted e-mail to people you want to really work with. Don’t mass e-mail, we can spot mass e-mails (even if you paste a paper title of ours).
Keep it short and to the point. Five sentences at most.
Don’t chat-gpt us. We don’t care for a wall of very well written but empty text.
Don’t get our research area wrong. This will almost certainly get your e-mail deleted.

Reflections on UpSet

Wed, 16 Oct 2024 10:00:00 +0000

UpSet has won the 10-year Test of Time Award at IEEE VIS. I’m deeply honored by this award, and I want to thank the committee for choosing our paper. But most importantly, I want to thank my co-authors Nils Gehlenborg, Hendrik Strobelt, Romain Vuillemot, and my PostDoc advisor Hanspeter Pfister!

As of October 2024 the UpSet paper has been cited 1900 times. Together with Jake Conway and Nils Gehlenborg, I also wrote a short follow-up paper introducing an R version of UpSet, which has been cited 2500 times. So clearly, those papers have hit a nerve – UpSet is now the de-facto standard for visualizing set data with more than 3 sets. Also, UpSet has been re-implemented many times, making it available on various platforms and in many programming languages, which certainly contributed significantly to its success.

In this post, I want to first give a brief history on how UpSet came to be and acknowledge the giants on whose shoulders we stood, and then share my thoughts on what made UpSet successful.

How UpSet Came to Be

The intention of UpSet plots was to solve the problem of set visualizations for more than three sets. Very specifically, it was inspired by the now infamous six-set banana
Venn diagram.

This chart was widely ridiculed on Twitter back then. When I saw this, I figured that we should be able to create a better visualization for set data than that. So I did some research and found various set vis methods, including radial sets. But the method that I liked the most was a response to a “Vennerable Challenge” for Autism data by Robert Kosara:

I liked that this chart uses a simple bar chart instead of the irregular shapes that Venn diagrams use, but I struggled a bit with parsing the tree – it always had to trace paths to the root and I couldn’t spot any trends with regards to the set intersection patterns.

Also, this chart doesn’t actually plot intersection sizes as bars but a different attribute (% of correct diagnostic tests), so it’s actually visualizing something very different.

Still, this chart got me thinking, and shortly before Christmas 2013, while I was a PostDoc in Hanspeter Pfister’s group at Harvard, I sketched this in my notebook:

As you can see, all of the principal ideas of UpSet are already in that sketch: the matrix layout for sets, the bar charts for set and intersection sizes, even advanced metrics such as “deviations from expected intersection sizes” and grouping and aggregation.

I pitched the idea to my office-mate at Harvard, Nicolas Bonneel (who is a computer graphics researcher), and he immediately got the idea, but was not very excited because it “was so simple”.

I went home for Christmas and kept thinking about it and decided to give it a go for VIS. I also wanted to do this on the web; I had only done Java-based visualizations up to this point, but there was this hot new thing called D3 that I wanted to take out for a spin. So I started out and pretty quickly got to a first prototype that you can see here:

At this point I realized that I’ll need help to pull this off, especially with only three months to go till the conference submission deadline. My advisor, Hanspeter Pfister, was supportive, so we assembled the “dream team” consisting of my PostDocs friends that were also doing visualization research at Harvard: Nils Gehlenborg (at the Harvard Medical School), Hendrik Strobelt (who had just joined the lab that month), and Romain Vuillemot (at the Harvard Kennedy School).

Everyone on this team was an experienced visualization researcher, and magically, everyone found the time to really pour their heart into the project. We had many lively discussions on our wall-spanning whiteboard, and lots of heated arguments about UpSet features in its early days.
We were able to pull the UpSet paper off in just three months – from idea, to design and refinement, to evaluation, to writeup – and you can still admire the result here: https://vcg.github.io/upset/

As you can see this is actually a fairly complicated interactive visualization system. While the basic design remained the same, everyone contributed immensely to make it all come together. It took a million design decisions to actually make this work. We also used interaction extensively to enable users to answer a variety of questions. The system includes supplementary visualizations, queries, aggregations of intersections, attribute visualizations, and so on.

We made the deadline and properly celebrated the submission!

The paper received mostly strong reviews on its first submission to InfoVis.
Of course, the dreaded “Reviewer 2” found that “The proposed software and its associated methodology does not go beyond what this area has been providing as standard for the last 15 years.” Nevertheless, the other reviews were sufficiently positive, and we were happy to present it that year in Paris at the 2014 Visualization conference.

What Made UpSet Successful?

So, what made UpSet more successful than some of my other visualization papers? I think it was a multitude of factors. Some of them might be unique to this project, but others may be relevant as a lesson also for other visualization projects.

UpSet Solved a Real and Pressing Need

There really wasn’t a good way to visualize set intersections of four, five or even more sets. Existing Venn or Euler diagram solutions just don’t work, it gets too complicated to understand what sets are involved in an intersection, and area-proportionality is incredibly hard for Venn diagrams with more than three sets. And especially in the biomedical domain we saw a lot of bizarre many-set Venn diagrams:

Other solutions required interaction, or were difficult to interpret. In contrast, UpSet plots are at least somewhat self-explanatory and solve the problem.

UpSet Met Users Where They Are

If we had just stopped with our UpSet implementation where we were with our InfoVis paper, I doubt that it would have been a big success. I’m sure people in our community would have appreciated it, and might even have built extensions based on our open source code. But we realized that for broad adoption, we have to meet users where they are, and we have to communicate with them.

So we set out to build the R version of UpSet – UpSetR. The idea was that the biomedical community is already working with R for their data visualization needs. Uploading data to our web tool and then taking a screenshot would have yanked them from their workflow, and would have resulted in a “dead-end” plot that they’d have to manually adapt if their data changed. An R version would make it simple to just integrate UpSet plots in their current workflow, significantly lowering the barrier to its use.

We also promoted the UpSet, and particularly the R version in two ways: we wrote a short two-page “Applications Note” in a widely read bioinformatics journal. There was no new scientific content (as judged by the VIS community) in that article, it was just a simplified re-implementation of the tool in R.

And then we also wrote a how-to guide style article for a then popular series: Nature Method’s “Points of View”. In that article we talk about set visualization in general: for example, we recommend using Venn diagrams for two or three sets, and matrices for very large numbers of sets. But we also point to UpSet plots for the “middle ground” of 4–10 sets.

At a later point, we also set up UpSet.app to document the general use of UpSet and published a Wikipedia article on UpSet. It was a small victory when the Wikipedia article was approved for inclusion by the Wikipedia admins instead of being deleted for “lack of notability”.

A Simple Idea is More Impactful than a Complicated System

While our IEEE InfoVis submission was a complicated system with sophisticated interaction. Our team, and our reviewers, are visualization researchers, and we like to think through all of the things that we can do, and how we can leverage interactions and have cool animations.

However, the version of UpSet that really took off is not our web-based version, but the more basic one that is implemented in UpSetR:

While some features, such as sorting made it into UpSetR, others didn’t. That basic idea is simple enough, and people started using and citing it in their papers. Soon we saw other versions of UpSet pop up, for example in Python, or even a different (better) R version. At this point we know of 13 re-implementations of UpSet – so clearly, the idea transcended the implementation.

While I do think software tools and libraries add tremendous value to our community, and my team invests a lot of energy in building and maintaining software, it is certainly nice to see that an idea takes off, not at least because you don’t have to maintain the software yourself!

UpSet is Useful for Communication, Not Only Discovery

Finally, UpSet hits the sweet-spot between being useful for discovery / exploration, but also for communication. Like scatter plots or histograms, UpSet can be used in the discovery process, but is still simple enough so that they can be used as a figure in scientific articles.

Nils often complained to me that he believes that a lot of complicated visualizations don’t get “credit” for the parts they play in the discovery process, but rather give ideas that are later tested using statistical methods or simpler plots; while the visualization tool that lead to the insight doesn’t get credit (for example, in the form of a citation).

This isn’t the case of UpSet plots, which are easy for scientists to generate, given the many implementations. And unlike our web-based tools, the static Python and R tools generate “paper-ready” figures, without UI elements that need to be edited out or other issues that make it difficult to include a figure. It probably also helped that we clearly instructed users on how to properly cite UpSet.

The Future of UpSet

While I’m certain that the reason for UpSet’s success is its simplicity, I still haven’t abandoned the interactive UpSet visualization system. In fact, we’ve just released UpSet 2.0 – a version that has the same analytical features as the original UpSet, but supports data upload, public sharing of interactive UpSet plots, and can be integrated in other tools as a React component. It also has a full provenance history (including undo/redo). And it’s enabling new research: for example, we are now generating (and evaluating) text descriptions for UpSet plots, so that they can become accessible to blind and low-vision users.

In closing: thanks again to my co-authors and friends for helping bring this project to life, and to the community for this recognition.

Lessons Learned from Visualizing Multimodal Data... with Aardvarks...

Mon, 30 Sep 2024 14:00:00 +0000

Understanding cancer is the key to fighting it.

Cancer is a terrible disease caused by your cells growing out of control. If we can understand exactly how cancer cells grow, move, and divide, we can develop strategies to prevent, diagnose, and treat cancer.

A fundamental way to further our understanding is to collect data to represent how cancer cells are growing. It turns out that this is pretty hard because…

Cancer is complex — we need complex ✨ multimodal ✨ data to represent it.

What is ✨ multimodal ✨ data? For us, it is data in different formats that represent different aspects of the same phenomenon. Specifically, we work with 🌌 images, 🌳 trees, and 📈 time-series data.

How is this data collected?

The datasets we worked with are from live-cell microscopy imaging; in other words, 🌌 images of cancer cells are recorded over time as those cells grow and divide.

Algorithms can track individual cells over time based on their position and other characteristics, such as size.

Then, derived attributes based on these images can be computed, such as the cell size or the amount of a specific protein in a cell. These attributes are calculated over time, resulting in 📈 time-series data.

During these experiments cells might divide into two daughter cells. If we record these divisions, we can construct a 🌳 tree of cell relationships or cell lineage.

Now that we have the data, we need to try to understand it. It turns out that…

Understanding multimodal data requires us to think about all the modalities together… and this is hard.

Each of the data modalities (🌳🌌📈) is necessary because they capture a different aspect of how cancer cells develop:

The 🌌 images show the spatial relationships between cells.
The 📈 time-series data shows how cells grow and change over time.
The 🌳 tree captures how cell attributes propagate across generations.

But one thing that quickly became clear was that to fully understand the phenomenon of interest (the spread of cancer cells), we needed to synthesize all three of these modalities together.

Right now, researchers are synthesizing or combining data modalities manually. In other words, they look at an image, then at time-series data, then back at an image, then at a tree, and mentally link data elements together. In the best case, this is tedious and taxing. In the worst case, it is impossible to relate elements from one modality to another.

So now what? Well, now is the part where the visualization nerds get to cheer and applaud as the hero of this story 🦸‍♀️ visualizations 🦸‍♂️ get to swoop in and save the day! This is because…

🍢 Composite visualizations 🍢 can show different data modalities together!

What are 🍢 composite visualizations🍢? In short, composite visualizations combine multiple visualizations together into the same view. (If you want a longer/better answer, Javed and Elmqvist do a great job “Exploring the design space of composite visualization.”)

This is great for our purposes because we have three different data modalities that are each represented best by different visualizations, and we are trying to link the elements across these modalities.

What’s the catch? Even though they are powerful…

Designing composite visualizations is… (you guessed it) …hard. Here’s our approach in three simple steps.

Selected a primary data modality. Even though all data modalities are needed, specific tasks prioritize certain modalities.
Choose the best visual encoding for it. This visualization serves as the host representation.
Embed secondary data modalities as client visualizations.

Of course, it’s a bit more complicated than this, but we’ll get to that. First, let’s give an example.

Here’s one of the composite visualizations we designed (this one is my 🤗 favorite 🤗).

First, we select the tree as the primary data type and, by extension, name this composite visualization the tree-first visualization.
We encode this tree as a node-link diagram.
Then, we embed the time-series data by nesting it within the nodes of the tree and superimpose the images of cells above the nodes either automatically or on demand.

Here are the other two composite visualizations!

We call these the time-series-first visualization and the image-first visualization.

What we learned.

Ok, I have a confession. These three simple steps for constructing composite visualizations are hiding a lot of complexity. Specifically, step three is carrying a lot of weight here. In reality, this single step is a much more iterative process.

The design space of composite visualizations can be intimidatingly large. You have to choose a visual encoding for each data type AND how those encodings get combined together. The choice of one affects the other, and there will be some back and forth while exploring this space.

That said, we found that the first two steps help anchor this exploration and reduce the design space. Selecting a primary data type and prioritizing a good encoding for that data type will help ensure that tasks for that primary data type can be done effectively. With that settled, figuring out how the other pieces fit into the puzzle becomes much more manageable.

We also made a tool!

We didn’t just design these visualizations; we implemented them into a tool! It’s an excellent tool! Our collaborators like it! I like it! You can try it out yourself! You can watch a video that demonstrates it!

If you’re an extreme visualization nerd, you can even read our research paper!! The paper also goes into more detail about the design and theory pieces I talked about in this blog.

I could talk about it more, but this blog is already long enough, and I’m tired of writing it, and you are probably tired of reading it. But I promise I am actually quite proud of the tool.

Are you still here? That must mean one of two things…

You are wondering what the deal is with the aardvarks.

Well, the research paper I mentioned is titled “Aardvark: Composite Visualizations of Trees, Time-Series, and Images”. Aardvark is the name for the tool we made.

But why aardvarks?? Is it some really awesome acronym?? I bet the “v” stands for visualization!!

Sorry to burst your bubble, but it is not an acronym, I just like naming my research tools after animals (see Ferret and Loon). As to why I picked aardvarks, I think they are a cool, weird animal. 🤷

…or you want to listen to us brag about our
✨🏆✨ ~ award ~ ✨🏆✨!

We are incredibly honored and excited to share that the research paper has received a Best Paper Award at IEEE VIS 2024.

I am amped up and just a touch terrified to present to the whole conference this year in Florida! I hope to see you there! 🏖️

Ok, actually three things; you want to see the “blooper” images.

And yes, I made all of the aardvark images with AI. Specifically Adobe Firefly. These monstrosities are the result of asking for an image of an aardvark, loon, and ferret…

Devin Lange Successfully Defends Dissertation

Mon, 29 Jul 2024 11:00:00 +0000

Devin Lange successfully defended his dissertation on “Is that Right? Data Visualizations for Scientific Quality Control”. Devin was advised by Alex Lex, with Kate Isaacs, Paul Rosen, Hanspeter Pfister, and Nils Gehlenborg serving on the committee.

Devin will join Nils Gehlenborg’s Lab at Harvard Medical School as a PostDoc, while also spending some more time with us as a PostDoc. Congrats, and good luck with your next steps!

reVISit: Taking Control of Your Online Studies!

Thu, 20 Jun 2024 01:00:00 +0000

You might have heard of reVISit before from our paper, or you might have seen a talk or participated in a meetup. But as of today, we’re excited to give the community a chance to run your own studies with reVISit with our 1.0 release – and CHI is just around the corner!

What is reVISit?

ReVISit is a software framework that enables you to assemble experimental stimuli and survey questions into an online user study. One of the biggest time-saving features of ReVISit is a JSON grammar, the reVISit Spec, used to describe the setup of your study. Stimuli are contained in components and can be either markdown, images, web pages, React components, or survey questions. The figure at the top shows the relationship of the reVISit Spec, the components, and how they are then compiled into a study.

Due to the different types of components, you can use reVISit for a diverse set of studies, spanning simple surveys, image-based perceptual experiments, and experiments evaluating complex interactive visualizations.

ReVISit is designed to accommodate sophisticated stimuli and study designs. Suppose you want to replicate the seminal Cleveland and McGill study. With reVISit you could implement a React-based set of visualizations (a bar chart, a stacked bar chart, a pie chart), and then pass parameters, such as the data, and the markers to highlight the marks, all via the study configuration.

Similarly, the reVISit spec enables designers to create controlled sequences defining in which order participants see stimuli. reVISit supports fixed, random, and Latin square designs that can be nested at various levels. For example, the overall study sequence (intro, training, experiment, survey) could be fixed. In the experiment arm, two conditions could use a Latin-square design. Within each condition, the experiment could randomly draw a small number of stimuli from a large stimuli pool while interspersing attention checks at random points and adding breaks.

Assembling and Deploying your Study

The components and your study configuration are then used to assemble a web-based study. You can first look at your study on your local browser, and if you want to share it deploy it to the web server of your choice. We recommend and document deploying to GitHub pages, but any web server you have access to will do.

You can then use the online version to direct participants to your study. You can use crowdsourcing platforms such as Prolific, Mechanical Turk or LabintheWild, or you can simply send a link to participants that you have recruited in other ways.

Data Collection

A typical study will have response fields, such as a text field or a slider, to provide the response. Such form-based responses are tracked by reVISit by default and can be downloaded in either JSON or a tidy tabular format. Similarly, you can provide response data out of interactive stimuli. For example, if a task is to click on a specific bar in a bar chart, you can log which bars were clicked. ReVISit tracks a diverse set of browser window events such as mouse moves, clicks, scrolls, resizes, which are time-stamped and can hence be used for basic log file analysis.

ReVISit also supports advanced provenance tracking based on trrack a provenance tracking library developed at our lab. If you instrument your study stimuli with trrack, you can recreate every state of your interface of every single participant! This can be incredibly useful to understand nuances of user behavior, as well as to help you debug your stimuli by exploring what went wrong in a particular session. In a future release, reVISit will also allow you to dynamically browse these events and fully “re-hydrate” all participants experiments.

Data Storage

ReVISit is implemented as a (mostly) server-less application, meaning that you don’t have to run, secure, and maintain a server to use reVISit. The only exception to this is data storage, as obviously, the data of online participants has to be stored somewhere.

If you’re running a local study, you can get away without this – you can just download the data from your browser after a study is complete. For online studies, we use Google Firebase to store data.

Currently, setting up Firebase for a reVISit study might be the most challenging part of working with reVISit. On the plus side, Firebase is a tried-and-true system where you have full control over your data. You even have options to choose the locale of your server so that you are compliant with your country’s regulations on data storage.

Data Analysis

ReVISit is not meant to replace your usual data analysis approaches. Instead, it aims to make it easy to export data in the formats you might use in R, Python, or your analysis platform of choice.

ReVISit, however, does provide a basic analytics interface that is most useful for monitoring the progress of your study. You can also use reVISit to identify participants that didn’t appropriately complete the study and reject them, which is most useful if you want to ensure that you have appropriate numbers of participants in your Latin square design.

What are the Benefits of Using reVISit?

So, why would you use reVISit over other approaches to running your study, such as Qualtrics, Survey Monkey, or even a custom experiment interface?

First, reVISit is open source with all the benefits you have of using open source software: it’s free; you can extend it; you can contribute to improving it.

Second, the open source nature and our approach of forking reVISit for your own study and storing your data in your own Firebase means that you have full control over your study and the data. Once you have forked the study, it will remain accessible and unchanged for as long as you like.

Third, reVISit has dedicated modes for quickly navigating your study, and you can also turn off data collection. This is great for both, developing your study, but also sharing your study with reviewers and readers of your research. That means that readers can see exactly what your participants saw, and hence may trust your study more. They could also fork your study and run a replication of your study with minimal effort! You can check out an example study and the associated results.

I’m Intrigued, but Can I Trust it for my Experiment?

reVISit is new, and we know that it’s fraught to bet on a new project if you don’t know whether it actually works or whether it will be maintained down the line. But we hope we can convince you to trust us!

First, we currently have multiple years of funding to continue development of reVISit. We’ve also ourselves run several successful studies, such as a study on guardrails against misinformation. Finally, we are committed to help you out if you run into issues! Join our slack team to get low-friction help, or write to us at contact@revisit.dev. We’re also happy to set up a meeting to answer any questions you may have; for example, to talk us through whether reVISit will work for your study design.

How Can I Learn More or Get Involved?

We’re grateful to all the community members who have shared their study needs and helped to make ReVISit 1.0 a reality, and we’re looking forward to bringing the community exciting new features in the coming year. Future releases will include better debugging tools through study rehydration, a way to capture and code think-aloud data, and improved analysis capabilities. Depending on community feedback we’re also interested in branching out to unconventional display devices (phones, AR/VR, etc.)

To take your first steps with reVISit, check out our getting started guide for instructions on how to install our software and to build a study.

Finally, if you are missing a feature or find a bug, let us know! Since reVISit is completely open source you could even submit a pull request!

Acknowledgements

We are very grateful to everyone who helped make reVISit a reality, including our wonderful community advisory board and the National Science Foundation for generous funding.