Gene-level pangenome analysis and the colorectal-cancer microbiome

Today I want to tell you about a recently published paper that I was a part of because it’s one that I’m really excited about, both for the findings and the methods. I’m going to start with the findings and then go back and explain how we got there.

The paper: Pangenome analysis reveals both niche-specific “specialists” and microbial “side hustlers” in colorectal cancer microbiomes. doi: 10.1080/19490976.2026.2728331

The biological question we were interested in was, “What is different about the bacteria present in the intestine (gut microbiome) of people with colorectal cancer (CRC) compared to people without this disease?” The motivation for this question is that we are looking for clues that will help us improve the state-of-the-art methods for preventing, detecting, and/or treating this disease. Of course, there are lots of unhelpful reasons why the gut microbiome could be different, such as the altered physiology of CRC making it easier for certain bacteria to grow, and so we think of these surveys as an initial step that can feed into controlled lab-based experiments. This question has been asked by many groups in lots of different ways over the years, and so what we did in this study was to apply a new data analysis approach to public datasets published by a handful of groups from around the world. Each of these datasets took considerable time, expense, and expertise to collect. We are grateful for the willingness of study participants to support scientific progress by making this contribution to the research community.

Previous work in this field has found that there are particular species of bacteria which tend to be found more commonly or abundantly in the gut microbiome of people with CRC, and other species which are found less commonly or abundantly. Other groups have identified differences in the prevalence of metabolic pathways shared across diverse organisms, and still others have measured differences down to the level of specific molecules. If I were to distill my understanding of the field into a vastly oversimplified soundbite, I would say that each study does find some difference in the CRC microbiome, but it is very hard to figure out from these observational studies which of these bacterial differences play a causal role and would therefore be the highest priority for developing preventions and therapeutics. For that, the observations from survey data must be transformed into falsifiable hypotheses which can be tested directly.

There has been groundbreaking work in the last few years that has identified specific bacteria and mechanisms which can reproducibly cause CRC in controlled experimental settings (animal models). While I cannot do justice to that field in this space, I will call out the work of Dr. Susan Bullman (MD Anderson Cancer Center) who has pioneered discoveries on a specific bacterium, Fusobacterium animalis, and its role in CRC. My thinking in this field was influenced greatly by a project in the Bulllman Lab that I was able to contribute some analysis tools to. In this work, Dr. Bullman’s group found that there was a previously-unrecognized subspecies of F. animalis which was far more carcinogenic than the other closely-related bacteria from the same species. In other words, they found that there was a biologically meaningful grouping of bacteria that did not correspond to any previously-recognized species or subspecies, and that group played a distinct role in causing human cancer.

In our recent paper, we expanded on our previous work and went looking for more cases like this, namely previously-unrecognized groups of bacteria hidden within species that had previously been identified as being associated in some way with CRC. We did end up finding some new examples that fit this pattern, with the top example being a lineage of Fusobacterium defined by the presence of a particular grouping of genes (an ‘operon’) encoding genes related to nutrient acquisition in low-glucose environments. We observed that this group of genes was present in a higher proportion of Fusobacterium genomes in the gut microbiome of participants with CRC, and lower proportions from healthy controls. This pattern was consistent across three independent studies performed in different parts of the world. While we have no data suggesting that this has identified any new cause of CRC, it would be possible to test that hypothesis by comparing specific examples of this bacterium with and without that group of genes.

Instead of listing lots of specific bacteria, I’d like to draw your attention to the way in which we ended up identifying them. The key identifying feature we used was whether those bacterial genomes encoded a particular set of genes. Bacteria tend to pass around groups of genes (horizontal gene transfer), and so it’s nice to track those genetic elements explicitly. More importantly, all of the results that we report from this survey are tied to groups of genes which are encoded by particular strains of a known species. The causal hypothesis can be tested in a very straightforward way – compare strains with those genes against strains lacking them.  The actual experimental testing isn’t straightforward at all, but the comparison (bacteria plus or minus certain genes) is the bedrock of molecular biology.

Lastly, let’s talk about how we analyzed the data. You’ll notice that there were multiple bacterial species and multiple microbiome surveys. The compute intensive steps (organizing the gene content for each species, aligning a single microbiome survey against a single species’ genes) are all completely isolated and modular. This is really nice because it means you can add a species or a survey to the analysis without having to redo any of the previous steps. All of the outputs only get pulled together in the final steps. The whole thing is performed using the gig-map suite of Nextflow workflows that I’ve been working on for a few years. Each step is configured to run within the Cirro Data Platform (which has now spun out of Fred Hutch, and is where I work), which makes it easy for non-computational scientists to kick off complex workflows. My hope is that others who want to use the same approach for their own projects will find this relatively easy to use.

If you’ve made it this far, I’ll leave you with a philosophical thought. Bacteria are incredibly diverse and constantly exchanging small pieces of their genomes. This gives us a huge opportunity to map associations of human disease down to individual genes and operons. Those genes and operons also happen to be the most experimentally tractable unit of information to work with. Getting that level of resolution in an animal model would require some immensely large number of generations on the scale of the Collaborative Cross, but we get it with bacteria for free. What this approach gives us is an incredibly efficient way of generating hypotheses that can be tested in the lab. If I had the funding and time to run a research lab, I would develop a meaningful model of an important disease (like CRC), fill up a few freezers with isolates of interesting bacteria, and then sequence all of their genomes. Every new microbiome survey that’s published would then just add more information to prioritize which genes to investigate, with the luxury of a complete inventory of which isolates contain the genes of interest. Of course, I’m not going to do that myself, but if you’d like to get started in your own lab I’d be more than happy to help!

Reactive Dashboards in the Browser (Without Writing JavaScript)

Things have gotten weird. With AI tools available, I find that I can build things I never really could before — web apps and tools in languages I never wanted to touch before. The range of what one person can put together has gotten a lot wider, and I think we’re all still figuring out how to adjust.

Speaking for myself, it used to be that building the thing was the barrier, and now I think the harder question is knowing what to build in the first place — where to actually point our prompts and tokens so that you end up with something useful instead of another half-baked idea.

If you're looking for somewhere to explore next, here's my suggestion: build a dashboard with Observable Framework.

I've spent a fair amount of time over the last year telling people about marimo, a reactive notebook for Python that I've become a bit of a broken record about. Observable is sort of the older relative of marimo — same reactive idea, but living in the web browser instead — and for the kind of AI-assisted tinkering I'm describing, it turns out to be a really good fit.

Compared to the languages I’m most comfortable with (Python and R), one of the things I like most about any JavaScript-based tool is just that the code runs in the browser. Everybody already has a browser, which sounds obvious but it's a big deal when you think about how hard it is to get someone else to run your fancy new tool using Python/R/Docker. When I want to share a result with a collaborator, I don't have to walk them through installing anything or ask which version of Python they happen to have — I can send them a link and it opens on their laptop or their phone and that's the end of it. For sharing scientific work, that ends up being really valuable.

What Do I Mean by "Reactive"?

Reactive — a style of programming where the system keeps track of which parts of your code depend on which other parts, and automatically re-runs the parts that need updating when something changes. You describe how things relate to each other, and it works out the order so you don't have to.

If you've spent most of your time in R or Python scripts, this takes a bit of getting used to. In a normal script/notebook the order matters a lot — line 40 needs to run after line 39, or things break. In a reactive setup you stop thinking about order. You just say that this plot depends on this slider, and then when the slider moves the plot updates on its own. You never have to lay out that dependency graph yourself; the runtime infers it while the code is running.

Marimo does the same thing over in Python. For me it's the part that makes both of them feel a bit different from normal coding — the relationships are inferred directly from the code and then you just watch it keep itself in sync.

Where the Idea Came From

I actually first heard about Observable back when I was getting into marimo. The marimo folks talked about having been inspired by it, but at the time it didn't really register. Observable meant writing JavaScript, and I wasn't confident enough to write much of anything in JS, so I sort of filed it away and never looked any closer. It's only more recently, with an assistant to lean on, that going back to the original felt like something I could actually do.

Under the hood they're doing a similar thing with that dependency graph. Where they really differ is in the language you write and in how you get the finished thing out to other people, and that difference matters more than you'd think.

Same Idea, Different Trade-offs

Marimo is Python, which makes it really fun to write if Python is already the language you think in (and if you’ve come up against the pain points of Jupyter). The tricky part is sharing it. Browsers don't run Python on their own, so to put a marimo app in front of someone you either have to run a server for it or bundle up the whole Python runtime and ship it into the browser with something like Pyodide. Both of those work fine, but they each have their own hassles.

Observable is JavaScript, so it just runs in the browser directly, with nothing sitting behind it. An Observable Framework project compiles down to ordinary static files — HTML, CSS, and JavaScript — that you can host just about anywhere, whether that's S3 or GitHub Pages or some cheap static host, with no server to keep running. I've spent an embarrassing amount of my career building static-site tooling to avoid running servers, so this is exactly the kind of thing I tend to get excited about.

So roughly speaking, marimo is easier to write and Observable is easier to share, and which of those matters more really just depends on what you're making and who needs to see it in the end.

Where the Data Comes In

For actual day-to-day work, though, the reactivity on its own isn't really the selling point for me. What I care about is what happens when you hook that reactivity up to a real dataset, and that's where a library called Mosaic comes in.

Mosaic wires your plots up to DuckDB, which can actually run right inside the browser through WebAssembly. So instead of dumping your entire dataset into JavaScript and making the browser chew through all of it, Mosaic hands the filtering and aggregation off to DuckDB and only draws whatever it needs at the moment. In practice that means you can cross-filter a chart sitting on top of a pretty large table and have it stay responsive, which would have been painfully slow in a browser not that long ago. When you drag a selection on one plot, the linked plots re-query the database and update, rather than slogging through a giant array loaded into memory.

Take Home: if you're trying to build a dashboard on top of data that's a bit too big to sit comfortably in a normal web page, the pairing of Observable's reactivity with Mosaic pushing the heavy lifting down to DuckDB is, in my opinion, a really nice way to go about it.

"But I Don't Write JavaScript"

For a lot of the people I talk to — scientists and analysts who mostly live in R or Python — the JavaScript thing has always been where the conversation ends. Observable is JavaScript, they don't write JavaScript, and so they never look any further. One of my closest collaborators tried to convince me to try JavaScript for years, and I always resisted.

And that's the barrier I think has mostly come down. A decent AI coding assistant is quite good at exactly this sort of thing — setting up a small web project, getting a plot working, fixing the layout, or just explaining what some error message actually means. Knowing JavaScript inside and out used to be the entry fee for any of this, and I don't really think it is anymore.

So we're back to that same question of what to actually build. If the building is mostly handled for you now, the more interesting problem is where to aim, and I think Observable is one of the better places to try. The documentation is good, and the whole thing happens to fit how an AI assistant likes to work, so you can lean on it for the JavaScript parts you don't know. You end up writing less code than you'd guess — a lot of it is just Markdown with small bits of JavaScript mixed in — and what you get at the end is a static site you can put anywhere.

Where to Start

If you want to actually try this out, here's roughly where I'd start:

  1. Skim through the Observable Framework docs. The getting-started walkthrough is pretty approachable.

  2. Have a look at one of the Mosaic examples so you can see the big-dataset cross-filtering working, and get a sense of what you're aiming at.

  3. Grab a small dataset of your own and ask your AI assistant to help you build a single interactive chart from it. Just start with one, get it working, and then add another once it does.

The point of a first attempt isn't really to get good at any of this. It's mostly just to get to the moment where you move some control on the page and everything else updates along with it, and you notice that you never actually had to connect those pieces together yourself. I've built a fair number of these by now, and that bit still gets me a little.

—-

If you've been curious about marimo, or about building for the web in general, or you're just tired of exporting static PNGs to drop into slides, I'd really encourage you to give Observable a try. The wall that used to keep the browser out of reach for people who don't write JavaScript is a lot lower now than I think most people realize. Your time and attention are appreciated, as always, dear reader — and if you do end up building something, please be in touch, I'd love to see it.

Publish Interactive Data Visualizations for Free with Python and Marimo

Working in data science, it can be hard to share insights from complex datasets using only static figures. All the facets that describe the shape and meaning of interesting data are not always captured in a handful of pre-generated figures. While we have powerful technologies available for presenting interactive figures — where a viewer can rotate, filter, zoom, and generally explore complex data  —  they always come with tradeoffs.

I recently wrote up an article for Towards Data Science presenting my experience using a recently released Python library — marimo — which opens up exciting new opportunities for publishing interactive visualizations across the entire field of data science.

Towards Data Science: Publish Interactive Data Visualizations for Free with Python and Marimo

Click on the image to open the app for yourself

Python-Based Data Viz (With No Installation Required)

In my work I’m constantly trying to find better ways to communicate the results of complex analyses to users, whether they are other scientists or the general public. As the tools for delivering sophisticated apps via the web have become better and better, it has even become possible for someone like me (who is not a web developer) to package up my Python code to be run directly in the web browser.

To try to help other people adopt this technique, I wrote a short tutorial which is now available on Towards Data Science:

https://towardsdatascience.com/python-based-data-viz-with-no-installation-required-aaf2358c881


Dispatches from the microbial frontier of cancer research

One of the aspects of my job that I enjoy the most is being able to support the stellar researchers working here at the Fred Hutch Cancer Center. My own personal goal is that my work analyzing data from the human gut microbiome will one day be used to improve the tools we have for preventing and treating cancer. As part of that effort, I have been collaborating with a brilliant physician-scientist, Dr. Neel Dey, who combines his clinical practice with a biomedical research program to identify the ways in with the microbiome influences colorectal cancer.

Our work together was recently featured in an article by Sabin Russell from the Fred Hutch press office, which I thought did a great job of capturing our recent advances.

Dispatches from the microbial frontier of cancer research



Microbial Pan-Genome Cartography

Many of the projects that I’ve been working on recently have led me to ask questions like, ”what bacteria encode this group of genes?” and “what genes are shared across this group of bacteria?” Even for a single species, the patterns of genes across genomes can be complex and beautiful!

Update: All of the maps shown below can now be viewed interactively here.

To make it easier to generate interactive displays to explore these patterns (which I call “genes-in-genomes maps”), I ended up building out a collection of tools which I would love for any researcher to use.

To explain a bit more about this topic, I recorded a short talk based on a presentation I gave at a local microbiome interest group.

In the presentation I walk through a handful of these pan-genome maps, which you can also download and explore for yourself using the links below.

I’m excited to think that this way of displaying microbial pan-genomes might be useful to researchers who are interested in exploring the diversity of the microbial world. If you’d like to talk more about using this approach in your research, please don’t hesitate to be in touch.

Looking Under the Lamppost - Reference Genomes in Metagenomics

In the world of research, a phrase that often comes up is “looking under the lamppost.” This refers to the fundamental bias in which we are more likely to pay attention to whatever is illuminated with our analytical methods, even though it might not be the most important. This phrase always comes into my mind when people are discussing how to best use reference genomes in microbial metagenomics, and I thought that a short explanation might be worthwhile.

The paper which prompted this thought was a recent preprint "OGUs enable effective, phylogeny-aware analysis of even shallow metagenome community structures” (https://www.biorxiv.org/content/10.1101/2021.04.04.438427v1). For those who aren’t familiar with the last author on this paper, Dr. Rob Knight is one of the most published and influential researchers in the microbiome field and is particularly known for being at the forefront of the field of 16S analysis with the widely-used QIIME software suite. In this paper, they describe a method for analyzing another type of data, whole-genome shotgun (WGS) metagenomics, which is based on the alignment of short genome fragments to a collection of reference genomes.

Rather than focus on this particular paper, I would rather spend my short time talking about the use of reference genomes in metagenomic analysis in general. There are many different methods which use reference genomes in different ways. This includes alignment-based approaches like the OGU method as well as k-mer based taxonomic classification, which is one of the most widely used approaches to WGS analysis. There are many bioinformatic advantages of using reference genomes, including the greatly increased speed of being able to analyze new data against a fixed collection of known organisms. The question really comes down to whether or not we are being misled by looking under the lamppost.

To circle back around and totally unpack the metaphor, the idea is that when we analyze the microbiome on the basis of a fixed set of reference genomes we are able to do a very good job of measuring the organisms that we have reference genomes for, but we have very little information about the organisms which are not present in that collection. Particular bioinformatics methods are influenced by this bias to different degrees depending on how they process the raw data, but the underlying principle is inescapable.

The question you might ask is, “is it a good idea for me to use a reference-based approach in my WGS analysis?” and like most good questions the answer is complex. In the end, it comes down to how well the organisms that matter for your biological system have been characterized by reference genome sequencing. Some organisms have been studied quite extensively, and the revolution in genome sequencing has resulted in massive numbers of microbial reference genomes in the public domain. In those cases, using reference genomes for those organisms will almost certainly give you higher quality results than performing your analysis de novo.

The harder question to answer is what organisms matter to your biological system. In many cases, for many diseases, we might think we have a very good idea of what organisms matter. However, one of the amazing benefits of WGS data is that it provides insight into all organisms in a specimen which contain genetic material. This is an incredibly broad pool of organisms to draw from, and we are constantly being surprised by how many organisms are present in the microbiome which we have not yet characterized or generated a reference genome for.

In the end, if you are a researcher who uses WGS data to understand complex microbial communities, my only recommendation is that you approach reference-based metagenomic analysis with an appreciation of the biases that they bring along with their efficiency and speed. As a biologist who studies the microbiome I think that there is a lot more diversity out there than we have characterized in the lab, and I am always excited to find out what lies at the furthest reaches from the lamppost.

What's Big about Small Proteins?

Something interesting has been happening in the world of microbiome research, and it’s all about small proteins.

What’s New?

There was a paper in my weekly roundup of microbiome publications which caught my eye:

Petruschke, H., Schori, C., Canzler, S. et al. Discovery of novel community-relevant small proteins in a simplified human intestinal microbiome. Microbiome 9, 55 (2021).

Reading through the abstract, the authors have “a particular focus on the discovery of novel small proteins with less than 100 amino acids.” While this may seem to be a relatively innocuous statement, I was very interested to see what they found because of some recent innovations in the computational approaches used to study the microbiome.

What’s the Context?

When people study the microbiome, they often only have access to the genome sequences of the bacteria which are present. This is very much the case for the type of metagenomic analysis which I focus on, as with any approach which takes advantage of the massive amounts of data which can be generated with genome sequencing instruments.

When analyzing bacterial genomes, we are able to predict what genes are contained in each genome using annotation tools designed for this purpose. The most commonly used tool for this task is Prokka, made by Torsten Seemann. Recently, researchers have started to realize that there are some bacterial proteins which were being missed by these types of approaches, since the experimental data used to build the predictive models did not include a whole collection of small proteins.

Then, in 2019 Dr. Ami Bhatt’s group at Stanford published a high-profile paper making the case that microbiome analyses were systematically omitting small bacterial proteins:

Sberro H, Fremin BJ, Zlitni S, Edfors F, Greenfield N, Snyder MP, Pavlopoulos GA, Kyrpides NC, Bhatt AS. Large-Scale Analyses of Human Microbiomes Reveal Thousands of Small, Novel Genes. Cell. 2019 Aug 22;178(5):1245-1259.e14. doi: 10.1016/j.cell.2019.07.016. Epub 2019 Aug 8. PMID: 31402174; PMCID: PMC6764417.

Around the same time, other groups were publishing studies which used other experimental approaches which supported the idea that bacteria encoded these small genes, which were also being transcribed and translated as bona fide proteins (a few quick examples).

What’s the Point?

The reason I think this story is worth mentioning is because it shines light on part the foundation of microbiome research. When we conduct a microbiome experiment, we can only make a limited number of measurements. We then do the best job we can to infer the biological features which are relevant to our experimental question. Part of the revolution of microbiome research from the last ten years has been the explosion of metagenomic data which is now available. This research is particularly interesting because it shows us how our analysis of that data may have been missing an entire class of genetic elements — genes which encode proteins less than 100 amino acids in length.

At the end of the day, the message is a positive one: with improved experimental techniques we can now generate more useful and accurate data from existing datasets. I am looking forward to seeing what we are able to find as the field continues to explore this new area of the microbiome!