Liquid biopsy is all the rage now for cancer screening. But screening – testing apparently healthy people for a disease – is the Great White Whale of cancer diagnostics. I predict it will lose more venture capitalists more money over the next decade than every other Dx play combined. But that’s OK—redistributing money from wealthy but clueless investors to deserving scientists is definitely a good thing.
False-positives are the bane of any test that screens low-prevalence diseases. In the US, about 0.5% of the population is diagnosed with cancer in any particular year. That’s not much.
Now let’s imagine a test for cancer that is 99% specific. That is, for every 100 patients without the disease, the test returns only one false positive. Sounds pretty good, no?
But in fact it would be awful. If 1000 US persons were screened for cancer, we would expect 5 true positives (assuming the test was 100% sensitive, a very generous assumption). Those 5 patients would benefit from getting the care they need sooner. But we would also get 10 false positives—patients who are told they have cancer even though they are healthy. What happens with them?
We know for sure that they will be deeply frightened. Probably they will undergo more tests. But we don’t really have any tests that are good at ruling out cancer. There will be NO POINT at which they can be told “You’re OK, no need to worry” with any degree of certainty. They will then be forced to decide whether to wait-and-see until some other sign or symptom of cancer appears, or undergo therapy. Cancer therapy is always expensive and usually toxic, carrying its own risks of harm – such as increased risk of cancer. Not something to which you can ethically subject healthy persons.
That’s the general problem for cancer screening. Let’s see how the new hotness in cancer screening – liquid biopsy based on differential methylation of cell-free DNA – stacks up.
Here is the key figure in the paper:
These are receiver-operator curves (so-called because they were originally devised as a way to optimize filtering signal from noise in radio transmissions), a standard way of evaluating diagnostic tests. They visualize the tradeoff between sensitivity and specificity. A perfect test would be a right angle with the corner in the upper left. A perfectly useless test would follow the diagonal.
You’ll note that these plots are for different specific cancers, not “cancer”. We will return to this point. LUC is lung, AML is leukemia, and PDAC is pancreatic cancer.
Let’s assume we want the test to have 80% sensitivity (meaning that we are willing to accept that 20% of cancers will be missed). This doesn’t sound great (and it’s not – most FDA-approved diagnostics have ≥95% sensitivity), but it is much better than other blood DNA tests for cancer, which have 50–70% sensitivity (see here and here).
Now draw a line from 0.8 on the y-axis to where it intercepts the blue ROC curve. Read down to the x-axis. That’s the false-positive rate at 80% sensitivity. For lung cancer it is about 7%, for leukemia 2%, for pancreatic 20%.
At an overall cancer incidence of 0.5%, this would mean that there would be many false-positives for every true positive. But since these are tests for specific cancers, the false-positive:true positive ratio is far worse.
The incidence of lung cancer diagnosis in the US population is about 0.05%/year, that of AML about 0.004%/year, and pancreatic cancer 0.01%/year. The ratios of false- to true-positives in our scenario of 80% sensitivity will thus range from 100 to 2000.
So for every patient helped, at least a hundred would be frightened if not actually harmed by the results from this test. I don’t think anyone considers this to be remotely acceptable.
And it’s actually worse than this. This is not a single test, but a battery of tests, each with its own independent false-positive rate. If we tested for the 10 most prevalent cancers and had an average false-positive rate of 1% (much better than current performance), then fully 10% of patients would test positive for at least one cancer. What a disaster.
I don’t want to dump on the science here. It looks very promising, and I’m sure the technology will get better. But I do want to give you an appreciation of just how very very hard this problem is. It’s unlikely that any one test technology is adequate to the task. Maybe this approach, which looks at DNA methylation patterns, can be combined with the CancerSEEK test (which has much better performance) to start making screening of at-risk populations feasible.
I hope that happens. I am sure it is not happening very soon.