There are a slew of COVID19 antibody tests being released into the market from various manufacturers. These tests don’t look for current infection but rather are designed to tell whether you have antibodies in your blood specific to the SARS-CoV-2 virus. If you have these antibodies it means that you were infected and recovered, and now your immune system is primed to fight the virus in the future (meaning you may have immunity to future infection for some period of time – note, however, that this conclusion hasn’t been confirmed scientifically yet – see here for WHO statement on this).
This IFOD gets a bit wonky below but relates a concept that is very important to understand these antibody tests and all screening tests relating to false positives. This is very important because if you get a positive antibody test you may think you have immunity and may act differently than if the test was negative. So – understanding the math behind these tests is essential.
Screening Test Accuracy
It is important for medical screening tests to be accurate and the rarer the disease, the more accuracy is needed. Accuracy is a term of art in medical testing and has two parts:
Sensitivity “is the test’s ability to correctly designate a subject with the disease as positive. A highly sensitive test means that there are few false negative results; few actual cases are missed.” Source.
Specificity “is the test’s ability to correctly designate a subject without the disease as negative. A highly specific test means that there are few false positive results.” Source.
Here’s a bit of wisdom from Scientific American that applies to all medical screening tests: “when one is looking for something relatively rare, a positive result is very often false.” A quick example – if a COVID antibody test has a 95% specificity (or a 5% false-positive rate) and about 5% of the population actually has the antibodies, a positive test result means there is only a 50/50 chance you actually have the antibodies. Let’s see some examples using actual data.
COVID-19 antibody test manufacturers typically report the sensitivity and specificity of their tests. For example, a test with FDA emergency use authorization and is in current use is from the firm Cellex. The company reports the sensitivity of its test at 93.8% and specificity of 95.6%. That sounds pretty good – if students got these scores on tests then they’d get “A”s. Let’s see how this test shakes out in the real world. An example from the periodical The Conversation is as follows:
“Consider what would happen if the test were given to 10,000 people as in the diagram below. Although (estimates vary significantly), the WHO suggested recently that as few as 3% of the global population may have had COVID-19 and recovered. This means that 9,700 of the 10,000 tested will not have had the disease and only 300 will have. Of the 300 recovered patients, 93.8% – or 281 – will be correctly told they have antibodies against the disease. Of the vast majority (9,700) of people who haven’t had the disease, 4.4% – or 427 – will be incorrectly told that they have had the disease and recovered.” Here’s the diagram:
That means that if COVID-19 incidence is 3% in your community and you receive a positive test result from the Cellex test, there is only about a 40% chance that you actually have the potentially protective antibodies.
Note that as more of the population actually have the disease, the test is more useful. For example, if 40% of the population has had COVID-19, that means of 10,000 people tested about 4,000 people actually have the antibodies and 6,000 do not. Using the above testing accuracy numbers means that 3,752 of the 4,000 who actually have the antibodies will get a true positive and 248 will get a false negative. Of the 6,000 without antibodies, only 264 will get a false positive, and 5,736 will get a true negative. So, in this scenario, if you get a positive test result there is a 93.4% chance it’s correct.
The lesson: if you get an antibody test you should ask what the sensitivity and specificity of the test is and also figure out what the rough estimates are for the amount of infected and recovered people in your community. Sit down and do that math. What are the chances your results are correct?
Here’s a link from Johns Hopkins that lists the sensitivity and specificity of the COVID antibody tests on the market: Antibody Tests. The accuracy of the tests varies widely. Supposedly the new test from Abbott Labs has very high accuracy.
Want to read in more detail about antibody screening tests? Here’s a great article from Scientific American: What COVID-19 Antibody Tests Can and Can’t Tell Us
How about breast cancer screening?
What is true of COVID antibody tests is also true of all medical screening tests – the accuracy matters. Here’s another example that is all too real for so many women: false positive mammograms. There is about 0.7% incidence of breast cancer for women over 40 according to this study. Mammograms have a sensitivity of 87% and specificity of 89%. Such low accuracy combined with a less than 1% incidence means that the vast majority of positive mammograms are false.
Let’s do the math: If we round up the 0.7% to a 1% incidence of breast cancer that means that out of 10,000 women screened, about 100 will actually have it and 9,900 will not. Using the sensitivity of 87% for mammograms, of the 100 women with breast cancer, 87 will get a true positive result and 13 will get a false negative. Of the 9,900 without breast cancer, at an 89% specificity, 8,811 will get a true negative but an astounding 1,089 will get a false positive. What that means is that if you have a positive mammogram there is only a 7.4% chance that you actually have breast cancer based on this data (note that age and other risk factors can change your actual chances).
The Lesson
The lesson is that even very accurate tests have a lot of issues when trying to correctly identify a relatively rare disease. If you get a positive or negative result from a medical screening test you have to do the math and figure out what it means. Also, you may want to be wary/double check your doctor’s advice with respect to screening tests as a study found that the majority of doctors didn’t understand this math.
Thanks John – the article still does not answer that very basic question: has anyone gotten COVID-19 more than once?
@Peter, there are no confirmed repeat COVID-19 cases, but it’s too early to tell if recovering from COVID-19 confers longterm immunity. There have been people that tested positive, recovered/tested negative, then tested positive, but those may be subject to the reality of test sensitivity and specificity, as outlined above.
Immunity is a gradient effect. On one end, there are diseases like chickenpox that, if caught once, confer lifelong antibodies that prevent a recurrence; on the other end, if smitten by a common viral cold, we can expect future viral colds. We don’t know enough about COVID-19 to know where along the gradient the protective effects of SARS-CoV-2 antibodies lie. There is also the possibility that cross-immunity from previous exposure to a similar virus, e.g. HCov-OC43, might confer some degree of immunity that reduces the severity of the illness.
Immunity is affected by viral mutation rates.
SARS-CoV-2 is a single strand RNA and is therefore subject to a greater mutation rate than the herpes simplex virus, a double-stranded DNA, whose mutation rate is about four orders of magnitude slower. (SARS-CoV-2 has an estimated mutation rate of 0.80 – 2.38 × 10-3 nucleotide substitution per site per year, whereas herpes simplex virus (HSV) has an estimated mutation rate of 1 × 10−7 to 2.7 × 10−8 nucleotide substitutions per site per year.)
https://bmcevolbiol.biomedcentral.com/articles/10.1186/1471-2148-4-21
https://en.wikipedia.org/wiki/Varicella_zoster_virus
John,
The WHO states that there is “no evidence” that having antibodies to SARS-CoV-2 will protect you from getting the disease again. Isn’t there a ton of anecdotal evidence? There are over 1 million people worldwide who have recovered – how many of them have gotten sick a second time?
https://explaincovid.org/post/2EUafbV3iwhUlflPxIqX
Thanks for clarifying the math on testing. I’m sure we will see politicians and reporters misrepresenting the effectiveness of tests because it isn’t straight forward – you have to do some math!
Dan, thanks also for the article on vaccines.
Good point, Michael. Teasing truth from numbers is probably not the first aim of most politicians citing statistical results, and I agree that innumeracy is rampant in the political class (or so it seems).
Thank you for taking the time to do this. Here’s an informative article on a related topic, vaccines. https://blogs.scientificamerican.com/observations/can-we-really-develop-a-safe-effective-coronavirus-vaccine/
Good topic to cover and you did it clearly. What is not often talked about in the Covid antibody testing are the tests looking for IgM antibodies (the immediate response Ab) or the IgG Ab (which confers long lasting immunity). And the question is: how long lasting? I’ve read that some people who survived Spanish flu were tested at the end of the 20th century and still have Ab.