How reliable is vaccination data for Sierra County?

by Debora Nicoll | October 15, 2021
8 min read
graphic satirizing data charts

Editor’s Note: The Sun’s story, “Who Enforces the Mask Mandate in New Mexico?”, published on Sept. 28, drew an inquiry from Sierra County Manager Charlene Webb. Debora Nicoll, the story’s author, had reported:

Sierra County’s vaccination rate falls far below that of the state as a whole. According to the CDC, as of Sept. 28, 52.2 percent of all eligible Sierra Countians were fully vaccinated. In the state as a whole, the comparable figure was 73.8 percent. 

Manager Webb wondered why the story cited the CDC’s data, rather than that reported by the New Mexico Department of Health, whose COVID-19 database recorded a significantly higher vaccination rate for Sierra County. “The DOH stats reflect a much more accurate picture of where Sierra County is regarding vaccination status,” Webb said in an email to Nicoll on Sept. 29. 

Aware that the CDC statistics she had cited were for Sierra County’s vaccination-eligible population, not for the county’s population as a whole, Nicoll wondered whether if that fact and other factors might have contributed to the discrepancies between the state and federal data. She embarked on an investigation whose results are reported here and call into question the comparability and reliability of both sets of COVID-19 data.

Local governments and individuals can make informed decisions on dealing with COVID-19 if given access to reliable, clear data on levels of vaccination and infections in their areas. The New Mexico Department of Health and the Center for Disease Control report significantly different numbers of COVID-19 vaccinations for the residents of Sierra County, making it difficult for local policy makers and the public to reach a consensus on how best to contain the spread of the virus.

Differences between state and national reporting of COVID-19 data are not confined to Sierra County. The Washington Post reported in a Sept. 30 article headlined “Messy, incomplete U.S. data hobbles pandemic response” that the “messiness” of coronavirus data at all government levels applies to reporting of infections, hospitalizations and deaths, among other variables that are needed to inform both national and local policy, as well as individual behaviors. “[S]cattered among local health departments, [the data] is often out of date and hard to aggregate at the national level,” the Post found, “and it is simply inadequate for the job of battling a highly transmissible and stealthy pathogen.”

Roll-out of vaccinations in Sierra County against COVID-19 began early this year for the most vulnerable individuals. Eligibility restrictions have slowly been reduced and currently anyone 12 or older may be vaccinated. Collection of vaccination data might be expected to be straightforward. For public release, New Mexico asks for three pieces of information from each person vaccinated: Is this your first or second shot; does it represent the completion of your vaccination series; and in which county do you live? 

According to NMDOH’s COVID-19 dashboard, as of Oct. 12, 6,233 residents of Sierra County who are 18 or older have obtained the complete vaccination series. This represents 66.7 percent of the county’s 18+ population, based on a population of 9,340 residents in this age range. NMDOH reports that 71.3 percent of New Mexico’s 18+ population has completed the vaccination series. 

However, the CDC’s COVID-19 data tracker reports (presumably for the same time period, although the date of the data is not clearly presented) that only 4,876 Sierra Countians who are 18 or older have been fully vaccinated. This, the CDC says, represents 53.6 percent of the county’s 18+ population. While not explicitly stated, the arithmetic tells us the CDC is using 9,087 as its metric of the population of the county that is age 18 or older.

Already, two discrepancies have been uncovered between the vaccination data NMDOH and CDC have published for Sierra County: the number of people age 18 or older who have been fully vaccinated, and the number of people age 18 or older who live in Sierra County.

It is not surprising that the two agencies have different population estimates for the county. NMDOH uses statistics from New Mexico’s Indicator-Based Information System, while the CDC uses 2019 Vintage Census Population Estimates. Thus, even if the NMDOH and the CDC vaccination data were the same, their calculations of the percentage of Sierra Countians who have been fully vaccinated would still vary because they use different counts of how many people live here.

Now, what are the reasons for the variance in the agencies’ reports of the absolute number of Sierra County residents 18 and older who have been fully vaccinated? NMDOH says that, as of Tuesday, that number is 6,233; the CDC says it is 4,876. The discrepancy is 1,357, a significant number for a county whose population is around 10,000.

In search of explanations, the Sun contacted both agencies. 

David Morgan, media and social manager at the NMDOH, replied to our email queries. When asked how NMDOH verifies vaccination status and patients’ personal information, including the county of residence, he explained that the providers of vaccinations are responsible for collecting that data and ensuring its accuracy. The providers enter this data into the New Mexico Statewide Immunization Information System. According to its website, NMSIIS is a “confidential and secure computer database designed to collect and maintain vaccination records of children and adults.”

Pharmacist Ron Golubski of Davis-Fleck United Pharmacy in Truth or Consequences confirmed that vaccination data is automatically downloaded to NMSIIS the night of its collection. Customers who have received a shot fill out a form that (among other things) asks for location data (city, state and zip). According to Golubski, customers are not required to present personal identification, meaning that, contrary to Morgan’s assertion, vaccine providers do not necessarily verify the county of residence of the person to whom the vaccine was administered.

NMDOH does not do any independent verification of the data it receives from NMSIIS. “We only determine the state data based on what is being reported to us by COVID-19 vaccine providers,” Morgan told the Sun. He nonetheless insisted: “[W]e are confident our numbers accurately reflect vaccination rates in Sierra County and statewide.”

If inaccuracies are introduced by provider-level reporting, those errors are passed along to other data collectors and users. “Public health departments nationwide routinely share data by county (i.e. number of full or partially vaccinated by county) as well as quantities of vaccine by manufacturer distributed across the state,” Morgan explained. “Information such as names and addresses of those vaccinated are not provided. Names and addresses are protected health information. We share only the date of birth of vaccinated persons.” 

Only this month did NMDOH begin to share the zip code of vaccinated individuals with other public health departments and agencies, Morgan later elaborated. 

Morgan provided the Sun with the raw vaccination data for Sierra County received by the department since vaccinations began early in the year, through Oct. 10. The supplied database, which clearly has not had duplicates removed, contains entries for 6,608 individuals. After subtracting entries for those under 18; those who have received a third dose (they were listed twice); and those for whom either no location data or location data that was not in Sierra County has been provided, the Sun arrived at a count of 5,961 Sierra County residents who have been fully vaccinated. This is several hundred people lower than the figure NMDOH reported on its COVID-19 dashboard.

NMDOH sends updates of its COVID-19 data to the CDC clearinghouse “every single day, including weekends and holidays,” according to Morgan. Before doing so, the department carries out “data quality efforts and cleanup including removing duplicates.” 

When asked why there is such a significant discrepancy between NMDOH and CDC vaccination data, Morgan could only speculate. “At the federal level,” he said, “we only see aggregate data, so we are not sure if there are duplicates or data clean up that needs to happen so the CDC is most likely over-reporting.”

However, if the NMDOH data are reasonably accurate, then the CDC appears to be under-reporting Sierra County vaccination rates. When the Sun pointed this out, Morgan said that the DOH is seeking input from the CDC to determine the discrepancy, but that a “timeframe for a response is difficult to pin down given their workload consists of every county in the country.”

In response to its own emailed inquiry to the CDC, the Sun received what appeared to be a boilerplate fact sheet. The CDC updates its vaccination data daily, the fact sheet stated, but the data it publicly reports on its database are typically 48 hours behind a state’s vaccination data reports. Health care providers are expected to report to local, state and federal authorities on doses administered within 72 hours of administration. Neither of these brief lag times explain the CDC’s possible under-reporting of Sierra County vaccination rates.

“Data will update as soon as they are reviewed and verified,” the CDC fact sheet stated, without explaining how those data are reviewed and what facts are verified. However, on the COVID-19 page of its website, in a section labelled “Historical Updates,” the CDC reports on corrections to its data. For example, the CDC acknowledges that, on June 6, 2021, the “total number of administered doses for New Mexico was incorrectly reported as 1,903,485 due to a data processing error, which has been amended. The correct total for NM on June 6, 2021, was 2,175,419 administered doses.” 

Despite claiming that its COVID-19 data tracker “allows for the exploration of standardized data across the county,” the CDC advises users to “visit the relevant health department website for the “most complete and up-to-date data for any particular county or state.” 

Sierra County is a sparsely populated county in a small state. It is not reassuring  that vaccination data cannot be gathered, verified and publicly presented in an apples-to-apples format by state and federal public health agencies for even such a tiny cohort as Sierra County residents. How can rational decisions be made about how best to protect Americans from the worst ravages of COVID-19 without trustworthy information? 

All that can be reliably said is, at best, about 6,000 Sierra Countians 18 or older have been fully vaccinated. At best, this is somewhere between of 57 and 67 percent of all county residents, depending on whether NMDOH or CDC population figures for the county are used. That leaves, at best, a third of our residents unvaccinated and unprotected, either by choice or necessity. The one clear conclusion is that we should continue to encourage people to get vaccinated and mask up indoors.

TAGS: | |
author

Debora Nicoll covers the Sierra County Commission for the Sun.

Share this:
HAVE YOU SEEN?

Understanding New Mexico's proposed new social studies standards for K-12 students

“The primary purpose of social studies is to help young people develop the ability to make informed and reasoned decisions for the public good as citizens of a culturally diverse, democratic society in an interdependent world.”
—National Council for the Social Studies 

Reader Michael L. Hayes of Las Cruces commented: What impresses me is that both the proposed standards and some of the criticisms of them are equally grotesque. I make this bold statement on the basis of my experience as a peripatetic high school and college English teacher for 45 years in many states with many students differing in race, religion, gender and socioeconomic background, and as a civic activist (PTA) in public education (My career, however, was as an independent consultant mainly in defense, energy and the environment.)

The proposed social studies standards are conceptually and instructionally flawed. For starters, a “performance standard” is not a standard at all; it is a task. Asking someone to explain something is not unlike asking someone to water the lawn. Nothing measures the performance, but without a measure, there is no standard. The teacher’s subjective judgment will be all that matters, and almost anything will count as satisfying a “performance standard,” even just trying. Students will be left to wonder “what is on the teacher’s mind?” or “have I sucked up enough.”

Four other quick criticisms of the performance standards. One, they are nearly unintelligible because they are written in jargon. PED’s use of jargon in a document intended for the public is worrisome. Bureaucrats often use jargon to confuse or conceal something uninformed, wrong or unworthy. As a result, most parents, some school board members and more than a few teachers do not understand them.

Two, the performance standards are so vague that they fail to define the education which teachers are supposed to teach, students are supposed to learn, and parents are supposed to understand. PED does not define words like “explain” or “describe” so that teachers can apply “standards” consistently and fairly. The standards do not indicate what teachers are supposed to know in order to teach or specify what students are supposed to learn. Supervisors cannot know whether teachers are teaching social studies well or poorly. The standards are so vague that the public, especially parents or guardians, cannot know the content of public education.

Three, many performance standards are simply unrealistic, especially at grade level. Under “Ethnic, Cultural and Identity Performance Standards”; then under “Diversity and Identity”; then under “Kindergarten,” one such standard is: “Identify how their family does things both the same as and different from how other people do things.” Do six-year-olds know how other people do things? Do they know whether these things are relevant to diversity and identity? Or another standard: “Describe their family history, culture, and past to current contributions of people in their main identity groups.” (A proficient writer would have hyphenated the compound adjective to avoid confusing the reader.) Do six-year-olds know so much about these things in relation to their “identity group”? Since teachers obviously do not teach them about these other people and have not taught them about these groups, why are these and similar items in the curriculum; or do teachers assign them to go home and collect this information?

Point four follows from “three”; some information relevant to some performance measures requires a disclosure of personal or family matters. The younger the students, the easier it is for teachers to invade their privacy and not only their privacy, but also the privacy of their parents or guardians, or neighbors, who may never be aware of these disclosures or not become aware of them until afterward. PED has no right to design a curriculum which requires teachers to ask students for information about themselves, parents or guardians, or neighbors, or puts teachers on the spot if the disclosures reveal criminal conduct. (Bill says Jeff’s father plays games in bed with his daughter. Lila says Angelo’s mother gives herself shots in the arm.) Since teacher-student communications have no legal protection to ensure privacy, those disclosures may become public accidentally or deliberately. The effect of these proposal standards is to turn New Mexico schools and teachers into investigative agents of the state and students into little informants or spies.

This PED proposal for social studies standards is a travesty of education despite its appeals to purportedly enlightened principles. It constitutes a clear and present danger to individual liberty and civil liberties. It should be repudiated; its development, investigated; its PED perpetrators, dismissed. No state curriculum should encourage or require the disclosure of private personal information.

I am equally outraged by the comments of some of T or C’s school board members: Christine LaFont and Julianne Stroup, two white Christian women, who belong to one of the larger minorities in America and assume white and Christian privileges. In different terms but for essentially the same reason, both oppose an education which includes lessons about historical events and trends, and social movements and developments, of other minorities. They object to the proposal for the new social studies standards because of its emphasis on individual and group identities not white or Christian. I am not going to reply with specific objections; they are too numerous and too pointed.

Ms. LaFont urges: “It’s better to address what’s similar with all Americans. It’s not good to differentiate.” Ms. Stroup adds: “Our country is not a racist country. We have to teach to respect each other. We have civil rights laws that protect everyone from discrimination. We need to teach civics, love and respect. We need to teach how to be color blind.”

Their desires for unity and homogeneity, and for mutual respect, are a contradiction and an impossibility. Aside from a shared citizenship, which implies acceptance of the Constitution, the rule of law and equality under the law, little else defines Americans. We are additionally defined by our race, religion, national origin, etc. So mutual respect requires individuals to respect others different from themselves. Disrespect desires blacks, Jews or Palestinians to assimilate or to suppress or conceal racial, religious or national origin aspects of their identity. The only people who want erasure of nonwhite, non-Christian, non-American origin aspects of identity are bigots. Ms. LaFont and Ms. Stroud want standards which, by stressing similarities and eliding differences, desire the erasure of such aspects. What they want will result in a social studies curriculum that enables white, Christian, native-born children to grow up to be bigots and all others to be their victims. This would be the academic equivalent of ethnic cleansing.

H.E.L.P.

This postmortem of a case involving a 75-year-old women who went missing from her home in Hillsboro last September sheds light on the bounds of law enforcement’s capacity to respond, especially in large rural jurisdictions such as Sierra County, and underscores the critical role the public, as well as concerned family and friends, can play in assisting a missing person’s search.

Reader Jane Debrott of Hillsboro commented: Thank you for your article on the tragic loss of Betsey. I am a resident of Hillsboro, a friend of Rick and Betsey, and a member of H.E.L.P. The thing that most distresses me now, is the emphasis on Rick’s mis-naming of the color of their car. I fear that this fact will cause Rick to feel that if he had only gotten the facts right, Betsey may have been rescued before it was too late. The incident was a series of unavoidable events, out of everyone’s control, and we will never know what place the correct color of her car may have had in the outcome. It breaks my heart to think that Rick has had one more thing added to his “what ifs” concerning this incident.

Diana Tittle responded: Dear Jane, the Sun undertook this investigation at the request of a Hillsboro resident concerned about the town’s inability to mount a prompt, coordinated response to the disappearance of a neighbor. From the beginning, I shared your concern about how our findings might affect Betsy’s family and friends. After I completed my research and began writing, I weighed each detail I eventually chose to include against my desire to cause no pain and the public’s right to know about the strengths and limitations of law enforcement’s response and the public’s need to know about how to be of meaningful assistance.

There was information I withheld about the state police investigation and the recovery. But I decided to include the issue of the car’s color because the individuals who spotted Betsy’s car emphasized how its color had been key to their identification of it as the vehicle described in Betsy’s Silver Alert. Because the misinformation was corrected within a couple of hours, I also included in this story the following editorial comment meant to put the error in perspective: “The fact that law enforcement throughout the state was on the lookout in the crucial early hours after Betsy’s disappearance for an elderly woman driving a “light blue” instead of a “silver” Accord would, in retrospect, likely not have changed the outcome of the search” [emphasis added].

I would also point to the story’s overarching conclusion about the inadvisability of assigning blame for what happened: “In this case, a perfect storm of unfortunate circumstances, many of them beyond human control, hindered the search that it would fall to Hamilton’s department to lead.”

It is my hope that any pain caused by my reporting will eventually be outweighed by its contribution to a better community understanding of what it will take in the future to mount a successful missing person’s search in rural Sierra County.

Scroll to Top