By Edward Copeland
The inimitable Samuel Clemens, better known to readers everywhere as Mark Twain, said many memorable things in his lifetime, but I've always been partial to his definition of the three types of lies. In the unlikely event that you're unfamiliar with Twain's list, his troika consisted of lies, damn lies and statistics. That statement came to my mind quite often as I watched the epic documentary project that HBO begins airing tonight, The Weight of the Nation, concerning the "obesity epidemic" in the United States. The series marks a collaboration between HBO and the Institute of Medicine in association with the Centers for Disease Control and Prevention and the National Institutes of Health and airs in four parts over the next two nights. Additionally, three episodes for families have been set for the
next three Wednesdays. The project, which also includes a dozen bonus shorts, a web site, a social media campaign and a book, all produced in partnership with The Michael and Susan Dell Foundation and Kaiser Permanente, a huge "not-for-profit" health care consortium whose tentacles integrate both the insurance business with physician-owned hospitals. I'll give most involved in this massive undertaking the benefit of the doubt that they mean well (especially HBO, which will allow free viewing of the program across all its platforms such as HBO.com, HBO GO, etc.) and truly fret over this "epidemic" that threatens the health of the nation far into the future unless we change our habits. Having spent nearly a full decade trapped in a Kafkaesque journey through this nation's health care system, I look skeptically upon most medical things, so while some of the series' points make logical, common sense, at the same time documentaries that don't even attempt to provide alternative points of views — if only to knock them down — trouble me. At that point, the label documentary seems less appropriate and I feel the outer letters of the word propaganda trying to crawl into my brain via my ear canal. It doesn't help in this day and age when simple computer archive searches turn up contradictory facts and statistics coming from the same organizations touted as authorities in this series. Now, you could easily presume that I've watched this ambitious venture with a jaundiced eye and a biased mind and you might be right. However, if those capable of rational thought have been paying attention to the world around them — whether that lifetime spans decades or just the first steps in a cognitive journey — having and honing that trait of suspicion probably serves a person better today than at any recent time since the media's morphing into little more than dictation machines, regurgitating as "facts" anything sources hand them, without vetting the information first. Fact checkin' — they don't need no stinkin' fact checkin'. Part One: "Consequences" debuts tonight on all HBO channels and services, including free streaming on HBO.com at 8 p.m. Eastern and Pacific/7 p.m. Central followed by Part Two: "Choices" at 9:10 p.m. Eastern and Pacific/8:10 p.m. Central. Now, don't misunderstand my intentions — this review of the first two parts of The Weight of the Nation today and parts three and four on Tuesday will not end with me endorsing a diet of Big Macs and intravenous Dr Pepper. I'm not against living a healthy lifestyle or exercise — I know I'd be walking if I could — but it's important to remember that the people who assembled this project have an agenda. It might be a noble agenda, but it remains an agenda, one in which anything that might contradict the message they want to convey won't be heard. That's what bothers me. I'm not an expert in the field, so I can't thoroughly debunk and verify everything, but I'll do what I can because in the time since I first got sick, before anyone diagnosed my multiple sclerosis and the years that followed, I've experienced a lot of the health care system first-hand. As a result, I know that the biggest mistake that all health care providers as well as this documentary series make occurs when they treat every patient as if he or she comes from a cookie cutter. That's why no two sets of fingerprints are alike. No two people match up exactly and the attempt to generalize can lead to delays in correct diagnoses, misdiagnosis and risks to people's lives. Neither a country nor an individual's health can be managed by rote and, to some extent, that message underlies The Weight of the Nation. When I began having my symptoms, which started with difficulty walking, my excess poundage became the prime suspect and my primary doctor referred me to a cardiologist. Keep in mind that my blood pressure always runs in the low to normal range, my EKG results tested fine and my blood sugar level gave no indication of diabetes. He referred me to a dietitian. I changed my diet, exercised, lost 100 pounds — and the legs got worse. It took a total of two years and several doctors before we found the right neurologist who diagnosed the M.S. — and a previous doctor still tried to get me to return to re-take a test with him that had proved negative when I'd already fired him. Before I get too far off track, I do want everyone to remember this: NEVER BE AFRAID TO FIRE YOUR DOCTOR. Remember, they are the employee, you are the employer. Most of them need a swift kick in the ego now and then anyway. The whole reason I began this part of the discussion had to do with a 1988 New York Times Magazine article by Robin Marantz Henig I found about doctors rethinking the treatment of mild cases of hypertension. It wasn't so much the content of the article that I found particularly relevant, but I loved the old saying she attributes to some physicians discussing the issue because it fits here as well: "If you give a kid a hammer, every problem starts to look like a nail."
To start my review of the series' actual content on a positive note, I must admit that the behind=the-scenes team that worked on The Weight of the Nation assembled a slick, attractive product. In fact, they implement so many charts and graphics, it's as if the original USA Today had come to life. "Consequences" serves, more or less, as an overview of issues that later installments explore in more depth.
It rattles off statistics that its various experts will reiterate time and again, which does make parts of the project redundant and repetitive at times. Out of the gate, part one did teach me something I didn't know: that a man named Adolphe Quetelet, a Belgian mathematician, astronomer, sociologist and statistician who invented the infamous Body Mass Index formula in 1835. Quetelet grew notorious for applying probability laws to social issues and generally discounted the concept of free will, alienating him from some scientists. His formula, amazingly still used today, takes a person's weight (in kilograms) and divides it by the square of their height (in meters). A person who registers a BMI of 18.5-24.9 would be considered healthy, 25-29.9 get categorized as overweight and 30 or higher earns the label obese. Only, not everyone thinks that relying on an index created 177 years ago by someone who figured chance into his equations (which isn't done today) might be outdated — especially when the formula fails to distinguish between fatty and lean tissue. This issue isn't raised in The Weight of the Nation. This raises a sore subject for the CDC because it went apeshit when two separate studies not only questioned the BMI but the idea that all overweight people suffer the same health risks as everyone else came out at the same time as the CDC planned the first Weight of the Nation conference in 2009 (What probably galled them the most was that one study found publication in The Journal of the American Medical Association). ROTE DIAGNOSIS OR BUST! Press articles on the studies and the BMI proliferated."Consequences" stays very busy tossing out its statistics at the viewer. According to the collective weight (pun intended) of its gathered experts, more than 68 percent of American adults fit the definition of being overweight or obese. Now, neither the documentary nor the experts who appear within it offer up data to show how exactly they calculated this figure. The U.S. finds it hard to get rock-solid Census figures every 10 years and the form fails to inquired as to the weight of every member of a household (as if it could be confirmed with
100 percent accuracy that the person answering the questions knows the truth or tells it if they do). Not everyone visits a doctor regularly — you might remember a couple of years ago a huge political fracas over trying to get affordable health insurance for the citizens of this country — so they can't go by that. Besides, I believe there happens to be something called doctor-patient confidentiality, so your physician better not be unloading information to just anyone unless you give permission. Good grief, I've had to sign enough forms verifying that I'm familiar with HIPAA. All those signatures damn well better not have been for naught. So — where did this 68 percent figure originate? I began with everyone's favorite free search engine — Google. (In reality, I'd love access to Lexis/Nexis, but they cater to corporate clients and since I live on Social Security Disability, $200 a month would be a steep price for me to pay.) After typing in "68 percent of Americans are overweight or obese" and winnowing out the NPRs and other news stories who just repeated the statistic without asking any questions as to how anyone arrived at it, I located a site that identified the source of the number. On the site of The National Heart, Lung and Blood Institute, which is a division of the National Institutes of Health (one of the partners in making The Weight of the Nation), itself a division of the Department of Health and Human Services, Google led me to a page labeled Why Obesity is a Health Problem. Under the heading Adult Obesity Rates and Statistics, it read, "According to the National Health and Nutrition Examination Surveys (NHANES) (2007-2008), approximately 68 percent of adults are overweight or obese, with 75 million adult Americans considered obese." Now, we're getting somewhere. We know it came from surveys conducted four and five years ago. Now, I just have to find that survey. Isn't this fun? It's like being a private eye, only in public. This page also offers the statistic I'm going
to come back to later beneath the heading Childhood Obesity Rates and Statistics where it says, "Children have become heavier as well. In the past 30 years, the prevalence of childhood obesity has more than doubled among children ages 2-5, has tripled among youth ages 6-11, and has more than tripled among adolescents ages 12-19. However, recent data suggest that the rate of overweight in children did not increase significantly between 1999 and 2008, except in the heaviest boys (BMI for age greater than or equal to the 97th percentile). This rate, though, remains alarmingly high. Statistics show about 17 percent of American children ages 2 to 19, or 1 in 6, are obese. Further, the latest data continue to suggest that overweight and obesity are having a greater effect on minorities, including blacks and Hispanics." Now, when I read this, I felt compelled to jump the gun a bit because, on this official government site run by a division of the NIH it says clearly, "recent data suggest that the rate of overweight in children did not increase significantly between 1999 and 2008, except in the heaviest boys (BMI for age greater than or equal to the 97th percentile)." Nowhere, in any of the four parts of The Weight of the Nation, including part three, "Children in Crisis," which airs Tuesday night do they bother to mention this little nugget. As I said at the outset when I expressed my concerns, if some data doesn't fit the message they seek to deliver, it wasn't going to be mentioned. That did not come from a newspaper, a rival scientific journal, a blog or any anecdotal evidence I might have offered. It came from the same government institution that helped create this project and lent experts to speak. That's when it gets disturbing. It might be a lie of omission for a noble cause (as opposed to lying to get us into a war), but it remains a government lie all the same in a way, though presumably they didn't have final edit. The two organizations listed as the actual authors of The Weight of the Nation and not just "associates" are HBO and The Institute of Medicine. 
Before I begin my quest to unravel the methodology used for The National Health and Nutrition Examination Surveys (which, from here on out, despite my training to avoid using obscure acronyms, I will refer to as NHANES), I wanted to delve deeper into The Institute of Medicine, since, unlike the Centers for Disease Control and Prevention and the National Institutes of Health, isn't a branch of a larger governmental agency. According to its own web site, The Institute of Medicine describes itself as, "an independent, nonprofit organization that works outside of government to provide unbiased and authoritative advice to decision makers and the public. Established in 1970, the IOM is the health arm of the National Academy of Sciences, which was chartered under President Abraham Lincoln in 1863. Nearly 150 years later, the National Academy of Sciences has expanded into what is collectively known as the National Academies, which comprises the National Academy of Sciences*, the National Academy of Engineering, the National Research Council, and the IOM. The IOM asks and answers the nation’s most pressing questions about health and health care." I put the last sentence in bold since my blog's style places quotes in italics and the institute italicizes that sentence on its web site. Changing it to plain type just didn't underscore the emphasis it attempts to make. (*NOTE: The link to The National Academy of Sciences on the institute's page doesn't work. I had to find it by going through the link from the National Academies site.) Additionally, the Institute of Medicine boasts that, "Our aim is to help those in government and the private sector make informed health decisions by providing evidence upon which they can rely. Each year, more than 2,000 individuals, members, and nonmembers volunteer their time, knowledge, and expertise to advance the nation’s health through the work of the IOM. Many of the studies that the IOM undertakes begin as specific mandates from Congress; still others are requested by federal agencies and independent organizations. While our expert, consensus committees are vital to our advisory role, the IOM also convenes a series of forums, roundtables, and standing committees, as well as other activities, to facilitate discussion, discovery, and critical, cross-disciplinary thinking." So, though the institute isn't connected officially to the bureaucracy, much of its work gets funded by taxpayers. Unfortunately, I ran out of time to discern how much. If you click on the studies link though, you will discover
that the Institute of Medicine proves quite eclectic in the issues it researches. Back to the investigation about NHANES. Turns out that the CDC conducts those surveys. On its site, they describe the surveys like this: "The National Health and Nutrition Examination Survey (NHANES) is a program of studies designed to assess the health and nutritional status of adults and children in the United States. The survey is unique in that it combines interviews and physical examinations." OK, that's vague enough. Doesn't explain how many people take part and how they extrapolate the 68% number from it, but let's dig further. Beneath that description a link written in blue reads, "Selected Participants." Let's click on that and see where that takes us. "Welcome NHNES Particpants," it greets us, followed by an explanatory paragraph that reads, "You, or a member of your family, may have a chance to take part in an important national health survey. The National Center for Health Statistics, a part of the Centers for Disease Control and Prevention, is responsible for this survey — The National Health and Nutrition Examination Survey (NHANES). This survey teaches us about the health and diet of people in the United States. Over the years, this survey has led to improvements in the food we eat and the health care we receive." After I stopped laughing at the idea of improvements in the health care we receive, I returned to scanning the page to see if this page would offer clues to methodology or only how to become a potential guinea pig. I have to
admit that I was intrigued that one of the frequently asked questions was "What do you do with my blood and urine samples that I consented to have stored?" On the left-hand side of the page, several links look promising. Beneath National Health and Nutrition Examination Survey, they have a link for Questionnaires, Datasets and Related Documentation and another for Survey Results and Products. Below that box a list of related sites includes links to Surveys and Data Collection Systems and Research Data Center. Boy, this has started to take almost as much time for me to do as a recap of an episode of Boardwalk Empire or Treme. I decided to start at the top, selecting Questionnaires, Datasets and Related Documentation first. To paraphrase what the old knight told Indy toward the end of Indiana Jones and the Last Crusade, "I chose well." The page rolled out a long list of links starting with NHANES Analytic Guidelines and including a NHANES report for 2009-2010, newer than the one in The Weight of the Nation. I'm checking the analytics first and it contains precisely what I'm looking for in terms of methodology. Unfortunately, it involves government bureaucracy, so they spell the methods out across 14 pages in PDF form. Let me add though that those pages turned out to be a delight and made me look even more askance at what these "experts" were peddling. Why do so many people feel the need to be less than forthcoming? It's not as if the agenda they push is an evil one. Eat less, eat healthier, exercise, take care of your body — sound advice anytime — but I guess they feel that Americans show themselves far too often to be too mentally deficient to hear unvarnished truth without employing scare tactics and talking down to them as if the median age of the viewer were 4."Data collected in NHANES comes from interviews, examinations, and laboratory tests based on blood and urine samples. There may also be measures taken in the home, such as dust or tap water collection. The source of a data item (interview, MEC*, sera**) is important for both assessment of quality of information and for determining the appropriate sampling weight to be used for producing statistical estimates. As with any data set, NHANES data are subject to sampling and non-sampling errors (including measurement error). Interview (questionnaire) data are based on self-reports and are therefore subject to non-sampling errors such as recall problems, misunderstanding of the question, and a variety of other factors. Examination data and laboratory data are subject to measurement variation and possible examiner effects. The NHANES program maintains high standards to insure non-sampling and measurement errors are minimized.…Despite the rigorous quality control standards, estimates produced from any data set are subject to sampling and non-sampling variation and interpretation of analysis must proceed accordingly. Data content and data collection protocols may change over time; this is another reason to read the documentation in order to understand any issues in comparability of data over time. Changes in methods may occur at any time and the user should not assume they have remained the same (especially in the continuous NHANES, conducted since 1999)."
— NHANES Analytic and Reporting Guidelines, Part 1 (last correction September 2006)
*Mobile Examination Centers **Subsample of stored serum of specimens from NHANES (1999–2000) tested to estimate
the prevalence and specificities of selected autoantibodies in the U.S. population in 2003-2004.
As most of you probably realized once the word survey appeared, the statement that categorized 68 percent of Americans as overweight or obese isn't a factual one, not in the way we say, "Two plus two equals four" and know it to be true. The survey only estimates 68 percent of U.S. adults fall into the obese and overweight classifications (based on the outdated, kooky BMI, remember). However, The Weight of the Nation never presents that possibility. In this election year, polls have exploded with alarming frequency for quite some time — and it only will get worse. Despite the endless unending onslaught, even the lazy media remembers to remind us of the margin of error. The documentary never mentions the notion of sampling errors — or methodology — or any evidence that supports that figure. They pronounce that 68 percent of Americans fall into that weight class and the discussion ends and we must accept their word as gospel truth. I imagine
that those involved didn't follow the analytic guidelines of that NHANES report. Part 1 begins in larger, bold, italic type saying, "The first and over-riding analytic guideline is that the data user, prior to any analysis of the data, should read all relevant documentation for the survey and for the specific data items to be used in an analysis." When it downsizes the type size and resorts to lightface, it adds, "Many analytic problems and misinterpretation of the data can be avoided by reading the documentation, examining the data collection protocols and data collection instruments, and conducting preliminary descriptive evaluation of the data. The documentation will indicate how the data were collected, how the data are coded and the amount of missing data." It makes me sad because these people didn't need to play this way. Michael Moore doesn't hide his point-of-view and he usually comes off making stronger cases in his films such as Sicko. The people and organizations behind The Weight of the Nation don't want to admit that they've constructed propaganda when they didn't need to do so and they end up undermining themselves. I wasn't exaggerating when I said how interesting the NHANES document turned out to be. The CDC unit used to conduct these surveys periodically and then release the results as a single set of data covering six years. Beginning in 1999, NHANES turned into a continuous, annual survey released in increments of two years. Here comes the important point, from Part 2: "For a two-year analysis, sample size is smaller and the number of geographic units in the sample is more limited…Sample size and statistical power consideration should be used to determine if a two-year sample is sufficient for a particular analysis or if 4 (or even 6) years of the survey need to be combined to produce statistically reliable analysis."
By going by the 2007-2008 figure, The Weight of the Nation not only neglects to mention the possibility of sampling errors, it ignores the survey makers' own advice to combine samples from the previous two-year or four-year samples to paint a more accurate picture. Why do I feel I'm doing more research writing this review than some people involved in the series did? Later, in Part 3 of NHANES' Guidelines, it gets serious and pulls out bold italic again.
"Because NHANES is a complex probability sample, analytic approaches based on data from simple random sample are usually not appropriate. Ignoring the complex design can lead to biased estimates and overstated significance levels. Sample weights and the stratification and clustering of the design must be incorporated into an analysis to get proper estimates and standard errors of estimates."
Just in case you missed the point so far, they raised the size of the type again when they used it as the start of Part 5.
"5. In order to produce estimates with greater statistical reliability for demographic sub-domains and rare events, combining two or more 2-year cycles of the continuous NHANES is encouraged and strongly recommended. For two-year cycles, the sample size may be too small to produce statistically reliable estimates for very detailed demographic sub-domains (e.g. sex-age-race/ethnicity groups) or for relatively rare events. The sample design for NHANES makes it possible to combine two or more “cycles” to increase the sample size and analytic options. Each two-year cycle and any combination of those two years cycles is a nationally representative sample."
Could the CDC's National Health and Nutrition Examination Survey analytical guidelines be any clearer? The two-year sample, which The Weight of the Nation uses to yell fire in the movie theater about 68 percent of all Americans being overweight or obese, lacks a sample sizable enough to be reliable and without combining it to another cycle of NHANES surveys, the result can't be extrapolated nationally. Perhaps they tried that but the figures didn't turn out the way they wanted, especially since the CDC's NHANES report released in 2007, as reported by Gina Kolata on Nov. 27, 2007, in The New York Times, found obesity rates leveling off. Kolata wrote:
"Obesity rates in American women have leveled off and stayed steady since 1999, a long enough time for researchers to say the plateau appears to be real. And, they say, there are hints that obesity rates may be leveling off for men, too.
The researchers’ report, published online today by the Centers for Disease Control and Prevention, used data from the centers’ periodic national surveys that record actual heights and weights of a representative sample of Americans. Those surveys, according to Cynthia L. Ogden, an epidemiologist at the National Center for Health Statistics and the lead author of the new report, are the only national ones that provide such data.
Dr. Ogden said the trend for women was “great news.” Obesity rates have remained steady at about 35 percent since 1999, a long enough time to persuade her that the tide has changed. “I’m optimistic that it really is leveling off,” she said.
Men’s rates increased until 2003, when they hit 33 percent and stayed there through the 2005-6 survey. Dr. Ogden said she would like to see a few more years of data before declaring that men's rates have stopped increasing."
As much as I would like to move on, I've burrowed so deep into NHANESland that I feel I must share the actual demographic breakdown of participants in the 2007-08 survey. They go into incredible detail and include some categories that I'll admit I don't comprehend. You can click on that link for the full list, but some highlights beyond what I've sprinkled as screenshots already.
This certainly marks a first for me. I've never had to divide a review of an HBO documentary into two parts, but I have more to say about "Consequences" (some positive things even) and the discussion of the second part, "Choices," has yet to be broached — and as a viewing experience, "Choices" ends up being the more satisfying hour.
Tweet
No comments:
Post a Comment