Skip to main content

Data anomalies and apparent reporting errors in ‘Randomized controlled trial testing weight loss and abdominal obesity outcomes of moxibustion’

Abstract

Randomized Controlled Trials (RCTs) are the best method to determine causal effects for treatments if they are well done and well reported. Good evidence about proposed treatments for obesity is needed, and Hsieh et al. (Biomed Eng Online 17:149, 2018) are to be commended for putting moxibustion to the test. However, careful evaluation of the paper reveals inconsistencies and apparent reporting errors, which raise doubts about conclusions from the study.

Randomized controlled trials (RCTs) are the best method to determine causal effects for treatments if they are well done and well reported. Good evidence about proposed treatments for obesity is needed, and Hsieh et al. [1] are to be commended for putting moxibustion to the test. However, careful evaluation of the paper, similar to a prior review of another paper on moxibustion [2], reveals inconsistencies and apparent reporting errors, which raise doubts about conclusions from the study.

The primary concern relates to the randomization process. Table 1 in Hsieh et al., which describes baseline anthropometrics of both groups in their study, showed that baseline body weight (BW) differed with a high degree of statistical significance (p < 0.001) based on the comparison of groups using a t test [1], which, by definition, is very unlikely to occur under randomization [3,4,5,6]. We also note that the baseline variances for BW were significantly heterogeneous (Bartlett’s test; \(\chi^{2} = 6.86, \;df = 1, \;p = 0.0088\)), which is again, by definition, highly unlikely under randomization. These baseline distribution differences by group raise questions as to whether the randomization was properly performed. Results might be influenced by any problem in the randomization process, which could bias the estimated treatment effects [6]. The details of the randomization process were not clear, and we found no clinical trial registration for the study; randomization details should be reported as per the CONSORT statement [7].

A collection of secondary concerns, unrelated to our core concern of randomization, comprises some inconsistencies and errors found in the paper that affect the understanding of the study design and confidence in the results. First, Figure 1 of Hsieh et al. indicates “Discontinued Intervention = 30” of the control group for follow-up, which should presumably be zero because other details in the paper indicate the control group sample size is n = 30 enrolled and n = 26 analyzed. Second, the third paragraph of the Results section indicates a p-value of 0.002 for BW and 0.003 for WC, whereas Table 1 reports the p-values as 0.0002 and 0.0003, respectively. Third, Tables 3 and 4 show the results of the WC and WHR in kg units instead of cm for Table 3 and unitless for the ratio in Table 4. Finally, we note that error terms are missing for some slopes but not others in Tables 2–4.

Although the lack of an attention-placebo control group matched for amount and type of study staff contact limits the ability to determine whether it is the treatment per se that is responsible for any effects of treatment assignment (if there are any such effects), we have chosen to focus here on the issue of proper randomization, without which there cannot be any determination of effects at all, let alone their mechanism.

We believe these anomalies should be corrected or explained, particularly with respect to the unlikely baseline imbalances, which raise concerns about the randomization process.

Availability of data and materials

Not applicable.

Abbreviations

BW:

Body weight

CONSORT:

Consolidated standards of reporting trials

RCTs:

Randomized clinical trials

WHR:

Waist-to-hip ratio

WC:

Waist circumference

References

  1. 1.

    Hsieh CH, Tseng CC, Shen JY, Chuang PY. Randomized controlled trial testing weight loss and abdominal obesity outcomes of moxibustion. Biomed Eng Online. 2018;17(Suppl 2):149. https://doi.org/10.1186/s12938-018-0571-8.

    Article  Google Scholar 

  2. 2.

    Thomas DM, Brown A, Dawson JA, Li P, Heymsfield SB, Allison DB. Letter to the Editor: exceptional data in paper on “The effect of meridian massage on BM, BMI, WC and HC in simple obesity patients: a randomized controlled trial”. World J acupunct Moxibustion. 2015;25(1):66–7.

    Article  Google Scholar 

  3. 3.

    Senn S. Testing for baseline balance in clinical trials. Stat Med. 1994;13:1715–26.

    Article  Google Scholar 

  4. 4.

    Roberts C, Torgerson DJ. Baseline imbalance in randomised controlled trials. BMJ. 1999;319(7203):185.

    Article  Google Scholar 

  5. 5.

    Carlisle JB. Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals. Anaesthesia. 2017;72(8):944–52.

    Article  Google Scholar 

  6. 6.

    Kang M, Ragan BG, Park JH. Issues in outcomes research: an overview of randomization techniques for clinical trials. J Athl Train. 2008;43(2):215–21.

    Article  Google Scholar 

  7. 7.

    Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.

    Article  Google Scholar 

Download references

Acknowledgements

Supported in part by R25HL124208. The opinions expressed are those of the authors and not necessarily of the NIH or any other organization.

Funding

Supported in part by R25HL124208. The opinions expressed are those of the authors and not necessarily of the NIH or any other organization.

Author information

Affiliations

Authors

Contributions

All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Luis M. Mestre or David B. Allison.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

In the last 12 months, Dr. Allison has received personal payments or promises for same from for-profit organizations including: Biofortis; California Walnut Commission; Fish & Richardson, P.C.; IKEA; Medpace; Gelesis; Law Offices of Ronald Marron; Sage Publishing; Tomasik, Kotin & Kasserman LLC; Nestle; WW (formerly Weight Watchers International, LLC). Dr. Allison has been an unpaid member of the International Life Sciences Institute North America Board of Trustees. In the last 12 months, Dr. Brown has received personal payments or paid travel from: American Society for Nutrition; Indiana University; Kentuckiana Health Collaborative; and Rippe Lifestyle Institute, Inc. Indiana University has received funds to support the work of these authors collectively from: Alliance for Potato Research and Education; Mead-Johnson; American Federation for Aging Research; Dairy Management, Inc.; Herbalife; Laura and John Arnold Foundation; NIH; National Cattlemen’s Beef Association, Oxford University Press; Sloan Foundation; and University of Alabama at Birmingham.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mestre, L.M., Dickinson, S.L., Golzarri-Arroyo, L. et al. Data anomalies and apparent reporting errors in ‘Randomized controlled trial testing weight loss and abdominal obesity outcomes of moxibustion’. BioMed Eng OnLine 19, 11 (2020). https://doi.org/10.1186/s12938-020-0753-z

Download citation

Keywords

  • Randomization
  • Clinical trials
  • Moxibustion therapy
  • Weight reduction
  • Waist circumference
  • Waist-to-hip ratio