Grading instruments are an important part of evidence-based medicine and are used to inform health policy and the development of clinical practice guidelines. They are extensively used in the development of clinical guidelines and the assessment of research publications, having particular impact on health care and policy sectors. The positive effects of using grading instruments are, however, potentially undermined by their misuse and a number of shortcomings. This review found eight key concerns about grading instruments: (1) lack of information on validity and reliability, (2) poor concurrent validity, (3) may not account for external validity, (4) may not be inherently logical, (5) susceptibility to subjectivity, (6) complex systems with inadequate instructions, (7) may be biased toward randomized controlled trial (RCT) studies, and (8) may not adequately address the variety of non-RCTs. This narrative review concludes that there is a need to take into account these criticisms and domain-specific limitations, to enable the use and development of the most appropriate grading instruments. Grading systems need to be matched to both the research question being asked and the type of evidence being used.