Purpose We examined how the choice of historic medication use criteria for identifying prevalent users may bias estimated adherence changes associated with a medication copayment increase. Methods From pharmacy claims data in a retrospective cohort study, we identified 6,383 prevalent users of oral diabetes medications from four VA Medical Centers. Patients were included in this prevalent cohort if they had one fill both 3 months prior and 4–12 months prior to the index date, defined as the month in which medication copayments increased. To determine whether these historic medication use criteria introduced bias in the estimated response to a $5 medication copayment increase, we compared adherence trends from cohorts defined from different medication use criteria and from different index dates of copayment change. In an attempt to validate the prior observation of an upward trend in adherence prior to the date of the policy change, we replicated time series analyses varying the index dates prior to and following the date of the policy change, hypothesizing that the trend line associated with the policy change would differ from the trend lines that were not. Results Medication adherence trends differed when different medication use criteria were applied. Contrary to our expectations, similar adherence trends were observed when the same medication use criteria were applied at index dates when no copayment changes occurred. Conclusion To avoid introducing bias due to study design in outcomes assessments of medication policy changes, historic medication use inclusion criteria must be chosen carefully when constructing cohorts of prevalent users. Furthermore, while pharmacy data have enormous potential for population research and monitoring, there may be inherent logical flaws that limit cohort identification solely through administrative pharmacy records.