Supervisor and coworker ratings (i.e., “observer ratings”) remain a common manner of measuring counterproductive work behavior (CWB) despite long‐standing doubts that observers have the opportunity to witness the work behaviors they are expected to rate. We conducted 2 studies that evaluated the observability of CWB items and consequences of observability. First, we show that many CWBs are unlikely to be witnessed by supervisors or coworkers—specifically, behaviors such as “spends too much time fantasizing or daydreaming instead of working” and “discussed confidential company information with an unauthorized person” were found to be lowest in observability, whereas “cursed at someone at work” or “acted rudely toward someone at work” were relatively higher in observability (though observability was generally low). Second, a meta‐analysis revealed variability in item‐level relationships (correlations and mean differences) between self‐ratings and observer ratings for specific CWB scale items (i.e., items from Bennett & Robinson, 2,000). Important, this variability was partially explained by observability—behaviors with low self–observer convergence tend to have low levels of observability, whereas behaviors with higher levels of convergence tend to have higher levels of observability. This study demonstrates that supervisor and coworker ratings of CWB may be susceptible to an observability bias resulting from rating behaviors they have not likely witnessed.