A dominant feature of the olfactory bulb response to odour is fast synchronized oscillations at beta (15–40 Hz) or gamma (40–90 Hz) frequencies, thought to be involved in integration of olfactory signals. Mechanistically, the bulb presents an interesting case study for understanding how beta/gamma oscillations arise. Fast oscillatory synchrony in the activity of output mitral cells (MCs) appears to result from interactions with GABAergic granule cells (GCs), yet the incidence of MC‐GC connections is very low, around 4%. Here, we combined computational and experimental approaches to examine how oscillatory synchrony can nevertheless arise, focusing mainly on activity between “non‐sister” MCs affiliated with different glomeruli (interglomerular synchrony). In a sparsely connected model of MCs and GCs, we found first that interglomerular synchrony was generally quite low, but could be increased by a factor of 4 by physiological‐levels of gap junctional coupling between sister MCs at the same glomerulus. This effect was due to enhanced mutually synchronizing interactions between MC and GC populations. The potent role of gap junctions was confirmed in patch‐clamp recordings in bulb slices from wild‐type and connexin 36‐knockout (KO) mice. KO reduced both beta/gamma local field potential oscillations as well as synchrony of inhibitory signals in pairs of non‐sister MCs. These effects were independent of potential KO‐actions on network excitation. Divergent synaptic connections did not contribute directly to the vast majority of synchronized signals. Thus, in a sparsely connected network, gap junctions between a small subset of cells can, through population‐effects, greatly amplify oscillatory synchrony amongst unconnected cells. This article is protected by copyright. All rights reserved