Emphysema is a typical component of chronic obstructive pulmonary disease (COPD), a progressive and inflammatory airway disease. However, no effective treatment currently exists. Here we show that keratan sulfate (KS), one of the major glycosaminoglycans (GAGs) produced in the small airway, decreased in lungs of cigarette-smoke-exposed mice. In order to confirm the protective effect of KS in the small airway, a disaccharide repeating unit of KS designated L4 ([SO3--6]Galβ1-4[SO3--6]GlcNAc) was administered to two murine models: elastase-induced-emphysema and LPS-induced exacerbation of a cigarette smoke-induced emphysema. Histological and micro-computed tomography (micro-CT) analyses revealed that, in the elastase-induced emphysema model mice, administration of L4 attenuated alveolar destruction. Treatment with L4 significantly reduced neutrophil influx and the levels of inflammatory cytokines, tissue-degrading enzymes (MMPs), and myeloperoxidase (MPO) in bronchoalveolar lavage (BAL) fluid, suggesting that L4 suppressed inflammation in the lung. L4 consistently blocked the chemotactic migration of neutrophils in vitro. Moreover, in the case of the exacerbation model, L4 inhibited inflammatory cell accumulation to the same extent as that of dexamethasone. Taken together, L4 represents one of the potential glycan-based drugs for the treatment of COPD through its inhibitory action against inflammation.