Adaptive divergence occurs even between insufficiently isolated populations when there is a great difference in environments between their habitats. Individuals present in an intermediate zone of the two divergent populations are expected to have an admixed genetic structure due to gene flow. A selective pressure that acts on the genetically admixed individuals may limit the gene flow and maintain the adaptive divergence. Here, we addressed a question whether selection occurs in the genetically admixed individuals between two divergent populations. Arabidopsis halleri is a perennial montane plant, which has clear phenotypic dimorphisms between highland and lowland habitats in Mt. Ibuki, central Japan. We obtained the whole-genome sequences of Arabidopsis halleri plants along an altitudinal gradient of 359-1,317 m with a high spatial resolution (mean altitudinal interval of 20 m). We found a zone where the highland and lowland genes were mixing (intermediate subpopulation). In the intermediate subpopulation, we identified 5 and 13 genome regions, which included 3 and 8 genes, that had a high frequency of alleles that are accumulated in highland and lowland subpopulations, respectively. In addition, we also found that the frequency of highland alleles of these selected genome regions was smaller in the lowland subpopulation compared with that of the non-selected regions. These results suggest that the selection in the intermediate subpopulation might limit the gene flow and contribute to the adaptive divergence between altitudes. We also identified 7 genome regions that had low heterozygote frequencies in the intermediate subpopulation. We conclude that different types of selection in addition to gene flow occur at the intermediate altitude and shape the genetic structure across altitudes.