A paper by Robinson et al provides a data point against, evaluating the post-mortem brains of around 150 people 90+ years old. They split their cohort three groups: those with dementia, those with cognitive impairment but no dementia, and those without cognitive impairment. Here is their main result table.
Thus, in their study, we can see that
a) there are pathological differences between dementia and other cognitive impairment, and
b) of the pathology types that they measured, there are no clear pathological differences between cognitive impairment and non-cognitive impairment.
To me this seems to be pretty good data emphasizing the standard theory that Alzheimers is distinct from normal aging. And that normal aging can also present with cognitive impairment, although these individuals can still convert to Alzheimers later in life.