Windmills rely on the wind, which is always there. I bet that's what he means by 'my little windmill'. She'll be there even if he doesn't want her to, but she'll stop if he stops existing (Like a windmill would stop if there's no wind)

And I think this song is sort of him thanking his girlfriend for being there and showing him she cares, and even if he doesn't show it, he cares about her too.