I agree bluebirds can be awfully tolerant. We don’t give them enough credit sometimes.
That said maybe you could approach him if is a supper regular occurrence? Just let him know what the boxes are for and request he not disturb the boxes directly or move slightly away from the boxes if possible.
