After the birds fledge you should take the nestbox down and clean it. I put my nestbox up again for those BB’s that stay over the winter. Have you noticed the bluebirds in the winter where you are. We are about 200 mi south of you and I have birds here year around. A group of them use the nestbox in the winter to roost at night. Now would be the time to invest in a pole to mount the nestbox on away from the fence.
As for the HOSP, I am not sure how much of a threat they are after the fledge. The young are usually with the parent birds in the trees where there is more protection from predators. Others may have more experience with this.
David
Stafford, VA