Borrow-checker hindered me to keep the same HashSet around all the time and just clear it, so it feels a little inefficient to have a new set every time.
