A close-ish pass while parallel with or going away from isn’t scary because no line change is needed by the slower rider; a close-ish pass while the slower rider is coming toward you often stimulates a steering input by the slower rider to avoid what they think will be a collision, or in fact would be a collision.