Once you've unfollowed someone, their profile will say Follow instead of Following. People won't be notified when you unfollow them.

Note: If your account is set to private, you can remove people from your followers list. You can also block them. When you block someone, the blocked person can't see your photos or videos or search for your Instagram account. The blocked person isn't notified when they're blocked.