On your own posts, you can delete both your and other people's comments. On other people's posts, you can only delete comments that you've written.
To delete a comment:

Instagram app for Android and iPhone

Instagram Lite app
- Tap
below the post. - Tap the comment you'd like to delete.
- Tap Delete Comment.