Explanation of Feature based Descriptions for Image Recognition (SIFT/SURF)

You have to explain me the concept of how does the feature descriptors recognize an image? As per my knowledge and study I came to know that the detectors are used to detect the keypoints and descriptor matchers are used to match the descriptors of one image to all the other images and return the image with a less distance values. I capture a photo of a book and I also have a book, bag, and pen photos already in a database. Then the descriptor matcher will match 1 image to the 3 images in a database Book to Book (450 Matches) Book to Bag (100 Matches) Book to Pen (20 Matches) As there are 450 matches with the object of book in the database it returns the data related to book. Is this the approach or I just mislead the concept of it? How many minimum matches should be there inorder to identify the object? Can anyone of you explain me about this concept