High-contrast study of the candidate planets and protoplanetary disk around HD 100546