Monoclonal antibodies emerged as an important therapeutic drug class with remarkable specificity and binding affinity. Nonetheless, these heterotetrameric immunoglobulin proteins come with high manufacturing and therapeutic costs which can take extraordinary proportions, besides other limitations such as their limited in cellulo access imposed by their molecular size (ca. 150 kDa). These drawbacks stimulated the development of downsized functional antibody fragments (ca. 15–50 kDa), together with smaller synthetic peptides (ca. 1–3 kDa) derived from the antibodies’ crucial complementarity-determining regions (CDR). Despite the general lack of success in the literal translation of CDR loops in peptide mimetics, rational structure-based and computational approaches have shown their potential for obtaining functional CDR-based peptide mimetics. In this review, we describe the efforts made in the development of antibody and nanobody paratope-derived peptide mimetics with particular focus on the used design strategies, in addition to highlighting the challenges associated with their development.