There isn't a minimum audience size for retargeting on Facebook. You can start retargeting with a large or small audience as long as it meets Facebook's minimum platform requirements. When you set up a retargeting campaign, you can choose to target people who've visited your website, interacted with your mobile app, or interacted with your Facebook page or Instagram profile. In addition, you can build an audience with custom audiences, video view targeting, and lookalike audiences.
For any retargeting campaign, you can narrow your audience further. For instance, you have the option of targeting your audience by age, location, interests, behavior, and more. Such targeting options help you make sure your campaign is being seen by the right people.
There are also best practices that you should follow when setting up a retargeting campaign - such as by avoiding segmenting your audiences too much, testing various ad formats, targeting people in different ways, and more. So, while there is no minimum size for a retargeting audience, the performance of your campaign will depend on how well you plan and optimize it.