We are reaching diminishing returns on smaller dimension sentence level vector models. I'm almost convinced the scores on the HuggingFace leaderboard are being rigged by fine tuning using the test sets. More creative solutions for retrieval are needed.
Originally posted on Twitter.