Will Google have the best LLM before 2024?
If Google deploys an LLM that reaches the top of the Elo leaderboard on https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard by December 31, 2023, 11:59 PM ET, this market will resolve to "Yes". Otherwise, this market will resolve to "No". Only models accessible to the public (e.g. developers or public sign-ups) will count for this market. If Google deploys a new model in 2023 that could reach the top of the leaderboard but is not yet ranked at year's end due to time required for evaluation, this market's resolution may be delayed until February 1, 2024, 11:59 PM ET. If at this point the resolution source still does not list the new model's ELO, the underlying script(s) will be used to determine whether google's model tops the leaderboard. Note that the model must be released by the resolution date to be considered (if they release a new version of the model after the resolution date, it will not be considered for this market). This market will resolve based on the leaderboard on Elo (https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard). If the resolution source becomes permanently unavailable by the resolution date, the underlying script to calculate google's model ELO on Chatbot Arena will be used directly.