from the widget to the exclusive technology – Yandex at

Alexander Ganshin, head of the Yandex weather forecast team, says the service has become the most popular in Russia.

“Yandex.Weather” appeared in 2000, was one of the first services of the company. At the beginning of the trip, it was a weather widget from the external Meteo-TV service. Later, a dedicated manager and developer appeared on the team, who set up the screen on the Yandex website of the Finnish company Foreca for a limited list of cities. A lot has changed since then.

How Yandex learned to predict the weather on its own

The weather forecast you see now on the site started growing when it looked like the service was already up – it was 14 years old. In 2015, we introduced the first version of Meteum technology. Since then, Yandex.Weather has ceased to be a broadcast of other people’s predictions. Initially, the service operated on a pilot basis: forecasts using Meteum technology were only available in the Central and Ural Federal regions, as the main development forces were in Moscow and Yekaterinburg. The team literally checked their own predictions for themselves, which made it possible to correct mistakes faster.

At the heart of our Meteum was a machine learning model that sought out and corrected inaccuracies in the forecasts of Foreca, the American Meteorological Center, and its own. We calculated our own prediction on a cluster of hundreds of computers using the Intermediate Weather Research and Prediction (WRF) model and the Matrixnet machine learning method, which we developed in Yandex and used at the time in the search.

A forecast that changes over time

The weather can change as quickly and unpredictably as the mood without a cup of coffee. It is especially changeable in summer, when the sun can suddenly give way to a downpour. Traditional meteorological forecasts do not always go hand in hand with such events, which is why they say “rain in some places”.

To enable our users to plan their day, we created an interactive rainfall map in Weather based on weather radar measurements and machine learning. First we connected 20 devices and then ten more. This allowed the coverage to increase and, consequently, the number of users.

Meteorological radars take measurements within a radius of 250 km from the installation site, but we now use only measurements taken within a radius of 170 km: there may already be inaccuracies in the line of sight. Radar gives us information about the power of rainfall (direct rain near the Earth’s surface) and the reflectivity, in other words, of the attenuation or scattering of the radar signal in water droplets in the atmosphere up to a height of 10 km – us allows us to improve the prediction of the occurrence and disappearance of rainfall.

On the interactive map, you can see the forecast for the next two hours with an accuracy of 10 minutes and learn about the rainfall in various parts of the city to plan your route and avoid getting caught in the rain.

Weather radar can not, the neural network will help

Although the number of weather radars increased, they were not always enough for an accurate forecast and in addition do not cover all settlements. Therefore, in areas without a radar network, we began to analyze satellite images using a neural network.

It works like this: first, the neural networks find places in satellite images where it is currently raining. After that, the neural networks come back into play: based on the information received about the movement of rainy areas, they make a prediction of how the weather will change over the next two hours.

Weather station on every smartphone

Now we also used the collective power of Yandex users. In order to evaluate the quality of our forecasts and make them even more accurate, we invited people to report rainfall through the application. User messages are displayed on the rainfall map as umbrellas.

If we talk about traditional ways of observing the weather, then from meteorological stations in Russia you can receive about 8 thousand reports of rain per day. Our users leave more than a million messages a day, especially on rainy days – up to three million. Not everything is reliable, but the volume of incoming information allows high accuracy due to the collection and comparison of messages coming from the same area.

Also, professional devices do not always respond quickly to sudden rains, but people signal it immediately: according to our calculations, thanks to their messages, we were able to reduce the number of errors by 20%. Users can also write a detailed message via the feedback form. We monitor the number of complaints for incorrect predictions and re-validate models to identify potential problems.

Meet Meteum 2.0

The use of user messages has allowed us to create a fundamentally new rainfall forecasting system. To reduce errors, we use our own weather data and information from four weather companies: American, European, Japanese and Canadian. Radar and satellite data are processed by a neural network and combined using the CatBoost machine learning algorithm. The machine learning model looks for patterns and learns to reproduce messages from our users. So we get rain information based on all the objective factors.

If the first Meteum was based on hydrodynamic models and machine learning, then the second added messages from people. This made it possible to increase the accuracy of rainfall forecasts by 5-15%, depending on the weather and the region. This approach allows prediction even when there are few observations or active users: Meteum 2.0 algorithms can be based on data from areas for which there is enough information.

Plan that speaks

To make it easier for our users to report the weather, we changed the design of the service. We brought the rainfall map to the fore so that as many people as possible know it.

UX research and testing has shown that people are more willing to participate in something if they see their contribution to the common goal and receive feedback. Therefore, we decided to add umbrellas to the rainfall map. As soon as the user leaves a report about the rainfall, an umbrella appears on the map for him and other users.

Before Meteum 2.0, collecting messages for the weather was a bit confusing. We had a lot of questions about cloudiness, wind, temperature – and people did not want to waste time on them. And not everyone knew that only one question could be answered. Then we made a button “Is that so?” below the weather icon and on the rainfall map there is a simple interface where you have to choose whether it is raining or not. This allowed us to increase the flow of messages dozens of times. As a result, we have been able to develop a new forecasting technology in which everyone can participate.

Leave a Comment