Azure Auto ML: The Solution for Faster Machine Learning Development

We used our Smart Workspace solution, explained above, as our data source. The illustration below shows meeting rooms in one of our offices as an example; the small dots in each room indicate the overall number of seats as well as whether they are occupied (red) or free (green).

By analyzing the data, we see that the data set consists of a series of observations recorded at regular intervals over a period of time. The task is to use this data to predict a future value based on past trends and patterns. The variables of interest in the data set include time-related factors such as the date, hour, and minute of each observation, so we will perform a time series analysis on our data set.

Building ML models with time series data is often tedious and complex, with many factors to consider. When working with time series data, it is important to apply appropriate preprocessing and feature engineering techniques in order to extract meaningful insights and build accurate models. That can include time-based indexing techniques such as date shifting and resampling, as well as decomposing the data into its seasonal, trend, and residual components. Feature engineering can also help create additional features that capture relevant patterns and trends in the data, such as lagging and rolling window statistics. Applying these techniques helps you better understand and model your time series data, leading to more accurate predictions and informed decisions.

These steps were applied by transforming the data with Python. Finally, the data was visualized, and a decomposition was applied to find out if our data has seasonality or a trend. The results show that the seasonality is strong; residuals can be disregarded for the multivariant time series analysis, since a second model will exclusively focus on statistical outliners.

Our time series data for the training data shows no clear trend, so we will apply the Auto ML features to search for interesting patterns.

Getting started with Auto ML, we first create a Machine Learning Studio Workspace, which is the top-level resource for Azure Machine Learning. It provides a centralized place to work with all the artifacts you create when you use Azure Machine Learning. The workspace keeps a history of all the training runs, including logs, metrics, output, and a snapshot of your scripts. You can use this information to determine which training run produces the best model. Moreover, it offers “Pro Code” capabilities by the Azure Machine Learning Python SDK as well as No-Code/Low-Code ML processes within the Azure Machine Learning Studio. In our case, we chose the latter. Read more about the configuration options in Microsoft’s Configuration documentation. 

Next, starting a run with the previously prepared data, we choose the task of “time series forecasting” among the options shown in Figure 1. For our time series forecast, we must select a time column. Auto ML also automatically preselects an appropriate metric to measure our model’s performance for us.

Root mean squared error (RMSE) is a common metric for evaluating the performance of time series forecasting machine learning models. This metric measures the deviation of the predicted values from the true values, which is important for forecasting tasks where accuracy is critical. RMSE is also sensitive to the scale of the data, making it a suitable choice for time series data that may have different scales or units. In addition, RMSE is easy to interpret because it is expressed in the same units as the original data, and it is well understood and widely used by practitioners, making it a reliable choice for evaluating the performance of time series forecasting models. When you select “View additional configuration settings,” the performance metric can also be adjusted.

Running automated ML experiments remotely requires computational targets. Azure Machine Learning Compute is a managed computing infrastructure that can be used to create single-node or multi-node computing. For more information on setting and using compute targets for model training, see Microsoft’s documentation. 

Once the automated Machine Learning (Auto ML) run is finished, we can examine the results from all the trained models. A big advantage of Auto ML is that it automatically sorts the results according to the best performance achieved. 

The best-performing model is automatically listed at the top of the Models table, but you can also look at each model individually to see if another model might be better suited for your needs. In our case, the lower the Normalized Root Mean Square Error (RMSE) value, the better our model performs. A smaller RMSE indicates that the predicted responses are closer to the true responses, which means the model is performing better. As a general rule of thumb, an RMSE value below 0.5 is considered good, but the acceptable RMSE value can vary depending on the specific requirements of your application.

(Source: Microsoft)

Finally, to deploy a trained model for consumption, we have two options when choosing our model, as shown in Figure 3:

1. Deploy directly in Azure
2. Download the models in .h5 format, tweak and deploy anywhere

In our case, the Voting Ensemble algorithm showed the best performance results, and we picked the first option to consume the model directly.

Once deployed via Azure, the Machine Learning Studio easily creates a consumable REST endpoint for us, which is embedded into the Smart Workspace bot. That allows users to consume predictions about the future office occupation status shown in Teams, like the example in Figure 5 below.