Step by step: Azure ML Job to predict using Azure endpoints (No coding needed)

 



Guys, today I am gonna present to you one of the coolest features of Azure ML studio: a predictive modelling machinery leveraging which you can gauge an outcome, by letting a suitable model be trained with modelling data. It's simple, easy and pretty handy to consume. Let me present to you: The Azure Automated ML jobs and their associated Endpoints. This article can help you in understanding the entire process step-by-step. 

So buckle your seatbelt Dorothy, 'cause Kansas is saying bye-bye.

In our example, I would be training Azure ML studio with heavy dataset of apartment prices at Mumbai, using Regression Algorithms. Once ready, we would be choosing the correct model to create an endpoint, that could be used to predict the price of any apartment, for any combination of area, location and room details. Uh, oh -- before we begin, I would request you to read by previous post: https://subsd365.blogspot.com/2024/08/train-your-data-model-using-azure-ai.html

This could give you a basic understanding of what is Regression algo is all about. We need regression here, becuase we are gonna predict numbers. If would have needed to predict something in yes-no, true or false: then we would have gone for Classification algo. If we needed to check for values based on time based predictibility, we should consider using time-series forecasting algo. Hence it all depends, the type of algo you need, the work that you would like to do.

Step 1: Let us begin by creating a Azure ML resource: this I have already covered in the above link. So nonce you get into the Azure ML studio, you can hover all the way down to Assets >> Data >> And click on create to create the data, which you would like to analyse.  


Fill out the following as a first step. Click on Next to continue. 

Select 'From Web files' in the next step to continue:


In the next step, fill out the URL from the below site to continue:

https://raw.githubusercontent.com/Mega-Barrel/Bombay_Rent_House_Analysis/master/data_sets/cleaned_data.csv




It's essentially a very large data site, containing several instances of apartments, their types, their rents and agent/constructors' names. Click on Next. It will validate the data for a couple of moments, before taking you the the next page:


This contains the synopsis of the data supplied, in a tabular form, showing you the first 50 odd records of the total records. Click on Next to continue.

This screen will show you the schema of the data entered:


And finally the following screen will show up ton indicate the data supplied:


Click on 'Create' to finish the wizard. If everything is alright, this would create the data as shown in the below list:



Step 2: Now that your data is ready, let us jump to create an Azure automated  ML job, that can be used to train the model with the data.


Click on the Icon to start. 
Give a proper name as a inidcated below (select new expreiment) radio button:


In the next step you select the dataset which you created from the above step:

Select 'Regression' as Task type. Click on Next to continue.
In the next step, it will ask for the Target column, which you would like to evaluate. In our case, we are gonna predict the Price of the apartment:


Additonally you can click on 'View additonal configuration settings' >> Click on Blocked Models, to restirct what all the models you don't want to run your data on:

This depends from project-to-project. There are some obvious models which are irrelevant on certain machine training models, and we can safely block them: consequently they can save a good amount of money.

Click Next to continue.
In the next screen, it will ask for the Compute type. Select Compute-cluster. Click on New to create a new Compute cluster option:
Select the following options:
I have chosen a low price computing option like below:

Click on Next:


Give a proper name. Change the idle timeout as 30, in order to shut down the cluster when not in use. Click on Create to complete the wizard. It will couple of minutes to finish off the cluster creation. 
You can now come back to your AML job creation form and select the clusster you just created:



Click on Submit Training job.  It will take a couple of moments to get the job be running and the training to start.
You need to be patient here, 😅😅😅 -- depending on dataset and the models you have selected, it might take several hours to conslude the training.
For the example given above, it took almost 6+ hours to finish the training:

Step-3: Once done, you can click on Jobs >>  click on the job which you created:


 Click on Model + child jobs >> it will show you a list of all the jobs (parallel threads) of 'epoch's which the training model has gone through, suggetsing which model has taken what precision  (Normalized root mean square error). More close to one, more is the precision. Select the best one which you think, click on Deploy >> Real time deploy>>

 In the following screem select as following and click on Deploy.


It will take several moments to make the endpoint ready. It might get errors like: HttpResponseError: (SubscriptionNotRegistered) Resource provider [N/A] isn't registered with Subscription [N/A]..

To fix the same go to your subscriptions:



 Go to Resource providers:


You need to add Microsoft.PolicyInsights and Microsoft.Cnd policies. Now you retry deployment.
This time the deployment would succeed without much of an ado.

You can see the Endpoint which got created:


Click on it, and you could see the details, with the status as Succeess:


Click on Test to test the Endpoint.

I tried with the following data (note the format of the data):

{

  "input_data": {

    "columns": [

      "area",

      "bathroom_num",

      "bedroom_num",

      "dev_name",

      "furnishing",

      "locality",

      "post_date",

      "type",

      "user_type",

      "year_posted",

      "month_posted"

    ],

    "index": [1],

    "data": [

         [1000, 1,1,"Veena Group","Semi-Furnished", "Vasai", "2020-01-11", "Multistorey Apartment",  "Agent", "2020", "01"]

      ]

  }

}

Click on 'Test' button:


Which is a near approximate data to what we have feed in. 

If you are not satisfied, you need to change the algorithm and try the whole process over and over again.


Whew!!! Isn't that exciting? Am gonna come back soon with many such cool features of Azure AI, on forthoming blogs. Till then take care, much Love and Namaste 💓💓💓



Comments

Post a Comment

Popular posts from this blog

X++ : mistakes which developers commit the most

Image
   The most important thing is to remember where and when to write the code. a. Developers have an inherent habit of writing erratic select statements: select * from custTable where custTable.AccountNum == <SomeValue> Where in essence we need to make use of optimized choice of fields in the select statement. b. Making use of 'continue' keywords: While select custTrans where <someConditions> { if (cusTTrans.CustGroup != mappingTable.CustGroup) { continue; } } 'Continue' statements are precisely a very unhygienic way of designing your code. The above statement could have been easily ornamented by using exists/not exists joins. Using Continue statements would eventually result in a high performance impact. Also select statements should be made punctuated with a 'firstonly' clause -- just not from best practise standpoint, but also from performance considerations. c. Writing nested loops: This is a common mistake that perha
Read more

Make your menu items visible on main menu, conditionally,, using this cool feature of D365FO

Image
  Wassup guys? I am sure you must have encountered this situation where you needed to display your menuItems, on main menu, based on some condition, haven't you? This was only possible by using config keys and flights. Things get a bit tricky,  when suppose you have a requirement like this: you need to make it dynamic, by introducing a toggle switch on a field on a paremeter form.  Suppose on Accounts Receivables Parameters >> You have a field called ' Show US customer '. You have a requirement that says when you turn on this toggle, then only the menuItem ' Input SSN details ', under AR module. This is now highly acheivable by leveraging subscription to 'SysMenuNavigationObjectFactory' class, simply by saying a. Create a new class USMenuItemsVisibilityManager b. Copy the following method, with proper documentation (don't change signature/anything else) [SubscribesTo(classstr(SysMenuNavigationObjectFactory), staticdelegatestr(SysMenuNavigationObjec
Read more

Speed up your execution performance by using SysGlobalCaches

Image
  Ever come across a situation where you needed to  a. Using a multi-threaded batch to import a journal header and lines, from a staging table. The table already contains the Journal Ids which you need to assign to the header and allocate the lines under that header. b. You need to continuously check after each line being imported, if the journal with the Id has been created. If not, it will create the header and create the lines under it;  If yes, it will directly create the lines for the header (Find or create). c. As you could rightly guess, this architecture has an inherent sluggishness associated with it -- each time you keep on checking if the header has been created or not, it does mean a lot of performance effects, as a consequence. Also as you can understand it's a multithreaded code, which means each thread has no direct relation with the other -- which does mean there is no way I can talk between the threads/pass on any information. SysGlobalCaches to the rescue: I perso
Read more