Predict lung cancer malignancy, using Azure AI ML studio, using Deep learning (low code-no code)

 


Amigos, in this blog, I will try to show how we can implement Azure ML Studio to predict complex forecasting answers, with the help of Azure Pipelines and ML jobs. I have covered how to setup Azure ML learning studio and pieplines, earlier in my blogposts:

https://subsd365.blogspot.com/2024/08/implement-no-code-predicitve-modelling.html

https://subsd365.blogspot.com/2024/09/step-by-step-azure-ml-job-to-predict.html

This post would give you an impression as to how leverage large datasets to evaluate a complex possibility (any patient having chances of relapses of a disease like Cancer/ its malignancy), by doing Classification based algorithm, without practically writing a single line of code.

Here goes the steps:

Step 1: Creating a data-ML job --

Once you have created An Azure ML Studio Workspace, you can crawl down to Automated ML Jobs section as shown below:




Click on Add and the following screen would come:

Fill it out, as shown. In the experiment, I have selected New experiment; Alternately, you can select an existing one.
Click Next. 
Under Basic settings tab >> select Task Type as Classfication (as becuase you are going to classify the outcome as Cancerous or non-cancerous).

Click on Create to proceed:

Fill out the screen as follows; do select Type as Tabular:


Select Source as web files in the next screen:

This will take you to the following screen:
I took the data from the following Github URL repo:
https://raw.githubusercontent.com/rohitinu6/Lung_Cancer_Prediction_Using_Machine_Learning/refs/heads/main/Dataset/survey%20lung%20cancer.csv
Hence I am pasting the same out here:


Clicking on Next would take some time to get things validated and take you to the next screen, giving you how the system has interpreted the data in a tabular format:


Click on next; in the next screen it will give you the columns involved. You can include/exclude any cloumn as per you need, however please do not change anything, as each of these components are pretty useful in prediciting maliganancy:


And in the next step, click on Create to create the data step:


 Once the data gets created, you would come back to the screen, from where you started it all:

Select the dataset you just create and Click on Next. 

And in the following screen, we are selecting the column we need for prediction:

Also do enable Deep learning. For additional setitngs, click on Addtional configuration settings: and select the algorithms which you do not want to get executed here:


I have exluded them purposely, as they are not really related to classification algorithms. excluding them, would just save some cost.

Click on Save.

Click on Validation Type as: K-Cross validation, and test data as 30% of data, as a result 70% of the data would be kept for training and 30% as testing:


Click Next to continue. Here you can select Computer cluster (they are cheaper and quite cost saving: able to scale down when not in need). I created one before hand and am using the same here:

 


Once done, now you can put it for training job:



 Here you need to wait, till the time the algo completes the training, by going through the model again and again (called as Epochs).

Step 2: Create and endpoint

Once the training is complete, it looks like:



Click on Model+Child Jobs, to see the list of associated jobs created along with.


Select the one with AUC weight that is close to 1 >> Click Deploy to continue:


Choose Real time time endpoint. The following scteen will appear:



Choose the necessary Virtual machine and click on Deploy button to continue. This will initialise the deployment and will consequently turn you back to the following screen:

Click on Endpoints and it will show you the endppoint which you just deployed. Will be under provisioning state:



It will be shown as Creating state:


Once done, teh deployment looks like this:



Uh, now it's ready -- let us test it.

Step 3: testing the endpoint --

Click on Test tab: 


I tested with the following JSON:

{

  "input_data": {

    "columns": [

      "GENDER",

      "AGE",

      "SMOKING",

      "YELLOW_FINGERS",

      "ANXIETY",

      "PEER_PRESSURE",

      "CHRONIC DISEASE",

      "FATIGUE",

      "ALLERGY",

      "WHEEZING",

      "ALCOHOL CONSUMING",

      "COUGHING",

      "SHORTNESS OF BREATH",

      "SWALLOWING DIFFICULTY",

      "CHEST PAIN"

    ],

    "index": [1],

    "data": ["M",69,1,2,2,1,1,2,1,2,2,2,2,2,2]

  }

}

And the answer came: YES

You can automate the testing by calling from a web service/API/Azure function directly -- like the example given in my previous post.


Comments

Popular posts from this blog

Make your menu items visible on main menu, conditionally,, using this cool feature of D365FO

Are you still using macros? Be sure you read this.

Modifying a DMF data entity query dynamically, from another query, for filteration in D365FO