Finding Decision boundary for Logistic Regression – Step by Step

In order to solve a binary classification problem, the basic use of logistic regression is to establish a decision boundary. The initial step is to establish a binary decision boundary, while the method can also be applied to multi-class classification or circumstances where there are many categorization classes.

Introduction To Decision Boundary

In the aforementioned example, the dashed line may be identified as such since we will find instances of a particular class on each half of the decision boundary. Our objective in logistic regression is to choose the best fit towards the decision boundary so that we can predict which class would fall into. An intriguing feature of logistic regression is the utilization of the sigmoid function. The sigmoid function is used as the predicted class estimator. Let’s look at the justification for this decision.

The decision boundary, which separates the data variable points into different groups and identifies the moments at which the algorithm switches from one group to another, must be built before using a specific classification technique. The classification technique should employ to train a classifier(predictor) on a dataset. A data point appears to be more likely to fall into the class A category than the class B category on the opposite side of the barrier.

Let’s look at the Logistic Regression example. The objective of logistic regression is- how to determine and divide the data points in order to accurately forecast the class of a particular observation using the data contained in the features. Assume we create a line to represent the decision boundary. As a result, all of the data points on one edge of the boundary must correspond to class A, and all other data points on the other side must belong to class B.

Consider that we create a line along a decision boundary that is identical to zero. In this approach, the line’s points are all assigned positive values on one side and negative ones on the other. In the graph below, for instance, the decision boundary at z=6×1 means that any value of x1 greater than 6 will result in a negative value for z, and any value of x1 less than 6 will cause a positive value for z.

Any linear statistical method, with or without extra polynomial characteristics, can be extended to reflect decision boundaries. 

We can build nonlinear partitions in the data and define more precise decision boundaries by including polynomial features. Consider z(x) from now on as a measurement of how far a particular data point is from the decision border.

Logistic Function

Let’s now examine the logistic function. 

• “g(z) = Output between 0 and 1 (probability estimate)
• z = Input to the function (z= mx + b)
• e = Base of natural log”

 The range of this function’s asymptotic bounds is 0 to 1. Additionally, the output will seems be near to 1 for extremely positive inputs and close to 0 for really negative inputs. We will essentially be able to convert the value we get from z into a forecast of the right class as a result of this. When an input is close to zero (and hence close to the decision boundary), it means that we are not confident in our ability to forecast whether the observation will be 0 or 1. The function g(z)=0.5 at the decision border, where z=0. As a threshold setting the prediction criterion, we’ll choose this value: The likelihood score that is produced by our present prediction function ranges from 0 to 1. We choose a specified threshold or turning point. The selected threshold will be used to categorize data into categories A and below and we will categorize values into class B in order to convert this to a distinct class (A/B).

Also Read: History of Convolutional Network (1940-Present)

probability>=0.5 à class=A

probability <=0.5 à class=B

We would be able to characterize this information as belonging to class A if the selected threshold was .5. We would label the observation as belonging to class B if our prediction had been less than 0.5. Therefore, a binary forecast for the class of the input observation is represented by the function’s output.

FAQs