eG Monitoring
 

Measures reported by AzrNonIntrctTest

The Sign-in logs provided by the Azure Active Directory (AD) portal is a treasure-chest of information about user sign-ins to the Azure organization and how signed-in users use the organization's resources.

One of the four types of sign-in logs offered by Azure AD is the Non Interactive Sign-in log. Non-interactive user sign-ins are sign-ins that were performed by a client app or OS components on behalf of a user. Like interactive user sign-ins, these sign-ins are done on behalf of a user. Unlike interactive user sign-ins, these sign-ins do not require the user to provide an authentication factor. Instead, the device or client app uses a token or code to authenticate or access a resource on behalf of a user. In general, the user will perceive these sign-ins as happening in the background of the user's activity.

Frequent non-interactive sign-in failures can degrade the overall Azure sign-in experience. To assure users of an above-par experience, administrators should be able to spot sign-in failures rapidly, find out the reasons for the same, and fix them.

Besides sign-in failures, administrators should also watch out for suspicious actions related to user accounts. Additionally, administrators should also keep an eye out for disturbing sign-in patterns. For instance, a specific user/IP address/location may be attempting to sign-in more often than normal. Likewise, a unusually large number of sign-in attempts may be made for a specific application/service principal/resource. Such abnormal activities could hint at an attempted security breach. To avert the breach, administrators should be able to spot such deviant sign-in patterns quickly and investigate them.

It is also important for administrators to know which authentication method was popularly used at sign-in - legacy authentication or modern authentication. Less secure authentication methods need to be identified, so that administrators can disable them for a tenant. With the help of the AzrNonIntrctTest, an administrator can achieve all of the above!

This test monitors Azure non-interactive sign-in logs for failed sign-ins and reports their count and details. With the help of these details, administrators can effectively troubleshoot the failures. The test also promptly captures and reports ‘risky sign-ins’, so that dubious sign-in attempts can be investigated and prevented. Additionally, the test reveals whether any sign-ins were made using unsecure legacy authentication protocols. Since such authentication protocols are a security threat, administrators may want to disable them. The test also helps administrators closely scrutinize the sign-ins to isolate abnormal patterns, such as the following:

  • Are there an unusually high number of sign-ins coming from specific IP addresses/locations/users/client applications?

  • Are any applications/resources making a suspicious number of sign-in attempts?

  • Are sign-in attempts from specific users/IP addresses/locations failing often?

  • Are specific applications/service principals/resources seeing more sign-in failures than others?

This way, the test sheds light on sign-in attempts that are 'suspect', so their authenticity can be verified, and any potential security risks pre-empted.

Outputs of the test : One set of results for the Azure Active Directory tenant being monitored

The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
Total_sign_in Indicates the total number of interactive sign-ins attempted. Number  
Success_sign_in Indicates the number of sign-in attempts that were successful. Number Use the detailed diagnosis of this measure to know which sign-in attempts succeeded.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Failure_sign_in Indicates the number of sign-in attempts that failed. Number Ideally, the value of this measure should be 0.

Use the detailed diagnosis of this measure to know which sign-in attempts failed.
Risky_sign_in Indicates the number of risky interactive sign-in attempts. Number In Azure AD Identity Protection, risk detections include any identified suspicious actions related to user accounts in Azure AD.

Ideally, the value of this measure should be 0. If a non-zero value is reported, then use the detailed diagnosis of this measure to know the risky sign-in attempts.
Success_ip_address Indicates the number of IP addresses from which successful sign-in attempts were made. Number Use the detailed diagnosis of this measure to know from which IP addresses successful sign-in attempts were made.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_location Indicates the number of locations from which successful sign-in attempts were made. Number Use the detailed diagnosis of this measure to know from which IP addresses successful sign-in attempts were made.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_app_id Indicates the number of applications that successfully used managed identities to sign into Azure. Number Use the detailed diagnosis of this measure to know from which applications signed into Azure successfully.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_resrc_name Indicates the number of services that were used in successful sign-ins. Number Use the detailed diagnosis of this measure to know which services were used in successful sign-ins.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_user_name Indicates the number of users who successfully signed in. Number Use the detailed diagnosis of this measure to know which users' sign-in attempts succeeded.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_client_app Indicates the number of client apps that successfully signed into Azure using the non-interactive sign-in method. Number Use the detailed diagnosis of this measure to know which client apps signed in successfully.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Failed_percent Indicates the percentage of sign-in attempts that failed. Number Ideally, the value of this measure should be low. Use the detailed diagnosis of this measure to know which sign-in attempts failed.
Unique_ip_address Indicates the number of IP addresses from which sign-in attempts failed. Number Use the detailed diagnosis of this measure to know from which IP addresses the maximum number of failed sign-in attempts were made. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_location Indicates the number of locations from which sign-in attempts failed. Number Use the detailed diagnosis of this measure to know from which locations the maximum number of failed sign-in attempts were made. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_app_id Indicates the number of applications that were unable to sign-into Azure. Number Use the detailed diagnosis of this measure to know which applications failed to sign into Azure the maximum number of times. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_resrc_name Indicates the number of services that were used in failed sign-ins. Number Use the detailed diagnosis of this measure to know which services were used in the maximum number of failed sign-ins. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_user_name Indicates the number of users whose sign-in attempts failed. Number Use the detailed diagnosis of this measure to know which users experienced the maximum number of failed sign-ins. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_client_app Indicates the number of client apps that could not sign-in using the non-interactive sign-in method. Number Use the detailed diagnosis of this measure to know which client apps failed to sign in. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Single_fctr_signin Indicates the number of sign-ins made using the single-factor authentication method. Number Use the detailed diagnosis of this measure to view the sign-ins made using single-factor authentication.
Multi_fctr_signin Indicates the number of sign-ins made using the multi-factor authentication method. Number Use the detailed diagnosis of this measure to view the sign-ins made using multi-factor authentication.
Modern_auth_signin Indicates the number of sign-ins that used modern client authentication techniques. Number Modern authentication is a method of identity management that offers more secure user authentication and authorization. Modern authentication is an umbrella term for a combination of authentication and authorization methods between a client (for example, your laptop or your phone) and a server, as well as some security measures that rely on access policies that you may already be familiar with.

Use the detailed diagnosis of this measure to view the sign-ins made using modern client authentication.
Legacy_auth_signin Indicates the number of sign-ins that used legacy client authentication techniques. Number Legacy authentication refers to basic authentication, which was once a widely used industry-standard method for passing user name and password information through a client to an identity provider.

Use the detailed diagnosis of this measure to view the sign-ins made using legacy client authentication.
Not_applied_cndtnl_acc Indicates the number of sign-ins during which no conditional access policy applied to the user and application. Number Conditional Access policies at their simplest are if-then statements, if a user wants to access a resource, then they must complete an action. Example: A payroll manager wants to access the payroll application and is required to do multi-factor authentication to access it.
Success_cndtnl_acc Indicates the number of sign-ins during which one or more conditional access policies applied to the user and application. Number  
Failed_cndtnl_acc Indicates the number of sign-ins that satisfied the user and application condition of at least one Conditional Access policy and grant controls are either not satisfied or set to block access. Number Use the detailed diagnosis of this measure to know which conditional access policies failed.