eG Monitoring
 

Measures reported by AzrIntrctSignTest

The Sign-in logs provided by the Azure Active Directory (AD) portal is a treasure-chest of information about user sign-ins to the Azure organization and how signed-in users use the organization's resources.

One of the four types of sign-in logs offered by Azure AD is the Interactive Sign-ins log. Interactive user sign-ins are sign-ins where a user provides an authentication factor to Azure AD or interacts directly with Azure AD or a helper app, such as the Microsoft Authenticator app. The factors users provide include passwords, responses to MFA challenges, biometric factors, or QR codes that a user provides to Azure AD or to a helper app.

In the interest of secuity, it would do an administrator a lot of good to know what type of authentication was used for the interactive sign-ins - legacy authentication or modern authentication. This analysis will point them to authentication methods that are less secure, so they are prompted to disable them for a tenant, if required.

Also, a sign-in failure always leaves a bad taste; if it happens too frequently, it can damage the total Azure sign-in experience of users. To assure users of a superlative experience, an administrator should be able to spot sign-in failures rapidly, find out the reasons for the same, and fix them.

Moreover, its not always about whether a sign-in succeeds/fails. An administrator should also pay attention to risky sign-ins and deviant sign-in patterns. For instance, if an unusually large number of sign-ins are coming from a specific user/IP address/location/protocol or for a specific application/service principal/resource, it could imply that someone is trying to hack into your Azure organization and take control of its resources. Administrators should be able to spot such patterns quickly and scrutinize them, so as to avert any potential security disasters.With the help of the AzrIntrctSignTest, an administrator can achieve all of the above!

This test monitors Azure interactive sign-in logs for failed sign-ins and reports their count and details. With the help of these details, administrators can effectively troubleshoot the failures. The test also promptly captures and reports 'risky sign-ins', so that dubious sign-in attempts can be investigated and prevented. Additionally, the test reveals whether any sign-ins were made using unsecure legacy authentication protocols. Since such authentication protocols are a security threat, administrators may want to disable them. The test also helps administrators closely scrutinize the sign-ins to isolate abnormal patterns, such as the following:

  • Are there an unusually high number of sign-ins coming from specific IP addresses/locations/users/protocols?

  • Are any applications/service principals/resources making a suspicious number of sign-in attempts?

  • Are sign-in attempts from specific users/IP addresses/locations failing often?

  • Are specific applications/service principals/resources seeing more sign-in failures than others?

This way, the test sheds light on sign-in attempts that are 'suspect', so their authenticity can be verified, and any potential security risks pre-empted.

Outputs of the test : One set of results for the Azure Active Directory tenant being monitored

The measures made by this test are as follows:

Measurement Description Measurement Unit Interpretation
Total_sign_in Indicates the total number of interactive sign-ins attempted. Number  
Success_sign_in Indicates the number of sign-in attempts that were successful. Number Use the detailed diagnosis of this measure to know which sign-in attempts succeeded.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Failure_sign_in Indicates the number of sign-in attempts that failed. Number Ideally, the value of this measure should be 0.

Use the detailed diagnosis of this measure to know which sign-in attempts failed.
Risky_sign_in Indicates the number of risky interactive sign-in attempts. Number In Azure AD Identity Protection, risk detections include any identified suspicious actions related to user accounts in Azure AD.

Ideally, the value of this measure should be 0. If a non-zero value is reported, then use the detailed diagnosis of this measure to know the risky sign-in attempts.
Success_ip_address Indicates the number of IP addresses from which successful sign-in attempts were made. Number Use the detailed diagnosis of this measure to know from which IP addresses successful sign-in attempts were made.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_location Indicates the number of locations from which successful sign-in attempts were made. Number Use the detailed diagnosis of this measure to know from which IP addresses successful sign-in attempts were made.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_app_id Indicates the number of applications that successfully used managed identities to sign into Azure. Number Use the detailed diagnosis of this measure to know from which applications signed into Azure successfully.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_resrc_name Indicates the number of services that were used in successful sign-ins. Number Use the detailed diagnosis of this measure to know which services were used in successful sign-ins.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_user_name Indicates the number of users who successfully signed in. Number Use the detailed diagnosis of this measure to know which users' sign-in attempts succeeded.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Success_auth_prtcl Indicates the number of authentication protocols that were used in successful sign-ins. Number Use the detailed diagnosis of this measure to know which protocols were used in successful sign-ins.

Note that this measure will report detailed diagnostics only if the Successful Signin DD parameter of this test is set to Yes.
Failed_percent Indicates the percentage of sign-in attempts that failed. Number Ideally, the value of this measure should be low. Use the detailed diagnosis of this measure to know which sign-in attempts failed.
Unique_ip_address Indicates the number of IP addresses from which sign-in attempts failed. Number Use the detailed diagnosis of this measure to know from which IP addresses the maximum number of failed sign-in attempts were made. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_location Indicates the number of locations from which sign-in attempts failed. Number Use the detailed diagnosis of this measure to know from which locations the maximum number of failed sign-in attempts were made. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_app_id Indicates the number of applications that were unable to sign-into Azure. Number Use the detailed diagnosis of this measure to know which applications failed to sign into Azure the maximum number of times. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_resrc_name Indicates the number of services that were used in failed sign-ins. Number Use the detailed diagnosis of this measure to know which services were used in the maximum number of failed sign-ins. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_user_name Indicates the number of users whose sign-in attempts failed. Number Use the detailed diagnosis of this measure to know which users experienced the maximum number of failed sign-ins. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Unique_auth_prtcl Indicates the number of authentication protocols that were used in failed sign-in attempted. Number Use the detailed diagnosis of this measure to know which were used in the maximum number of failed sign-ins. You may want to investigate these attempts to figure out if they were geniuine attempts or malicious attacks.
Single_fctr_signin Indicates the number of sign-ins made using the single-factor authentication method. Number Use the detailed diagnosis of this measure to view the sign-ins made using single-factor authentication.
Multi_fctr_signin Indicates the number of sign-ins made using the multi-factor authentication method. Number Use the detailed diagnosis of this measure to view the sign-ins made using multi-factor authentication.
Modern_auth_signin Indicates the number of sign-ins that used modern client authentication techniques. Number Modern authentication is a method of identity management that offers more secure user authentication and authorization. Modern authentication is an umbrella term for a combination of authentication and authorization methods between a client (for example, your laptop or your phone) and a server, as well as some security measures that rely on access policies that you may already be familiar with.

Use the detailed diagnosis of this measure to view the sign-ins made using modern client authentication.
Legacy_auth_signin Indicates the number of sign-ins that used legacy client authentication techniques. Number Legacy authentication refers to basic authentication, which was once a widely used industry-standard method for passing user name and password information through a client to an identity provider.

Use the detailed diagnosis of this measure to view the sign-ins made using legacy client authentication.
Not_applied_cndtnl_acc Indicates the number of sign-ins during which no conditional access policy applied to the user and application. Number Conditional Access policies at their simplest are if-then statements, if a user wants to access a resource, then they must complete an action. Example: A payroll manager wants to access the payroll application and is required to do multi-factor authentication to access it.
Success_cndtnl_acc Indicates the number of sign-ins during which one or more conditional access policies applied to the user and application. Number  
Failed_cndtnl_acc Indicates the number of sign-ins that satisfied the user and application condition of at least one Conditional Access policy and grant controls are either not satisfied or set to block access. Number Use the detailed diagnosis of this measure to know which conditional access policies failed.