Description of indicators

Public indicators

• User experience score

  According to the eight index dimensions of application startup time, interactive time, first screen time, operation time, request response time, request error rate, crash rate and stutter rate, the user experience in the process of using the application is comprehensively scored.

  Scoring algorithm: indicator score = 100 * (indicator mean-indicator 90th percentile)/ (indicator 15th percentile-indicator 90th percentile)

  Through linear transformation, the value of each index is converted into a score of [0, 100] according to the high and low thresholds.

  Apply Score = sum (Metric Weight * Individual Performance Metric Score)

  Wherein, each performance index score is the sum of each weight multiplied by each performance index score as the total performance score.

• Number of active devices

  The number of active devices (de-duplicated) for this app during the selected time period.

• Start time

  The average startup time for the application over the selected time period.

  Calculation formula: (total time for first startup + total time for cold startup + total time for hot startup)/ (times of first startup + times of cold startup + times of hot startup)

• Interactable time

  The average interactive time for this application during the selected time period.

  Calculation formula: total interactive time/total page opening times

• First screen time

  The average first screen time of the app during the selected time period.

  Calculation formula: total first screen time/total page opening times

• Operating time

  The average operating time for this app during the selected time period.

  Calculation formula: total operation time/total operation times

• Request response time

  The average request response time for this app over the selected time period.

  Calculation formula: total request response time/total number of successful requests

• Request error rate

  The average request error rate for this application over the selected time period.

  Calculation formula: total request error times/total request times

• Collapse rate

  The crash rate of the application in the selected time period (Note: the number of startup times of the crash rate is calculated according to the number of initialization times of the Tingyun SDK).

  Calculation formula: number of crashes/total number of starts

• Stuck rate

  The app's stutter rate for the selected time period.

  Calculation formula: number of stutters/number of starts

Network indicators

• Request response time

  The time from when the client initiates the request to when the request is completed.

  Calculation formula: average response time = response time/number of successful requests

• Client time

  The amount of time the client spends waiting for the request response to be processed.

  Calculation formula: average client time = total client time/number of successful requests

• Network time

  Time spent by the request on the network, excluding client time.

  Calculation formula: average network time = (response time – client time)/number of successful requests

• DNS time

  The time at which the client converts the domain name to a numeric IP.

  Calculation formula: average DNS time = total DNS time/total DNS occurrences

• Number of DNS occurrences

  The number of DNS resolutions that actually occurred.

• SSL time

  The elapsed time to establish a Secure Sockets Layer (SSL) connection.

  Calculation formula: average SSL time = total time consumed by SSL/total number of SSL occurrences

• Number of SSL occurrences

  The number of times the SSL handshake actually occurred.

• TCP time

  The time at which the TCP/IP connection was established.

  Calculation formula: average TCP time = total TCP time/total TCP occurrences

• Number of TCP occurrences

  The number of TCP connections that actually occurred.

• First package time

  The time from the end of sending an HTTP request to the receipt of the first packet returned by the server.

  Calculation formula: average first packet time = total first packet time/number of successful requests

• Packet time remaining

  The elapsed time for the client to accept a packet that is not the first packet returned by the server.

  Calculation formula: average remaining packet time = total remaining packet time/number of successful requests

• Throughput rate

  The average number of HTTP requests per minute, in RPM (Requests per minute).

  Calculation formula: throughput rate = total requests/time (min)

• Reuse rate

  The multiplexing policy of requesting TCP connection establishment. The higher the multiplexing rate is, the higher the efficiency of the request is.

  Calculation formula: Reuse rate = (1-number of connections/total number of requests) * 100

• Number of requests

  The total number of times a client initiated a request during the currently selected time period.

• Number of successful requests

  Number of requests with a response code of 200.

• Number of normal requests

  The number of successful requests slows down the number of requests.

• Number of bad requests

  The number of HTTP errors (4xx, 5xx) plus the number of network errors (request timeout, connection failure, etc.).

• Number of slow requests

  The number of requests that met the slow request threshold. If the number of downloaded bytes is greater than 50kb, a request with a transmission rate less than 10kb/s is a slow request. If the number of bytes downloaded is less than 50kb, a request with a response time greater than 2000 ms is a slow request.

• Number of HTTP errors

  Number of times an HTTP error (4xx, 5xx) occurred.

• Number of network errors

  The number of times a network error (request timed out, connection failed, etc.) occurred.

• Percentage of slow requests

  The ratio of the number of slow requests to the number of successful requests during the selected time period.

• HTTP error rate

  The ratio of the number of HTTP errors to the total number of requests over the selected time period (common HTTP errors: 4 XX, 5 XX).

• Network error rate

  The ratio of the number of network errors to the number of requests over the selected time period.

Response Code	Error type	Possible cause of error
901	Unknown Host	An error occurred during DNS resolution
902	Connection Establishment Failure	Error connecting to server
903	Connection Timeout	Wait for server to exceed response duration
904	Client Protocol Error	Network protocol error, such as: HTTP is written as HTT
905	Client Canceled	An error thrown when forcibly terminating a network connection (download)
906	Invalid Response Content	Request content and response content are inconsistent
907	SSL Authentication Required	Exception occurred during SSL certificate validation
908	SSL Certificate Error	SSL local certificate is inconsistent with server certificate

• Average amount of data transferred

  The ratio of total traffic consumption to the number of successful requests over the selected time period.

• Upstream traffic consumption

  The ratio of upstream traffic consumption to the number of successful requests over a selected period of time.

• Downlink traffic consumption

  The ratio of the downstream traffic consumption to the number of successful requests in the selected time period.

• Transmission rate

  The ratio of traffic consumption to total response time over the selected time period.

  Calculation formula: transmission rate = total traffic consumption/total response time

• CDN network time

  Time spent on the network by requests marked as CDNs, excluding client time.

• CDN availability

  The availability of requests marked as CDNs.

  Calculation formula: CDN availability = 1- (total number of CDN request errors/total number of CDN requests)

• CDN transfer rate

  The transfer rate of the requests marked as CDN.

• Network latency

  The time required for a message or packet to travel from one end of a network to the other.

• Packet loss rate

  The ratio of the number of packets lost to the number of packets sent in the test.

User experience metrics

• Number of starts

  Record the number of complete boot processes.

• First Start Time

  The first startup after App installation is a special cold startup.

  Calculation method:

  • Android: Application. AttachbaseContext () starts and MainActivity. OnResume () ends.
  • IOS: end of main function startup time (SDK startup time) to FirstVC. ViewDidAppear ().

• Cold start time

  App ends the process, or exits to the background. After the process is recycled by the system, it starts again. Calculated in the same way as "First Start".

• Hot start time

  • Android: Home key: Activity. OnRestart () starts and Activity. On Resume () ends; Back key: Activity. OnCreate () starts and Activity. OnResume () ends.

  • IOS: application WillEnterForeground () begins and application DidBecomeActive () ends.

• Start time

  The time at which the application started.

  Calculation formula: startup time = total startup time/total startup times

• Initialization time

  • Android: End of Application. AttachbaseContext () to end of Application.OnCreate ().
  • IOS: main function startup time (SDK startup time) to application Delegate.

• Build time

  • Android: MainActivity init, end of Application.OnCreate () to end of MainActivity. OnCreate ().
  • IOS: application Delegate. DidFinishLaunching WithOptions () start to FirstVC. LoadView () start.

• Page load time

  • Android: If it is the home page, it is from the end of the Application. OnCreate () to the end of the MainActivity. OnResume (); If it is not the first page, it is from the end of the MainActivity. OnCreate () to the end of the MainActivity. OnResume ().
  • IOS: Start of FirstVC. LoadView () and end of FirstVC. ViewDidAppear ().

• Application preparation time

  Android: Application. AttachbaseContext () start to end. This metric is unique to Android and is only presented in the single-sample detail data of the Android launch experience analysis.

• View Fill Time

  End of Application. OnCreate () to end of MainActivity. OnCreate ().

• Interaction preparation time

  End of MainActivity. OnCreate () to end of MainActivity. OnResume ().

• Number of times the page has been opened

  The total number of times the current page has been opened.

• First screen time

  Time from page opening to content presentation (including network time).

  Calculated from the start of the first life cycle method to the end of the last asynchronous invocation method back to the main thread.

• Interactable time

  The time from when the page opens to when the user can interact normally.

  Calculation time:

  • Android: from onCreat/onStart to onResume.
  • IOS: Start with viewDidLoad/view WillAppear and end with the last viewDidAppear.

• Ratio of slow first screen

  The page whose presentation time exceeds the established threshold is used to measure whether the content of the current page is presented smoothly. The default threshold is 3000 ms, which can be configured.

  Calculation formula: slow first screen times/page opening times

• Proportion of slow interaction

  The page whose interaction time exceeds the established threshold is used to measure whether the current page interaction process is complete. The default threshold is 1000ms, which is configurable.

  Calculation formula: slow interactive times/page opening times

• Number of operations

  The total number of times the operation was performed.

• Number of available operations

  Available operations = Total operations-Failed operations-Crash operation

• Availability

  Availability = number of operations available/total number of operations

• Belonging page (operation)

  The name of the page to which the action belongs (Android: Activity or Fragment name; iOS: VC name).

• The operation is time consuming

  The total elapsed time of the operation (including the execution time of the asynchronous method).

• Blocking takes time

  The execution of the entry function takes time during the operation (the user cannot do anything during the blocking process).

• Proportion of smooth operation (%)

  Operations that take less time than the set threshold (default threshold 3s, configurable).

  Calculation formula: proportion of smooth operation = times of smooth operation/total times of operation

• Proportion of slow operation (%)

  Operations that take longer than the set threshold (default threshold 3s, configurable).

  Calculation formula: proportion of slow operations = number of slow operations/total number of operations

• Proportion of stuttering (%)

  Blocking operations that take longer than the set threshold (default threshold 3s, configurable).

  Calculation formula: proportion of stuttering operations = number of stuttering operations/total number of operations

• Proportion of crash operations (%)

  Crash/flashback of the application during the operation.

  Calculation formula: proportion of crash operations = number of crash operations/total number of operations

• Percentage of failed operations (%)

  An operation in which the request error rate reaches 100% during the operation (three requests are issued during the operation, and if all three requests have errors, the operation is an error operation).

  Calculation formula: proportion of failed operations = number of failed operations/total number of operations

Anomaly analysis

• Number of starts

  Startup times in exception analysis are counted according to SDK initialization times, including cold startup times and warm startup times.

  Explain: The number of boots counted in the crash will be greater than the number of boots recorded in the boot experience analysis.

• Number of crashes

  The number of crashes that occurred.

• Collapse rate (%)

  Apply a breakdown rate over time for a selected period of time.

  Calculation formula: crash rate = number of crashes/number of starts

• Proportion of crashed devices

  Calculation formula: proportion of crashed devices = number of affected devices/number of active devices

• Number of affected devices

  Number of devices that have crashed (deduplicated) during the selected time period.

• Number of active devices

  The number of devices that were active during the selected time period.

• Number of stutters

  The number of times a stutter occurred.

• Stuck rate (%)

  The application's stutter rate over time for a selected period of time.

  Calculation formula: stutter rate = number of stutters/number of starts

• Proportion of stuttering equipment

  Calculation formula: proportion of stuttering devices = number of affected devices/number of active devices

• Number of errors

  The number of times an error occurred during the selected time period.

• Custom Error Rate (%)

  The custom error rate applied over time for the selected time period.

  Calculation formula: Custom Error Rate = Affected Devices/Active Devices

User statistics

• Number of new users

  The number of new devices added during the selected time period.

• Number of active devices

  The number of devices that were active during the selected time period.

• Number of starts

  According to the statistics of SDK initialization times, including cold boot times and warm boot times.

• Average length of use

  Record the user's usage time in the application.

  Calculation formula: average usage time = total usage time/number of sessions

• Number of daily active devices

  Record the number of devices that are active on the current day (de-duplicate).

• Number of monthly active devices

  Record the number of devices that are active in the current month (de-duplicate).