Once it is established that the bug tracking system is an attribute measurement system, the next step is to look at the terms precision and accuracy in relation to the situation. First of all, it is useful to understand that precision and accuracy are terms borrowed from the world of continuous (or variable) measuring instruments. For example, it is desirable that a car`s speedometer measures just the right speed over a speed range (e.B. 25 mph, 40 mph, 55 mph and 70 mph), no matter who reads it. The absence of distortion over a range of values over time can usually be called accuracy (distortion can be considered false on average). The ability of different people to interpret and match the same meter value multiple times is called accuracy (and accuracy problems can come from a problem with the meter, not necessarily from the people who use it). Before proceeding, the hiring manager wants to ensure that the team conducting the interviews is able to reach a high level of agreement. Otherwise, the team may select inappropriate candidates. The reasons why agreements (consistencys) were weak could be as follows: In addition to the problem of sample size, logistics, which ensure that evaluators do not remember the original attribute they assigned to a scenario when they see it for the second time, can also be challenging. Of course, this can be somewhat avoided by increasing the sample size and, better yet, waiting a while before giving the reviewers the scenarios for the second time (perhaps one to two weeks). Randomizing executions from one notice to another can also be useful.
In addition, evaluators also tend to work differently when they know they are being examined, so the fact that they know that it is a test can also skew the results. Hiding this in any way can help, but it`s almost impossible to achieve, despite the fact that it borders on immorality. And in addition to being marginally effective at best, these solutions add complexity and time to an already difficult study. Unlike a continuous meter, which may be (on average) but not accurate, a lack of precision in an attribute measurement system necessarily also leads to accuracy problems. If the error encoder is unclear or undecided on how to encode an error, different codes are assigned to multiple errors of the same type, making the database inaccurate. In fact, the inaccuracy of an attribute measurement system contributes significantly to the inaccuracy. Whenever someone makes a decision – such as ”Is this the right candidate?” – it is important that the decision-maker chooses the same choice again and that others come to the same conclusion. The analysis of award agreements measures whether or not several persons who make a judgment or assessment of the same subject have a high degree of agreement with each other. Modern statistical software such as Minitab can be used to collect study data and perform analysis. Kappa graphical output and statistics can be used to examine the efficiency and accuracy of operators in performing their assessments.
Attribute match analysis can be a great tool for uncovering sources of inaccuracies in a bug tracking system, but it should be used with great care, consideration, and minimal complexity, if any. To do this, it is best to first examine the database and then use the results of this audit to create a targeted and optimized analysis of repeatability and reproducibility. The audit should help to identify which specific people and codes are the main sources of problems, and the evaluation of the award agreement should help determine the relative contribution of repeatability and reproducibility issues to those specific codes (and to individuals). Also, many bug tracking systems have problems with precision records that indicate where an error was created because the location where the error is found is saved, not where the error was caused. Where the error is found doesn`t help much in identifying the causes, so the accuracy of the site assignment should also be an element of the audit. If the audit is planned and designed effectively, it can provide sufficient information on the causes of accuracy issues to justify a decision not to use the analysis of award agreements at all. In cases where the audit does not provide sufficient information, the analysis of attribute agreements allows for a more detailed investigation that provides information on how to use training and fail-safe modifications to the measurement system. First, the analyst must firmly determine that there is indeed attribute data.
It can be assumed that the assignment of a code – that is, the classification of a code into a category – is a decision that characterizes the error with an attribute. Either a category is correctly assigned to a defect or it is not affected. Similarly, the error is assigned to the correct source location or not. These are the answers ”Yes” or ”No” and ”Correct assignment” or ”Wrong assignment”. This part is quite simple. The accuracy of a measurement system is analyzed by dividing it into two main components: repeatability (the ability of a particular evaluator to assign the same value or attribute several times under the same conditions) and reproducibility (the ability of several evaluators to agree among themselves on a particular set of circumstances). With an attribute measurement system, repeatability or reproducibility issues inevitably lead to accuracy issues. .