On-line version ISSN 0718-5073
Rev. ing. constr. vol.27 no.1 Santiago 2012
Revista Ingeniería de Construcción Vol. 27 No1, Abril de 2012 www.ricuc.cl PAG. 75 - 92
Conceptual development of an integrated system for quality control of skid resistance measurements
Hernán de Solminihac*, Marcelo Bustos**1, Tomás Echaveguren***, Alondra Chamorro*, Sergio Vargas****
* Pontificia Universidad Católica de Chile. CHILE
** Universidad Nacional de San Juan. ARGENTINA
*** Universidad de Concepción. CHILE
****Universidad del Bio Bio. CHILE
Safety is one of the main issues that must be considered when evaluating the quality of service provided by a road. Therefore, skid resistance measurement procedures are quite important components of a road management system. Currently, many equipments and procedures are used to measure skid resistance in pavement surfaces, like British Pendulum, GripTester, SCRIM, among others. Skid resistance data collected with such devices is afterwards processed and compared against minimum threshold values, predefined for the road management procedures. However, to ensure an acceptable reliability and quality level of such results, continuous monitoring of measurement procedures and also revision and calibration of devices must be performed. In this paper, a general methodology is proposed, to be used as a quality control system for devices and skid resistance measurements. Such methodology has been designed by using checklists, and applying statistical concepts of repeatability and reproducibility. The methodology was therefore applied to evaluate the quality level achieved by skid resistance devices and measurement procedures used in Chile, with good results. The quality of the evaluation procedures included in the methodology were also verified applying Six-Sigma statistical analysis techniques, which contributed significatively to improve the proposed quality control system.
Keywords: Quality control, skid resistance measurement, repeatability, reproducibility, skid resistance devices.
Safety conditions are one of the most important issues of the level of quality service provided by a paved road to the users. It depends on many variables like geometric design, transverse section and road signs but also depends on the road surface conditions, more specifically, on the skid resistance and road - tire interactions.
Skid resistance is a functional property of the pavement that contributes to maintain dynamic stability of the vehicles traveling along horizontal curves, and to provide an adequate braking distance between vehicles. For this reason, skid resistance is periodically monitored using high-performance devices like SCRIM, Grip Tester or ASTM trailers, among others. Such devices allow obtaining data series of skid resistance measurements in long sections.
Quality control of skid resistance measurement procedures and used devices is very important to ensure a minimum level of reliability for the results obtained in the field. It also allows verifying that the safety conditions required by the vehicles to travel or brake safely are fulfilled. Therefore, it is essential to ensure that the measurement devices and also the procedures for data collection, processing and evaluation reach minimum levels of quality, according to local and international standards.
The main objective of is paper is to present and briefly explain the concepts, contents and scope of a quality control system for skid resistance in roads. First, a short discussion of repeatability and reproducibility concepts, who are key indicators in the control system, is outlined. Afterwards, there is a general explanation of the proposed certification system, and finally the systems developed to certificate devices and measurement procedures are described with more detail.
2. Main concepts about quality measurements
The quality of a measurement system can be described by characteristics such as statistical stability, low variability and low skew. This can be quantified by indicators of accuracy, precision and stability (Kenett and Zacks, 2000). The level of precision can be estimated by repeatability and reproducibility indexes.
Repeatability means the capacity of a specific procedure to obtain measurements with a reduced level of dispersion. A measurement system has good repeatability if multiple measures of the same variable in the same place, with the same device and under similar conditions are statistically equivalent.
Reproducibilty, instead, is associated to the variance between different procedures that measure the same variable. A measurement system has good reproducibility if many measurements collected with different devices in the same place and under the same conditions are statistically equivalent (Montgomery, 1991). The procedure to determine both indicators is conceptually similar, based on ANOVA (Analysis of Variance) tests, which are available in most of the commercial statistical software.
3. Quality control system
The structure of a control system designed to verify and certify an acceptable level of quality in skid resistance measurement and processing for pavement surfaces is described in this paper as follows. Such structure is based on the calculation of statistical indexes as repeatability and reproducibility. But these concepts are also applied to verify the quality and appropriateness of the evaluation process in itself, to achieve a mechanism of evaluation coherent and homogeneous, as independent as possible regarding the personal characteristics and conditions of the evaluators involved in that process of quality certification. The designed system described in this paper has been developed to evaluate quality of the British Pendulum and GripTester devices (Figure 1).
Figure 1. Skid resistance measurement devoces considered in the proposed Certification Systems
There is a particular need that motivates to develop such types of system in Latin America. In the countries of this continent, equipments to measure skid resistance like the ones shown in Figure 1 have to be periodically sent to their original factories (located in Europe in most cases) to be checked and re-calibrated. But it implies an excessively long period of time until these devices return to Latin America, as it would also happen to any country located far away from the European factories, and the organizations that own them usually don't have additional devices to make such measurements meanwhile the others are being verified and calibrated. For that reason, the re-calibration and checking processes are usually delayed more time than advisable. This situation incorporates uncertainty in the level of accuracy and calibration of the measurement devices, and in the reliability of the data measured with them. Therefore, as an alternative, the proposed Quality Certification Systems are intended to provide a similar type of procedures to control the quality level of the SR measurements obtained with these devices, without having to send them to their origin factories to be checked and calibrated.
4. Skid resistance measurements certification system
The main objective of the skid resistance Measurements Certification System (MCS) is to define a simple and precise methodology to evaluate and certify the quality of SR measurement and processing procedures. Also, MCS has the following secondary objectives:
• Define the most important aspects having to be verified in each stage of the methodology, and the different attributes or conditions to fulfil in each case.
• Assign different weighing levels for aspects and attributes, according to its relative importance on the quality of the procedures.
• Elaborate criteria for evaluating and assigning global qualifications to the quality level achieved for the analysed procedures.
• Apply the designed criteria to evaluate operators and procedures, back-feeding the evaluation mechanism.
For a better comprehension, the following definitions are established:
• Processes: generic procedures whose quality has to be evaluated, like SR measurements or the calibration of devices.
• Aspects: corresponding to different global stages of the processes. For instance, preparation for measurements, protective measures for the elements used in field, and checking safety conditions, all these are aspects of the measurement process.
• Attributes: inside each aspect there is an important number of specific attributes, corresponding to conditions that must be accomplished. Attributes can be "critical" or "not critical". An attribute is critical when its omission or inadequate fulfillment affects significantly the global quality of the aspect to which pertains, or also when it is essential to verify subsequent attributes.
• Checklists: the lists of aspects and attributes to be evaluated in each process, specifically designed to verify the quality level achieved for each one.
• Operators: persons who operate and use the devices to measure SR on the pavements, and who are also evaluated as a part of the certification process.
• Devices: the SR measurement equipments that have to be evaluated, in this specific case, British Pendulum and GripTester.
• Evaluators: persons who inspect and qualify the execution of the processes applied by the operators, and also who certify the quality and calibration of the devices.
4.1 General structure of Measurements Certification System MCS
MCS basically consists on the application of a set of checklists to evaluate operators, by independent evaluators. The methodology is integrated by the following steps (Figure 2):
Figure 2. General structure of the SR Measurements Certification System
a) Operator instruction phase: the evaluators provide the operators the current measurement standards and inform them about the different guidelines and evaluation criteria to be applied during the evaluation.
b) Measurements evaluation phase: measurement tests are applied on paved sections previously selected. The evaluators should qualify each one of the attributes incorporated in the checklists, to assign a global qualification to the aspects applied by the operators, according to MCS guidelines and their own criteria.
c) Post-processing evaluation phase: by analysing the measurement reports provided by the operators, the evaluators should assess each attribute and aspect defined in the checklists.
d) Global Qualification phase: each evaluator should calculate and provide a global qualification about the level of quality achieved by the operator during the application of the evaluated processes.
e) Certification phase: certifications will be provided to each process approved with an acceptable level of quality in the evaluations.
4.2 Checklists to evaluate processes
Operator evaluation is performed with checklists that represent a set of conditions to be accomplished during the measurement processes. Two main classes of lists have been incorporated in MCS.
Checklists for measurement procedures
They are designed to evaluate if the operators follow both measurement standards and procedures defined by device constructors. To do that, each list has three main aspects to be evaluated:
• Aspect 1: Device adjustment and preparation before measurements
• Aspect 2: Measurements execution and data collection
• Aspect 3: Safety measures adopted during data collection procedures
Each of these aspects is composed by "critical" and "not critical" attributes. Critical attributes have a higher weight than the not critical ones, regarding to the final qualification. Also, each aspect has a different weight.
Data processing checklists
These lists are designed to evaluate if the operator performs correctly the sequence of steps established in the standards to process and filter the data collected in field, and to determine the representative indexes (average values and standard deviations) that have to be presented in the reports. There is only one aspect included in the checklists, who is composed by a set of attributes defined according the methodology outlined in the measurement standards for each device.
An example of the checklists developed for the MCS is presented in Figure 3, to evaluate a certain aspect, to describe the way as the weighed qualifications are calculated. Critical attributes are marked in gray, having double value of its weighing coefficient if compared against the non critical attributes.
Figure 3. Example of checklist application, to evaluate a specific aspect of measurement process using GripTester device (de Solminihac et al., 2006)
• All the attributes defined as "critical" must have a qualification equal or higher tan "acceptable", depending on the scale used for qualifications.
• For each aspect, there couldn't be more than one "not critical" attribute with qualification lower than "acceptable".
• Final Qualification (FQ) must be equal to or higher than "acceptable".
Quality checking of evaluation procedures
The global quality of the evaluation methodology previously described was verified with field tests carried out in Chilean road with collaboration of National Roads Laboratory of Chile, where different evaluators applied the checklists to qualify the performance shown by operators measuring skid resistance with British Pendulum and GripTester. Afterwards, statistical tests were performed with the results of the evaluations, as follows:
• Test of Average Values: to evaluate the difference between the average of qualifications calculated by different evaluators, for measurement and data processing procedures.
• Repeatability and Reproducibility Tests: These tests are to analyse global qualifications and also by each aspect, to verify the level of independence of the checklists regarding to each evaluator.
• Capacity Analysis: It allows evaluating the quality of the process in itself, analysing the level of dispersion of the results with respect to predefined tolerance thresholds. The index is the "potential capacity" Cp of the process, calculated considering the standard deviation of the process; as the value of Cp increases, it also shows that the quality of the process in augmented, because its variation is being reduced.
These types of tests correspond to Six Sigma techniques to improve quality (Escalante, 2004), frequently used to evaluate industrial processes. Applying such methodologies, after many adjustments of the checklists, the tolerances predefined for each quality indicator were finally achieved, both for GripTester and British Pendulum checklists.
The test of Average Values showed relative differences below 10% in all cases, both for British Pendulum and GripTester checklists evaluation.
The normalized repeatability obtained for measurements performed with Grip Tester was 0.06 and the corresponding normalized reproducibiliy calculated was 0.02, in both cases below 0.07 which was the maximum threshold. Similar results were achieved for British Pendulum.
Capacity analysis showed a Cp value between 0.54 and 0.64 for British Pendulum, depending on the operator evaluated, and Cp = 1.35 was obtained for GripTester; in all cases, above the minimum predefined value of Cp = 0.45. The final results show that evaluation checklists for measurement and data processing are well designed and are highly independent from the individual evaluators.
5. System certification for devices
Following a similar structure, a Devices Certification System (DCS) was also conceptually designed. This system applies checklists to certify the level of calibration of the devices, to verify if the devices are adequately set and prepared following the recommendations made by their constructors, and the levels of precision and accuracy achieved by such devices. The potential users of this system would be those organizations interested in certifying the appropriate functioning of their devices for measurement purposes, such as road management consultants, academic institutions and road laboratories.
The checklists designed for this system are also based on weighed aspects and attributes that have to be evaluated by experts, who analyse and qualify each attribute to calculate finally a global qualification indicating the quality level achieved by the device.
5.1 Evaluation methodology for DCS
A global diagram for the whole evaluation procedure is shown in Figure 4, and it is integrated by the following stages:
a) Operators instruction phase: operators need to know what is the DCS and how the evaluations are performed. Only those operators previously certified by the Measurements Certification System can be evaluated in the DCS.
b) Evaluation of calibration procedures phase: the evaluators must verify the correctness of the routine calibration activities performed by the operators.
c) Precision and accuracy evaluation phase: once the SR measurements have been performed, as indicated in the DCS methodology, the evaluators should check if the requirements of repeatability and reproducibility (R&R), and also accuracy, are being accomplished. For accuracy evaluation, a "control" or "Master" device has to be defined firstly, against which the rest of the measurements performed with other devices Hill be compared. A good policy is to adopt as the Master device the equipment owned by the government road agencies, to ensure a well-known reference for all the other owners of similar devices.
d) Certification phase: All the devices that have been approved by the evaluators, shall receive certifications to accredit that they achieve more than a minimum quality level regarding the calibration procedures and also for precision (R&R) and accuracy during the measurement processes.
Figure 4. Global structure of the proponed Devices Certification System DCS
5.2 Requirement evaluation using checklists
Three main types of checklists have been constructed for the proposed DCS:
• Checklists to verify requirements predefined for test lanes
• Checklists to verify application of device calibration procedures
• Checklists to verify accuracy and precision achieved
Requirements for test lanes
For evaluation purposes in the DCS, skid resistance measurements should be executed in paved test lanes, that can be specifically constructed, or lanes of existent road can be used instead. The lanes selected for DCS must accomplish a set of requirements regarding their geometric characteristics, minimum length, pavement surface conditions, as follows:
• Straight alignment, without horizontal curves.
• Longitudinal slope as low as possible, and in any case, lower than 4%.
• If road lanes are used, AADT must be lower than 1000 veh/day, and also less than 30% of heavy trucks.
• Enough visibility and good safety conditions.
• Surface distress very low, for not affecting quality of measurements.
Detailed definitions of these requirements are presented in the corresponding checklists.
British Pendulum Calibration Checklists
The checklists designed to evaluate this type of devices consider three main aspects:
• Aspect 1: Mechanic verification of all parts of the device and its whole functioning
• Aspect 2: Calibration procedures
• Aspect 3: Repeatabilty check for spot measurements
The first aspect considers attributes related with functioning verification, device global condition and integrity of the different parts than compound a British Pendulum, like screws, pointer, rubber slider, etc. In the second aspect, attributes regarding calibration procedures are checked (alignment, weights, spring stress). Finally, the third aspect is intended to verify acceptable repeatability levels for spot measurement. Maximum standard deviation for five consecutive measurements in each spot has been fixed as 0.89 BPN units.
GripTester Calibration Checklists
In this case, the corresponding checklist evaluates also three main aspects:
• Aspect 1: Routine check and rapid verification procedures usually performed before measurements and also before calibration, revising relative movement of systems, alignment, transmission systems, tires, etc.
• Aspect 2: Check and adjustment procedures for different parts of the device, that can be verified annually.
• Aspect 3: Main calibration procedures (vertical and horizontal load).
Checklists to evaluate GripTester measurements precision and accuracy
Again, three main aspects to be accomplished by the set of measurements obtained with GripTester have to be evaluated.
• Repeatability Test: it compares two sets of SR measurements obtained in subsequent runs over the same test lane by a same GripTester device. The repeatability index is calculated using ANOVA statistical tests for both sets of data. That index is therefore divided by the lowest average value for both sets of measurements, and the result is the normalized or standardized repeatability, which must be under the specified tolerances. In the proposed system the maximum value acceptable for standardized repeatability is 0.15, to compare two subsequent runs.
• Reproducibility Test: the same tests are applied for two sets of measurements performed on the same lane by two different GripTester. The limit value suggested in this case for standardized reproducibility is 0.08, according to British Standards BS7941-2:2000, adopted as a reference in this case (BSI, 2000).
• Accuracy Test: the sets of measurements obtained by the Master device and the other device being evaluated are compared using the statistical test "Analysis of Average Values" for one factor, calculating confident intervals for the difference of averages for both sets of measurements (Escalante, 2004). If the zero value is included in those confident intervals, therefore the averages are statistically equivalent, and consequently the evaluated device shows a good level of accuracy, similar to the level achieved by the Master device.
5.3 Conditions for procedures certification
As it was defined for the Measurements Certification System MCS, in the case of devices the certifications are granted only if all the requirements are completely accomplished alter the corresponding evaluations. The conditions to be fulfilled are the same:
• All the critical attributes qualified at least as "acceptable".
• For each aspect, it must be no more than one attribute qualified as "not acceptable"
• Final global qualification must be higher than or at least equal to "acceptable".
If the requirements are not accomplished in a first evaluation, the device must be deeply inspected, and their pieces re-checked. Also, if in the phase of precision and accuracy evaluation the devices can't achieve acceptable results, a same type of deep re-checking of the device has to be performed. Once everything has been corrected and re-calibrated, a second evaluation takes place, and if there are still problems to approve the evaluations, at that moment it would be considered to send the device to their factory of origin, to be checked again and repaired if necessary.
Lack of precision can also be due to factors that are external to the device (non-adequate driving, bad weather conditions, safety problems, etc.), that have to be carefully verified during the evaluation process. And if it were the case, a new evaluation must be developed for the same device, taking care of not repeating the same unfavourable factors.
5.4 Quality verification for evaluation procedures
The checklists designed for the Device Certification System DCS were also evaluated to ensure the quality of the evaluation procedure in itself, analysing their level of applicability and their degree of independence regarding to different evaluators.
An experimental test was performed in Chilean roads for that purpose, to determine precision indicators for the evaluation procedure, using also statistical tests to check potential capacity, accuracy and reproducibility, according to Six-Sigma techniques. In all cases the results lead to an iterative process to refine and modify the checklists until a satisfactory final result was achieved, ensuring a good level of quality and independence of the evaluation procedures proposed for the DCS.
Potential capacity was evaluated for each checklist considering the qualificationes assigned by the evaluators. Table 1 show the final qualifications obtained using each designed checklist and for each evaluated device, and also the corresponding Cp value. All Cp's were clearly above the minimun acceptable value (Cp > 0.48).
Table 1. Final qualifications and Cp values obtained for each checklist in DCS
Accuracy was evaluated through an analysis of normalized differences in average values obtained between the assigned qualifications, using each checklist. The values obtained for normalized differences were inside the range 0 - 0.05 in all cases, which fulfilled the objectives in that case.
Normalized reproducibility was calculated for each checklist, and the results were in all cases under 0.03, which is below the pre-adopted maximum value of 0.04.
This paper presents the main characteristics of two quality control systems designed to certify the quality of processes related to skid resistance measurements in paved roads, named Measurements Certification System and Devices Certification System, respectively. These systems are based on checklists that allow evaluating specific aspects of the SR data measurement and processing, device calibration and quality of data reports. Evaluators use these checklists to verify if the evaluated procedures achieve an acceptable level of quality.
Also, the quality of the evaluating procedures was verified through specific tests, obtaining statistical indexes as potential capacity, repeatability, reproducibility and accuracy. All those indicators fulfilled the tolerances predefined, clearly showing that the evaluation procedures are efficient and independent on the evaluators. Therefore, those systems allow to verify, in the practice, if the conditions required to ensure a safe displacement or circulation of the vehicles are adequately accomplished, both analysing devices and measurements performed by them. From the road safety and infrastructure management viewpoints, these systems constitute a very useful tool for quality assurance, taking into account that a good measurement of surface skid resistance in a pavement, is a key factor to achieve a satisfactory degree of safety in the actual traffic using the roads.
Authors wish to thank FONDEF, of CONICYT Chile, for financial support provided along the project FONDEF D03I-1042. The authors are also grateful for the support and colaboration provided by the National Roads Administration, Ministry of Public Works of Chile, as a counterpart organisation during the project. They recognize the technical and economical support brought by the CINTRA group of road concessionary firms, that were also part of the project team.
BSI (2000), Surface friction for pavements - Part 2: Test method for measurement of surface skid resistance using the GripTester braked wheel fixed slip device. British Standards, BS 7941-2:2000. ISBN 0-580-33210-1. UK, September 2000. [ Links ]
De Solminihac H., Bustos M., Echaveguren T., Chamorro A. (2006), Servicio de Acreditación de Mediciones de Fricción con Péndulo Británico y GripTester: Resultado Final. Proyecto FONDEF D03I-1042, Santiago de Chile, Marzo 2006. [ Links ]
Escalante E. (2004), Sigma-Seis: Metodología y Técnicas. Editorial Limusa, Noriega Editores, México D.F., 2004. [ Links ]
Kenett R. and Zacks S. (2000), Modern Industrial Statistics, Thompson Learning, Mexico. [ Links ]