Sniffer dogs performance is stable over time in detecting COVID-19 positive samples and agrees with the rapid antigen test in the field | Scientific Reports – Nature.com

The study was approved both by the University of Milan Ethical Committee (CE_26_21 and CE 84/21) and the Institutional Review Board of the European Institute of Oncology (R90/14-IEO102), in accordance with the relevant guidelines and regulations, and individual written informed consent was obtained from all the study participants after appropriate information concerning the study was provided. Eligible participants were people of either gender, older than 7 years; in the case of minors, written informed consent was provided by a legal tutor. Participation to provide a sweat sample or to be sniffed by trained dogs was volunteer. Anti-COVID-19 vaccination was not an exclusion criterium. Whenever possible, the type, number of dose and date of vaccine administration were also recorded after the information was provided on a voluntary basis.)

Phase 1: Laboratory training and testing

The first phase of the study consisted of training dogs to discriminate in the laboratory between sweat samples from patients with COVID-19 and sweat samples from healthy controls.

Participants and sample sourcing

The samples were collected from: the III Infectious Diseases Unit, L. Sacco Hospital, ASST Fatebenefratelli-Sacco, Milan, Italy; the Division of Thoracic Surgery of the European Institute of Oncology (IEO), Milan, Italy; COVID-19 screening stations across the North of Italy.

Participants were divided into two groups as shown in Table 6:

1. 1.

   Cases, N. 146 patients COVID-19 positive, who had tested positive by RT-PCR for SARS-CoV-2 within 24 h prior collection, regardless of COVID-19 symptoms.

2. 2.

   Controls, N. 571 patients COVID-19 negative, who had tested negative by RT-PCR for SARS-CoV-2 within 24 h prior collection.

The number of samples was calculated based on the number of training and test–retest sessions required according to the training technique we developed, so that dogs could be presented with several samples while ensuring a case: control ratio of 1:5. Samples collected were used multiple times only during training.

Sample collection and handling

Armpit sweat samples were collected following the same standard protocol for all volunteers and using the same media across all locations. The volunteers were asked to hold an inert polymer tube (3.6 cm length, 0.8 cm thickness) commonly used for adsorbing VOCs for explosive, drug, or criminology detection (Getxent, Neuchâtel, Switzerland) simultaneously under each armpit for 20 min. Each individual polymer tube was immediately placed in a sealed envelope, bearing the subject’s ID. The samples were then shipped while refrigerated (+ 4 to + 8 °C) to the laboratory of the dog training/testing centre, in separate packaging for cases and controls, and kept stored at a temperature of 4 °C. The choice to work with sweat samples was made, in accordance with what reported by Grandjean et al.⁴⁹, because sweat does not appear to be an excretion route for SARS-CoV-2 virus and can be easily collected in a non-invasive way. It appears however to carry COVID-19 VOCs⁴⁹. Moreover, sweat allows the anticipation of practical applications of trained dogs to detect SARS-CoV-2 virus on positive people.

Animals

Dogs were provided by local dog owners and screened for inclusion in the study. The initial screening phase included the assessment of the effectiveness of food as a reinforcer for each dog and the dog’s suitability for work in a laboratory environment in the presence of non-familiar people. Out of the 6 dogs that met the inclusion criteria, 3 dogs underwent the entire training session, while the others were exempted, due to either health or management problems, or low motivation for working in the scent line-up. One 8 years old Belgian Malinois female dog (Nala), an 8-year-old female mixed-breed dog (Helix), and a 5-year Dachshund male dog (Otto) underwent a training period (two weekly training sessions) to discriminate between the sweat of people with COVID-19 and the healthy controls. The training lasted 5 months for Helix and 7 months for Nala and Otto. All dogs had previous experience with bio-detection work (Helix: human lung cancer¹⁵, Nala and Otto: phyto aromas and Cimex lectularius).

The dogs were not used for other tasks during the study. All dogs lived at home with their owners, they were handled by professional dog trainers or behaviour scientists during the training and testing sessions, and were cared for by their owners between sessions.

Sniffing room and experimental equipment

Dog training and testing was conducted at the laboratory of Animal PhysioEthology of the Department of Veterinary Medicine and Animal Sciences, University of Milan, Lodi, Italy. Two tracks of three samples, especially designed for this study, and constructed of aluminium, were positioned in a single straight line, spaced 40 cm apart, on the floor, as in Fig. 1. A sterile container with the sweat-stained cylinders was put into each dedicated holder, which was protected by a perforated metal cover, for the dogs to sniff. The cylinder was not visible or accessible to the dogs other than by olfaction. For self-protection and to prevent cross-contamination, the study team wore powder-free nitrile gloves and masks when handling the containers and sample cylinders.

(a) The two especially designed stainless steel and aluminium apparatuses positioned in a single straight line on the floor, spaced 40 cm apart; (b) detail of the apparatus.

Full size image

Training procedure

The training aimed to teach the dogs to discriminate Case and Control sweats and indicate the cases. The training procedure was carried out by Medical Detection Dogs Italy’s bio-detection trainers and was based on operant conditioning, with a food reward provided for correct behaviour as described previously¹⁵. Briefly, two test days per week were scheduled, each with two sessions of at least five trials, depending on the dog’s level of motivation. In each trial, samples were selected to fill the 2 three-slot tracks so that one sample from the Case group was compared with five samples from the Control group, always varying the positive (Case) donor’s identity. To minimise position-related interference, the location of each sample was randomly changed throughout the trials. At the end of each session, the slots were changed and the sample tracks were thoroughly cleaned with a vapor machine (Vaporetto PRO 90 Turbo, Polti, Italy). On each trial, the dogs were required to indicate the positive samples by sitting, lying, or scratching directly in front of the holder containing the COVID-19 sample (Fig. 2). In the last part of training, empty runs implying presentation of all negative samples were also performed. The training was considered completed when the dog was able to detect one positive sample out of six samples, as confirmed by achievement of a success rate ≥ 80% in two subsequent sessions of five trials. All dogs reached this threshold.

Helix (a) Nala (b) and Otto (c) indicate the correct position respectively by sitting, lying, and scratching directly in front of/on the sample station.

Full size image

Double-blinded test–retest reliability assessment

The test and the re-test aimed to measure the accuracy of the training. Each dog was naïve to the samples they used during both the test and retest phases, meaning that they had never been presented before with one of those samples. However, more than one dog could be presented with the same sample from a given volunteer and dogs could sniff during the test or the re-test samples that another dog had used in training. For this reason, during these phases the three dogs were presented with a total of n. 360 unfamiliar samples (n. 300 negatives and n. 60 positives), who were collected from a total of n. 282 participants (n. 233 RT-PCR negative, n. 43 RT-PCR positive). During testing and re-testing, only the trainer was inside the room, and she remained hidden by the wooden screen (Fig. 1) while the dog performed the research work.

At the beginning of the test, before the dog and the trainer entered the room, one positive and five negative samples were selected and placed in the six holders by an experimenter, who left the laboratory soon after. The location of each sample was pre-determined by this experimenter in a pseudo-random order (designed using http://www.random.org), so that the positive sample was in a different location in each trial. A second experimenter was in an adjacent room, avoiding any interaction with the trainer and observing the dogs’ work and behaviour through a digital video-camera (GoPro HERO7, Italy), which was mounted on a wall and operated remotely. The study was double-blinded setting, since the type and position of the samples in the sniffing stations were unknown to both the trainer and the second experimenter.

The test took place on 2 days, 1 week apart, on which every dog performed 1 session of 5 trials. For each trial, a new set of samples of volunteers was used. However, some of the samples might have been used during training by another dog. At the end of each trial, the dog was rewarded verbally regardless of whether the marking was correct or not.

To assess the repeatability (or test–retest reliability)⁵⁰ of canine olfactory detection COVID-19 related VOCs in human sweat samples, a retest was conducted in two additional and successive sessions, 1 week apart, within 3 weeks after the second test session and under the same conditions. During this time, the dogs did not work in olfactory research.

Phase 2: field study

Animals

Five dogs underwent the training process for the field study. One of these was Nala, the Belgian Malinois who was also involved in the laboratory testing phase. The other dogs were Chaos (Golden retriever, male, 4.5 years old), Hope (Border collie, female, 6 years old), Iris (Golden retriever, female, 3 years old) and Nim (mixed breed, female, 12 years old). Only Nala and Nim had previous experience of scent work (phyto aromas and Cimex lectularius). These dogs were recruited based on their behavioural characteristics, such as search ability and fearfulness.

In field training procedure

The trainers and the method (operant conditioning with positive reinforcement and a food reward provided for correct behaviour) employed were the same as in laboratory testing except they did not use the apparatuses to hold the samples. Training for the field screening was carried out from September 2021 to mid-January 2022, and passed through two steps, as described below, during which the dogs learnt to identify COVID-19 patients by sniffing the volunteers directly. Step one training sessions were conducted at various drive-thru COVID-19 testing points after volunteers had received their negative antigen test result. This was necessary so that, adding a positive scent or not, could reliably allow for presenting the dogs with positive and negative scents on humans. Round-shaped metallic stainless-steel boxes with 6 holes (height: 1.2 cm; diameter: 3.6 cm), typically employed in nose-work, were used during the training. The boxes contained either a positive or negative sweat-stained cylinder. The boxes used for positive samples were not mixed with the boxes used for negative samples. In each training session, the dogs had to enter a room where a maximum of 5 volunteers lined up having a box hidden in their clothing, preferring areas where people are expected to sweat the most (e.g., sleeves, shoes, etc.). The location of the person holding the positive sample was always randomised in the line-up. The volunteers could either be sitting or standing in a queue, thus simulating a typical situation in a testing point, including hospitals and pharmacies, and dogs were required to indicate that they recognised the scent of COVID-19 by stopping in front of the person holding the box with the target odour (positive scent). As in the training phase, the scent was obtained from patients with a diagnosis confirmed by RT-PCR. Overall, the dogs were trained using 131 positive samples. None of the samples presented to Nala (who had participated in phase 1) came from volunteers who had participated in the laboratory testing phase. This training step lasted 2 months for each dog.

In step 2 of the field study, 192 incoming clients at community pharmacies volunteered for the study. In each trial, the dogs could sniff any part of the body of one volunteer and they were allowed to touch with their noses the body of the volunteer. This training step lasted 2.5 months for each dog.

The trials in step 1 were unblinded (e.g., the handlers knew the position and number of the sniffers containing the positive cylinders) to observe the spontaneous behavioural cues offered by each dog when alerting on the target odour (positive scent). In step 2, the trials were single-blinded, as the handlers were unaware of who had received a positive diagnosis. By contrast, the volunteers and the director of training, who was present during both phases, were always unblinded, and the director informed the dog-trainer dyad of every correct alert. The results of this training step are reported in Table 7 and supplementary Table 5.

In field screening procedure

In mid-January 2022, in-field screening begun on human volunteers in North Italy’s pharmacy COVID-19 testing queues, in order to screen the general population. Sessions were carried out once a week, between February 2022 and May 2022. During each daily session, each dog was asked to investigate a maximum of 10 people, depending on their motivation. The director of training was always present as an external controller, ensuring that the predetermined protocol was followed. For each session, the tests were triple-blinded with the diagnosis being unknown to everyone involved (handler, director of training, and participants). As in the in field study by Vesga et al.²⁷, the director had to interpret the behaviour of each dog to decide if a food reward was due. In the screening, the volunteers were sniffed out by a dog shortly after their COVID-19 rapid antigen test, and before receiving the results. Since we wanted to evaluate performance under real-life conditions, the dogs worked on leash as, in Italy, this is how they are required to be handled in all public areas.

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics for Windows, version 27.0 (Armonk, NY: IBM Corp).

As for phase 1 (laboratory testing), the sensitivity (Se) and specificity (Sp) of a dog’s indication of samples compared with the true diagnosis confirmed by RT-PCR were calculated. The sensitivity refers to the conditional probability of the dog indicating COVID-19 when the condition was present, and specificity refers to the conditional probability of the dog ignoring a sample from a healthy donor. Both sensitivity and specificity were expressed as proportions. Point estimates were calculated with 95% confidence intervals. The probability of a perfect test trial (finding the right sample and ignoring the controls) by chance was 1/6 (17%). Similarly to a previous study²², positive predictive values (PPVs) and negative predictive values (NPVs) with 95% confidence intervals were also calculated based either on our prevalence or on a high (40%) and low (3%) hypothetical scenarios of prevalence^40,51. PPVs and NPVs were obtained from Se, Sp, and prevalence according to the Bayes’ rule⁵²:

Percentage of agreement⁵³ and Spearman’s correlation³⁵ were computed to measure the degree to which dogs’ detection ability to detect COVID-19 was identical to the RT-PCR, and to establish temporal reliability, between the test and the retest, respectively. According to Schober et al.⁵⁴, Spearman’s correlation coefficient was interpreted as follows: 0.00–0.10 = negligible correlation, 0.10–0.39 = weak correlation, 0.40–0.69 = moderate correlation, 0.70–0.89 = strong correlation, and 0.90–1.00 = very strong correlation. A generalised linear model (GzLM) with binary response analysis was performed to identify either test- or volunteer-related factors (dog, RT-PCR diagnosis, testing phase, participant gender) influencing a dog’s correct response. The dogs’ response (yes/no) was entered as dependent variable and all factors were measured for main effects. The strength of the associations was expressed as odds ratio (OR) and 95% confidence interval (95% CI).

As for phase 2 (field work), Se and Sp were calculated compared with the RAD result. In each trial, the probability of success or failure was 50%. Thus, we calculated 95% CI of the Se and Sp and considered statistically different from a random choice those that did not overlap 50%, which is the randomness region.

Cohen’s Kappa was used to measure the agreement of the two methods (sniffer dog and RAD) to screen people. Results were interpreted as follows: values ≤ 0 as indicating no agreement, 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement⁵⁵.

For all the analyses, a two-sided p < 0.05 was considered statistically significant.

Ethical statement

We confirm that the procedures comply with national and EU legislation. Research was performed in accordance with the Declaration of Helsinki. The study was approved by the Animal Welfare Committee (OPBA) of the University of Milan (OPBA_06_2021). Before participating in the olfactory detection test, each dog owner gave informed written consent for using their dogs’ test results in research. Reporting of results follows the recommendations of the ARRIVE guidelines. Informed consent was given by each subject for publication of identifying images in an online-access publication.