Introduction
Digital dermatopathology has demonstrated clinical efficacy in large academic centers, however, has yet to develop wide scale adoption in private laboratories. Proposed reasons for delayed adoption have included cost and availability of resources in a practically resourced setting. Historically, implementation and validation of the methodology has been a hinderance to adoption, however, several studies have demonstrated diagnostic equivalence of digital pathology (DP) to traditional microscopic (TM) methods using published and updated College of American Pathologists (CAP) guidelines. DP has been validated in the digital dermatopathology space, as dermatopathology is uniquely positioned for digitization, given the small tissue sample sizes and fewer tissue blocks required per case.
Discussion of specific technical validation requirements are discussed in the literature with varying results, including; proprietary FDA-Approved devices, specifications regarding diagnostic reading rooms, DP specific viewing monitors, and technically experienced staff. Previous publications demonstrate “improved technical performance” with relatively expensive technical hardware and software, though initial validation studies and the current CAP guidelines recommend validation of the system as a whole, not specific instrumentation. Suggesting the specific technical upgrades available may be minor non-clinical improvements to the overall DP system.
For private laboratories, identifying a return on investment (ROI) for capital expenditures is an commonly reported for delayed adoption. Few studies have demonstrated DP cost-benefit analysis, however, each DP installation requires its own independent review of its existing resources and ROI.
Motic is a cost-effective digital pathology provider with a desktop scanner for use in conjunction with companion cloud-based software, MoticFlow, for case visualization, database management, and report generation. (Motic USA, San Francisco, CA) Herein we describe the implementation, validation, and cost analysis of a digital dermatopathology solution in a private dermatology lab, using the Motic suite of instrumentation.
Methods
In accordance with CAP Guidelines, the study utilized routine dermatopathology cases selected by non-reviewers. The cases were distributed to three board-certified dermatopathologists for TM and DP review with a minimum 2-week washout period. TM and DP reviews were non-uniformly performed to reduce observer bias. The entirety of the process, including slide preparation, digital scan quality review, and diagnostic slide review/reporting was time tracked and reported via an Excel (Microsoft, Bellevue, WA) reporting worksheet. The scanner and associated computer workstation were placed in a private four provider dermatopathology laboratory (index laboratory) in which the technical component (TC) occurs locally, and professional component (PC) occurs at an offsite location by a board-certified dermatopathologist.
Figure 1: The DP arm of the study used MoticEasyScan Pro scanner, MoticFlow software, and an Amazon cloud-based server for storing retrieving, and reporting slide and clinical data. The slides were scanned at 40x magnification using the vendor default specifications.
Data and slide management for the two arms of the study, DP and TM, were performed in a simulated clinical environment, presenting simulated dermatopathology cases with matched histology slides and clinical data. A MoticEasyScan Pro scanner, MoticFlow v2.0, and an Amazon cloud-based server were utilized for slide and clinical data storage, retrieval, and reporting for the DP arm of the study. Slides were scanned at 40x with default vendor specifications. The TM component of the study utilized a laboratory information system-analog Excel database which imported clinical information when the slide number was entered. Diagnoses in both arms were rendered in a blinded fashion and amongst dermatopathologist reviewers. Staff and dermatopathologists were appropriately trained to utilize the digital slide scanner and software, prepare slides for scanning, assess scans for slide quality, accession cases in MoticFlow, and review and report simulated cases in MoticFlow. Staff received 3 hours of onsite training while reviewing pathologists received a 30-minute training on access and use of both reporting software (MoticFlow and Excel based LIS-analog).
Task | Rate (if applicable) | Cost |
|---|---|---|
Motic Pro6 Scanner + PC | One time | $30,000 |
MoticFlow Software Licensing Fee and Cloud Based Server Space (10TB) | Annual | $7,000 |
Motic Training/Installation | One time | $1,700 |
Technician | Hourly Rate | $30 |
Pathologist | Hourly Rate | $225 |
Shipping (overnight) | Daily | $95 |
Table 1 – Associated tasks with associated costs and assumed estimates
A total of sixty-four (N=64) cases with a combined one hundred and thirty-four (N=134 slides) slides were randomly selected with relatively even distribution amongst histologic categories (melanocytic N=20, inflammatory lesions N=20, and non-melanoma skin cancers (NMSC) N=24), respectively. The case and slide number represents an average one-day case volume at the index dermatopathology laboratory. Each case was de-identified by non-reviewers and clinical history including age, gender, site, and clinical impression were retained, see supplemental table 1. No clinical photos were included in the study. Each case retained the original histologic diagnostic material (H&E, special stains, and immunohistochemical stains) available at the time of primary TM diagnosis and was made available for reviewing pathologists. The cases were reviewed for tissue and scan quality by a trained technician who routinely performs all other laboratory duties. TM was reviewed first for two reviewers, while one reviewer initiated the study with DP.
Concordance was measured similarly to previously published studies, with minor and major disagreements. Overall, concordance was assigned based on interpreted clinical outcome rather than diagnostic subcategorization. Minor disagreement demonstrated a diagnostic disagreement without clinical impact while major disagreements included clinical impact. Given, the nuance of interpretation between diagnostic categories (melanocytic, NMSC, and inflammatory cases) slight modifications to the interpretation of disagreement was utilized. Briefly, melanocytic lesions required an additional comment if re-excision was required based on the dermatopathologist’s interpretation. If there was diagnostic disagreement, but no change in the treatment, the case was reported as a minor disagreement. If there was treatment impact, a major disagreement was recorded. Inflammatory cases were deemed concordant if diagnoses were given the same histologic pattern subcategorization, ie: spongiotic dermatitis, psoriasiform dermatitis, etc, as the clinicopathologic correlation by the reviewing provider would allow an accurate final diagnosis. NMSC were concordant only if histologically concordant.
Cost analysis was performed using the vendor provided price and time-based cost calculation based on capital expense and performing all necessary steps following slide generation in this laboratory environment. Comparison of the time (hh:mm:ss) and financial impact on a dollar per slide ($/slide) and per workday ($/workday) basis were used. A 250-workday calendar was used given the index laboratory operational cycle. The time duration for histologic processing was consistent between the two methodologies and no modifications to the baseline TM process were required for digitization. For DP, the slide preparation, slide scan time, upload time, and quality review were determined to be discrete additive steps relative to TM. The added time cost of digitizing and evaluating a single histologic slide assumed an estimated locum hourly rate for technician time ($30/hr) and pathologist time ($225/hr). Notably in this example, slide shipment was also a discrete component of TM, as the index laboratory utilizes an offsite dermatopathologist with appropriate shipment and handling. The list of activities encompassed in this study and actual associated time costs are seen in table 1.
Statistical analysis for diagnostic concordance included overall intraobserver concordance TM-DP, overall TM interobserver concordance, and DP interobserver concordance. Concordance for each histologic category was also performed. Comparative interpretation used 95% confidence intervals (CI) using score method incorporating continuity correction.
Concordance (Overall) | Concordance (Major Disagreements only) | |
|---|---|---|
TM-DP Intraobserver | 93.8% (95% CI 89.3-96.7) | 97.9% (95% CI 94.8-99.4) |
Melanocytic Cases (N=20) | 83.3% (95% CI 71.5-91.7) | 93.3% (95% CI 83.8-98.2) |
Inflammatory (N=20) | 98.3% (95% CI 91.1-99.9) | 100% |
NMSC (N=24) | 98.6% (95% CI 90.3-99.7) | 98.6% (95% CI 90.3-99.7) |
DP-DP Interobserver | 95.3% (95% CI 91.3-97.8) | 99.5% (95% CI 97.1-99.1) |
TM-TM Interobserver | 93.8% (95% CI 89.3-96.7) | 96.4% (95% CI 94.0-99.2) |
Table 2 – Intra- and Interobserver Concordance Comparison
Results
Sixty-four cases were reviewed by three dermatopathologists, rendering a total of 192 interpretations, with an overall TM-DP intraobserver concordance of 93.8% (95% CI 89.3-96.7), see Table 2. Histologic category TM-DP intraobserver concordance for melanocytic 83.3% (95% CI 71.5-91.7), inflammatory 98.3% (95% CI 91.1-99.9), and NMSC 98.6% (95% CI 90.3-99.7). (Table 3) Of the minor (N=7) and major disagreements (N=5, Table 4), four major disagreements (80%) occurred within the melanocytic category, while the remaining case was from the NMSC (N=1) category. Only 1 discordant case occurring the inflammatory category, a minor disagreement, see figure 4. Including only major disagreements (cases with treatment impact), overall concordance is 97.9% (95% CI 94.8-99.4). Overall DP interobserver concordance is 95.3% (95% CI 91.3-97.8) while overall TM interobserver concordance is 93.8% (95% CI 89.3-96.7).
Category | TM Diagnosis | DP Diagnosis | Consensus Diagnosis |
|---|---|---|---|
NMSC | Irritated and Inflamed Seborrheic Keratosis | Keratoacanthoma/Invasive Squamous Cell Carcinoma (margins negative) | Keratoacanthoma/Invasive Squamous Cell Carcinoma (margins negative) |
Melanocytic | Compound Dysplastic Nevus with Severe Atypia | Inflamed compound dysplastic nevus with mild Atypia (halo effect) | Inflamed compound dysplastic nevus with mild atypia |
Melanocytic | Compound Dysplastic Nevus with Severe Atypia | Compound dysplastic nevus with moderate atypia | Compound dysplastic nevus with moderate atypia |
Melanocytic | Compound Dysplastic Nevus with moderate-severe Atypia | Compound dysplastic nevus with moderate atypia | Compound nevus |
Melanocytic | Invasive melanoma | Compound dysplastic nevus with moderate atypia | Compound dysplastic nevus with moderate atypia |
Table 3 – Interobserver TM-DP Discordant Cases – Major Disagreement Only
One hundred and thirty-four slides (N=134) slides were scanned with total slide preparation and scan time measured as 475 minutes (mean: 00:03:55 mins/slide). Upload time for all slides was 327 mins (mean: 00:02:26 mins/slide) with a final quality review time of 120 mins (mean: 00:00:54 sec/slide). Total added preparation time 0:06:52/slide. Two dermatopathologists had comparative DP time tracking relative to the TM, as the dermatopathologist TM diagnoses were not time stamped. Pathologists completed the digital review with an additional 36 mins and 76 mins, respectively, for an added average review time of 56 mins (+00:00:25 /slide).
The MoticEasyScan was installed with training for $38,700* USD in year one, with a recurring $7,000 annual MoticFlow service plan. Using 250 working days per year, the capital cost per day over 1 year is $154.80/working day or $58.96/working day over 5 years. Comparative logistics cost (for TM in the index laboratory example) the daily estimated shipping cost (FedEx, Nashville TN) is $95/day for next day delivery. Using this shipping expense as a breakeven point, breakeven on the complete digital pathology solution ($38,700 first year and $7,000/annually thereafter) comparative to traditional microscopic solution ($23,750/annually for shipping) is realized in just under 2 years (1.92 years, using 250 working days), with cost savings in year 3 (+18,550/year 3).
[*The price mentioned for the MoticEasyScan in this white paper reflects the 2021 pricing and may be subject to change.]
The added cost of preparation time per scanned slide with DP is +$3.43. The DP dermatopathologist review time cost +$1.01/slide and $2.13/slide (mean $1.57/slide), respectively. The mean total added cost for DP is $5.00/slide (ranges $4.44-$5.56/slide). Turn-around-time savings was not a definitive endpoint of the study, however, the total around time savings when comparing DP to TM are >24:00:00 given the shipping delay and requested stain/IHC shipping delay.
Figure 2: Results of intraobserver concordance in digital slide scanning for dermatopathology: 93.8% overall concordance with minimal disagreements, as demonstrated in 64 cases by three dermatopathologists.
Discussion
The study demonstrates the Motic system (MoticEasyScan Pro 6, MoticFlow) with cloud-based server is a robust and capable system of handling a dermatopathology practice with overall TM-DP intraobserver concordance of 93.8% (95% CI 89.3-96.7) and 97.9% (95% CI 94.8-99.4) if including major disagreements (with treatment impact) only. The study also demonstrates non-inferiority of DP interobserver concordance 95.3% (95% CI 91.3-97.8)) to the TM microscopy system (interobserver concordance 93.8% (95% CI 89.3-96.7)) across multiple board-certified dermatopathologists, while meeting the CAP required validation requirements. This concordance is in alignment with previously published studies.
Figure 3: Melanocytic neoplasm concordance, especially the grading of dysplastic nevi, is well-documented to exhibit low levels of diagnostic interobserver correction. This study confirms these prior findings, as grading of nevi demonstrated the highest disagreement among observers in our analysis.
Diagnostic subcategories demonstrated similar concordance and reasons for discordance as previously published studies (melanocytic 83.3% (95% CI 71.5-91.7), inflammatory 98.3% (95% CI 91.1-99.9), and NMSC 98.6% (95% CI 90.3-99.7)). Melanocytic neoplasm concordance, particularly grading of dysplastic nevi, is known to harbor low diagnostic interobserver and intraobserver correlation. These findings are mirrored within this study with grading of nevi being the highest discordant feature (N=4/4 melanocytic lesions), example shown in figure 2. Unsurprisingly, this is seen in interobserver concordance of TM and DP cases. Interestingly, this discordance was identified via TM and not DP, the significance uncertain with this limited sample size. In the inflammatory category, only one discordant case was identified as a minor disagreement, see figure 4. An annular erythematous pustular plaque on the leg, diagnosed as Majocchi’s granuloma. However, with two observers re-reviewing the digitized PAS-D, the tissue biopsy fragment with intrafollicular fungi was overlooked on re-review and a diagnosis of spongiotic epidermis with folliculitis without fungus was made. This is to be considered without patient impact, as folliculitis was identified. This discordance is a known pitfall of DP and TM methodologies. Other identifiable causes of discordance is “degree of atypia”, as seen in the discordant NMSC case, see figure 3. In which a diagnosis of inflamed verrucal keratosis was made via TM however via DP a keratoacanthoma type squamous cell carcinoma was rendered.
Figure 4: As instance of discordance in the diagnosis of a non-melanoma skin cancer case, attributed to differences in the assessment of the ‘degree of atypia’. While one observer diagnosed an inflamed verrucal keratosis using the traditioanl method, another observer rendered a diagnosis of keratoacanthoma-type squamous cell carcinoma using the digital pathology approach.
Cost analysis reveals a total one time added expense for digital pathology to range from $10,000 – $95,000 (range based on low, medium, or high throughput MoticEasyScan models) with an additional $7,000/annual cost for software and data management. In the index example laboratory setting with recurring shipping expense, breakeven on logistics of digitization is achieved in less than 2 years. Other previously published studies report a capital expense breakeven in approximately 7 years. Notably, in this cited example authors also include storage and file retrieval costs and subsequent savings for a tertiary large scale pathology practice. The capital cost per day over 1 year is $154.80/working day or $58.96/working day over 5 years. In this example, by year 3 the digitization process becomes cost conservative to the current practice starting at $18,550 saved beginning year 3. This analysis assumes existing local and remote practice has a suitable internet connection and infrastructure and does not require alteration of their histology practice. Examples could include restricted field of view to encompass relevant tissue requiring additional tissue sections or changes required for LIS/electronic health record integration or additional cost for higher speed internet connections at CLIA-certified reviewing sites.
Time-cost analysis demonstrates an added preparation time +0:06:52/slide with an estimated associated cost of +$3.43/slide for digitization. Diagnostic review time digitally, added an average review time of 56 mins (+00:00:25/slide) and mean cost +$1.57/slide. Notably, this study was the first installation and attempt to digitize histologic specimen at this laboratory. As such, added expense could be reduced due to efficiencies in practice. One example includes, significant time added (~02:00:00 hrs) to the quality review and re-review with re-scan given poor slide preparation. In a prior study by Stratman et al, they demonstrate a savings of 13.4% time savings to a pathologists workday with a time and motion study by automating case assembly, queries, requests, retrieval, and delivery. Mills et al, demonstrate a 00:00:05s/case increase relative to surgical pathology specimen, however, this is not well studied in dermatopathology. They did recognize that with prolonged adoption, reviewer times significantly decrease.
In this labs example, an annualized added expense per slide with digital pathology yields an estimated $167,500 total preparation and review cost annually (assuming 134 slides/day for 250 days). Logistics expense savings of $18,550 in year 3 and $16,750 per year thereafter. The estimated added total annual cost of approximately $150,750. Prior studies do not specifically identify time/cost analysis of producing a high-quality digital image for review. Potential cost saving and quality improving measures not directly evaluated in this study, but reported by others include turn-around time savings, reducing microscope expenses, quality improvement and reduced cost from intradepartmental/extradepartmental consultation/re-review and slide retrieval, improved quality of diagnosis by subspecialists and subsequent reduction of error and over/undertreatment, and reduction of potential slide storage cost. Many of the additionally previously cited savings are not relevant to the private dermatopathology laboratory, such as reduction of laboratory and dermatopathologist FTE due to efficiencies in practice and expanded workloads, though workflow throughput may be expandable without FTE addition. Added future benefits could also include reference technical component work for other providers (digitized H&E, IHC, special stains) and potential for artificial intelligence algorithms with diagnostic and workflow improvement. These algorithms have demonstrated significant promise for the future, however, have not significantly impacted dermatopathology at the time of this publication.
Limitations do exist in this study, notably, the index dermatopathology practice has an approximate annual case volume of 17,500 cases. The MoticEasyScan Pro 6 is a desktop 6-slide scanner which is best suited for relatively low volume scanning and an appropriate assessment of workload would be required for additional histology work. Another theoretical limitation is the cloud-based server solution, as this may not be available in all physical settings (hospital based, community, independent lab, etc) and require extensive discussion and initial work to establish an appropriate IT solution for each application. Motic does allow for a private server-based connection to MoticFlow however each application is customized for the users needs. With respect to the diagnostic concordance and cost assessment, the study is retrospective in nature with limited sample size limiting annual cost estimates.
Conclusion
The digitization of dermatopathology in a private laboratory using the Motic system (MoticEasyScan Pro 6, MoticFlow) with cloud-based server is a robust and diagnostically capable system with added cost relative to the TM methodology.
Reference:
- Lee, J.J., et al., Validation of Digital Pathology for Primary Histopathological Diagnosis of Routine, Inflammatory Dermatopathology Cases. Am J Dermatopathol, 2018. 40(1): p. 17-23.
- Shah, K.K., et al., Validation of diagnostic accuracy with whole-slide imaging compared with glass slide review in dermatopathology. J Am Acad Dermatol, 2016. 75(6): p. 1229-1237.
- Retamero, J.A., J. Aneiros-Fernandez, and R.G. Del Moral, Complete Digital Pathology for Routine Histopathology Diagnosis in a Multicenter Hospital Network. Arch Pathol Lab Med, 2020. 144(2): p. 221-228.
- Evans, A.J., et al., Validating Whole Slide Imaging Systems for Diagnostic Purposes in Pathology. Arch Pathol Lab Med, 2022. 146(4): p. 440-450.
- Hanna, M.G., et al., Validation of a digital pathology system including remote review during the COVID-19 pandemic. Mod Pathol, 2020. 33(11): p. 2115-2127.
- Montezuma, D., et al., Digital Pathology Implementation in Private Practice: Specific Challenges and Opportunities. Diagnostics (Basel), 2022. 12(2).
- Ho, J., et al., Can digital pathology result in cost savings? A financial projection for digital pathology implementation at a large integrated health care organization. J Pathol Inform, 2014. 5(1): p. 33.
- Hanna, M.G., et al., Implementation of Digital Pathology Offers Clinical and Operational Increase in Efficiency and Cost Savings. Arch Pathol Lab Med, 2019. 143(12): p. 1545-1555.
- Al-Janabi, S., et al., Whole slide images for primary diagnostics in dermatopathology: a feasibility study. J Clin Pathol, 2012. 65(2): p. 152-8.
- Vodovnik, A., Distance reporting in digital pathology: A study on 950 cases. J Pathol Inform, 2015. 6: p. 18.
- Clarke, E.L., et al., Display evaluation for primary diagnosis using digital pathology. J Med Imaging (Bellingham), 2020. 7(2): p. 027501.
- Norgan, A.P., et al., Comparison of a Medical-Grade Monitor vs Commercial Off-the-Shelf Display for Mitotic Figure Enumeration and Small Object (Helicobacter pylori) Detection. Am J Clin Pathol, 2018. 149(2): p. 181-185.
- Mills, A.M., et al., Diagnostic Efficiency in Digital Pathology: A Comparison of Optical Versus Digital Assessment in 510 Surgical Pathology Cases. Am J Surg Pathol, 2018. 42(1): p. 53-59.
- Williams, B.J., et al., Future-proofing pathology part 2: building a business case for digital pathology. J Clin Pathol, 2019. 72(3): p. 198-205.
- Newcombe, R.G., Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med, 1998. 17(8): p. 857-72.
- Duncan, L.M., et al., Histopathologic recognition and grading of dysplastic melanocytic nevi: an interobserver agreement study. J Invest Dermatol, 1993. 100(3): p. 318S-321S.
- Stratman, C., L. Drogowski, and J. Ho, Digital pathology in the clinical workflow: A time and motion study. Pathology Visions, 2010.
- Hartman, D.J., et al., Enterprise Implementation of Digital Pathology: Feasibility, Challenges, and Opportunities. J Digit Imaging, 2017. 30(5): p. 555-560.
