Reproducible Experimental Design and Sample Prep

Designing a rigorous microscopy experiment: Validating methods and avoiding bias

Jost AP, Waters JC
J Cell Biol. 2019 May 6;218(5):1452-1466.

Images generated by a microscope are never a perfect representation of the biological specimen. Microscopes and specimen preparation methods are prone to error and can impart images with unintended attributes that might be misconstrued as belonging to the biological specimen. In addition, our brains are wired to quickly interpret what we see, and with an unconscious bias toward that which makes the most sense to us based on our current understanding. Unaddressed errors in microscopy images combined with the bias we bring to visual interpretation of images can lead to false conclusions and irreproducible imaging data. Here we review important aspects of designing a rigorous light microscopy experiment: validation of methods used to prepare samples and of imaging system performance, identification and correction of errors, and strategies for avoiding bias in the acquisition and analysis of images.

 

Link to paper

 

A biologist's guide to the field of quantitative bioimaging

BA Cimini et al

Technological advancements in biology and microscopy have empowered a transition from bioimaging as an observational method to a quantitative one. However, as biologists are adopting quantitative bioimaging and these experiments become more complex, researchers need additional expertise to carry out this work in a rigorous and reproducible manner. Here we provide a navigational guide for experimental biologists to understand quantitative bioimaging from sample preparation through image acquisition, image analysis, and data interpretation. We discuss the interconnectedness of these steps, and for each, we provide general recommendations, key questions to consider, and links to high-quality open access resources for further learning. This synthesis of information will empower biologists to plan and execute rigorous quantitative bioimaging experiments efficiently.

 

Link to preprint

FPbase: a community-editable fluorescent protein database

TJ. Lambert
Nature Methods volume 16, pages277–278 (2019)

FPbase is a free and open-source, community-editable database for fluorescent proteins (FPs) and their properties. The primary objective is to aggregate structured and searchable FP data that is of interest to the imaging community and FP developers. Each protein in the database has a dedicated page showing amino acid sequence, accession IDs (e.g. GenBank, UniProt), evolution lineages and mutations, fluorescence attributes, structural data, references that introduced or characterized the protein, and more. Excerpts from primary literature can be entered to store key information about a protein that is otherwise difficult to capture within the current database schema. 

 

Link to FPbase

Hypothesis-driven quantitative fluorescence microscopy - the importance of reverse-thinking in experimental design

Wait EC, Reiche MA, Chew TL
J Cell Sci. 2020 Nov 5;133(21)

One of the challenges in modern fluorescence microscopy is to reconcile the conventional utilization of microscopes as exploratory instruments with their emerging and rapidly expanding role as a quantitative tools. The contribution of microscopy to observational biology will remain enormous owing to the improvements in acquisition speed, imaging depth, resolution and biocompatibility of modern imaging instruments. However, the use of fluorescence microscopy to facilitate the quantitative measurements necessary to challenge hypotheses is a relatively recent concept, made possible by advanced optics, functional imaging probes and rapidly increasing computational power. We argue here that to fully leverage the rapidly evolving application of microscopes in hypothesis-driven biology, we not only need to ensure that images are acquired quantitatively but must also re-evaluate how microscopy-based experiments are designed. In this Opinion, we present a reverse logic that guides the design of quantitative fluorescence microscopy experiments. This unique approach starts from identifying the results that would quantitatively inform the hypothesis and map the process backward to microscope selection. This ensures that the quantitative aspects of testing the hypothesis remain the central focus of the entire experimental design.

 

Link to Paper

When light meets biology – how the specimen affects quantitative microscopy

Reiche MA, Aaron JS, Boehm U, DeSantis MC, Hobson CM, Khuon S, Lee RM, Chew TL.
J Cell Sci. 2022 Mar 15;135(6)

Fluorescence microscopy images should not be treated as perfect representations of biology. Many factors within the biospecimen itself can drastically affect quantitative microscopy data. Whereas some sample-specific considerations, such as photobleaching and autofluorescence, are more commonly discussed, a holistic discussion of sample-related issues (which includes less-routine topics such as quenching, scattering and biological anisotropy) is required to appropriately guide life scientists through the subtleties inherent to bioimaging. Here, we consider how the interplay between light and a sample can cause common experimental pitfalls and unanticipated errors when drawing biological conclusions. Although some of these discrepancies can be minimized or controlled for, others require more pragmatic considerations when interpreting image data. Ultimately, the power lies in the hands of the experimenter. The goal of this Review is therefore to survey how biological samples can skew quantification and interpretation of microscopy data. Furthermore, we offer a perspective on how to manage many of these potential pitfalls.

 

Link to Paper

SearchLight Spectral Modeling & Analysis Tool

Online Spectral Modeling & Analysis Tool to chose fluorophores based on filter and hardware specifications. 

Link to SearchLight 

Detailed and Accurate methods reporting to improve reproducibility

Best practices and tools for reporting reproducible fluorescence microscopy methods

P Montero Llopis et al
Nat Methods 18, 1463–1476 (2021).

Although fluorescence microscopy is ubiquitous in biomedical research, microscopy methods reporting is inconsistent and perhaps undervalued. We emphasize the importance of appropriate microscopy methods reporting and seek to educate researchers about how microscopy metadata impact data interpretation. We provide comprehensive guidelines and resources to enable accurate reporting for the most common fluorescence light microscopy modalities. We aim to improve microscopy reporting, thus improving the quality, rigor and reproducibility of image-based science.

 

Link to paper

List of selected reagents, resources and initiatives in microscopy

Supplementary Information from P Montero Llopis et al, Nat Methods 18, 1463–1476 (2021).
Selected Reagents/ToolsWhere to find itPurpose
Diffraction-limited fluorescent beads, 100 nmhttps://www.thermofisher.com/order/catalog/product/F8803#/F8803PSF characterization: XYZ resolution
Assess and measure stage precision and reproducibility Assess vibration issues
Tetraspeck beads (multi-colored beads), 100-200
nm
https://www.thermofisher.com/order/catalog/product/T7279?SID=srch-srp-T7279#/T7279?SID=srch-srp-T7279Chromatic aberration, registration shift (XY), dispersion, camera/lightpath alignment
Fluorescent silica particles (sicastar®-F)https://www.micromod.de/en/produkte-22-fluorescent_sil.htmlChromatic aberration, registration shift (XY), dispersion, camera/lightpath alignment
GATTA-beadshttps://www.gattaquant.com/products/gatta-beads-w.htmlChromatic aberration, registration shift (XY), dispersion, camera/lightpath alignment
Gold beadshttps://www.nanopartz.com/bare_gold_nanorods.asp
http://hestzig.com/index_files/Page497.htm
Chromatic shift (XY), dispersion, camera/lightpath alignment
Calibration slideshttp://argolight.com/ https://www.zeiss.com/carl-zeiss-
jena/services/calibration.html?gclid=CjwKCAjwiaX8BRBZEiwAQQxGx943YM_05K3vkPF59r8cd1xpAmlMW3rA3ggE-
0nRqWiKE7DAm3UYZBoC6sEQAvD_BwE
https://www.gattaquant.com/products/localization-based/gatta-paint-nanoruler.html
Assess resolution, linearity of detectors, general calibration and performance assessment
Resolution targethttps://www.edmundoptics.de/f/nbs-1963a-resolution-target/12203/;
https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=4338
Assess resolution
Power meterhttps://www.excelitas.com/product/x-cite-optical-power-measurement-system https://www.thorlabs.com/navigation.cfm?guide_id=37
http://argolight.com/products/argo-power/
power density in sample, performance of light source
Fluorescein and other quenching dyeshttps://www.sigmaaldrich.com/catalog/product/aldrich/330000?lang=en&region=US;
https://www.sigmaaldrich.com/catalog/product/sial/f6377?lang=en&region=US
flatfield correction
Fluorescent plastic slideshttps://www.thorlabs.com/thorproduct.cfm?partnumber=FSK5, https://www.chroma.com/products/accessories/92001-autofluorescent-plastic-slidesflatfield correction
Micrometerhttps://www.thorlabs.com/thorproduct.cfm?partnumber=R1L3S2P; https://www.amscope.com/microscope-stage-calibration-slide-for-usb-camera-0-01mm-stage- micrometer.html?gclid=Cj0KCQiAkuP9BRCkARIsAKGLE8UGSO_5ylS-iBtL7vrlzB-
Eb01lZLApF0nYFy1sWAP2qaPIWr2bU1MaAiMLEALw_wcB&gclsrc=aw.ds
Calibrate distances/pixel size
Selected SoftwareWhere to find itPurpose
FPbasehttps://www.fpbase.org/Experimental design and fluorescence protein selection
PFSjhttps://github.com/cmongis/psfjEvaluate PSF and overall performance
MetroloJhttps://imagejdocu.tudor.lu/plugin/analysis/metroloj/startInstrument performance
Trackmatehttps://imagej.net/TrackMateSingle particle/cell tracking, displacement evaluation
UnwarpJ pluginhttps://imagej.net/UnwarpJChannel registration
MethodsJhttps://github.com/tp81/MethodsJMethods writing assistance
MicroMetaApphttp://big.umassmed.edu/omegaweb/2020/05/11/micro-meta-app-beta-release/Microscope configuration and metadata
IJ1 MacroDiel, E. E., J. W. Lichtman and D. S. Richardson (2020). "Tutorial: avoiding and correcting sample-induced spherical aberration artifacts in 3D fluorescence microscopy." Nat Protoc 15(9): 2773-2784Selection of appropriate Z step and correction of axial distortions
Confocal Checkhttps://journals.plos.org/plosone/article?id=10.1371/journal.pone.0079879Confocal performance automated software
Daybook QChttp://argolight.com/Instrument performance
uListhttps://www.microlist.org/Database of light microscopy resources
uForumhttps://forum.microlist.org/Light microscopy forum
ImageSChttps://forum.image.scImage analysis forum
Selected InitiativesWhere to findPurpose
QUAREP-LiMihttps://quarep.org/Qualitive assessment and quality control
Bioimaging North America (BINA)https://www.bioimagingna.org/qc-dm-wgQuality control and data management
Association of Biomolecular Resource Facilities (ABRF-LMRG)https://abrf.org/research-group/light-microscopy-research-group-lmrgScientific exchange, standards for the field of microscopy
Global Bioimaginghttps://www.globalbioimaging.org/Network of imaging infrastructures, collaboration
European Light Microscopy Initiative (ELMI)https://elmi.embl.org/Network between imaging scientists and vendors
Royal Microscopy Societyhttps://www.rms.org.uk/Training, networking, facilities acknowledgment
German BioImaginghttps://www.gerbi-gmb.de/Access, training and data management services
Euro Bioimaging (and Nodes)https://www.eurobioimaging.eu/Develop and provide access to technology
Defining Our research Methodology (DORy)https://www.doryworkspace.org/Develop of standards for data collection and facilitate data sharing
CZI Imaging Programhttps://chanzuckerberg.com/science/programs-resources/imaging/Networking, support and imaging tool development
OMEROhttps://www.openmicroscopy.org/omero/Data management
NEUBIAShttp://eubias.org/NEUBIAS/Network of bioimage analysis, education and training
FocalPlanehttps://focalplane.biologists.com/Microscopy community
COBAhttps://openbioimageanalysis.org/Quantitative image analysis
Data RepositoriesWhere to findPurpose
OMEROhttps://www.openmicroscopy.org/about/Data management and sharing for microscopy
Bioimage Archivehttps://www.ebi.ac.uk/bioimage-archive/Image storage and distribution for life sciences research
FAIR sharinghttps://fairsharing.org/Maximize access to scientific data making them findable, accessible, interoperable and reusable
Brain Initiative-Cell Consensus Networkhttps://biccn.org/High-resolution atlas of different brain cell types from diverse organisms
Brain Imagine Libraryhttps://www.brainimagelibrary.org/Data repository, sharing and analysis of large data sets
figsharehttps://figshare.com/Citable and sharable data and figure repository
Image Data Resource (IDR)https://idr.openmicroscopy.org/Data repository of image datasets from published studies

MicCheck - Microscopy Metadata Checklist Generator

This tool guides researchers through simple questions related to their imaging choices and dynamically generates a checklist of essential and optional metadata that can then be copied and pasted into a text editor or downloaded in pdf format. In addition to online use, by downloading and modifying the example text file, core facilities or laboratories are also able to create their own versions of MicCheck with custom metadata examples specific to their microscope systems.

 

Link to Original Version

Link to Version 2.0

Micro-Meta App: an interactive tool for collecting microscopy metadata based on community specifications.

Rigano, A., Ehmsen, S., Öztürk, S.U. et al.
Nat Methods 18, 1489–1495 (2021). https://doi.org/10.1038/s41592-021-01315-z

For quality, interpretation, reproducibility and sharing value, microscopy images should be accompanied by detailed descriptions of the conditions that were used to produce them. Micro-Meta App is an intuitive, highly interoperable, open-source software tool that was developed in the context of the 4D Nucleome (4DN) consortium and is designed to facilitate the extraction and collection of relevant microscopy metadata as specified by the recent 4DN-BINA-OME tiered-system of Microscopy Metadata specifications. In addition to substantially lowering the burden of quality assurance, the visual nature of Micro-Meta App makes it particularly suited for training purposes.

 

Link to paper

MethodsJ2: a software tool to capture metadata and generate comprehensive microscopy methods text

Ryan, J., Pengo, T., Rigano, A. et al.
Nat Methods 18, 1414–1416 (2021).

Proper reporting of metadata is essential to reproduce microscopy experiments, interpret results and share images1,2. The lack of methods reporting in microscopy is evident in that few research articles pass a test for the minimal information required to reproduce experiments1 (about 17% of 240 articles containing 1,500 figures with images). The problem is compounded by the number and variety of microscope modalities, options and associated components. Automation has distanced researchers from the technical parameters, so it can be difficult for them to know what information needs to be reported. MethodsJ2 is an ImageJ/Fiji-based software tool that aims to improve reproducibility in microscopy by capturing image metadata from multiple sources, consolidating it and automatically generating methods text for publication.

 

Link to Paper

MDEmic in a use case for microscopy metadata harmonization: Facilitating FAIR principles in practical application with metadata annotation tools

Kunis, S., Hänsch, S., Schmidt, C., & Wong, F.
ArXiv. /abs/2103.02942

While the FAIR principles are well accepted in the scientific community, the implementation of appropriate metadata editing and transfer to ensure FAIR research data in practice is significantly lagging behind. On the one hand, it strongly depends on the availability of tools that efficiently support this step in research data management. On the other hand, it depends on the available standards regarding the interpretability of metadata. Here, we introduce a tool, MDEmic, for editing metadata of microscopic imaging data in an easy and comfortable way that provides high flexibility in terms of adjustment of metadata sets. This functionality was in great demand by many researchers applying microscopic techniques. MDEmic has already become a part of the standard installation package of the image database OMERO as OMERO.mde. This database helps to organize and visualize microscopic image data and keep track of their further processing and linkage to other data sets. For this reason, many imaging core facilities provide OMERO to their users. We present a use case scenario for the tailored application of OMERO.mde to imaging data of an institutional OMERO-based Membrane Dye Database, which requires specific experimental metadata. Similar to public image data repositories like the Image Data Resource, IDR, this database facilitates image data storage including rich metadata which enables data mining and re-use, one of the major goals of the FAIR principles.

 

Link to Paper

A guide to accurate reporting in digital image acquisition – can anyone replicate your microscopy data?

Heddleston JM, Aaron JS, Khuon S, Chew TL
J Cell Sci. 2021 Mar 30;134(6)

Recent technological advances have made microscopy indispensable in life science research. Its ubiquitous use, in turn, underscores the importance of ensuring that microscopy-based experiments are replicable and that the resulting data comparable. While there has been a wealth of review articles, practical guides and conferences devoted to the topic of maintaining standard instrument operating conditions, the paucity of attention dedicated to properly documenting microscopy experiments is undeniable. This lack of emphasis on accurate reporting extends beyond life science researchers themselves, to the review panels and editorial boards of many journals. Such oversight at the final step of communicating a scientific discovery can unfortunately negate the many valiant efforts made to ensure experimental quality control in the name of scientific reproducibility. This Review aims to enumerate the various parameters that should be reported in an imaging experiment by illustrating how their inconsistent application can lead to irreconcilable results.

 

Link to Paper

Data Visualization and Analysis

Community-developed checklists for publishing images and image analysis

Schmied, C., Nelson, M., Avilov, S., Bakker, G., Bertocchi, C., Bischof, J., Boehm, U., Brocher, J., Carvalho, M., Chiritescu, C., Christopher, J., Cimini, B., Ebner, M., Ecker, R., Eliceiri, K., Gaudreault, N., Gelman, L., Grunwald, D., Gu, T., . . . Jam
ArXiv. /abs/2302.07005

Images document scientific discoveries and are prevalent in modern biomedical research. Microscopy imaging in particular is currently undergoing rapid technological advancements. However for scientists wishing to publish the obtained images and image analyses results, there are to date no unified guidelines. Consequently, microscopy images and image data in publications may be unclear or difficult to interpret. Here we present community-developed checklists for preparing light microscopy images and image analysis for publications. These checklists offer authors, readers, and publishers key recommendations for image formatting and annotation, color selection, data availability, and for reporting image analysis workflows. The goal of our guidelines is to increase the clarity and reproducibility of image figures and thereby heighten the quality of microscopy data is in publications.

 

 

Link to Paper

Effective image visualization for publications - a workflow using open access tools and concepts

Schmied C, Jambor HK.
F1000Res. 2020 Nov 26;9:1373.

Today, 25% of figures in biomedical publications contain images of various types, e.g. photos, light or electron microscopy images, x-rays, or even sketches or drawings. Despite being widely used, published images may be ineffective or illegible since details are not visible, information is missing or they have been inappropriately processed. The vast majority of such imperfect images can be attributed to the lack of experience of the authors as undergraduate and graduate curricula lack courses on image acquisition, ethical processing, and visualization. 
Here we present a step-by-step image processing workflow for effective and ethical image presentation. The workflow is aimed to allow novice users with little or no prior experience in image processing to implement the essential steps towards publishing images. The workflow is based on the open source software Fiji, but its principles can be applied with other software packages. All image processing steps discussed here, and complementary suggestions for image presentation, are shown in an accessible “cheat sheet”-style format, enabling wide distribution, use, and adoption to more specific needs.

 

link to Paper

Creating Clear and Informative Image-based Figures for Scientific Publications

Jambor HK. et al

Scientists routinely use images to display data. Readers often examine figures first; therefore, it is important that figures are accessible to a broad audience. Many resources discuss fraudulent image manipulation and technical specifications for image acquisition; however, data on the legibility and interpretability of images are scarce. We systematically examined these factors in non-blot images published in the top 15 journals in 3 fields; plant sciences, cell biology, and physiology (n = 580 papers). Common problems included missing scale bars, misplaced or poorly marked insets, images or labels that were not accessible to colorblind readers, and insufficient explanations of colors, labels, annotations, or the species and tissue or object depicted in the image. Papers that met all good practice criteria examined for all image-based figures were uncommon (physiology 16%, cell biology 12%, plant sciences 2%). We present detailed descriptions and visual examples to help scientists avoid common pitfalls when publishing images. Our recommendations address image magnification, scale information, insets, annotation, and color and may encourage discussion about quality standards for bioimage publishing.

 

Link to Paper

CyLinter

Baker, G. (2021)
https://github.com/labsyspharm/cylinter

QC FOR MULTIPLEX MICROSCOPY

Although quality control (QC) methods have long been associated with analysis tools for single-cell genomics and transcriptomics research, analogous tools have lagged in the area of quantitative microscopy. There are now at least 9 different multiplex imaging platforms capable of routine acquisition of 20-40 channel microscopy data and each is sensitive to microscopy artifacts. Current tools for microscopy-based QC act on pixel-level data. CyLinter differs in that it allows users to work with both pixel-level and single-cell data to identify and remove cell segmentation instances corrupted by visual and image-processing artifacts that can significantly alter single-cell data quality.

 

Link to webpage

BIAflows

A Bio Image Analysis workflows benchmarking platform.
BIAFLOWS helps comparing bio image analysis workflows by benchmarking them on annotated datasets and simplifying their reproducible deployment.

https://biaflows-sandbox.neubias.org/#/

MCMICRO

Multiple-choice microscopy pipeline

An end-to-end processing pipeline that transforms multi-channel whole-slide images into single-cell data. This website is a consolidated source of information for when, why, and how to use MCMICRO.

 

Link to Website

 

Believing is seeing - the deceptive influence of bias in quantitative microscopy

Believing is seeing - the deceptive influence of bias in quantitative microscopy. J Cell Sci. 2024 Jan 1;137(1):jcs261567. doi: 10.1242/jcs.261567. Epub 2024 Jan 10. PMID: 38197776. Lee RM, Eisenman LR, Khuon S, Aaron JS, Chew TL.

Microscopy Communities and Resources

BioImaging North America

Engaging bioimaging scientists across North America by creating an inclusive and supportive community to share, advance and succeed together.

 

https://www.bioimagingnorthamerica.org/

QUAREP-LiMi

Group of enthusiastic light microscopists from Academia and Industry all interested in improving quality assessment (QA) and quality control (QC) in light microscopy. 

https://quarep.org/

Global BioImaging

Global BioImaging is an international network of imaging infrastructures and communities, which was initiated in 2015 by a european (Horizon 2020) funded project.

Recognizing that scientific, technical and data challenges are universal rather than restricted by geographical boundaries, it brings together imaging facility operators and technical staff, scientists, managers and science policy officers from around the globe, to network, exchange experiences and build capacity internationally.

https://globalbioimaging.org/