We introduce a deep learning model for resolution enhancement and prediction of super-resolved biological structures, which is based on a Generative Adversarial Network (GAN) assisted by a complementary segmentation task. It is applied to predict biological nanostructures from diffraction-limited images and to guide microscopists for quantitative fixed- and live-cell STimulated Emission Depletion (STED) microscopy. More specifically, we show that the use of a complementary segmentation task improves the accuracy of the predicted nanostructures, allowing quantitative analysis of the sub-diffraction structures in the resulting generated images.

Source Code

The source code is publicly available on GitHub.


Bouchard, C., Wiesner, T., Deschênes, A., Lavoie-Cardinal, F. & Gagné, C. Task-Assisted GAN for Reoslution Enhancement and Modality Translation in Fluorescence Microscopy. bioRxiv (2021).

The paper is available here.


We provide the download links to the datasets used in the paper below.

All datasets used for this paper are in-house datasets. If you use the provided dataset please cite the following papers: Neuronal activity remodels the F-actin based submembrane lattice in dendrites but not axons of hippocampal neurons for the Axonal Rings dataset and the Dendritic F-actin Rings and Fibers Dataset, Activity-Dependent Remodeling of Synaptic Protein Organization Revealed by High Throughput Analysis of STED Nanoscopy Images for the Synaptic Proteins Dataset. These datasets are available for general use by academic or non-profit, or government-sponsored researchers. This license does not grant the right to use these datasets or any derivation of it for commercial activities.

Axonal Rings
Dendritic F-actin Rings and Fibers
Synaptic Proteins
Live F-Actin
Fixed to Live F-Actin


All the trained models that were used for the results presented in the paper are available on the project's GitHub repository, except the segmentation model for live F-actin which is available to download here.


Francine Nault and Sarah Pensivy for neuronal cell culture. Annette Schwerdtfeger for careful proofreading of the manuscript. Anthony Bilodeau for the design of the website. Funding was provided by grants from the Natural Sciences and Engineering Research Council of Canada (F.L.C. and C.G.), the CERVO Foundation (F.L.C.), and the Neuronex Initiative (National Science Foundation, Fond de recherche du Québec - Santé) (F.L.C.). C.G. is a CIFAR Canada AI Chair and F.L.C. is a Canada Research Chair Tier II. C.B. is supported by a PhD scholarship from the Fonds de Recherche Nature et Technologie (FRQNT) Quebec, an excellence scholarship from the FRQNT strategic cluster UNIQUE, and a Leadership and Scientific Engagement Award from Université Laval.