Strategies for effective transfer learning for HRTEM analysis with neural networks

Transfer learning provides a practical opportunity to adapt machine learning models to new domains by retraining models on a new, typically smaller set of data. This technique is especially useful for supervised learning problems in electron microscopy, where high-quality, labeled experimental data are scarce and experimental conditions can change significantly across experiments. Using simulated benchmark datasets, we investigate a series of transfer learning protocols for developing segmentation models for locating nanoparticles in high-resolution transmission electron microscopy (HRTEM) data. We train over 10,500 neural networks and evaluate model performance and out-of-distribution generalization capabilities after both pretraining and transfer learning phases, covering shifts in imaging conditions, noise conditions, and structural distributions. Via our high-throughput data generation and model training approach, we demonstrate that transfer learning can be an effective strategy to tackle domain shifts in machine learning problems for HRTEM and that transfer learning can be beneficial for model generalization. Lastly, we provide some practical training recommendations for adopting transfer learning protocols into machine learning development pipelines.