Skip to main content Accessibility help



  • Access
  • Cited by 1


      • Send article to Kindle

        To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        A self-driving microscope and the Atomic Forge
        Available formats

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        A self-driving microscope and the Atomic Forge
        Available formats

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        A self-driving microscope and the Atomic Forge
        Available formats
Export citation


This material is based upon work supported by the US Department of Energy, Office of Science, Division of Materials Science and Engineering, Basic Energy Sciences, and was performed at the Oak Ridge National Laboratory’s Center for Nanophase Materials Sciences, a US Department of Energy, Office of Science User Facility.

We welcome comments and feedback on this article via email

The electron microscope predates the transistor and the charge-coupled device (CCD). The opportunity to integrate these advancements into the electron microscope was seized and revolutionized in the modern (scanning) transmission electron microscope (STEM). Real-time analysis became possible and is now routine. Consider the efforts one would have to undertake to perform a Fourier transform in the days when images were acquired by exposing photographic plates. Indeed, it was real-time data analysis that enabled the measurement and correction of aberrations in modern instrumentation.Reference Pennycook and Nellist1 What are the next opportunities on the horizon for the STEM?

With the impressive progress made in the fields of deep learning, computer vision, and automation, we posit that the next revolution in microscopy will stem from the integration of these tool sets: a self-driving microscope. Such a machine will “understand” what it is looking at and automatically document features of interest. The microscopist will have high-level tools to tell the microscope to “look for distortions at that interface” or “obtain a tomographic reconstruction of this structure.” The microscope will know what various features look like by referencing databases, or it can be shown examples on the fly. Consider the trainable Weka segmentation plugin for ImageJ.Reference Arganda-Carreras, Kaynig, Rueden, Eliceiri, Schindelin, Cardona and Seung2,Reference Schneider, Rasband and Eliceiri3 This plugin allows the user to highlight regions of an image to inform the computer which features belong in which categories. As examples are added the computer becomes increasingly accurate at classifying the rest of the image automatically.

Using the Atomic Forge, Oak Ridge National Laboratory (ORNL) researchers brought two, three, and four silicon atoms together to build clusters (green) and make them rotate within a layer of graphene (blue). Photo credit: ORNL.

Advances in deep learning to interpret atomically resolved images are already beginning. For example, Ziatdinov et al. applied deep convolutional neural networks to automate the detection of molecular orientation, defect identification, and classification.Reference Ziatdinov, Dyck, Maksov, Hudak, Lupini, Song, Snijders, Vasudevan, Jesse and Kalinin4,Reference Ziatdinov, Dyck, Maksov, Li, Sang, Xiao, Unocic, Vasudevan, Jesse and Kalinin5 Maksov et al. illustrated the untangling of lattice dynamics involving the interaction of thousands of defects using deep learning and unsupervised unmixing strategies,Reference Maksov, Dyck, Wang, Xiao, Geohegan, Sumpter, Vasudevan, Jesse, Kalinin and Ziatdinov6 and Vasudevan et al. showed strategies for automated classification of Bravais lattice symmetries.Reference Vasudevan, Laanait, Ferragut, Wang, Geohegan, Xiao, Ziatdinov, Jesse, Dyck and Kalinin7 There is no fundamental impediment for implementing tools such as these in the microscope to tell the computer what to pay attention to and where to gather data. As these fields progress and as the techniques are refined and proven, integration into the microscope itself becomes increasingly attractive and powerful.

A parallel development is the recognition that the STEM can be used to tailor materials at the atomic level, termed the Atomic Forge.Reference Kalinin, Borisevich and Jesse8 The electron beam can alter materials but historically, steps have been taken to avoid beam alterations, as they typically preclude characterization of a pristine sample by introducing defects. Recently, however, these beam-induced modifications have been leveraged to produce remarkable demonstrations of materials manipulation at the nanometer and atomic scales. Jesse et al. demonstrated the controlled layer-by-layer growth from an amorphous precursor in strontium titanateReference Jesse, He, Lupini, Leonard, Oxley, Ovchinnikov, Unocic, Tselev, Fuentes-Cabrera, Sumpter, Pennycook, Kalinin and Borisevich9 and Si.Reference Jesse, Hudak, Zarkadoula, Song, Maksov, Fuentes-Cabrera, Ganesh, Kravchenko, Snijders, Lupini, Borisevich and Kalinin10 Hudak et al. showed how individual Bi atoms can be moved with the electron beam through a crystalline Si lattice to form patterned structures with atomic column precision.Reference Hudak, Song, Sims, Troparevsky, Humble, Pantelides, Snijders and Lupini11 Dyck et al. illustrated the introduction of single dopant Si atoms in graphene with near lattice site precision.Reference Dyck, Kim, Kalinin and Jesse12 Susi et al. laid the groundwork for predicting and demonstrating controlled Si dopant motion through a graphene lattice.Reference Susi, Kotakoski, Kepaptsoglou, Mangler, Lovejoy, Krivanek, Zan, Bangert, Ayala, Meyer and Ramasse13,Reference Susi, Meyer and Kotakoski14 And Dyck et al. demonstrated the assembly of primitive structures with Si dopants in graphene.Reference Dyck, Kim, Jimenez-Izal, Alexandrova, Kalinin and Jesse15

These demonstrations of such exquisite precision harken back to the initial atomic motion demonstrated in a scanning tunneling microscope by Eigler,Reference Crommie, Lutz and Eigler16Reference Eigler and Schweizer19 which ignited the field of nanotechnology. However, the range of beam-driven materials transformations observed in STEM covers the gamut, including particle fragmentation, precipitation, nanotube/nanowire growth, catalytic etching and growth, phase changes, deposition, sculpting, and welding.Reference Dyck, Ziatdinov, Lingerfelt, Unocic, Hudak, Lupini, Jesse and Kalinin20 Most of these are not performed with single atom precision, but the examples mentioned above show that this level of precision is certainly possible. What remains is to establish the techniques and tools that will enable such modifications to be performed routinely for the construction of atomically precise extended structures. This possibility is further enhanced by the idea of a self-driving microscope. If a microscope can drive itself, knows what it is looking at, and can respond “intelligently” to this information, perhaps it can also build structures at the atomic scale.

Both the self-driving microscope and the Atomic Forge are in the early stages of development, with the concepts themselves just beginning to crystalize. Many questions and challenges remain, and these ideas could easily be dismissed as unattainable. Aberration correction itself was repeatedly dismissed after multiple failed attempts;Reference Pennycook and Nellist1 nevertheless, we now enjoy the fruits of those who persisted in the face of doubt, and have almost achieved Feynman’s vision of a microscope with 100 times higher resolution than his time.Reference Pennycook21,Reference Feynman22 Similarly, the self-driving microscope and the Atomic Forge will fulfill another Feynman prophecy: “What would happen if we could arrange the atoms one by one the way we want them?”Reference Feynman22


1.Pennycook, S.J., Nellist, P.D., Scanning Transmission Electron Microscopy: Imaging and Analysis (Springer Science & Business Media, Berlin, Germany, 2011).
2.Arganda-Carreras, I., Kaynig, V., Rueden, C., Eliceiri, K.W., Schindelin, J., Cardona, A., Seung, H.S., Bioinformatics 33, 2424 (2017).
3.Schneider, C.A., Rasband, W.S., Eliceiri, K.W., Nat. Methods 9, 671 (2012).
4.Ziatdinov, M., Dyck, O., Maksov, A., Hudak, B.M., Lupini, A.R., Song, J., Snijders, P.C., Vasudevan, R.K., Jesse, S.,Kalinin, S.V., “Deep Analytics of Atomically-Resolved Images: Manifest and Latent Features,” preprint, arXiv:1801.05133 (2018).
5.Ziatdinov, M., Dyck, O., Maksov, A., Li, X., Sang, X., Xiao, K., Unocic, R.R., Vasudevan, R., Jesse, S., Kalinin, S.V., ACS Nano 11, 12742 (2017).
6.Maksov, A., Dyck, O., Wang, K., Xiao, K., Geohegan, D.B., Sumpter, B.G., Vasudevan, R.K., Jesse, S., Kalinin, S.V., Ziatdinov, M., “Deep Learning Analysis of Defect and Phase Evolution during Electron Beam Induced Transformations in WS2,” preprint, arXiv:1803.05381 (2018).
7.Vasudevan, R.K., Laanait, N., Ferragut, E.M., Wang, K., Geohegan, D.B., Xiao, K., Ziatdinov, M.A., Jesse, S., Dyck, O.E., Kalinin, S.V., “Mapping Mesoscopic Phase Evolution during E-beam Induced Transformations via Deep Learning of Atomically Resolved Images,” preprint, arXiv:1802.10518 (2018).
8.Kalinin, S.V., Borisevich, A., Jesse, S., Nature 539, 485 (2016).
9.Jesse, S., He, Q., Lupini, A.R., Leonard, D.N., Oxley, M.P., Ovchinnikov, O., Unocic, R.R., Tselev, A., Fuentes-Cabrera, M., Sumpter, B.G., Pennycook, S.J., Kalinin, S.V., Borisevich, A.Y., Small 11, 5895 (2015).
10.Jesse, S., Hudak, B.M., Zarkadoula, E., Song, J., Maksov, A., Fuentes-Cabrera, M., Ganesh, P., Kravchenko, I., Snijders, P.C., Lupini, A.R., Borisevich, A., Kalinin, S.V., “Direct Atomic Fabrication and Dopant Positioning in Si Using Electron Beams with Active Real Time Image-Based Feedback,” preprint, arXiv:1711.05810 (2017).
11.Hudak, B.M., Song, J., Sims, H., Troparevsky, M.C., Humble, T.S., Pantelides, S.T., Snijders, P.C., Lupini, A.R., ACS Nano 12 (6), 5873 (2018).
12.Dyck, O., Kim, S., Kalinin, S.V., Jesse, S., Appl. Phys. Lett. 111, 113104 (2017).
13.Susi, T., Kotakoski, J., Kepaptsoglou, D., Mangler, C., Lovejoy, T.C., Krivanek, O.L., Zan, R., Bangert, U., Ayala, P., Meyer, J.C., Ramasse, Q., Phys. Rev. Lett. 113, 115501 (2014).
14.Susi, T., Meyer, J.C., Kotakoski, J., Ultramicroscopy 180, 163 (2017).
15.Dyck, O., Kim, S., Jimenez-Izal, E., Alexandrova, A.N., Kalinin, S.V., Jesse, S., “Assembling Di- and Multiatomic Si Clusters in Graphene via Electron Beam Manipulation,” preprint, arXiv:1710.09416 (2018).
16.Crommie, M.F., Lutz, C.P., Eigler, D.M., Science 262, 218 (1993).
17.Crommie, M.F., Lutz, C.P., Eigler, D.M., Nature 363, 524 (1993).
18.Eigler, D.M., Lutz, C.P., Rudge, W.E., Nature 352, 600 (1991).
19.Eigler, D.M., Schweizer, E.K., Nature 344, 524 (1990).
20.Dyck, O., Ziatdinov, M., Lingerfelt, D.B., Unocic, R.R., Hudak, B.M., Lupini, A.R., Jesse, S., Kalinin, S.V., Nat. Rev. Mater. (2019), doi:10.1038/s41578-019-0118-z.
21.Pennycook, S.J., MRS Bull . 40 (1), 71 (2015).
22.Feynman, R.P., Eng. Sci. 23 (5), 22 (1960).