Introduction
Conventional electron backscatter diffraction (EBSD) is a scanning electron microscope (SEM)-based technique used to determine the three-dimensional orientation of individual grains in crystalline materials. At present, the collected electron backscatter diffraction patterns (EBSPs) are primarily utilized to construct representative maps of the microstructure, given user-defined phases. The resulting maps and EBSPs can be further utilized to study dislocation evolution (Zhu et al., Reference Zhu, Harrington, Gray and Vecchio2018) and geometrically necessary dislocations (Zhu et al., Reference Zhu, Harrington, Livescu, Gray and Vecchio2016), among others (Wilkinson et al., Reference Wilkinson, Meaden and Dingley2006; Schwartz et al., Reference Schwartz, Kumar, Adams and Field2009; Engler & Randle, Reference Engler and Randle2010; Wilkinson & Britton, Reference Wilkinson and Britton2012). These analyses are currently achieved in commercial EBSD utilizing a Hough transform on each diffraction pattern combined with a look-up table of interplanar angles constructed from a set of selected reflectors as specified by the user (typically up to five phases). In the process, the Hough transform strips most of the information about the crystal structure and chemistry from the image (Michael & Goehner, Reference Michael and Goehner1999; Dingley & Wright, Reference Dingley, Wright, Schwartz, Kumar, Adams and Field2009; Li & Han, Reference Li and Han2015; Nolze et al., Reference Nolze, Hielscher and Winkelmann2017). The small subset of information being utilized for differentiating the phases often results in mis-classification of similar crystal structures (Chen & Thomson, Reference Chen and Thomson2010) and orientations that produce similar diffraction maxima in different crystal systems (McLaren & Reddy, Reference McLaren and Reddy2008; Karthikeyan et al., Reference Karthikeyan, Dash, Saroja and Vijayalakshmi2013). Despite its shortcomings, the Hough transform has continued to be the standard in pattern indexing, even though computing power has increased exponentially, largely because computer algorithms have previously been incapable of Kikuchi band detection, or the more challenging problem of autonomously parsing relevant information for determining chemistry and the crystal structure.
The inability for state-of-the-art Hough-based EBSD systems to make determinations about individual components of or the entire crystal structure of a phase has not gone unnoticed in the scientific community. One proposed solution is a dictionary-based approach (Park et al., Reference Park, Wei, De Graef, Shah, Simmons and Hero2013; Chen et al., Reference Chen, Park, Wei, Newstadt, Jackson, Simmons, De Graef and Hero2015; Singh & De Graef, Reference Singh and De Graef2017; Ram & De Graef, Reference Ram and De Graef2018). Libraries of selected materials can be selected and simulated with software, such as EMSoft (Callahan & De Graef, Reference Callahan and De Graef2013), for the purpose of comparing the experimentally collected diffraction patterns with those simulated. Through this process, the most similar patterns in the dictionary determine the phase and orientation of each EBSD pattern, even for deformed or fine-grained materials (Ram & De Graef, Reference Ram and De Graef2018). These approaches have helped to allay major challenges in phase differentiation, such as separating martensite from cementite (Ram & De Graef, Reference Ram and De Graef2018). Other types of phase identification methodologies have been demonstrated, wherein EBSD is combined with other analytical techniques, such as energy-dispersive X-ray spectroscopy (EDS) or wavelength-dispersive X-ray spectroscopy (WDS), given that the chemical and structural information of the phase exists in a theoretical model or crystal database (Goehner & Michael, Reference Goehner and Michael1996; Nowell & Wright, Reference Nowell and Wright2004; Dingley & Wright, Reference Dingley, Wright, Schwartz, Kumar, Adams and Field2009). The problem remains that the collected EBSD images are traditionally processed via the Hough transform when attempting to perform phase identification. As mentioned previously, the down sampling of this data allows for patterns from multiple crystal structures to be mistaken for others since the computer algorithms are capable of finding orientations which may only appear to be correct (Karthikeyan et al., Reference Karthikeyan, Dash, Saroja and Vijayalakshmi2013). In commercial phase identification systems, the user must also select a limited number of elements, typically up to 3, from the EDS results before the computer narrows down the list of possible candidates. The combination of these two reductions in information results in a multitude of phases, from a variety of crystal structures, being returned for the user to decide which is the best fit for each phase present in the microstructure.
The recent advent of machine learning techniques, such as the convolutional neural network (CNN), offer an opportunity to address many of the challenges to autonomously extracting information from diffraction data (Ziletti et al., Reference Ziletti, Kumar, Scheffler and Ghiringhelli2018; Oviedo et al., Reference Oviedo, Ren, Sun, Settens, Liu, Hartono, Ramasamy, DeCost, Tian, Romano, Gilad Kusne and Buonassisi2019). The reason CNNs are of particular interest is due to their multiple advantages over classical computer vision techniques, which require a multitude of heuristics (Wang et al., Reference Wang, Dong, O'Daniel, Mohan, Garden, Kian Ang, Kuban, Bonnen, Chang and Cheung2005; Alegre et al., Reference Alegre, Barreiro, Cáceres, Hernández, Fernández and Castejón2006; Lombaert et al., Reference Lombaert, Grady, Pennec, Ayache and Cheriet2014; DeCost & Holm, Reference DeCost and Holm2015; Zhu et al., Reference Zhu, Wang, Kaufmann and Vecchio2020) — such as detecting Kikuchi bands, accounting for orientation changes, determining bandwidth, and searching for higher-order Laue zones (HOLZ) rings, and carrying the burden of developing the logic that defines these abstract qualities. Instead, this deep learning technique determines its own internal representation of the data, via backpropagation (Rumelhart et al., Reference Rumelhart, Hinton and Williams1986), potentially coupled with one of many possible hyperparameter tuning strategies (Bergstra & Bengio, Reference Bergstra and Bengio2012), such that it maximizes performance at the discrimination task. This is the underlying principle behind deep representation learning (i.e., deep neural networks) (LeCun et al., Reference LeCun, Bengio and Hinton2015). Such methods allow the model to find patterns that may be unintuitive or too nuanced for humans to discern. Other discovered features might be obvious to experts but difficult to translate into specific logic (e.g., linking the background signal to the chemical composition). CNNs are further advantageous over other machine learning models since they operate on the image data directly. As an example of other machine learning models applied to EBSD, a nearest neighbor machine learning model has been previously explored by Goulden et al. (Reference Goulden, Mehnert, Thomsen and Jiang2017) to address the challenge of separating ferrite and martensite by machine learning aided pattern quality analysis, but pattern quality is too rigid a metric for general use, and the method was reliant on human analysis and confirmation over multiple rounds of indexing a single map. Another advantage is the flexibility of these CNNs, allowing for the transfer of knowledge learned from discriminating images in other contexts (Gonzalez et al., Reference Gonzalez, Bhowmick, Beltran, Sankaran and Bengio2019; Thompson et al., Reference Thompson, Schonwiesner, Bengio and Willett2019), the development of models suitable for application in a highly specific materials space, or deployment in an application where the phases present are completely unknown. For example, during the initial analysis of a material, a pre-trained CNN could be utilized for the determination of which Bravais lattices or space groups are present (Kaufmann et al., Reference Kaufmann, Zhu, Rosengarten, Maryanovsky, Harrington, Marin and Vecchio2020). Other studies utilizing CNNs in EBSD have demonstrated the orientation determination (Jha et al., Reference Jha, Singh, Al-Bahrani, Liao, Choudhary, De Graef and Agrawal2018) and phase classification of several materials that may be easily confused in traditional EBSD (Foden et al., Reference Foden, Previero and Britton2019). The work by Foden et al. demonstrates the application of simulated patterns for achieving this goal. Simulated EBSPs may alleviate the challenges associated with finding suitable materials for data collection, similar to what is being done in other applications (Ziletti et al., Reference Ziletti, Kumar, Scheffler and Ghiringhelli2018; Oviedo et al., Reference Oviedo, Ren, Sun, Settens, Liu, Hartono, Ramasamy, DeCost, Tian, Romano, Gilad Kusne and Buonassisi2019). In well-known classes of materials, application-specific models could be developed to distinguish phases traditional Hough-based EBSD finds nearly indistinguishable, such as martensite and ferrite. Moreover, modern computing hardware allows for real-time classification in line with recent advancements in EBSD technology (Goulden et al., Reference Goulden, Trimby and Bewick2018).
Herein, it is demonstrated that CNNs can be constructed to recognize phases based on their structure/symmetry, chemistry, and even lattice parameter variations; some of these well beyond the scope of what the Hough transform is capable of, even with user supplied information about the sample. Several of these CNN-based demonstrations are meant to serve as proof-of-concept models for application-specific use cases, while others such as the Bravais lattice identification model can readily be applied to materials outside the training set. Further automation and improved accuracy of the proposed process could be achieved by the development of diffraction pattern databases, inclusion of simulated diffraction patterns into the training process, and the development of standard models for various use cases.
Materials and Methods
EBSD Pattern Collection
Six different multiphase materials were selected for demonstrating the proposed phase-mapping methodology: (1) a rutilated quartz sample, (2) a sample of the Campo del Cielo meteorite, from the Chaco Province, Argentina, (3) an arc-melted ingot of Ni80.8B13.6Si5.4Fe0.2 (at%) blended with 40 wt% eutectic tungsten carbide (a metal matrix composite) (Maroli & Liu, Reference Maroli and Liu2017), (4) an Fe–Al metallic–intermetallic laminate (MIL) composite (Wang et al., Reference Wang, Harrington, Zhu and Vecchio2019), (5) a thermally cycled MCrAlY-based thermal barrier coating (Evans et al., Reference Evans, Mumm, Hutchinson, Meier and Pettit2001; Mercer et al., Reference Mercer, Faulhaber, Yao, McIlwrath and Fabrichnaya2006; Pollock et al., Reference Pollock, Lipkin and Hemker2012), and (6) a sample of 430 stainless steel. Diffraction patterns from 28 other materials, detailed in Kaufmann et al. (Reference Kaufmann, Zhu, Rosengarten, Maryanovsky, Harrington, Marin and Vecchio2020), were utilized for training the CNN-based model for Bravais lattices demonstrated herein. EBSD patterns (EBSPs) were collected in a Thermo Fisher (formerly FEI) Apreo scanning electron microscope (SEM) equipped with an Oxford Symmetry EBSD detector and an Oxford X-MaxN EDS detector. The Oxford Symmetry EBSD detector was utilized in high-resolution (1244 × 1024) mode with frame averaging and EDS maps were collected simultaneously. The Hough indexing parameters were 12 Kikuchi bands, a Hough resolution of 250, and band center indexing. After collecting high-resolution EBSPs from each material, all patterns collected were exported as tiff images. Figure A1 in Appendix shows example images of the high-resolution diffraction patterns collected from the six materials utilized in this study. The collected data was not filtered for pattern quality via any means, each pattern was individually assessed by the neural network, and the collection of images for each sample may contain partial or low-quality diffraction patterns, which will decrease the accuracy of their identification.
Neural Network Architecture and Phase-Mapping Procedure
The well-studied CNN architecture Xception (Chollet, Reference Chollet2017) was selected for fitting a model that determines from which phase or crystal structure a diffraction pattern originated. Figure A2 in Appendix details a schematic of the CNN operating on an EBSP. Training was performed using Adam optimization (Kingma & Ba, Reference Kingma and Ba2014), with batches of 32 images, and a minimum delta of 0.001 as the validation loss employed for early stopping criteria. Categorical cross-entropy was used as the loss function. All training data labels were created using validated phases in combination with combined EBSD/EDS and, where applicable, X-ray diffraction (XRD). In the cases where training data was collected from multiphase materials, such as the MIL composite, it is possible that a small number of diffraction patterns could have been attributed to the incorrect class. If a substantial number of patterns were mislabeled, we expect to see poor performance from the model. The CNNs were implemented using Python 3.5 with TensorFlow (Abadi et al., Reference Abadi, Barham, Chen, Chen, Davis, Dean, Devin, Ghemawat, Irving, Isard, Kudlur, Levenberg, Monga, Moore, Murray, Steiner, Tucker, Vasudevan, Warden, Wicke, Yu, Zheng and Brain2016) and Keras (Chollet, Reference Chollet2015). The conversion from neural network predictions to phase maps is accomplished using MATLAB R2018B. The histogram plot and pattern quality phase map were also made using MATLAB R2018B.
The rutilated quartz specimen was mapped using a CNN previously trained to differentiate the crystal symmetry of the 14 Bravais lattices (Kaufmann et al., Reference Kaufmann, Zhu, Rosengarten, Maryanovsky, Harrington, Marin and Vecchio2020). This model was trained using the same hyperparameters used for all other models in this work. The model was not trained using quartz diffraction patterns.
For the Campo del Cielo meteorite sample, EBSPs were collected from taenite, schreibersite, iron, and regions without any evidence of a diffraction pattern (i.e., containing only background noise). The model was trained on 500 randomly selected patterns from each of these phases.
Diffraction patterns for three of the five phases present in the Ni80.8B13.6Si5.4Fe0.2 plus 40 wt% eutectic tungsten carbide sample were collected from separate samples of the pure phases (Ni, W2C, and WC), similar to a “standard” library. Diffraction patterns from the other two phases, Ni3B and W2NiB2, were collected directly from the sample owing to the challenge of making pure specimens of these two phases. The model was trained to differentiate 500 randomly selected patterns from each of these phases.
The Fe–Al MIL composite is presented herein as four distinct phases. This material was fabricated via diffusion controlled growth from a “multiple-thin foil” configuration and a two-stage reaction process described elsewhere (Wang et al., Reference Wang, Harrington, Zhu and Vecchio2019). This fabrication strategy yields layers of pure iron, an Al-enriched α-Fe layer, an Fe-enriched FeAl B2 layer, and a near-equiatomic (~48 at.% Fe) FeAl B2 layer. Multiple layers of the sample were mapped using Hough-based EBSD alone, with reference chemistries from EDS for each phase, and separately with the convolutional neural network (CNN-based) EBSD approach. Diffraction patterns for training the CNN were collected from each of these regions and separated into four groups based on the chemistry at that location. The model was trained to recognize the subtle differences caused by chemistry and structure changes using 500 randomly selected patterns from each of these phases.
The thermal barrier coating sample contains five face-centered cubic (FCC) phases, all space group 225, and one rhombohedral structure. The FCC phases are Ni, yttria-stabilized zirconia (YS–ZrO2), TaC, Cr23C6, and Hastelloy X. The rhombohedral phase is the thermally grown oxide (TGO) Al2O3. The sample was phase mapped using standalone Hough-based EBSD, EBSD combined with chemistry information from EDS assigned to each phase, and separately with the CNN-based EBSD approach. Five hundred diffraction patterns were randomly selected from each phase to train the model.
A section of 430 stainless steel (max 0.12 wt% carbon) was selected to study the methodology's ability to separate martensite from ferrite. The lattice parameters for phase mapping via traditional EBSD were determined from XRD and the well-defined dependence of the lattice parameter c on carbon content (Xiao et al., Reference Xiao, Fan, Jinxiu, Mingxing, Mokuang and Zhenqi1995). Diffraction patterns used to train the model were collected from out-of-sample “standards” of pure Fe and rapidly quenched lath martensitic steel (Dougherty et al., Reference Dougherty, Cerreta, Gray, Trujillo, Lopez, Vecchio and Kusinski2009). The training patterns from the lath martensitic steel (max 0.15 wt% carbon) were filtered such that they were of the same high quality as the pure Fe patterns. Five hundred high-quality diffraction patterns from each phase were then selected to train the model.
For each of the six materials, each diffraction pattern collected in an EBSD map was evaluated in a random order by the corresponding trained CNN model without further information. The output classification of each diffraction pattern was recorded and saved in a (.csv) file. A custom Matlab script was used to assign an RGB color value to the predicted class for each diffraction pattern and assign the color value to the corresponding pixel in the EBSD map. Unlike the results obtained from the commercial Oxford Aztec software, no smoothing algorithm was employed for interface boundaries between phases, making the CNN-based maps look more pixelated than their commercial counterpart.
Results and Discussion
Figure 1 demonstrates the CNN-based EBSD method's ability to overcome the first major challenge traditional Hough-based EBSD is presented with: “What crystal symmetries are present in this material?” The electron image in Figure 1b clearly displays the two-phase nature of a rutilated quartz sample. When using a commercially available EBSD software package, such as Oxford's Aztec application, the user must first select a list of phases, which serve as the Hough transform libraries, for determining the phase and crystallographic orientation of each diffraction pattern. Assuming the phases are known, the user can select the quartz and rutile libraries to produce the phase map in Figure 1a. However, there is often uncertainty as to which crystal structures are present in a sample. To alleviate this concern, a CNN-based model that analyzes each diffraction pattern and returns the most likely Bravais lattice, out of the 14 possible choices, is developed. Figure 1c demonstrates the model applied to the same high-resolution diffraction patterns collected from the rutilated quartz specimen. The trained CNN can determine, autonomously, the correct Bravais lattice with a high degree of accuracy, reproducing the original user-selected phase map with a high degree of fidelity. This achievement is made more significant by the fact that the CNN was not trained on any diffraction patterns from quartz. When comparing the pixels that differ between the phase maps generated by the traditional Hough-based EBSD (Fig. 1a) and the CNN-based EBSD (Fig. 1c), it is immediately evident that the diffraction patterns classified as neither hexagonal nor primitive tetragonal tend to be located on the boundary between the two phases or in pores. This also tends to be the case for the non-indexed pixels (black) in the traditional Hough-based phase map. The CNN-based approach is also able to identify a number of these non-indexed pixels as belonging to primitive tetragonal (rutile) or hexagonal (quartz). Unlike the Hough-based method, this implementation is required to choose from one of the 14 Bravais lattices; it does not yet have the option to leave a pixel non-indexed based on pattern quality or other metrics. This is a feature that will be explored later in this work. It is important to note that this CNN-based EBSD approach is autonomous, wherein the user is not involved in any phase selection or the crystal structure decision-making process.
Fig. 1. Phase-mapping rutilated quartz based on Bravais lattices. (a) Hough transform EBSD map with rutile and quartz as user-selected phases. (b) Electron image showing the two-phase microstructure. (c) Phase map generated from the CNN-based model's analysis of each diffraction pattern individually. Scale bars = 250 μm. There are 11,700 total EBSPs (pixels).
In contrast to the previous example, which is designed to be the most broadly applicable, the following examples demonstrate the versatility of CNNs through a series of application-specific demonstrations, and their ability to address even more challenging crystal structure classification problems encountered in EBSD.
The first example explores a geological specimen with large variation in pattern quality. Figure 2 compares the performance of the traditional Hough-based approach versus our CNN-based EBSD approach on a sample of the Campo del Cielo meteorite. This specimen contains an Fe-rich matrix with isolated regions of taenite and schreibersite throughout. The first challenge is that the sample becomes deeply recessed in a section of the schreibersite phase as seen in the lower right corner of the electron image (Fig. 2b). This significantly impacts the pattern quality in this region and eventually the collected patterns contain only background noise. Therefore, it is important that the CNN-based model has a method for deciding when there is insufficient information in a captured diffraction pattern to make an accurate classification. This is accomplished by adding a zero-solution class as one of the available outputs, as detailed in the methods section. Comparing the output phase map from each method, it is immediately evident that the overall features are very similar. Upon further inspection, it is observed that the CNN-based method is able to fill in a greater percentage of the data from the recessed schreibersite phase, up to the point when the CNN method determines the diffraction patterns do not contain sufficient information to make a classification (Fig. 2c). Moreover, the sections of taenite are more complete along the zero-solution boundaries, including the section below the schreibersite phase, which traditional Hough-based EBSD was unable to map. In the middle left section of the electron image, it is evident that there exists a two-phase structure that traditional Hough-based EBSD identifies only as taenite. The EDS maps (Figs. 2d–2k), particularly Cu, Co, Fe, and Ni, confirm that the chemistry in this region is indicative of two phases. The CNN identifies the second-phase regions as schreibersite, which is a good match chemically and crystallographically, upon inspection of the diffraction patterns captured from that region. In the Hough-based phase map, there are four thin black lines that are not identified due to low-quality patterns and diffraction overlap (Fig. 2a). The CNN-based EBSD phase map identifies these regions as a mixture of taenite and schreibersite, and the EDS maps confirm these thin sections to be primarily taenite. In summary, the machine learning-based method is found to be robust to large variations in pattern quality and can be architected with a no solution class.
Fig. 2. Phase mapping the Campo del Cielo meteorite. (a) Hough transform EBSD map with user-selected phases. (b) Electron image showing the two-phase microstructure. (c) Phase map generated from the CNN-based model's analysis of each diffraction pattern individually. (d) Pattern quality map. (e–k) EDS maps of the analyzed region. Scale bars = 50 μm. There are 40,784 total EBSPs (pixels).
Figure 3 compares the performance of traditional Hough-based approach versus our CNN-based EBSD approach on an arc-melted metal matrix composite. This example demonstrates the ability to train the CNN-based model on diffraction patterns collected from “standards” of each phase (Ni, W2C, and WC) and apply the model to identify these phases in a complex multiphase sample (described in the Methods section). The 89,280 diffraction patterns collected for Figures 3a and 3c are independently analyzed by the CNN, which results in a high-fidelity map of the material. Between Figures 3a and 3c, only ~5,000 pixels (or 5.6%) have a different label and most of these differences are located where the traditional Hough-based method produced no solution. The CNN-based model excels at applying the information it learned from other systems, or “standards” onto a very different system with a high degree of accuracy.
Fig. 3. Phase mapping a metal matrix composite. (a) Hough transform EBSD map with user-selected libraries. (b) Electron image showing the multiphase microstructure. (c) Phase map generated from the CNN-based model's analysis of each diffraction pattern individually. (d–g) EDS maps of the analyzed region. Scale bars = 25 μm. There are 89,280 total EBSPs (pixels).
Furthermore, due to the significant difference in hardness (Maroli & Liu, Reference Maroli and Liu2017), the phases in the nickel-based matrix and tungsten carbide particles are susceptible to vastly different polishing rates. This typically reduces pattern quality in protected areas, such as within the WC particles seen in the electron image. The phase map generated via the traditional approach shows that much of this region produces partial EBSD patterns that the Hough-based approach cannot match to a look-up table of interplanar angles. However, the CNN-based model can utilize the information that is present to identify these EBSPs and primarily classifies them as either W2C or W2NiB2. The EDS maps in Figures 3d–3g, as well as the measured chemistry (not shown), support these classifications. The maps of W and C show that neither of the regions identified as W2C or W2NiB2 contain as much carbon as the regions mapped to WC in either phase map. Furthermore, the map of Ni shows that it does indeed diffuse into the large WC particles, where it causes the W2NiB2 phase to form near the perimeter. The combined achievements on this material show that the CNN is robust to partial patterns and can be trained using other samples as “standards”.
The next demonstration of the CNN-based EBSD approach is perhaps one of the most challenging types of problems encountered in EBSD. The four phases present in the Fe–Al sample are formed as the result of a diffusion couple style experiment between Fe and Al foils (described in the Methods section). This results in two body-centered cubic phases (space group 229), pure Fe and α-Fe from the Al substitution in the lattice, and two primitive cubic phases (space group 221), nearly equiatomic FeAl B2 and the non-equiatomic B2 solid solution. The backscattered electron image in Figure 4a shows the alternating layers of the material and gives an early indication that multiple phases are present in the sample. Figure 4b provides a reference Fe map and the bright columns indicate where pure Fe layers remain after the reaction. If the FeAl B2 and pure Fe diffraction libraries are selected by the user, the Hough-based EBSD is unable to differentiate between the two phases, and a phase map as shown in Figure 4c is produced. Because the diffraction maxima are so similar between these two phases, every diffraction pattern is identified as both phases with equal confidence, and the software selects, by default, the second phase on the user-selected list. Figure 4d demonstrates a representative result from a user assigning a reference chemistry from within each phase, using point EDS, to each of the four phases present. If the user selects the reference chemistry for each phase, with the best of intentions and accuracy, a resultant map as shown in Figure 4d can be achieved. While the four phases are now each present, the phase map does not accurately describe what is observed in the chemistry analysis. Instead, the pure Fe and α-Fe (Fe-SS) layers are interspersed randomly (rather than layers), and the B2-SS is scattered throughout the region known to be near-equiatomic FeAl B2. After considerable “optimization” (repeatedly changing the reference chemistry selected in point EDS for each phase), the user can eventually reach a phase map (as shown in Fig. 4e) that looks similar to the known microstructure; this of course can only be achieved because the microstructure in this case is known in advance.
Fig. 4. Phase mapping an Fe–Al MIL composite. (a) Electron image showing the multiphase microstructure. (b) EDS map of Fe. (c) Hough transform EBSD map with user-selected libraries. (d) Phase map generated by Hough transform combined with non-optimized EDS measurements. (e) Phase map generated by Hough transform combined with optimized EDS measurements as reference chemistry. (f) Phase map generated from the CNN-based model's analysis of each diffraction pattern individually. Scale bars = 500 μm. There are 10,164 total EBSPs (pixels).
By comparison, a CNN trained on a small subset of patterns belonging to each phase produces the phase map seen in Figure 4f. While the phases are not as perfectly linear as in Figure 4e, it is likely that the merging of phases at their borders is real to some degree according to the analysis performed by Wang et al. (Reference Wang, Harrington, Zhu and Vecchio2019) and their observation that crystal orientation influences the diffusion rates. Otherwise, the phase map is in good agreement with the expected results, and the CNN-based EBSD method is adept at separating diffraction patterns based on space groups and small changes in chemistry within the same space group.
CNNs applied to EBSD can resolve another challenging problem that phase identification including EDS reference chemistry for each phase does not completely resolve. The electron image for a cycled thermal barrier coating containing five FCC phases and one rhombohedral phase is shown in Figure 5c. The top layer is yttria-stabilized zirconia (YS–ZrO2), followed by the thermally grown oxide layer (Al2O3). In the middle section is a complex bond coat containing Ni, a chrome-carbide phase (Cr23C6), TaC, and a small amount of Al2O3. The bottom layer is the nickel-based superalloy Hastelloy X with Cr23C6 at the grain boundaries from thermal cycling. Except for the Al2O3 phase, each of these phases belongs to space group 225, meaning that their crystal symmetries are the same, but the lattice parameters can be different. As seen in Figure 5a, the Hough transform method alone is unable to reliably differentiate the FCC phases and indexes them primarily as Hastelloy X with some interspersed YS–ZrO2. The phase fraction of TaC (the brightest spots in the electron image) is much too large (based on the known chemistry) and is incorrectly included in the substrate. The Hough transform performs reasonably well at identifying the band of Al2O3, which is not surprising given its rhombohedral structure. Including reference chemistry for each of the phases improves the overall quality of the Hough-based EBSD phase map, as shown in Figure 5b. The band of TGO is further defined and the YS–ZrO2 layer contains much fewer erroneous pixels. However, the Hastelloy X substrate has large grains and twins that are being identified as TaC and the Hastelloy X is indexed as approximately 40% of the bond coat layer (center section). Furthermore, the phase fraction of TaC in the bond coat is still much higher than what is observed to be present in the electron image. Figure 5d is the result of training a CNN-based model to differentiate these phases. While not free of errors, the resulting phase map is observed to be a higher-fidelity mapping of the area shown in the electron image than traditional Hough-based EBSD offers. Comparison with the EDS maps in Figures 5e–5l further confirms the increased plausibility of the phase fractions and the credible identification of each diffraction pattern. The YS–ZrO2 layer is nearly completely indexed as such with most of the errors present residing at pores and cracks in the oxide. The Al2O3 is now indexed as a continuous band of thermally grown oxide in good agreement with the morphology of the Al EDS map. The central region is almost entirely indexed as Ni with interspersed TaC, Cr23C6, and Al2O3. Careful comparison with the EDS maps for Ta, Cr, Al, and O reveal the chemistry to overlap well with the presence, or absence, of the respective phase in the CNN-based phase map. The Hastelloy X substrate is well confined and interlaced with Cr23C6 particles that were not identified in Figures 5a and 5b but are clearly present in the Cr EDS map.
Fig. 5. Phase mapping a thermal barrier coating. (a) Hough transform EBSD map with user-selected libraries. (b) Phase map generated by Hough transform combined with optimized EDS measurements as reference chemistry. (c) Electron image showing the multiphase microstructure. The Hastelloy X substrate is at the bottom of the image. (d) Phase map generated from the CNN-based model's analysis of each diffraction pattern individually. (e–l) EDS maps of the analyzed region. Scale bars = 100 μm. There are 45,052 total EBSPs (pixels).
The final demonstration of the potential for CNNs to revolutionize EBSD technology is perhaps the most universally known limitation: the challenge of separating martensite from ferrite. This problem has traditionally been resolved by setting band contrast (or pattern quality) thresholds and accepting some degree of misindexing. The determination of the threshold value is biased by human criterion and can result in drastically different maps and phase fractions depending on the user-selected threshold. Another applicable method for mapping tetragonality in martensitic steels, called high (Angular) resolution EBSD (HR-EBSD), uses a reference pattern to measure the relative strain rate of all other patterns (Tanaka & Wilkinson, Reference Tanaka and Wilkinson2018). While this technique achieves high sensitivity (~10−4) and can be used to back calculate relative c/a ratios, the absolute strain and c/a ratio of the reference pattern is typically unknown. This method is not compared herein owing to the need for a dictionary of simulated patterns for the initial calibration. The microstructure of commercial 430 stainless steel is displayed in Figure 6a. While not yet obvious from the electron image alone, the martensite regions appear raised compared to the ferrite. The band contrast map in Figure 6b confirms both the location of the martensite as well as the overlap in pattern quality between the martensite and ferrite. This is the same problem encountered in the previous work (Goulden et al., Reference Goulden, Mehnert, Thomsen and Jiang2017) and explains why their pattern quality-based model was dependent on iterative phase discrimination and “locking” pixels the user deemed correctly identified before re-analysis. The region depicted in Figure 6a is mapped via traditional EBSD (Fig. 6c) using ferrite and martensite libraries generated based on the lattice parameter from XRD and well-established equations from the literature (Xiao et al., Reference Xiao, Fan, Jinxiu, Mingxing, Mokuang and Zhenqi1995) (refer to the Methods section for more information). The Hough-based method results in nearly all the 12,236 collected diffraction patterns being identified as martensite. In contrast, the machine learning model trained on “standards” for martensite and ferrite produces a phase map (Fig. 6d) with strong correlation to what is observed in the band contrast map. As seen in Figure A1, the martensite and iron diffraction patterns used to train the model as out-of-sample “standards” are of considerable quality. Therefore, unlike the previous study (Goulden et al., Reference Goulden, Mehnert, Thomsen and Jiang2017), our CNN-based model is not reliant on pattern quality metrics as a singular variable and each diffraction pattern is indexed in one pass through the data. A histogram of the band contrast values is shown in Figure A3. A comparison to the common thresholding method is made in Figure 6e. A threshold of band contrast <45 is applied to map the lowest quality patterns to martensite. This threshold was selected using the histogram as a guide. The machine learning model compares favorably to this method and many of the most obvious regions overlap. However, the patterns on which the two models disagree are not strictly band contrast-dependent. In fact, several medium or low band contrast regions are identified as ferrite by the machine learning model (Fig. 6d), while the same regions are mapped to martensite when classified by pattern quality (Fig. 6e).
Fig. 6. Phase mapping 430 stainless steel. (a) Electron image showing the microstructure. (b) Band contrast map for the region depicted in (a). (c) Hough transform EBSD map with user-selected libraries for martensite and ferrite. (d) Phase map generated from the machine learning model's analysis of each diffraction pattern individually, using “out-of-sample” standards. (e) Phase map constructed using a threshold value (<45) for the band contrast in each EBSP. Scale bars = 25 μm. Arrow points to regions in (d) mapped as ferrite by the machine learning approach but mapped as martensite by pattern quality in (e). There are 12,236 total EBSPs (pixels).
The distinction between ferrite and martensite in EBSD has routinely been considered one of the grand challenges in EBSD. With support from well-designed machine learning models, the technique is demonstrated to be capable of highly accurate, autonomous classification of these two phases.
Conclusion
In this paper, a CNN-based approach to accessing latent signals in EBSPs is developed and demonstrated. The flexibility and utility of this methodology is established by designing training sets for a variety of cases that are challenging problems in traditional Hough-based EBSD, but for which the information is present in the diffraction pattern. The output of the trained CNNs is compared to phase maps generated using state-of-the-art Hough-based EBSD systems and practices. In each case, the model is demonstrated to perform well at the task of properly identifying phases, while validating the flexibility of the new methodology.
Major improvements offered by the CNN-based phase-mapping approach are the identification of crystal symmetry (e.g., Bravais lattices or space groups), the correct identification of phases from incomplete or otherwise low-quality diffraction patterns, and the ability to distinguish crystallographically similar phases with subtle differences in the chemical composition or the lattice parameter. Even martensite and ferrite, which have a nearly equivalent crystal structure, can be distinguished using this methodology. Moreover, these are accomplished without the need for additional inputs to the model. Further development of this technology is expected to yield a significant number of advancements to the EBSD platform, specifically this approach can move EBSD from a user-dependent methodology, to an autonomous phase identifying microstructure characterization platform. This work demonstrates these potential capabilities using in-house development of models and available diffraction patterns. The methodology can readily be applied by other researchers for the development of new models. Inclusion of simulated diffraction patterns may help accelerate the process. In the future, we foresee commercial EBSD systems being equipped with standardized models, and potentially even on-demand modeling capabilities, for working with specific use cases or elucidating information about unknown materials.
Acknowledgments
K.K. was supported by the Department of Defense (DoD) through the National Defense Science and Engineering Graduate Fellowship (NDSEG) Program. K.K. would also like to acknowledge the support of the ARCS Foundation, San Diego Chapter. K.V. would like to acknowledge the financial generosity of the Oerlikon Group in support of his research group.
APPENDIX
Fig. A1. Representative diffraction patterns from each phase in the six materials studied. All patterns shown are collected from the material studied. The sample number is noted to the left of each group of patterns.
Fig. A2. Schematic of the neural network. In convolutional layers, a learnable filter is convolved across the image and the scalar product between the filter and the input at every position is computed to form a feature map. Pooling layers are placed after convolutional layers to down sample the feature maps and produce coarse grain representations and spatial information about the features in the data. A traditional dense neural network is placed as the last layer, where the probability that the input diffraction pattern belongs to a given class is computed. These outputs are used to construct a phase map.
Fig. A3. Histogram of band contrast values for the 430 stainless steel map. Band contrast values are binned in groups in steps of 5, starting from 0. The left-edge value is included in the count for each bin, but not the right-edge value.
Open access
