Hostname: page-component-76fb5796d-45l2p Total loading time: 0 Render date: 2024-04-27T03:07:09.702Z Has data issue: false hasContentIssue false
  • English
  • Français

Active cloaking in Stokes flows via reinforcement learning

Published online by Cambridge University Press:  30 September 2020

Mehdi Mirzakhanloo Open the ORCID record for Mehdi Mirzakhanloo [Opens in a new window] ,
Soheil Esmaeilzadeh Open the ORCID record for Soheil Esmaeilzadeh [Opens in a new window]  and
Mohammad-Reza Alam Open the ORCID record for Mohammad-Reza Alam [Opens in a new window]
Mehdi Mirzakhanloo
Affiliation:
Department of Mechanical Engineering, University of California, Berkeley, CA94720, USA
Soheil Esmaeilzadeh
Affiliation:
Department of Energy Resources Engineering, Stanford University, Stanford, CA94305, USA
Mohammad-Reza Alam*
Affiliation:
Department of Mechanical Engineering, University of California, Berkeley, CA94720, USA
*
†Email address for correspondence: reza.alam@berkeley.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Hydrodynamic signatures at the Stokes regime, pertinent to motility of micro-swimmers, have a long-range nature. This implies that movements of an object in such a viscosity-dominated regime can be felt tens of body-lengths away and significantly alter dynamics of the surrounding environment. Here, we devise a systematic methodology to actively cloak swimming objects within any arbitrarily crowded suspension of micro-swimmers. Specifically, our approach is to conceal the target swimmer throughout its motion using cooperative flocks of swimming agents equipped with adaptive decision-making intelligence. Through a reinforcement learning algorithm, the cloaking agents experientially learn an optimal adaptive behaviour policy in the presence of flow-mediated interactions. This artificial intelligence enables them to dynamically adjust their swimming actions, so as to optimally form and robustly retain any desired arrangement around the moving object without disturbing it from its original path. Therefore, the presented active cloaking approach not only is robust against disturbances, but also is non-invasive to motion of the cloaked object. We then further generalize the proposed approach and demonstrate how our cloaking agents can be readily used, in any region of interest, to realize hydrodynamic invisibility cloaks around any number of arbitrary intruders.

JFM classification

Biological Fluid Dynamics: Micro-organism dynamics Biological Fluid Dynamics: Swimming/flying
Type
JFM Papers
Information
Journal of Fluid Mechanics , Volume 903 , 25 November 2020 , A34
Check for updates
Copyright
© The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Alam, M.-R. 2012 Broadband cloaking in stratified seas. Phys. Rev. Lett. 108 (8), 084502.CrossRefGoogle ScholarPubMed
Ariel, G., Sidortsov, M., Ryan, S. D., Heidenreich, S., Bär, M. & Be'er, A. 2018 Collective dynamics of two-dimensional swimming bacteria: experiments and models. Phys. Rev. E 98 (3), 032415.CrossRefGoogle Scholar
Bemporad, A., Morari, M., Dua, V. & Pistikopoulos, E. N. 2002 The explicit linear quadratic regulator for constrained systems. Automatica 38 (1), 3–20.CrossRefGoogle Scholar
Boyd, R. W. & Shi, Z. 2012 Optical physics: how to hide in time. Nature 481 (7379), 35.CrossRefGoogle ScholarPubMed
Bückmann, T., Kadic, M., Schittny, R. & Wegener, M. 2015 Mechanical cloak design by direct lattice transformation. Proc. Natl Acad. Sci. USA 112 (16), 4930–4934.CrossRefGoogle ScholarPubMed
Bullock, T., Atema, J., Fay, R. R., Popper, A. N. & Tavolga, W. N. 2008 Sensory Processing in Aquatic Environments. Springer Science & Business Media.Google Scholar
Camacho, E. F. & Alba, C. B. 2013 Model Predictive Control. Springer Science & Business Media.Google Scholar
Chattopadhyay, S., Moldovan, R., Yeung, C. & Wu, X. L. 2006 Swimming efficiency of bacterium Escherichia coli. Proc. Natl Acad. Sci. USA 103 (37), 13712–13717.CrossRefGoogle ScholarPubMed
Cichos, F., Gustavsson, K., Mehlig, B. & Volpe, G. 2020 Machine learning for active matter. Nat. Mach. Intell. 2 (2), 94–103.CrossRefGoogle Scholar
Colabrese, S., Gustavsson, K., Celani, A. & Biferale, L. 2017 Flow navigation by smart microswimmers via reinforcement learning. Phys. Rev. Lett. 118 (15), 158004.CrossRefGoogle ScholarPubMed
Cortez, R., Fauci, L. & Medovikov, A. 2005 The method of regularized stokeslets in three dimensions: analysis, validation, and application to helical swimming. Phys. Fluids 17 (3), 031504.CrossRefGoogle Scholar
Cummer, S. A., Popa, B. -I., Schurig, D., Smith, D. R., Pendry, J., Rahm, M. & Starr, A. 2008 Scattering theory derivation of a 3D acoustic cloaking shell. Phys. Rev. Lett. 100 (2), 024301.CrossRefGoogle ScholarPubMed
Darnton, N. C., Turner, L., Rojevsky, S. & Berg, H. C. 2007 On torque and tumbling in swimming Escherichia coli. J. Bacteriol. 189 (5), 1756–1764.CrossRefGoogle ScholarPubMed
Drescher, K., Dunkel, J., Cisneros, L. H., Ganguly, S. & Goldstein, R. E. 2011 Fluid dynamics and noise in bacterial cell–cell and cell–surface scattering. Proc. Natl Acad. Sci. USA 108 (27), 10940–10945.CrossRefGoogle ScholarPubMed
Drescher, K., Goldstein, R. E., Michel, N., Polin, M. & Tuval, I. 2010 Direct measurement of the flow field around swimming microorganisms. Phys. Rev. Lett. 105 (16), 168101.CrossRefGoogle ScholarPubMed
Elgeti, J., Winkler, R. G. & Gompper, G. 2015 Physics of microswimmers – single particle motion and collective behavior: a review. Rep. Prog. Phys. 78 (5), 056601.CrossRefGoogle ScholarPubMed
Ernst, D., Glavic, M., Capitanescu, F. & Wehenkel, L. 2008 Reinforcement learning versus model predictive control: a comparison on a power system problem. IEEE Trans. Syst. Man Cybern. 39 (2), 517–529.CrossRefGoogle ScholarPubMed
Fields, D. M. & Yen, J. 2002 Fluid mechanosensory stimulation of behaviour from a planktonic marine copepod, Euchaeta rimana Bradford. J. Plankton. Res. 24 (8), 747–755.CrossRefGoogle Scholar
Gallagher, M. T. & Smith, D. J. 2018 Meshfree and efficient modeling of swimming cells. Phys. Rev. Fluids 3 (5), 053101.CrossRefGoogle Scholar
Gazzola, M., Tchieu, A. A., Alexeev, D., de Brauer, A. & Koumoutsakos, P. 2016 Learning to school in the presence of hydrodynamic interactions. J. Fluid. Mech. 789, 726–749.CrossRefGoogle Scholar
Ghose, S. & Adhikari, R. 2014 Irreducible representations of oscillatory and swirling flows in active soft matter. Phys. Rev. Lett. 112 (11), 118102.CrossRefGoogle ScholarPubMed
Guenneau, S., Amra, C. & Veynante, D. 2012 Transformation thermodynamics: cloaking and concentrating heat flux. Opt. Express 20 (7), 8207–8218.CrossRefGoogle ScholarPubMed
Jalali, M. A., Alam, M. -R. & Mousavi, S. 2014 Versatile low-Reynolds-number swimmer with three-dimensional maneuverability. Phys. Rev. E 90 (5), 053006.CrossRefGoogle ScholarPubMed
Kim, S. & Karrila, S. J. 2013 Microhydrodynamics: Principles and Selected Applications. Courier Corporation.Google Scholar
Lauga, E. & Powers, T. R. 2009 The hydrodynamics of swimming microorganisms. Rep. Prog. Phys. 72 (9), 096601.CrossRefGoogle Scholar
Li, J., de Avila, B. E.-F., Gao, W., Zhang, L. & Wang, J. 2017 Micro/nanorobots for biomedicine: delivery, surgery, sensing, and detoxification. Sci. Robot. 2 (4), 6431.CrossRefGoogle ScholarPubMed
Liu, B., Breuer, K. S. & Powers, T. R. 2013 Helical swimming in Stokes flow using a novel boundary-element method. Phys. Fluids 25 (6), 061902.CrossRefGoogle Scholar
Martel, S., Felfoul, O., Mathieu, J.-B., Chanu, A., Tamaz, S., Mohammadi, M., Mankiewicz, M. & Tabatabaei, N. 2009 a MRI-based medical nanorobotic platform for the control of magnetic nanoparticles and flagellated bacteria for target interventions in human capillaries. Intl J. Rob. Res. 28 (9), 1169–1182.CrossRefGoogle ScholarPubMed
Martel, S., Mohammadi, M., Felfoul, O., Lu, Z. & Pouponneau, P. 2009 b Flagellated magnetotactic bacteria as controlled mri-trackable propulsion and steering systems for medical nanorobots operating in the human microvasculature. Int. J. Rob. Res. 28 (4), 571–582.CrossRefGoogle ScholarPubMed
Mirzakhanloo, M. & Alam, M.-R. 2018 Flow characteristics of chlamydomonas result in purely hydrodynamic scattering. Phys. Rev. E 98 (1), 012603.CrossRefGoogle ScholarPubMed
Mirzakhanloo, M. & Alam, M.-R. 2020 Stealthy movements and concealed swarms of swimming micro-robots. Phys. Fluids 32 (7), 071901.CrossRefGoogle Scholar
Mirzakhanloo, M., Jalali, M. A. & Alam, M.-R. 2018 Hydrodynamic choreographies of microswimmers. Sci. Rep. 8 (1), 3670.CrossRefGoogle ScholarPubMed
Nelson, B. J., Kaliakatsos, I. K. & Abbott, J. J. 2010 Microrobots for minimally invasive medicine. Annu. Rev. Biomed. Engng 12, 55–85.CrossRefGoogle ScholarPubMed
Niv, Y., Daw, N. D. & Dayan, P. 2006 Choice values. Nat. Neurosci. 9 (8), 987.CrossRefGoogle ScholarPubMed
Pané, S., Puigmartí-Luis, J., Bergeles, C., Chen, X.-Z., Pellicer, E., Sort, J., Počepcová, V., Ferreira, A. & Nelson, B. J. 2019 Imaging technologies for biomedical micro- and nanoswimmers. Adv. Mater. Technol. 4 (4), 1800575.CrossRefGoogle Scholar
Park, J., Youn, J. R. & Song, Y. S. 2019 Hydrodynamic metamaterial cloak for drag-free flow. Phys. Rev. Lett. 123 (7), 074502.CrossRefGoogle ScholarPubMed
Pécseli, H. L. & Trulsen, J. K. 2016 Plankton's perception of signals in a turbulent environment. Adv. Phys. X 1 (1), 20–34.Google Scholar
Pendry, J. B., Schurig, D. & Smith, D. R. 2006 Controlling electromagnetic fields. Science 312 (5781), 1780–1782.CrossRefGoogle ScholarPubMed
Pimponi, D., Chinappi, M., Gualtieri, P. & Casciola, C. M. 2016 Hydrodynamics of flagellated microswimmers near free-slip interfaces. J. Fluid Mech. 789, 514–533.CrossRefGoogle Scholar
Pozrikidis, C. 2002 A Practical Guide to Boundary Element Methods with the Software Library BEMLIB. CRC Press.CrossRefGoogle Scholar
Pozrikidis, C. 1992 Boundary Integral and Singularity Methods for Linearized Viscous Flow. Cambridge University Press.CrossRefGoogle Scholar
Reddy, G., Celani, A., Sejnowski, T. J. & Vergassola, M. 2016 Learning to soar in turbulent environments. Proc. Natl Acad. Sci. USA 113 (33), E4877–E4884.CrossRefGoogle ScholarPubMed
Ryan, S. D., Berlyand, L., Haines, B. M. & Karpeev, D. A. 2013 A kinetic model for semidilute bacterial suspensions. Multiscale Model. Simul. 11 (4), 1176–1196.CrossRefGoogle Scholar
Ryan, S. D., Haines, B. M., Berlyand, L., Ziebert, F. & Aranson, I. S. 2011 Viscosity of bacterial suspensions: hydrodynamic interactions and self-induced noise. Phys. Rev. E 83 (5), 050904.CrossRefGoogle ScholarPubMed
Saadat, M., Mirzakhanloo, M., Shen, J., Tomizuka, M. & Alam, M.-R. 2019 The experimental realization of an artificial low-Reynolds-number swimmer with three-dimensional maneuverability. In 2019 American Control Conference (ACC), pp. 4478–4484. IEEE.CrossRefGoogle Scholar
Schurig, D., Mock, J. J., Justice, B. J., Cummer, S. A., Pendry, J. B., Starr, A. F. & Smith, D. R. 2006 Metamaterial electromagnetic cloak at microwave frequencies. Science 314 (5801), 977–980.CrossRefGoogle ScholarPubMed
Smith, D. J. 2018 A nearest-neighbour discretisation of the regularized stokeslet boundary integral equation. J. Comput. Phys. 358, 88–102.CrossRefGoogle Scholar
Spagnolie, S. E. & Lauga, E. 2012 Hydrodynamics of self-propulsion near a boundary: predictions and accuracy of far-field approximations. J. Fluid Mech. 700, 105–147.CrossRefGoogle Scholar
Sutton, R. S. & Barto, A. G. 2018 Reinforcement Learning: An Introduction. MIT Press.Google Scholar
Tesauro, G. 1995 Temporal difference learning and TD-Gammon. Commun. ACM 38 (3), 58–68.CrossRefGoogle Scholar
Tsang, A. C. H., Demir, E., Ding, Y. & Pak, O. S. 2020 a Roads to smart artificial microswimmers. Adv. Intell. Syst. 2, 1900137.CrossRefGoogle Scholar
Tsang, A. C. H., Tong, P. W., Nallan, S. & Pak, O. S. 2020 b Self-learning how to swim at low Reynolds number. Phys. Rev. Fluids 5 (7), 074101.CrossRefGoogle Scholar
Tuttle, L. J., Robinson, H. E., Takagi, D., Strickler, J. R., Lenz, P. H. & Hartline, D. K. 2019 Going with the flow: hydrodynamic cues trigger directed escapes from a stalking predator. J. R. Soc. Interface 16 (151), 20180776.CrossRefGoogle ScholarPubMed
Urzhumov, Y. A. & Smith, D. R. 2011 Fluid flow control with transformation media. Phys. Rev. Lett. 107 (7), 074501.CrossRefGoogle ScholarPubMed
Walker, B. J., Ishimoto, K., Gadêlha, H. & Gaffney, E. A. 2019 Filament mechanics in a half-space via regularised stokeslet segments. J. Fluid Mech. 879, 808–833.CrossRefGoogle Scholar
Watkins, C. J. C. H. & Dayan, P. 1992 $Q$-learning. Mach. Learn. 8 (3–4), 279–292.CrossRefGoogle Scholar
Yen, J., Lenz, P. H., Gassie, D. V. & Hartline, D. K. 1992 Mechanoreception in marine copepods: electrophysiological studies on the first antennae. J. Plankton Res. 14 (4), 495–512.CrossRefGoogle Scholar
Yen, J., Murphy, D. W., Fan, L. & Webster, D. R. 2015 Sensory-motor systems of copepods involved in their escape from suction feeding. Integr. Compar. Biol. 55 (1), 121–133.CrossRefGoogle ScholarPubMed
Zhang, S., Genov, D. A., Sun, C. & Zhang, X. 2008 Cloaking of matter waves. Phys. Rev. Lett. 100 (12), 123002.CrossRefGoogle ScholarPubMed
Zhang, S., Xia, C. & Fang, N. 2011 Broadband acoustic cloak for ultrasound waves. Phys. Rev. Lett. 106 (2), 024301.CrossRefGoogle ScholarPubMed

Mirzakhanloo et al. Supplementary Movie

(a) Time evolution of a crowded suspension of intruders (shown in green), each of which actively cloaked by a pair of smart micro-swimmers (cloaking agents shown in blue). Here the intruders are freely moving toward arbitrary directions in space. Periodic boundary conditions are imposed on the presented panel -- i.e. it represents just a window of an infinite domain. As evident in the system's time evolution, using their adaptive decision-making intelligence, our agents are able to robustly maintain the cloak formation in the presence of complex hydrodynamic interactions. They are also able to immediately restore the desired formation after any sever close encounters which cause major disruptions to the cloak. (b) Magnitude of the disturbing flows (i.e. flow signatures) induced by the actively cloaked system, monitored throughout the presented time evolution.

Download Mirzakhanloo et al. Supplementary Movie(Video)
Video 4.5 MB