Are museums places about a community or for the community? This article addresses this question by bringing into conversation Jewish museums and Indigenous museum theory, with special attention paid to two major institutions: the Jewish Museum Berlin and the National Museum of the American Indian. The JMB’s exhibitions and the controversies surrounding them, I contend, allow us to see the limits of rhetorical sovereignty, namely the ability and right of a community to determine the narrative. The comparison between Indigenous and Jewish museal practices is grounded in the idea of multidirectional memory. Stories of origins in museums, foundational to a community’s self-understanding, are analyzed as expressions of rhetorical sovereignty. The last section expands the discussion to the public sphere by looking at the debates that led to the resignation of Peter Schäfer, the JMB’s former director, following a series of events that were construed as anti-Israeli and hence, so was the argument, anti-Jewish. These claims are based on two narrow conceptions: First, that of the source community that makes a claim for the museum. Second, on the equation of Jewishness with a pro-Israeli stance. Taken together, the presentation of origins and the public debate show the limits of rhetorical sovereignty by exposing the contested dynamics of community claims. Ultimately, I suggest, museums should be seen not only as a site for contestation about communal voice, but as a space for constituting the community.