Resistive switches are being explored as a candidate for ultra-dense memory as well as logic circuits. The advantages of the resistive switches include high switching speed and excellent scaling potential. Here, we report for the first time the switching behavior of anti-parallel connected resistive switches (APS), which is a composite device exhibiting bi-directional switching properties. Under the opposite voltage biases, the two anti-parallel cells are alternatively set and reset, rendering the APS switched in both directions. For appropriate ON resistance values and set and reset voltages the two anti-parallel switches can be both set in conductive states. An APS device can be realized in a single switch by two coexisting Cu and oxygen vacancy nanofilaments which are formed and ruptured under opposite voltage polarities. The described APS behavior is of interest to logic applications and in neural networks.