Extinction maps at 8μm from the Spitzer Space Telescope show that many Class 0 protostars exhibit complex, irregular, and non-axisymmetric structure within the densest regions of their dusty envelopes. Many of the systems have highly irregular and non-axisymmetric morphologies on scales ~1000 AU, with a quarter of the sample exhibiting filamentary or flattened dense structures. Complex envelope structure is observed in regions spatially distinct from outflow cavities, and the densest structures often show no systematic alignment perpendicular to the cavities. We suggest that the observed envelope complexity is the result of collapse from protostellar cores with initially non-equilibrium structures. The striking non-axisymmetry in many envelopes could provide favorable conditions for the formation of binary systems. We then show that the kinematics around L1165 as probed with N2H+ are indicative of asymmetric infall; the velocity gradient is not perpendicular to the outflow.