Securing by a separate resilient retaining piece supported by base or case