Switch Hidden Filter addon allows you to quickly toggle on or off the Permanent Filters feature from the menus.

You can access it from View > Switch hidden filter menu entry