Agree. This would make for a really simple improvement - just add a red dot (e.g. the same way how it appears on the plugins icon when there’s an update available) to the filter icon when a filter is applied
As an additional related improvement, I’d suggest making the filter icon floating when scrolling through active references. This would make it easier to change filters when you realise you’re looking for the wrong related page