Right re-tested it and it was user error on my behalf. It must cache all the threads it reads when a certain user, so if I give a test user mod permissions then remove them, it will still show threads that were downloaded when it was a mod. Clearing the cache between changes meant it followed the security properly
So I've renabled it. If you have the app, search for "350z" and you will see the forum on the list