Federal budget watchdog’s comments are ‘just wrong,’ says one of his predecessors