The meeting house where Priestley rose in 1780 owned-upon its destruction by a ‘Church and King’ mob in 1791-123 kneeling cushions and 319 psalm books.
If they were to return to freedom, they would receive a bible, sermon book, and psalm book, respectively.
When reading this passage, our religious friends visualize church psalm-books. In their opinion, these psalm-books could include any Old Testament psalm set to music.