Sara Benson, who has the podcast Copyright Chat, recently did a 10-minute episode on reading books aloud and Fair Use. The episode is available where you get your podcasts and also on her library's website (along with a transcript). She also points to a guide written earlier this year by eight people entitled "Reading Aloud: Fair Use Enables Translating Classroom Practices to Online Learning."
As Benson states, since the start of our stay-at-home orders in March and the need for libraries and teachers to work remotely, people have had questions about reading books aloud online. We're used to do this in-person in the library or classroom, but what happens when we do this online? The good news is is that Fair Use still applies.
If you are relying on Fair Use, do you need permission or guidance from the publisher? No. While some publishers have provided such guidance, you don't need that. You need to look at, understand, and rely on Fair Use.
Are there instance where Fair Use does not apply? Some. The one that Benson notes is when read alouds are posted to YouTube and those videos have ads. Rather than posting your videos for everyone to see - and in a platform that you cannot control - I would hope that you would consider how to deliver your read alouds to the group that you normally do this with. Stay focused on your audience (which helps you keep your use fair).