So if I go back to Tom’s words: ‘You can try to be helpful, you can try to educate. It’s not that you have to not interact, it’s just that you have to not demand, you have to not manipulate. You have to let it be‘. Can anybody clarify this approach to me? How would this work?
Here, Tom is speaking generally to a general audience about personal evolution where manipulation is being done out of fear/ego. In your case you, you have a specific goal to accomplish by educating these people. That approach from Tom can help you, on a personal level, but it will not necessarily help the situation in the classroom. Letting go of anger and frustration is a good thing but letting go entirely of trying to influence things(manipulate) is not necessarily going to be a more optimal approach to the task, especially if a large part of your intent to have them stop talking and apply themselves more is to help them.
My interpretation is that the problem is the educational structure that most students are coerced into which doesn't fulfill or appeal to any of their interests or ways of learning and also your restrictions to a conventional way of teaching for conventional test-taking as a metric of success. However, it's not just a matter of ineffective teaching methods, these kids, I'm assuming largely millennials, are coming from a certain culture and exist in the school system within a certain social system that influences their behavior and their personal value of learning in class. Some teachers use the threat of punishment because fear of consequences can be a motivator to students not motivated to learn which can garner test results and test results are also a measure of a teacher's success. Other teachers try to be relatable and try to be a friend and that can work for some but often doesn't lead to much to kids who use politeness as a doormat.
Being a teacher, especially in this point in time, presents complex social dynamics and situations that I personally don't have an answer for but in these social situations with these students, you have the ability to act in a certain way and receive genuine feedback. My only humble advice is to detach from the expectations of test results, the pressure of parents, who have zero clue about what's going on, and the hierarchy and politics of school administration. Relate on a genuine level, not just as a teaching figure, dare to explore other teaching methods and use your intuition to guide your interactions with students.