If you are lucky, you have direct access to your customers. In our experience very few Product Owners have this luxury – especially if they are in large organisations. Most Product Owners we come across have various business stakeholders and face the daunting task of managing their often conflicting demands and schedules.
Product Owners know that their job is to maintain the backlog, and this means prioritising the backlog as well. Many POs don’t feel they have the authority to do this and so prioritisation by decibel factor wins.
(Prioritisation by decibel factor = whoever shouts the loudest gets their stuff at the top of the backlog)
But you don’t need to have authority to make these decisions, what you need are great facilitation and negotiation skills.
Firstly – you need to know who your stakeholders are and what they care about. Draw up a mindmap listing everyone who you think is/might be a stakeholder. Use people’s names, not just a department or job title. Now jot down what valuable information that person can provide. What are their concerns?
You can also mark which stakeholders are the most important in terms of business value and company strategic views.
Use this mindmap to check if the time you spend with your stakeholders is being spend effectively. Also think of this as a living document. Update it when your stakeholders change or if your product focus changes.
[box] 
This is a technique we use in our book “Growing Agile: A Coach’s Guide to Mastering Backlogs”. The book contains workshops you can run with your team and stakeholders to help them.