Know how to listen, and you will profit even from those who talk badly. (Plutarch)

Modern mathematics is very much a collaborative activity rather than an individual one. You need to know what’s going on elsewhere in mathematics, and what other mathematicians find interesting; this will often give valuable perspectives on your own work. This is true not just for talks in your immediate field, but also in nearby fields. (For much the same reason, I recommend studying at different places.)  An inspiring talk can also increase your motivation in your own work and in the field of mathematics in general.

You also need to know who’s who, both in your field and in neighboring ones, and to acquaint yourself with your colleagues. This way you will be much better prepared when it does turn out that your work has some new connections to other areas of mathematics, or when it becomes natural to work in collaboration with another mathematician.  Talks and conferences are an excellent way to acquaint yourself with your mathematical community.

(Yes, it is possible to solve a major problem after working in isolation for years – but only after you first talk to other mathematicians and learn all the techniques, intuition, and other context necessary to crack such problems.)

Oh, and don’t expect to understand 100% of any given talk, especially if it is in a field you are not familiar with; as long as you learn something, the effort is not wasted, and the next time you go to a talk in that subject you will understand more. (One can always bring some of your own work to quietly work on once one is no longer getting much out of the talk.)

See also Tom Korner’s “How to listen to a maths lecture“, and Katherine Ott’s “Weekend Getaway Guide: A Mathematics Research Conference” from the Feb 2011 issue of the Notices of the AMS.