I would highly recommend it. You could easily spend a whole day at Wawel alone. I also recommend off season, considering how busy it was for us even then (OK, it was a Sunday so I guess it's a major destination for Polish families).
We deliberately skipped the dark parts (memorials related to Nazi occupation and holocaust, Schindler's factory, Auschwitz wouldn't have been far) because it was supposed to be fun but if I'd had more time, I would have picked a day dedicated to that.
The weather was pretty much perfect for wandering around a city so this time of year (or spring) would be ideal.