vnc - "connection refused"

I'm trying to figure out if vnc requires additional steps, or what I'm forgetting about installing vncserver and vcviewer. When running vncviewer from the client, i.e., vncviewer 192.168.xx.xx to a computer on my home network, I get "connection refused." I set up a password on the server with vncpasswd. Should I have done that on the client instead? Is there a config file which allows or denys connections? --