SSH Prompts for Password After Keys Setup Successfully

[Rynok]

Note: I setup my keys properly. (public key of ServerA added to authorized_keys of ServerB and chmod 700 used on .ssh directory and the files contained inside)

After setting up keys, I am having an issue with SSH where it prompts me for a password when I never actually setup the password on my user profile. (Please note, there is a difference between "not setting a password" and explicitly saying "passwd -d" to remove a password). SSH only has this error when I haven't setup a password on the user.

I was wondering, is there a configuration setting that I'm missing that is causing SSH to prompt for a password that doesn't exist? (It shouldn't prompt for a password at all since keys are setup, but apparently it does. I know it's not a problem with my ssh files though because setting a password on the user on the system I'm SSHing to fixes it)

Any help you can offer would be much appreciated.

[lomcevak]

The problem may be that the permissions of the files are 700. I believe they have to be 600.

[Rynok]

If that was the problem, then it would also not work when I set the password on the user account on the server I'm connecting to. All I change is I setup the password and it magically works. Chmod isn't the problem I'm afraid

[Rynok]

Bumping this to see if anyone has encountered this problem before or not.
To re-summarize for the forum browsers:
SSH Keys Setup so that rynok@ServerA->rynok@ServerB
No password is set on 'rynok' user.
SSH Prompts for a Password.
I set a password on 'rynok@ServerB'
SSH Works w/o a Prompt!

Any idea why it only prompts me because there is no password set?
Is this normal and I'm missing a config that gets around this?