I'd make a complaint with your bank and the one that took your card if you had ID. I can understand them not hanging it over if they didn't know who you were, but if you had something like a drivers license then there's no reason for them to have witheld it. It's no exactly easy to get cash out on a weekend!
That may be because, as it's not your bank, they can't confirm your ID with their database or bank account number. Your card doesn't have much more info than an account number and a name. They can't see who the account number belongs to, and the name on the card could belong to anyone. Banks are obligated to be very strict with their security and identification.