Live Tournament, down to the final two tables (final table gets paid). Blinds 500-1,000. I'm in the BB with 4,900 + 1,000 chips.

.
It's folded round to the button who pushes with 5,400 total (so calling would leave me 500). SB immediately folds.
A bit of background on the table: Quite loose since the blinds went up, average chips are around 6BB. The button pushed on the previous hand, too, when it was folded round to him. BB passed, he didn't show.
Basically I sat there for a good five minutes, not really giving him credit for much of a hand, especially at he had (1) only about 5 BB left (2) was the button (3) had pushed the previous hand, too .... and ended up calling. He had AQo and I failed to suck-out.
In retrospective, I think it was a pretty horrendous call from my part, looking at a coin-flip at best and being down to the felt it I lost. Folding this and the SB would have left me with 4,400 and a full round of opportunities to get my money in first / catch a hand and double up, &c. &c.
So two things: Please confirm that it was a horribly fishy call (1) and any advice on how I can stop talking myself into calling in situations like that (2). Any advice would be appreciated, just quite annoyed with myself, as I felt I had played very good poker until that point, being CL for a long time, had lost a good amount of chips a couple of rounds earlier on, but that was one of those situations where you can't do much (QQ v AK - all in preflop, A on the turn ..), and with about average chips I think I would have had a fair shot at making it at least into the money, with some cards maybe even quite deep into it ..