Presuming you raised 4BB pre-flop, once he has matched your flop call there is approx 22BB in the pot.
Against the made hands you mention (set, TPTK and 2 pair) we are dogs of approx 28%, 43% and 36% respectively. If we consider each hand equally likely then we are approx 2/1 against to win the hand.
Therefore calling an all-in against anyone with more than another 22BB behind them is negative EV.
Realistically I would imagine his range would be slightly bigger, so I'd probably call another 30BB or so, but never the 90BB+ that he has.
