If you are leaving from Tokyo, transporatation to the ship by taxi will be quite a bit less than Silversea's quote. Don't forget that there is no tipping in Japan so all you are paying is the taxi fare.
If the ship actually leaves from Yokohama, then you are just paying for the convenience with the Silversea transfer.
From Narita to hotel will cost you 3,000 JPY (about $35) each on limousine bus; by far the easiest way to go. Do not take a taxi; will cost you around $400 or more.