In Canada, tap water is perfectly fine to drink as is. There are some exceptions when it comes to more remote places, especially in some camping grounds, but they’ll tell you or write it somewhere if that’s the case.
I went to Iceland last year and the water’s very clean and safe! There’s no problem drinking it from the tap.
And if you want to travel cheaper, youth hostels are very affordable. Though you are not garanteed to have a private room, it’s perfectly ok if you just want a place to sleep imho