If you get stuck going to the Wharf, you can still do yourself a favor and stay out of the core: (a) on the east side, consider Pier 39 as the extreme tip of North Beach; (b) on the west side, if you've ridden the Powell Street cable car line out to where it ends on Hyde street, you should then make a *left* turn, and walk through the aquatic park, then go up over the hill-side trail to Fort Mason Center. You should definitely skip the Ghiardelli's trap.

Fort Mason Center is an old WWII era military supply base (You know, Liberty Ships, Henry Fonda's "Mr. Roberts", etc), long since re-purposed as a convention center plus. "Plus" includes the famous "Greens" vegetarian restaurant (with a very reasonable coffee/food take-out counter, which you should note is near some convenient benches right on the docks); a good used books store, and the "Long Now Foundation" offices, with a small museum/gallery attached.

There's one problem with this scheme of just skipping the Wharf: you'll miss the Museé Méchanique and their collection of antique music boxes and arcade attractions. They relocated into the middle of the wharf area, on pier 45. The original electro-mechanical music (Reference).