Why does my Steam profile need to be public?
Steam Bookmarks uses the official Steam Web API to determine the game you are currently playing. If your profile is private, the GetPlayerSummaries endpoint doesn't return your current game.
Where is my data stored?
All your settings and bookmarks are stored locally, in your browser. That's why loading your Steam Bookmarks URL in another browser doesn't display your bookmarks.