From the research I've been doing it looks like the alternator is not even necessary, a DC motor will output DC when spun. Then you just need to run it through something like a charge controller and into a battery. There are a bunch of videos on YouTube where DIYers have built this kind of set-up.
Alkalines have been in the back of my mind from the start as a fallback option. It just seems like a shame to resort to such a cheap solution when I wouldn't mind spending more for a somewhat better solution. Which is why I was curious if I could take a heftier AGM battery and follow the recommended maintenance procedure of charging it to capacity with low current every few months, and have a lot more power available.
Another option I considered for a fair bit of time was a portable fold-out solar panel. I may live in an apartment, but I could take one of those out to a park and read a book or something while charging. Downsides are, you still need a battery to charge, or you need to bring your electronic devices that need charging outside and pray that the weather cooperates.
Overall I'm kind of liking the bicycle generator idea, mostly because it sounds the coolest
. It also doesn't tie me to the weather conditions, and is completely sustainable over the long-term. I'm probably still going to research a little more before I make any decisions though. I also have a feeling I may get some alkalines as well anyway, just because they're so cheap.