Your 5V/500mA solar panel would produce that in best case conditions. Outside, pointed at the sun (following the sun's track in the sky), no clouds, etc.
These are significant variables. If you expect it to charge well when the sun comes up, you'll need to aim it before you go to sleep, and keep re-aiming it.
Let's suppose you have a good day and get an average of 1.5W/hr out of it while the sun is up. That might be too optimistic, you'll have to try it and see.
Once you determine what power you're getting out of it in your worst case "needed" charged scenario, then you can figure out what charger you can use, or you can put more solar panels in parallel for more current, or more in series with a boost regulator board.
Since you don't want to have to monitor it, it does seem easiest to get a voltage boost regulator board, boost the voltage to 12VDC, to input to a charger that accepts that, which is *almost* any charger that comes with an AC/DC adapter that plugs into a barrel jack on the charger. Other input voltages are possible but 12VDC is by far the most common for a charger using an external PSU. Some of these chargers also include a car cigarette lighter adapter which is also a sign it can take 12VDC input.
Once you have such a charger, one that comes with an AC-DC 12V switching power supply, you can measure the current it consumes from that PSU while charging the needed battery(s).
You'd produce an equation, watts output from the solar cell, into a boost 12VDC regulator board, calculate the output as watts multiplied by the efficiency of the boost regulator board, divided by the 12VDC output to get amps, and then you see if the output current in amps is enough to power the charger based on the measurements you made of it running off a PSU instead.
My replies have been in a fairly random disorganized state because all the information wasn't presented up front in a concise manner so we could just do calculations instead of guesswork.
Frankly if this is only being used at home then I don't understand the need to charge batteries with a solar panel. It would only take a couple cents worth of power to charge a battery from an AC/DC switching PSU running on the AC mains power grid.
Another alternative that would give you a more consistent result would be using a solar panel, with a boost voltage charge controller, to charge up a 12V battery, then that battery is connected to the charger so it is in a known good, ready state whenever you want to charge batteries.
I think I'm about done with this topic because there's only so much I can add without wasting your time if you don't already have enough to go on.
It all starts with taking some measurements.
