Stucatz,
It sounds like you might have a problem with your battery as with everything turned off to battery should not be discharging unless there is a problem.
In regards to the solar panel, as you have said that it has 4 watts of output. This is when it is working to its full capacity and in bright sunlight directly above it. If your battery is capable of producing 700 amps of cranking power it needs at least 8,400 watts of power to deliver that. By simple mathematics this means that if the battery was dead flat you would need 2,100 hours of sunlight to achieve this.
A 4 watt solar panel delivers about 1/3 of amp of current. As you can see the panels do not delivery anywhere near enough charge to bring a battery up to scratch.
The solar panels are good for keeping a battery topped up but are not designed to charge a flat battery.
I hope this helps.