Kao Atlantis and Look have the right idea here. If what you're saying is true it sounds like your account may be compromised and someone may be playing it without your knowledge.
Player Support has the tools necessary to investigate and see what might be going on. Depending on the circumstances, most damage done to an account as a result of a compromise can be undone. But time is pretty important here to mitigate further damage, so contact us as soon as you can.
In addition, if you share that password with any other web apps or games (ie: facebook or e-mail) I'd encourage you to update it.
I generally try to update my passwords every 60 days or so to something new. The risk isn't really tied to any particular web app or feature, but if you share your password among many different services it can lead to compromises in others. For example, usually following any leak of account information from popular games we see an increase in Account recoveries in our own playersupport (despite the leak being for a different game or service unrelated to League). Since many share their password between League and other services, if that other service gets compromised, hackers know to try the same password in other services to see how far they can get..