I don't think it's either yours or the ADC's fault, it's just something that requires practice. Generally, CSing isn't as important of a skill to supports as something like vision control is, but it's still a valuable skill to learn so you can help better manage wave control, assist your ADC in getting last hits when under turret, last hit properly when you are in lane alone (or if you want to take some CS for yourself. This is frowned on by a lot of players, but CS doesn't belong to anyone), etc.
So should you worry about it as a support? Imo not -really- , though it's still an important skill that you should try to work on which will become more important when you're against tougher opponents. There are a lot of variables that can effect your CSing, and it's difficulty is also based off of who you're playing and who you're against.
If you are interested in improving your skills, I suggest trying to look up some videos on how to CS and then practicing them a bit in a more controlled environment, whether that be practice tool or AI games.