I need advice on my relationship.?

I have been dating my boyfriend for over 2 years. We have a 1 year old daughter. He also has a 8 year old girl, and 9 year old boy from his ex wife. He is always busy with working or spending time with the kids. His kids live with us during the summer and he sees them every other weekend plus one day a week during the school year. He always wants to spend more and more time with his kids. And let me state that during the summer he coaches his son's little league team so that is a few nights during the week and every weekend. Then his daughter plays volleyball and he has to go to all her games or he feels guilty. Lastly, he himself plays ball every weekend. So needless to say i end up feeling a little short changed all summer long. His divorce/custody agreement stated he would have them every weekend during the school year, but luckly his ex wife changed her mind after the first year and wanted them on some weekends. What i'm upset about is he just informed me that if it wasnt for me he would see his kids more, and that he missed them.. and he stated he wants to play more ball for himself. How is there any time left for ';our'; relationship as a couple? I'm sad he never wants to just spend more time with me.. its either time with the kids and of course i'm always included, but i like adult time tool. Or he just wants to be alone and play ball, and he says i'm invited to watch... but thats just me sitting there on a bench holding our 1 year old little girl. I just keep thinking it may be time to move on.. but i'm reluctant because we have built our lives together over past 2 years and have the children involved.I need advice on my relationship.?
Sadly, he isn't giving you the attention and love that you so richly deserve! He is not worthy of you, baby! You should give him an ultimatum - tell him things MUST change or it's over! Why remain in a marriage where you feel under-loved? Life is far too short to be unhappy! He's probably not even aware of your true feelings - guys can be like that sometimes! Suggest marriage counseling so that a neutral person can hear you two out and offer professional advice to you both. If it still doesn't work out, at least you tried everything that you could! Good luck, baby!