Fathers rights over child?

do i have rights over my son I have been to court for contact. however,the judge agreed this access because he felt she was doing this out of despite & not because I was a bad parent.I also have a 6 year old and she knows i am a good parent and have 60/40 pr over my eldest I am on the birth cert was rewarded weekend access through court.my ex partner was not recommended at the magistrates by any solicitor.. i started having him weekends witch was agreed then straight after on the next weekend the mother broke the agreement witch was made by the court and i started having him more than what was aggreed through the court. however she has granted me access within one week without consulting my solicitor witch i took the rite path and told him exactly that! I pay CSA for my son not much but was agreed by the judge himself. now I have my son every weekend and she now is starting to leave him with me and not attending to pick him up without letting me know within 2 DAYS after number times I phoned her phone I have no emergency contact ... I want to know what rights I have after this stage and so on...