Several children were born to them, all of whom unfortunately died in their infancy except the Princess Mary, afterwards Queen Mary of England. Even before there was any question of separation from his wife, Henry's relations with some of the ladies at court were not above suspicion. By one, Elizabeth Blount, he had a son whom he created Duke of Richmond and to whom at one time he thought of bequeathing the crown of England. In a short time Mary, the eldest sister of Anne Boleyn, succeeded to Elizabeth in the affections of the king. The fact that Catharine was some years older than her husband, that infirmity and sorrow for the death of her children had dimmed her charms, and that there could be no longer any hope for the birth of an heir to the throne, preyed on Henry's mind and made him not unwilling to rid himself of a wife, whom, however, he could not but admire even though she had forfeited his love. Were he to die there was no one to succeed him but the Princess Mary, and her right to the throne might be contested. Even though she succeeded, her marriage must inevitably create great difficulties. Were she to marry a foreign prince, he feared that England might become a province; were she to accept the hand of an English nobleman, a disputed succession ending in civil war was far from being improbable. His gloomy anticipations were shared in by many of his advisers; and Wolsey, who had set his heart on uniting the forces of England and France against the Emperor, was not unwilling to set a seal on the new French anti-imperial alliance by repudiating Henry's marriage with the Emperor's aunt, if such a dissolution could be brought about without infringing the laws of God.
Though it would seem that doubts had long since arisen in Henry's mind regarding the lawfulness of his marriage to his deceased brother's wife, and that questions of policy may have influenced the attitude of his advisers towards the projected separation, yet it is certain that it was the charms of the young and accomplished Anne Boleyn, that brought matters to a crisis. With her experience of the gay and corrupt court of France, she was not likely to be mistaken about the influence of her charms or the violence of the king's passion. She would be the king's wife if he wished; but she would not be, like her sister, the king's mistress. Overcome by the force of his desires, he determined to rid himself of a wife of whom he was tired, in favour of her young and more attractive rival. The fact that Catharine had been married to his brother Arthur was seized upon by him to furnish a decent pretext for the projected separation. His conscience, he averred, reproached him for such an ******uous alliance, and for his own peace of mind it was necessary, he maintained, to submit the validity of his marriage to the decision of the Church.
There is no convincing evidence that the idea of a separation from Catharine originated with Cardinal Wolsey, though the latter, longing for a matrimonial alliance of his king with a French princess, and not aware of Henry's intention with regard to Anne, was probably not sorry when he learned of Henry's scruples; and it is not true to say that the first doubts regarding the illegitimacy of the Princess Mary were raised by the French Ambassador in 1527. The whole story of the negotiations with France regarding Mary's marriage at the time, makes it perfectly clear that her legitimacy was assumed. The divorce proceedings originated in Henry's own mind, and the plan of marrying Anne Boleyn was kept a secret from Wolsey and from most of the royal advisers. When exactly the question of a separation from Catharine was first mooted is uncertain; but there can be no doubt that early in 1527 active steps were taken to secure a condemnation of the marriage.