Oh, Bernadette, how I can sympathize with this story. Mine is slightly different but with the same result. I didn't leave but my daughter and grandchildren did when she remarried. For the prior 6 years they lived with us. I helped raise the children. They left as little kids and now, two years later, one is a teenager and the other thinks she is. I miss them so much - both the people and the life we had together - that it's better to limit my remembering so I don't sink into a hole of depression.
Your Diamond is a lovely woman, and I have no doubt she cherishes her memories of those summers spent with you.