Five years ago, Harry Connick Jr.‘s world changed forever.

In October 2012, the multiplatinum recording artist, host of the daytime talk show Harry and actor’s wife, Jill Goodacre, had a routine annual mammogram that came back clear.

“They said, ‘Okay, looks good. Since you have dense breasts, just go across the hall for your sonogram,’ ” Goodacre, 53, tells PEOPLE in this week’s issue. But during the sonogram, something was detected. After undergoing a biopsy, Goodacre received the harrowing news — during breast cancer awareness month — that she had Stage 1 invasive ductal carcinoma and would need to immediately undergo a lumpectomy, followed by radiation.

“I was scared I was going to lose her, absolutely,” says Connick Jr., 50, whose mother died of ovarian cancer when he was 13. “I wasn’t going to let her see that, but I was. I know from losing my mom that the worst can happen. She’s my best friend, and I really don’t know what I would do without her.”

For Goodacre, one of the hardest parts of her cancer battle was telling the couple’s three daughters — Georgia, 21, Sara Kate (who goes by Kate), 20, and Charlotte, 15 — about the diagnosis. “It broke my heart,” she shares.

Although the mother of three did not have to undergo chemotherapy, her treatment has been grueling.

“The lumpectomy didn’t come back with clean margins,” she explains. Pathology tests showed she also had extensive ductal carcinoma in situ, a less invasive form of the disease. “So I had to go in for a second surgery the very next day. And then radiation absolutely wiped me out. And since then there’s been the Tamoxifen, which I’ve now been taking for five years.”

Tamoxifen, an estrogen modulator taken in pill form that helps prevent the development of hormone receptor-positive breast cancers, can have difficult side effects, including weight gain, which Goodacre — a former Victoria’s Secret model — has admittedly struggled with.

“I’ve always been a pretty fit person, and so to be just rounder and heavier and not to really be able to do much about it — that’s been hard. It’s taken a lot out of my self-confidence,” she says.

“It’s a part of how the cancer and the treatment impacted her, and it was a real issue, even though she will always be the most beautiful woman in the world,” adds Connick Jr.

Now, as she approaches the five-year mark of remission, Goodacre is looking forward to stopping Tamoxifen soon and preparing to tell the world what few outside her family knew.

“It wasn’t like we were superstitious, like if we said something about being in the clear we’d somehow jinx it,” Goodacre says. “But we wanted to be well on the other side of things before we told everybody. The doctors all say that after the five-year mark, things look optimistic, so we’re starting to feel pretty good.”

“It’s not something that’s just going to go away like it never happened,” adds Goodacre. “I’ll always be a little nervous, always having to get checked, always hoping it doesn’t come back.”

On Thursday’s episode of Harry, the couple will candidly discuss her cancer journey and the day she was diagnosed — “It’s one of the hardest days of my life,” she recalls — in a heart-to-heart discussion.

“All I wanted to do was grow old with you and have as many years as possible as I could with you,” Connick Jr. tells Goodacre in a PEOPLE exclusive sneak peek of the interview.

Goodacre tells him, “You always used to say that: ‘I just want to grow old with you.’ ”

“It’s true,” Connick Jr. says. “I wanted to know what you would look like older. … I made the right decision.”

Harry airs weekdays (check local listings or visit Harrytv.com).