Rory Feek Says He Drove to Visit Daughters amid Dispute, But They Wouldn't Talk Without Mediator: 'We're at an Impasse'

Rory Feek is continuing to speak out amid his family dispute over the care of his youngest daughter Indiana — and says a recent trip to meet with his two older daughters did not go according to plan.

Rory, 59, shared a blog post titled “canceled” on Sept. 17, nearly three weeks after daughters Heidi, 37, and Hopie, 35, took legal action against their father, who they claim has cut them off from contacting Indiana, 10, and put her in harm’s way (Details of the filing remain confidential, as Indiana is a minor; the legal proceedings remain ongoing).

The singer-songwriter — who in July married Indiana’s former teacher, a woman named Rebecca — wrote that he knows his family “isn’t perfect,” and that they’ve “always been a work in progress.”

“There are things that my older daughters and I need to work out together,” he wrote. “Right now we’re at an impasse on how that’s ever going to happen.”

Rory wrote that a week and a half ago, he drove down to Alabama, where Heidi and Hopie live, to try and hash out their disagreement.

“[I had] hopes that we might be able to sit down and talk, or better yet, I’d just sit and listen to whatever they wanted to say and maybe have the chance to give them a hug and at least try to show them how much I love them,” he wrote. “But even though their cars were in the driveway and they were inside, no one answered.”

The musician said he eventually left the two bouquets of peace lily flowers that he’d brought on the doorstep and drove back home. On his way back, he allegedly received a text from Heidi that read, “We are only willing to talk with you with a licensed therapist or attorney or both.”

“That is where the impasse comes in. They believe that it’s the job of someone with a doctorate, legal or masters degree to repair what is broken in our family. And I believe that it’s our job,” Rory wrote.

Heidi tells PEOPLE she wasn’t home when Rory arrived, but her husband Dillon and sister Hopie were. Dillon says that upon realizing the unannounced truck in the driveway was Rory, Hopie began having a panic attack.

After a brief call to both Heidi and to their lawyer, which they estimate took about four minutes, Dillon went to answer the door to find Rory had already left on the two-hour drive home.

“Regardless of the intention, it came off as intimidating. We’ve set really clear boundaries. We have such a hard time communicating with him and feeling heard that we really don’t feel like it’s possible without a third party,” says Heidi. “[Visiting] was never for us, because if he was trying to reach us, he would do it in the way that we’ve requested of him, or at least text us and say, ‘Hey, I’m coming.’”

She also confirms that she filed legal documents against her father earlier this month, and that Rory was served the day after he drove to Alabama.

In his post, Rory alleged that he’s had events and writing jobs canceled in the wake of the family’s disagreements being made public, and that “people began to distance themselves” from him.

“Unsure whether what they’ve heard or seen online might be true, and even if it’s not, they need to make sure they don’t personally get caught in any of the crossfire directed at me,” he wrote. “All that said… I don't feel canceled. I feel the opposite of that. While all this has been taking place online, here at our farm in our actual physical lives, we’ve been having the greatest time settling into our new life as a married couple and family.”

Heidi previously expressed her frustrations with her father in an open letter shared with PEOPLE and later published on Instagram. In the letter, she wrote that she and Hopie had been trying to reach Rory offline “for months,” but that he had not been responding.

She included a screenshot of multiple unanswered text messages to her dad that were sent at various points in July asking about Indiana, and also explained why she felt the need to have a therapist or attorney present for any communications.

“The last time we sat down to have this conversation, without a mediator as you requested, you told us you were done being our father,” she wrote. “We forgive you.”

She added, “Our hope is that this letter will find you, so you can know where our hearts are. Love is action, not words. We forgive you for your unkind words. You called us cowardly in your blog post for not facing you. We forgive you.”

Heidi also spoke at length of her concerns about Homestead Heritage, an “agrarian and craft-based intentional Christian community” that Rory and Rebecca are a part of. Multiple members in the past have been arrested and charged with child abuse; Rory admitted there can be “a few bad apples” in large groups, while the community itself said in a statement that “in every case of abuse we’ve ever encountered, it was our ministry that exposed and reported the crime.”

"I really just want to be able to be in my little sister's life and I can't," Heidi tells PEOPLE. "I'm trying to do the right thing."

Meanwhile, in his blog post, Rory said that Indiana, who has special needs, is doing “super well,” and has recently started 5th grade with Rebecca as her homeschool teacher.