Philippians 3:13 "Brethren, I do not regard myself as having laid hold of it yet; but one thing I do: forgetting what lies behind and reaching forward to what lies ahead"
One of the deepest struggles a person can have is the discouragement that comes from regret. I believe the reason why it's so deep and difficult for many is because of one reason: the past is unchangeable.  Anxiousness and fear can be dealt with in a sense - the Lord can give us hope for the future that whatever we fear or are anxious about will not happen.   There's hope things can still work out.  The future can be changed, but the past cannot.  Whatever mistakes we've made in the past can never be unmade.  Many things we left undone can never be done again.  Whatever passed yesterday cannot be attained again.  Yesterday is gone forever, and for many of our decisions - the possibility to change them is gone too.  For many of them, we are now left to live with the consequenses of them, some for the rest of our lives.
Yes, regret can be one of the heaviest weights for a person, even a Christian can carry for their whole life because of that reason: the past is unchangeable.  One example I've had in my life in past years is the regret of not spending more time, or talking more, or praying more for loved ones who've passed away.  Sometimes when I've thought of them, the thought comes into my mind, "I wish... just one more day."  But yesterday is gone, I can never get it back.  When the Lord promises in Joel that He will make up for the years the locusts have eaten (Joel 2:25), how can he restore a field that's totally gone from my life?
I don't have all the answers, but the Lord has tremendously comforted me in the area of the discouragement of regret, and by His grace I can say He freed me from slavery to it. Not just in the instance of loved ones who have passed away, but many other things too.  I just wanted to share one or two things that He spoke to me along the way that were words of life for me, that stayed with me to set me free.
One thing that's really comforted me deeply in the area of regret is this truth:
The Lord loves me, He is sovereign, and in His sovereignty and love He did not stop me from making my mistake.  This means that He has in His Divine wisdom intentionally chosen to make my mistakes part of His plan for my life.
Here's a question: was Peter's denial of Jesus God's will?  Definitely not.  It was a deep sin.  Jesus even went so far as to say once that whoever denies Him He will deny before the Father (Matt 10:33).  It was a serious thing, and Peter knew it.  How could he not carry regret and discouragement for his whole life after that?
But that's not the whole story.  We also have to look at one other question: Was Peter's denial of Jesus part of God's plan?  I believe we can say without a doubt, that it was not God's will, but God knew Peter would deny Him, and so He used it.  Peter was so proud, and God had a great plan for Peter, but the only way to accomplish this was to break Peter's pride by allowing him to fall.
It’s very significant that Jesus did not pray for Peter not to fall, but He did pray that Peter’s faith wouldn’t fail (Luke 22:31-32).  After that point, the always-confident Peter, when he preached at Pentecost and thousands of Jews were saved, he could  remain humble because he carried a reminder with him of how unworthy he was on his own - he never forgot that fall of his.  And for something so ugly (when you focus on that one terrible action of his), the Lord worked it so perfectly into the tapestry of Peter's life when you "Zoom out" to see the whole picture.  Only the power of God can do something like that.  God is such a Master at what He does that He can take an ugly piece of cloth, and sew it into the quilt of someone's life in such a way that the quilt is even more beautiful in the end than it would have been without it.  Peter died beautifully humble, with the honor of being martyred for the Lord (he even requested to be crucified upside-down instead of right-side up because he felt unworthy to be killed like his Savior!).
To me, this tells me that - God will not stop me from making mistakes, even sins, if He needs to do it to accomplish His plan for my life.  Maybe I need to be broken deeper than I thought, or maybe He will correct it in some miraculous way which I never would have imagined - but either way, He allowed it.  He could have made Peter so sick that night of the betrayal, that he was bedridden and nowhere near being questioned by anybody.  He could have let one of the soldiers hit Peter on the head when they arrested Jesus so that he would be knocked out for a few days - no denial by Peter, problem solved!  He could have sent an angel to wake me up at 2AM the day of some mistake of mine to speak to me, "Bobby - God says do this" or "Bobby, at all cost, don't do this",  and stopped me in my tracks, but He didn't.  He allowed me to make the mistake.  Looking back, I should repent with all my heart of my sins, and at the same time I should never believe that God's eternal plan and purpose is ruined and irrecoverable.  If He allowed it - that means that He has decided to include it into His plan for me, I need never ever look back in despair thinking, "Why did I do this, why didn't I do that?  Everything is ruined forever because of me!"
If it would have derailed God’s eternal plan for my life, the Lord would have stopped me immediately.  He will not let go so easy what He paid such a high price for.
It's true that my past may cause years or even decades of grief, sorrow, and difficulty for me in the future.  This can magnify the temptation to give into regret and dwell in deep discouragement.  What do we do in this case?  If this is the case - I believe we can look at 2 Corinthians 12 for encouragement, and take these consequences like Paul looked at his thorn: "If God allowed this, I need this."
Paul said, "I need this thorn to keep me humble, from exalting myself" (2 Cor 12:7).  Life may be difficult from here on out, but it need not be void of complete fullness of joy and rest, and a peace that passes understanding.  In many ways, this can give us even more capacity for joy, because if we're carrying a deep physical or earthly burden it can help us from being deceived by the joy this world offers, and thinking that we really have the joy of the Lord when in fact we don't.  If we have everything going perfectly for us, it’s very easy to be deceived that we are walking just fine with the Lord while all this time our spiritual life may be in shambles (Rev 3:17).
If you're struggling with regret, remember: there's a very, very important reason that the Lord did not prevent what you did (or neglected to do), and He may not show you the reason yet, or maybe never in this life.  But there's a reason.  Endure the consequences faithfully, being totally honest with Him about your past, and allow the Lord to lift you up above the storm of whatever consequences may come (Matt 14:30-31).
Going back to my struggle of wishing I could do differently with loved ones who've passed, my comfort is this: God loves them much more than I will ever love them.  And what He has done in the past (so loved the world and sent Jesus that they may be saved - John 3:16) is infinitely more important than what I’ve done (or not done) in the past!
Anything which I should have done that I've missed, the Lord could have (and may have) worked that in some other way for this person I loved. Maybe even in the last seconds of their life, like the thief on the cross. He knows I loved them, and knew my heart the whole time - that the love itself I have for someone else is actually a manifestation of God’s heart and love for this person that I’m partaking of.  We love because He first did (1 Jn 4:19).  And I believe the Lord is such an intimate Father, even our whispering inward desires He can take as prayers.  
Comparing my life to driving a car on a road: I will look forward through the windshield with an ambition to do things totally differently in the future wanting to stay in God's perfect path for my life.  But when I look backward through the rearview mirror, I'll repent honestly and whole-heartedly, and then praise the Lord for His mercy which covers all the times I missed His will, and for working those times into a great plan He has for me when I "Zoom out" to look at the entire picture eternally.
2 Cor 5:17 "Therefore if anyone is in Christ, he is a new creature; the old things passed away; behold, new things have come."
Carrying regret does not glorify God.  For us in Christ, the old things have passed away, even the ones yesterday.  The day is new today.  Be totally free from regret from today on, do not carry it even one more second.  Instead, turn your regret into praise.  Praise - that the Lord's plan is much, much bigger than your past.