Should I tell my boyfriend that I slept with his dad?

I'm 18 and my boyfriend is 20. I knew him since I was little. He's my neighbor and we have been friends for as far as I could remember. His parents are divorced and he lives with his dad who isn't around all that much. I always thought his dad is a really nice guy but he is always telling me that he's really scared of him for whatever reason. Anyway his dad pretty much watched us grow up together. Last month I came home tipsy from a party and I wanted to crash at his place (my parents get mad at me) but he wasn't picking up his phone so I had to knock. His dad said he was gone for the weekend to visit his mom and I was like can I come crash at his room anyway? He was like okay and one thing kinda led to the next and we hooked up a couple more times after that before I tried to ditch the dad but he's threatening to tell my bf. I don't think its fair to him and I think I should tell him plus I don't want him to hear this from his dad but I'm scared of losing him not only as a bf but as a lifelong friend. The other issue is that his dad is forcing me to keep sleeping with him or he's gonna rat me out. I'm soo confused. Help me!!!