I think you've got it spot on. The threat of a return violence is nothing more than a bluff, and is being shamefully used as a political lever.I have no problem with a border between the UK and the ROI, I also have no problem with 'no border' between the UK and ROI. The only border I have any sort of a problem with, is an internal one between constituent parts of the UK.
I do think that the previous Government let the EU set the agenda on this, whereas Boris has taken the right line: we won't have a border, you do what you want. Further, and I write as someone who was in NI during the 'troubles', i.e. terrorist campaign(s), I am not convinced that anyone will (re)start a terror campaign on the basis of our leaving the EU and not having a border, other than those already inclined that way.
On that last point I'll happily bow to the combined wisdom of my countrymen in NI and stand corrected, but it seems to me - from personal experience and the opinions of family / friends in NI - that despite there always being some idiots, the number of people who will now support a terror campaign has very, very greatly diminished.
Once we're out of the EU, and once it is seen that life will carry on across the border, I think the 'threat' will prove to be no more than a damp squib.