is there a way to script enabling / disable a smart rule?
No.
would it be possible to have the “Open URL in DT” check the
type
of the current open group?
Items aren’t contained in smart groups, so what would be the point of checking the group?
I think you’re overcomplicating this. I don’t even see a need for a script in this case.
Why don’t you just apply some specific metadata to items you want to operate this way?