Some friends and I have just had six attempts to get four players in to the same session on Xbox, resulting in a crash to dashboard every time for ALL players, regardless of who was hosting.
Everything seemed be fine until the fourth player joined, then regardless of who it was the game froze for all players, stuttering audio and then a crash to desktop.
We repeated this with different hosts and different joining order but the result was the same as soon as the last player joined.
We each had the DLC installed, 3 players on XB1X and 1 on XB1S.