Since your report of GZ lagging on Xbox is one of the very few ones ever posted to the forums, i think the issue is with your hardware rather than with the game itself. Moreover since it happens during the beginning of gaming session, when Xbox is accessing game data and loading up the game.
Since there is HDD inside all Xboxes, HDDs do wear out (just like any PC hardware) and get slower, as time moves onwards.
So, it could be that the issue is with aged hardware. However, since GZ is already very borderline game on consoles, consoles are seeing hard time to keep game running smoothly.
If you were on PC, i’d suggest replacing the HDD with SSD and add more RAM as well, and probably even to OC the CPU to fix the lag issues. But since you aren’t, you’re pretty much stuck with what you have.
I don’t know the following for sure, but: Can you change GZ graphical settings on Xbox? Since if you can, you could reduce the graphical settings, which in turn reduces the load on your Xbox and smoothens out the lag. I know that this can be done on PC but on consoles, i have no clue.