Your initial steps of adding a tiny z hop to the retractions seems an interesting one and it must have resolved a lot of troubles relating the first printed layer. Other than this, you could try slowing down the speed of your first layer print and see the difference it makes.