I think the root of the problem lies in the fact that the tool head get to where it needs to go, stops and then Retracts, the relatively long period the head is at a standstill, mother earth is messing things up, by having gravity pull filament from the nozzle. We need to remove the filament, so there is nothing left to pull out, when the head stops. The other main problem lies with the fact that filament is controlled by length, and not pressure/flow.
I am experiencing approximately the same issues with Coasting, The problem when coasting it that the constant identical pressure to the filament gives an even and predictable spread, but when you "release" the pressure by coasting, this lowers the pressure over the distance, and the flow during the coast gets unpredictable.
If you look at pressure for the beginning and end of a filament line, it should start moving slowly and accelerate as the pressure is getting to operating pressure, and the only way i can se for lowering the pressure sharply for endings, is by setting a "Pre-Retraction Distance" instead of Coast, so the flow is not only gradually and unpredictably lowered, but removed, while still applying the remaining distance through gravity, the amount of filament that can be laid down would limit the distance significantly, and will 0,1 mm be too much. That distance should be the same every time if the pressure is the same, when retracting. And that's when it gets hairy, if the previous line was short, did it get to full pressure or did it not, and then the under or over extrusion starts, due to unpredictable filament pressure.
If the temp of the filament is constant, and i assume it is, it should react the same way every time, so the pre-retraction distance should be the same, and you will never end up having spent filament while coasting, that the software has NO way of compensating for. With pre-retract, the result should be far more predictable. If retraction is setup for lifts, the lift should not occur at pre-retract, but at retract.
I guess a calibration tool would be helpful, as:
you would have to know exactly how much distance of filament you can put down after pre-retraction.
you would have to know how much distance is required to get to operating pressure. Though it should be the same as this will happen after pre-retraction, and distance of filament for pre-retract, tells you how much filament you lost and should compensate with by adding retract restart distance.
When i messed around with coasting, with some filaments (Wood specially), i needed a lot of coast, but that affected my infills, as they started out by being very under extruded, but if the layer was big enough, the pressure would be back up fine, but on small layers, the missing filament in the infill made the print weak, ALSO outlines suffers from coasting on small area layers.
The problem basically is, that there is no way of knowing how much filament is used during Coast, and how do you compensate for the unknown flow while coasting. If the filament to lay down is close to coasting distance will it ever add any filament, or will it take so long to add and remove pressure that it ends up adding way to much filament. Even moving while retracted, the pressure is changing down, so depending on the length of the move, the pressure is again unpredictable.
I believe that the whole issue is a question of timing of speed and pressure, but as pressure information is unavailable, we can only try to find the best solution. But gravity will ultimately be the biggest problem, it will pull filament out, no matter what we do.